System

Abstract

The system provided by the embodiment of the invention aims to provide an optimal playing posture and fingering means based on the physique and the finger length of the user. A system according to an embodiment includes an acquisition unit, an analysis unit, a generation unit, a provision unit, and an evaluation unit. The acquisition part is used for acquiring physique data. The analysis unit analyzes the physique data acquired by the acquisition unit. The generation unit generates an optimal performance posture and fingering means on the basis of the data analyzed by the analysis unit. The providing section provides the feedback generated by the generating section. The evaluation unit evaluates the progress of the user on the basis of the feedback provided by the provision unit, and proposes a new exercise method or subject.

Description

Technical Field

[0001] The technology disclosed herein relates to a system. Background Technology

[0002] Patent Document 1 discloses a personalized chatbot control method executed by at least one processor, the method comprising: receiving user speech; adding the user speech to a prompt containing instructions related to a chatbot role; encoding the prompt; and inputting the encoded prompt into a language model to generate chatbot speech in response to the user speech.

[0003] Patent document 1: Japanese Patent Application Publication No. 2022-180282. Summary of the Invention

[0004] In the current technology, it is difficult for beginners of musical instruments to find practice methods that are suitable for their own physique and finger length, and it is difficult to obtain personalized and optimized guidance. Such problems exist.

[0005] The system involved in this technical solution aims to provide the optimal playing posture and fingering techniques based on the user's physique and finger length.

[0006] The system described in this embodiment includes an acquisition unit, an analysis unit, a generation unit, a provision unit, and an evaluation unit. The acquisition unit acquires physical data. The analysis unit analyzes the physical data acquired by the acquisition unit. The generation unit generates optimal playing postures and fingering techniques based on the data analyzed by the analysis unit. The provision unit provides feedback generated by the generation unit. The evaluation unit evaluates the user's progress based on the feedback provided by the provision unit and proposes new practice methods or challenges.

[0007] The system described in this embodiment can provide optimal playing posture and fingering techniques based on the user's physique and finger length. Attached Figure Description

[0008] Figure 1 This is a conceptual diagram illustrating an example of the configuration of a data processing system according to the first embodiment.

[0009] Figure 2 This is a conceptual diagram illustrating an example of the main functions of the data processing apparatus and smart device according to the first embodiment.

[0010] Figure 3 This is a conceptual diagram illustrating an example of the data processing system configuration in the second embodiment.

[0011] Figure 4 This is a conceptual diagram illustrating an example of the main functions of the data processing device and smart glasses according to the second embodiment.

[0012] Figure 5 This is a conceptual diagram illustrating an example of the data processing system configuration in the third embodiment.

[0013] Figure 6 This is a conceptual diagram illustrating an example of the main functions of the data processing device and head-mounted terminal according to the third embodiment.

[0014] Figure 7 This is a conceptual diagram illustrating an example of the data processing system configuration in the fourth embodiment.

[0015] Figure 8 This is a conceptual diagram illustrating an example of the functions of the main parts of the data processing device and robot according to the fourth embodiment.

[0016] Figure 9 It represents an emotion graph that maps multiple emotions.

[0017] Figure 10 It represents an emotion graph that maps multiple emotions.

[0018] Explanation of reference numerals in the attached figures

[0019] Data processing systems 10, 210, 310, and 410

[0020] 12 Data processing device

[0021] 14 Smart devices

[0022] 214 Smart Glasses

[0023] 314 Head-mounted terminal

[0024] 414 Robot. Detailed Implementation

[0025] Hereinafter, an example of an implementation of the system involved in this disclosure will be described with reference to the accompanying drawings.

[0026] First, let's explain the terms used in the following description.

[0027] In the following embodiments, the processor (hereinafter referred to as "processor"), as indicated by the reference numerals, can be a single computing device or a combination of multiple computing devices. Furthermore, a processor can be a single computing device or a combination of multiple computing devices. Examples of computing devices include CPU (Central Processing Unit), GPU (Graphics Processing Unit), GPGPU (General-Purpose computing on Graphics Processing Units), APU (Accelerated Processing Unit), or TPU (Tensor Processing Unit), etc.

[0028] In the following embodiments, RAM (Random Access Memory), as indicated in the figures, is a memory for temporary information storage that is used by the processor as working memory.

[0029] In the following embodiments, the memory, as indicated by the reference numerals, is one or more non-volatile storage devices used to store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), disk (e.g., hard disk), or magnetic tape, etc.

[0030] In the following embodiments, the communication I / F (Interface) with reference numerals is an interface including a communication processor and an antenna, etc. The communication I / F is responsible for communication between multiple computers. Examples of communication standards applicable to the communication I / F include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0031] In the following implementation, "A and / or B" is synonymous with "at least one of A and B". That is, "A and / or B" means that it can be only A, only B, or a combination of A and B. Furthermore, in this specification, when "and / or" connects more than three items, the same approach as "A and / or B" applies.

[0032] First Implementation Method

[0033] Figure 1 An example of the configuration of the data processing system 10 according to the first embodiment is shown.

[0034] like Figure 1As shown, the data processing system 10 includes a data processing device 12 and an intelligent device 14. An example of the data processing device 12 is a server.

[0035] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, RAM 30, and memory 32. The processor 28, RAM 30, and memory 32 are connected to a bus 34. Furthermore, the database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. An example of the network 54 includes a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0036] The smart device 14 includes a computer 36, a receiver 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, RAM 48, and memory 50. The processor 46, RAM 48, and memory 50 are connected to a bus 52. In addition, the receiver 38, output device 40, and camera 42 are also connected to the bus 52.

[0037] The receiving device 38 includes a touchscreen 38A and a microphone 38B, etc., for receiving user input. The touchscreen 38A receives user input generated by contact with an indicator (e.g., a pen or finger). The microphone 38B receives user input generated by sound by detecting the user's voice. The control unit 46A sends data representing user input received via the touchscreen 38A and microphone 38B to the data processing unit 12. In the data processing unit 12, a specific processing unit 290 (see reference...) Figure 2 Get the data that represents the user input.

[0038] The output device 40 includes a display 40A and a speaker 40B, etc., and presents data to the user by outputting data in a user-perceptible form (e.g., sound and / or text). The display 40A displays visual information such as text and images according to instructions from the processor 46. The speaker 40B outputs sound according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0039] Communication I / F 44 is connected to network 54. Communication I / F 44 and 26 are responsible for sending and receiving various information between processor 46 and processor 28 via network 54.

[0040] Figure 2 An example of the main functions of the data processing device 12 and the smart device 14 is shown.

[0041] like Figure 2 As shown, in the data processing apparatus 12, specific processing is performed by the processor 28. A specific processing program 56 is stored in the memory 32. The specific processing program 56 is an example of a "program" as understood in this disclosure. The processor 28 reads the specific processing program 56 from the memory 32 and executes the read specific processing program 56 on the RAM 30. Specific processing is implemented by the processor 28 operating as a specific processing unit 290 based on the specific processing program 56 executed on the RAM 30.

[0042] The memory 32 stores a data generation model 58 and an emotion-specific model 59. The data generation model 58 and the emotion-specific model 59 are used by the specific processing unit 290. The specific processing unit 290 can use the emotion-specific model 59 to infer the user's emotions and perform specific processing based on the user's emotions. The emotion inference function using the emotion-specific model 59 (emotion-specific function) includes inferring and predicting the user's emotions, performing various inferences and predictions related to the user's emotions, but is not limited to this example. Furthermore, emotion inference and prediction also include, for example, emotion analysis (analysis).

[0043] In the smart device 14, specific processing is performed by the processor 46. A specific processing program 60 is stored in the memory 50. The specific processing program 60 is used in conjunction with the data processing system 10. The processor 46 reads the specific processing program 60 from the memory 50 and executes the read specific processing program 60 on the RAM 48. Specific processing is implemented by the processor 46 operating as a control unit 46A based on the specific processing program 60 executed on the RAM 48. Furthermore, the smart device 14 may also have the same data generation model and emotion-specific model as the data generation model 58 and the emotion-specific model 59, and use these models to perform the same processing as the specific processing unit 290.

[0044] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device (e.g., a generation server) may have the data generation model 58. In this case, the data processing device 12 obtains the processing results (prediction results, etc.) using the data generation model 58 by communicating with the server device that has the data generation model 58. Furthermore, the data processing device 12 may be a server device or a user-owned terminal device (e.g., a mobile phone, robot, home appliance, etc.). Next, an example of the processing performed by the data processing system 10 of the first embodiment will be described.

[0045] Implementation Method 1

[0046] The personalized performance coach described in this invention is an AI system designed to address the challenges faced by beginners and intermediate musicians. Many beginners struggle to find practice methods suitable for their individual physical characteristics, such as physique and finger length, often leading them to give up. For example, stories of people giving up the guitar because they can't press the F chord are common. While attending music classes is an option, personalized instruction is often dependent on the instructor's skill and experience, and the availability of optimized guidance is largely a matter of luck. The personalized performance coach utilizes multimodal AI to provide individualized guidance on optimal playing style and fingering techniques based on the user's physique and finger length. This addresses issues such as "tendonitis and other physical burdens caused by incorrect instrument holding" and "finger inability to reach or dexterous fingers," helping beginners who are prone to giving up at the initial stages of instrument playing. Furthermore, for intermediate learners who are prone to developing bad habits, the system also considers skeletal and joint range of motion to provide corrective guidance for skill improvement. For instance, users can use their smartphone cameras to photograph their physique, and the AI ​​will automatically analyze physical characteristics such as height, limb length, and joint position to create a personalized user profile. Next, users record videos of themselves playing their instruments using their smartphones. AI analyzes the videos and compares them to ideal playing postures and fingering techniques. Based on the AI's analysis, specific feedback is generated. This feedback is overlaid on the user's performance using AR technology, allowing for intuitive understanding. Furthermore, the AI ​​regularly evaluates the user's progress and suggests new practice methods or topics based on their skill level. Thus, even beginners who have mastered basic skills can continue to receive advanced feedback and playing topics geared towards intermediate learners. Personalized performance coaches prevent beginners from giving up and support continued instrument playing by providing personalized, optimized guidance, readily available support, and efficient skill enhancement opportunities. Users can practice according to their own topics and pace, truly experiencing the joy of playing an instrument. Therefore, personalized performance coaches can provide optimal playing postures and fingering techniques based on the user's physical data and achieve personalized, optimized guidance through progress evaluation.

[0047] The personalized performance coach described in this embodiment includes an acquisition unit, an analysis unit, a generation unit, a provision unit, and an evaluation unit. The acquisition unit acquires the user's physical data. For example, the acquisition unit can use a smartphone camera to acquire the user's physical data. The acquisition unit can automatically analyze the user's height, limb length, joint position, and other physical characteristics. The analysis unit analyzes the physical data acquired by the acquisition unit. For example, the analysis unit can generate a user-specific profile based on the acquired physical data. The analysis unit analyzes the user's physical data in detail to generate basic data for providing personalized optimization guidance. The generation unit generates optimal playing posture and fingering techniques based on the data analyzed by the analysis unit. For example, the generation unit can analyze the user's performance video to generate ideal playing posture and fingering techniques. The generation unit uses a generative AI to analyze the user's performance video and generate optimal playing posture and fingering techniques. The provision unit provides feedback generated by the generation unit. For example, the provision unit can overlay the feedback generated by the generative AI onto the user's performance in an AR manner. The provision unit uses AR technology to display feedback, allowing the user to intuitively understand the feedback. The evaluation unit evaluates the user's progress based on the feedback provided by the provision unit and proposes new practice methods or topics. For example, the evaluation department can periodically assess the user's progress and propose new practice methods or topics based on their skill level. The evaluation department provides detailed assessments of the user's progress to offer appropriate practice methods or topics according to their skill level. Thus, the personalized performance coach described in this embodiment can provide optimal playing posture and fingering techniques based on the user's physical data, and achieve personalized optimization guidance through progress evaluation.

[0048] The acquisition unit is used to acquire users' physical data. For example, the acquisition unit can use a smartphone camera to acquire users' physical data. Specifically, it uses the smartphone camera to photograph the user's full body and automatically analyzes the user's height, limb length, joint position, and other physical characteristics through image processing technology. For this purpose, deep learning image recognition algorithms can be used to acquire users' physical data with high accuracy. For example, if a user stands in front of a smartphone camera in a specific pose, the acquisition unit can capture images from multiple angles and generate a 3D model. Based on this 3D model, the user's physical data can be analyzed in detail. In addition, the acquisition unit can also acquire data from wearable devices used by users daily. For example, it can collect data such as heart rate, steps, and calories burned from smartwatches or fitness trackers and integrate this data with the user's physical data to create a more comprehensive profile. Thus, the acquisition unit can collect users' physical data from multiple angles, providing detailed and accurate data.

[0049] The analysis unit is used to analyze the physical data acquired by the acquisition unit. For example, the analysis unit can generate a user-specific profile based on the acquired physical data. Specifically, the analysis unit analyzes the data from the acquisition unit to gain a detailed understanding of the user's physical characteristics. To this end, machine learning algorithms are used to classify the user's physical data and extract feature quantities. For example, based on data such as the user's height, limb length, and joint position, basic data is generated to determine the most suitable playing posture and fingering techniques for the user's physique. The analysis unit uses this data to create a personalized guidance profile suitable for the user's physique. This profile includes not only the user's physical characteristics but also past performance data and practice records. Thus, the analysis unit can analyze the user's physical data in detail to generate basic data for providing personalized optimization guidance. In addition, the analysis unit can continuously monitor the user's physical data and update the profile when necessary. Therefore, the analysis unit can always provide optimal guidance based on the latest data.

[0050] The generation department generates optimal playing posture and fingering techniques based on the data analyzed by the analysis department. For example, the generation department can analyze a user's performance video to generate ideal playing posture and fingering techniques. Specifically, the generation department uses generative AI to analyze the user's performance video and generate optimal playing posture and fingering techniques. The generative AI uses deep learning video analysis technology to analyze the user's playing movements in detail. For example, it analyzes the user's hand movements, finger positions, body posture, etc., when playing an instrument to determine the ideal playing posture and fingering techniques. Based on this data, the generation department generates playing posture and fingering techniques most suitable for the user's physique and playing style. In addition, the generation department also generates feedback to provide to the user, including specific playing instructions and practice method suggestions. For example, the generation department can generate practice methods to improve specific playing techniques or specifically point out areas for improvement in playing posture. Thus, the generation department can generate playing posture and fingering techniques most suitable for the user's physique and playing style, providing personalized optimization guidance.

[0051] The Provision Department provides feedback generated by the Generation Department. For example, it can overlay AI-generated feedback onto the user's performance using AR technology. Specifically, AR technology is used to overlay feedback onto the user's performance video in real time. This allows the user to intuitively understand areas for improvement while watching their own performance. For instance, when a user plays an instrument, real-time feedback on hand position and finger movements can be displayed, indicating correct playing posture and fingering techniques. The Provision Department can use visual guidance or animation to help users intuitively understand the feedback. This allows users to easily grasp specific areas for improvement while watching their own performance. Furthermore, the Provision Department can record the user's reactions to feedback and reflect them in subsequent feedback sessions. This allows the Provision Department to provide appropriate feedback based on the user's progress, achieving personalized optimization guidance. In addition, the Provision Department can flexibly adjust the display method of feedback based on the environment and device the user uses to receive it. For example, it can provide corresponding feedback displays for various devices such as smartphones, tablets, and AR glasses. This ensures that the Provision Department can provide optimal feedback to users in any environment.

[0052] Based on feedback provided by the provision department, the evaluation department assesses user progress and proposes new practice methods or topics. Specifically, the evaluation department analyzes user performance data and feedback history to evaluate the user's skill level and progress in detail. To this end, machine learning algorithms are used to analyze user performance data, evaluating skill improvement and topic achievement. For example, it evaluates the time spent by the user to master a specific playing technique and the achievable playing accuracy. Based on this data, the evaluation department proposes new practice methods or topics according to the user's skill level. For example, it specifically points out practice methods to improve a particular playing technique or the next topic to be mastered. Furthermore, the evaluation department regularly evaluates user progress and provides feedback to support continuous skill improvement. Thus, the evaluation department can provide appropriate practice methods or topics based on the user's skill level, achieving personalized optimization guidance. In addition, the evaluation department can record user reactions to feedback and reflect them in the next evaluation, thereby conducting more precise evaluations. Therefore, the evaluation department can comprehensively evaluate user progress and generate the foundational data for providing personalized optimization guidance.

[0053] The provisioning department can overlay AI-generated feedback onto the user's performance using AR technology. For example, the provisioning department can use AI to analyze the user's performance video, generate ideal playing posture and fingering techniques, and display this feedback in AR. The provisioning department utilizes AR technology to display feedback, allowing the user to intuitively understand it. For example, the provisioning department can visually overlay the ideal playing posture and fingering techniques onto the user's performance video. This allows the user to practice while comparing their own performance with the ideal playing posture. Some or all of the above processing in the provisioning department can be achieved through AI, or it can be done without AI. For example, the provisioning department can use a system that takes AI-generated feedback as input and AR display as output to display the feedback. This allows the user to understand the feedback more intuitively.

[0054] The evaluation department can periodically assess users' progress and propose new practice methods or topics based on their skill levels. For example, the evaluation department can periodically analyze users' performance videos and evaluate their progress. Based on the user's skill level, the evaluation department proposes appropriate practice methods or topics. For instance, the evaluation department can propose basic practice methods for beginners, or advanced practice methods or topics for intermediate learners. The evaluation department provides detailed evaluations of users' progress, offering feedback appropriate to their skill level. Thus, users can obtain corresponding practice methods or topics based on their own skill level, efficiently improving their skills. Some or all of the above processing in the evaluation department can be implemented using AI, or it can be done without AI. For example, the evaluation department can input users' performance videos into AI, which will then perform progress evaluation. This allows for the provision of appropriate practice methods or topics based on the user's skill level.

[0055] The acquisition unit can utilize a smartphone's camera to acquire a user's physical data. For example, the acquisition unit can obtain this data by having the user photograph their own physique using their smartphone camera. The acquisition unit then automatically analyzes the user's physical characteristics, such as height, limb length, and joint position, using the smartphone camera. To facilitate easy acquisition of physical data by users, the acquisition unit provides a simple method based on the smartphone camera. For example, the acquisition unit can analyze image data captured by the smartphone camera to extract physical data. Thus, users can easily acquire physical data without special equipment. Some or all of the above processing in the acquisition unit can be implemented using AI, or it can be done without AI. For example, the acquisition unit can input image data captured by the smartphone camera into AI, which will then perform physical data analysis. This allows users to easily acquire physical data.

[0056] The analysis unit can generate user-specific profiles based on the acquired physical data. For example, the analysis unit can analyze the acquired physical data in detail to generate user-specific profiles. Based on the user's physical data, the analysis unit generates foundational data for providing personalized optimization guidance. For example, the analysis unit analyzes the user's height, limb length, joint position, and other physical characteristics to generate user-specific profiles. Thus, the user can obtain the most suitable playing posture and fingering techniques for their own physique. Some or all of the above processing in the analysis unit can be implemented using AI, or it can be done without AI. For example, the analysis unit can input the acquired physical data into AI, which will then execute the generation of user-specific profiles. Thus, personalized optimization guidance is achieved by generating user-specific profiles.

[0057] The generation unit can analyze a user's performance video to generate ideal playing posture and fingering techniques. For example, the generation unit can input the user's performance video into the generation AI, which then generates the ideal playing posture and fingering techniques. The generation unit uses the generation AI to analyze the user's performance video and generate optimal playing posture and fingering techniques. For example, the generation unit can analyze the user's performance video in detail to generate ideal playing posture and fingering techniques. Thus, the user can practice while comparing their own performance with the ideal playing posture. Some or all of the above processing in the generation unit can be achieved through the generation AI, or it can be done without using the generation AI. For example, the generation unit can input the user's performance video into the generation AI, which then generates the ideal playing posture and fingering techniques. Thus, by analyzing the user's performance video, ideal playing posture and fingering techniques can be provided.

[0058] The data acquisition department can analyze a user's past physical data and select the optimal acquisition method. For example, based on the user's past physical data, the department can choose the method that obtains the most accurate data. The department can also analyze changes in the user's past physical data and select a stable data acquisition method. Furthermore, the department can consider the frequency of past physical data acquisition and select the optimal acquisition time. Thus, physical data can be acquired in the optimal way based on past data. Some or all of the above processing in the acquisition department can be implemented using AI, or it can be done without AI. For example, the acquisition department can input the user's past physical data into AI, which can then select the optimal acquisition method. Thus, physical data can be acquired in the optimal way based on past data.

[0059] The data acquisition department can filter physical data based on the user's current health status and lifestyle habits. For example, it can consider the user's current health status and acquire physical data when they are in good physical condition. It can also analyze the user's lifestyle habits and acquire physical data at the most suitable time of day. Furthermore, the acquisition department can select the type of data to acquire based on the user's health status and lifestyle habits. Thus, data can be acquired according to the user's health status and lifestyle habits. Some or all of the above processing in the acquisition department can be implemented using AI, or it can be done without AI. For example, the acquisition department can input the user's health data into AI, which can then perform the filtering. Thus, data can be acquired according to the user's health status and lifestyle habits.

[0060] The acquisition department can consider the user's geographical location information when acquiring physical data, prioritizing the acquisition of highly relevant data. For example, when a user is at high altitude, data related to oxygen concentration and air pressure can be prioritized. When a user is in a city, data related to environmental noise and vibration can be prioritized. When a user is indoors, data related to indoor temperature and humidity can be prioritized. Thus, highly relevant data can be acquired based on the user's geographical location information. Some or all of the above processing in the acquisition department can be implemented using AI, or it can be done without AI. For example, the acquisition department can input the user's geographical location information into AI, which can then perform the acquisition of highly relevant data. Thus, highly relevant data can be acquired based on the user's geographical location information.

[0061] The acquisition department can analyze users' social media activities to obtain relevant data while acquiring physical data. For example, when a user posts sports-related content on social media, physical data related to that activity can be acquired. When a user shares health information on social media, physical data can also be acquired based on that information. When a user participates in a specific activity on social media, physical data related to that activity can also be acquired. Thus, relevant data can be acquired based on users' social media activities. Some or all of the above processing in the acquisition department can be implemented using AI, or it can be done without AI. For example, the acquisition department can input users' social media data into AI, which can then perform the acquisition of relevant data. Thus, relevant data can be acquired based on users' social media activities.

[0062] The analysis unit can adjust the level of detail in the analysis based on the importance of the physical data. For example, it can perform detailed analysis on important physical data, while simplifying the analysis of basic physical data. It can also perform detailed analysis on physical data relevant to a specific purpose, depending on that purpose. Thus, important data can be analyzed in detail. Some or all of the above processing in the analysis unit can be implemented using AI, or it can be done without AI. For example, the analysis unit can input physical data into AI, which can then adjust the level of detail in the analysis based on its importance. This allows for detailed analysis of important data.

[0063] The parsing unit can apply different parsing algorithms based on the category of the physical data during parsing. For example, a skeletal parsing algorithm can be applied to skeletal data. A muscle parsing algorithm can also be applied to muscle data. A joint parsing algorithm can also be applied to joint data. Thus, appropriate parsing can be performed according to the data category. Some or all of the above processing in the parsing unit can be implemented by AI, or AI can be used without AI. For example, the parsing unit can input physical data into AI, and the AI ​​can execute the application of category-based parsing algorithms. Thus, appropriate parsing can be performed according to the data category.

[0064] The parsing unit can determine the parsing priority based on when the physical data was acquired. For example, it can prioritize parsing the most recent physical data. It can also refer to previous physical data when parsing the most recent data. Alternatively, it can prioritize parsing physical data acquired within a specific period. Thus, the most recent data can be parsed first. Some or all of the above processing in the parsing unit can be implemented using AI, or it can be done without AI. For example, the parsing unit can input physical data into AI, which will then execute a parsing priority based on the acquisition time. Thus, the most recent data can be parsed first.

[0065] The parsing unit can adjust the parsing order based on the relevance of the physical data during parsing. For example, it can prioritize parsing important physical data, highly relevant physical data, or physical data related to a specific purpose. Thus, highly relevant data can be parsed first. Some or all of the above processing in the parsing unit can be implemented using AI, or it can be done without AI. For example, the parsing unit can input physical data into AI, which can then perform a relevance-based adjustment of the parsing order. This allows highly relevant data to be parsed first.

[0066] The generation unit can adjust the level of detail generated based on the importance of physical data during the generation process. For example, it can generate detailed playing postures and fingering techniques based on important physical data. It can also generate simplified playing postures and fingering techniques based on basic physical data. Furthermore, it can generate detailed playing postures and fingering techniques adapted to a specific purpose based on physical data relevant to that purpose. Thus, detailed playing postures and fingering techniques can be generated based on important data. Some or all of the above processing in the generation unit can be implemented using AI, or it can be done without AI. For example, the generation unit can input physical data into AI, which can then adjust the level of detail based on importance. Thus, detailed playing postures and fingering techniques can be generated based on important data.

[0067] The generation unit can apply different generation algorithms based on the category of physical data during generation. For example, it can generate suitable playing postures and fingering techniques based on skeletal data. It can also generate suitable playing postures and fingering techniques based on muscle data. Furthermore, it can generate suitable playing postures and fingering techniques based on joint data. Thus, appropriate playing postures and fingering techniques can be generated according to the data category. Some or all of the above processing in the generation unit can be implemented using AI, or it can be done without AI. For example, the generation unit can input physical data into AI, which will then execute a category-based generation algorithm. This allows for the generation of appropriate playing postures and fingering techniques based on the data category.

[0068] The generation unit can determine the generation priority based on when the physical data is acquired. For example, it can generate playing postures and fingering techniques based on the latest physical data. It can also refer to previous physical data and generate playing postures and fingering techniques based on the latest data. Alternatively, it can generate playing postures and fingering techniques based on physical data acquired within a specific period. Thus, playing postures and fingering techniques can be generated based on the latest data. Some or all of the above processing in the generation unit can be implemented using AI, or AI can be omitted. For example, the generation unit can input physical data into AI, which will then determine the generation priority based on the acquisition time. Thus, playing postures and fingering techniques can be generated based on the latest data.

[0069] The generation unit can adjust the generation order based on the relevance of physical data during generation. For example, it can prioritize generating playing postures and fingering techniques based on important physical data. It can also prioritize generating playing postures and fingering techniques based on highly relevant physical data. Furthermore, it can prioritize generating playing postures and fingering techniques based on physical data relevant to a specific purpose. Thus, playing postures and fingering techniques can be generated based on highly relevant data. Some or all of the above processing in the generation unit can be implemented using AI, or AI can be omitted. For example, the generation unit can input physical data into AI, which will then perform a relevance-based generation order adjustment. Thus, playing postures and fingering techniques can be generated based on highly relevant data.

[0070] The feedback department can refer to users' past feedback history to select the optimal display method when providing feedback. For example, the feedback department can select the most effective display method based on users' past feedback history. It can also analyze users' past feedback history to select an easy-to-understand display method. Furthermore, it can refer to users' past feedback history to adjust the display order of feedback. Thus, the optimal display method can be selected based on past feedback history. Some or all of the above processing in the feedback department can be implemented using AI, or it can be done without AI. For example, the feedback department can input users' feedback history into AI, which will then select the optimal display method. Thus, the optimal display method can be selected based on past feedback history.

[0071] The feedback provision department can customize the feedback content based on the user's current performance status. For example, the department can analyze the user's current performance and provide appropriate feedback. It can also adjust the level of detail in the feedback based on the user's performance status, and determine the priority of the feedback based on the user's performance status. Thus, appropriate feedback can be provided based on the current performance status. Some or all of the above processing in the provision department can be implemented using AI, or it can be done without AI. For example, the department can input the user's performance data into AI, which can then perform feedback customization. Thus, appropriate feedback can be provided based on the current performance status.

[0072] The feedback delivery department can consider the user's geographic location information when providing feedback and select the optimal feedback method. For example, visual feedback can be prioritized when the user is outdoors. Detailed feedback can be provided when the user is indoors. Concise feedback can be provided when the user is moving. Thus, the optimal feedback method can be selected based on geographic location information. Some or all of the above processing in the delivery department can be implemented using AI, or it can be done without AI. For example, the delivery department can input the user's geographic location information into AI, which will then select the optimal feedback method. Thus, the optimal feedback method can be selected based on geographic location information.

[0073] The feedback provision department can analyze users' social media activities and propose feedback methods when providing feedback. For example, when a user posts sports-related content on social media, feedback related to that activity can be provided. When a user shares health information on social media, feedback can also be provided based on that information. When a user participates in a specific activity on social media, feedback related to that activity can also be provided. Thus, appropriate feedback methods can be proposed based on social media activities. Some or all of the above processing in the feedback provision department can be implemented using AI, or it can be done without AI. For example, the feedback provision department can input users' social media data into AI, which can then propose feedback methods. Thus, appropriate feedback methods can be proposed based on social media activities.

[0074] The evaluation department can analyze a user's past performance data during the evaluation process to select the optimal evaluation method. For example, the evaluation department can choose the most effective evaluation method based on the user's past performance data. It can also analyze past performance data to select an easy-to-understand evaluation method. Furthermore, it can refer to past performance data to adjust the display order of evaluations. Thus, the optimal evaluation method can be selected based on past performance data. Some or all of the above processing in the evaluation department can be implemented using AI, or it can be done without AI. For example, the evaluation department can input the user's performance data into AI, which will then select the optimal evaluation method. Thus, the optimal evaluation method can be selected based on past performance data.

[0075] The evaluation department can customize evaluation criteria based on the user's current skill level during the evaluation process. For example, the evaluation department can analyze the user's current skill level and set appropriate evaluation criteria. It can also adjust the level of detail in the evaluation based on the user's skill level. Furthermore, it can determine the priority of the evaluation based on the user's skill level. Thus, appropriate evaluation criteria can be set based on the current skill level. Some or all of the above processes in the evaluation department can be implemented using AI, or they can be performed without AI. For example, the evaluation department can input the user's skill data into AI, which will then execute the customization of the evaluation criteria. Thus, appropriate evaluation criteria can be set based on the current skill level.

[0076] The evaluation department can consider the user's geographical location information during the evaluation process to select the optimal evaluation method. For example, when the user is outdoors, a visual evaluation can be prioritized. When the user is indoors, a detailed evaluation can also be provided. When the user is on the move, a concise evaluation can also be provided. Thus, the optimal evaluation method can be selected based on geographical location information. Some or all of the above processing in the evaluation department can be implemented using AI, or it can be done without AI. For example, the evaluation department can input the user's geographical location information into AI, which will then select the optimal evaluation method. Thus, the optimal evaluation method can be selected based on geographical location information.

[0077] The evaluation department can analyze users' social media activities during the evaluation process to propose evaluation methods. For example, when a user posts sports-related content on social media, evaluations related to that activity can be provided. When a user shares health information on social media, evaluations can also be provided based on that information. When a user participates in a specific activity on social media, evaluations related to that activity can also be provided. Thus, appropriate evaluation methods can be proposed based on social media activities. Some or all of the above processing in the evaluation department can be implemented using AI, or it can be done without AI. For example, the evaluation department can input users' social media data into AI, which can then propose evaluation methods. Thus, appropriate evaluation methods can be proposed based on social media activities.

[0078] The system involved in this embodiment is not limited to the above examples. For example, various modifications can be made as follows.

[0079] In addition to physical data, the acquisition department can also acquire user lifestyle data. For example, it can collect data on sleep patterns, diet, and exercise habits, and use this data to infer the user's physical condition and energy level. The analysis department can generate an optimal exercise plan that takes into account the user's physical condition and energy level based on the acquired lifestyle data. The generation department generates exercise content suitable for the user's physical condition and energy level based on the analysis results. The provision department provides the generated exercise content to the user, supporting continuous practice without pressure. Thus, personalized optimization guidance can be achieved based on the user's lifestyle.

[0080] The evaluation department can analyze users' past practice data to select the optimal evaluation method. For example, it can choose the most effective evaluation method based on past practice data. It can also analyze past practice data to select an easy-to-understand evaluation method. Furthermore, it can adjust the display order of evaluations by referring to past practice data. Thus, the optimal evaluation method can be selected based on past practice data. Some or all of the above processing in the evaluation department can be implemented using AI, or it can be done without AI. For example, the evaluation department can input users' practice data into AI, which will then select the optimal evaluation method. Thus, the optimal evaluation method can be selected based on past practice data.

[0081] In addition to physical data, the acquisition department can also acquire user health data. For example, it can acquire data such as heart rate, blood pressure, and body temperature, and infer the user's health status based on this data. The analysis department can generate an optimal exercise plan that takes into account the user's health status based on the acquired health data. The generation department generates exercise content suitable for the user's health status based on the analysis results. The delivery department provides the generated exercise content to the user, supporting continuous practice without pressure. Thus, personalized optimization guidance can be achieved based on the user's health status.

[0082] In addition to analyzing user performance videos, the generation department can also analyze user audio data to generate optimal playing posture and fingering techniques. For example, it acquires audio data from the user's performance, analyzing volume and rhythmic accuracy. Based on the analysis results, the generation department can adjust the user's playing posture and fingering techniques. The provision department then provides the user with the generated playing posture and fingering techniques, supporting the user in improving their musical expression. Thus, personalized optimization guidance can be achieved using audio data.

[0083] In addition to user performance data, the evaluation department can also consider user musical preferences and goals when making evaluations. For example, it can obtain the user's preferred music genres and target performance styles, and set evaluation criteria based on this information. The evaluation department can adjust the level of detail and feedback content of the evaluation according to the user's musical preferences and goals. Thus, appropriate evaluations can be provided based on the user's individual objectives. Some or all of the above processing in the evaluation department can be implemented using AI, or it can be done without AI. For example, the evaluation department can input the user's musical preferences and target data into AI, which will then execute the setting of evaluation criteria. This allows for the provision of appropriate evaluations based on the user's musical preferences and goals.

[0084] In addition to the user's physical data, the acquisition unit can also acquire the user's geographic location information. For example, when the user is at a high altitude, data related to oxygen concentration and air pressure can be acquired. When the user is in a city, data related to environmental noise and vibration can be acquired. When the user is indoors, data related to indoor temperature and humidity can be acquired. Thus, highly relevant data can be acquired based on the user's geographic location information. Some or all of the above processing in the acquisition unit can be implemented using AI, or it can be done without AI. For example, the acquisition unit can input the user's geographic location information into AI, which can then perform the acquisition of highly relevant data. Thus, highly relevant data can be acquired based on the user's geographic location information.

[0085] The following is a brief description of the processing flow of Implementation Method 1.

[0086] Step 1: The acquisition unit acquires the user's physical data. For example, the acquisition unit can use a smartphone camera to acquire the user's physical data. The acquisition unit automatically analyzes the user's height, limb length, joint position, and other physical characteristics.

[0087] Step 2: The analysis unit analyzes the physical data acquired by the acquisition unit. For example, the analysis unit can generate a user-specific profile based on the acquired physical data. The analysis unit analyzes the user's physical data in detail to generate the basic data used to provide personalized optimization guidance.

[0088] Step 3: The generation unit generates the optimal playing posture and fingering techniques based on the data analyzed by the analysis unit. For example, the generation unit can analyze the user's performance video to generate ideal playing posture and fingering techniques. The generation unit uses generation AI to analyze the user's performance video and generate the optimal playing posture and fingering techniques.

[0089] Step 4: The Provision Department provides feedback generated by the Generation Department. For example, the Provision Department can overlay the feedback generated by the AI ​​onto the user's performance using AR technology. The Provision Department uses AR technology to display the feedback, allowing the user to intuitively understand it.

[0090] Step 5: Based on the feedback provided by the supply department, the evaluation department assesses the user's progress and proposes new practice methods or topics. For example, the evaluation department may periodically evaluate the user's progress and propose new practice methods or topics based on the skill level. The evaluation department provides a detailed assessment of the user's progress in order to provide appropriate practice methods or topics based on the user's skill level.

[0091] Implementation Method 2

[0092] The personalized performance coach described in this invention is an AI system designed to address the challenges faced by beginners and intermediate musicians. Many beginners struggle to find practice methods suitable for their individual physical characteristics, such as physique and finger length, often leading them to give up. For example, stories of people giving up the guitar because they can't press the F chord are common. While attending music classes is an option, personalized instruction is often dependent on the instructor's skill and experience, and the availability of optimized guidance is largely a matter of luck. The personalized performance coach utilizes multimodal AI to provide individualized guidance on optimal playing style and fingering techniques based on the user's physique and finger length. This addresses issues such as "tendonitis and other physical burdens caused by incorrect instrument holding" and "finger inability to reach or dexterous fingers," helping beginners who are prone to giving up at the initial stages of instrument playing. Furthermore, for intermediate learners who are prone to developing bad habits, the system also considers skeletal and joint range of motion to provide corrective guidance for skill improvement. For instance, users can use their smartphone cameras to photograph their physique, and the AI ​​will automatically analyze physical characteristics such as height, limb length, and joint position to create a personalized user profile. Next, users record videos of themselves playing their instruments using their smartphones. AI analyzes the videos and compares them to ideal playing postures and fingering techniques. Based on the AI's analysis, specific feedback is generated. This feedback is overlaid on the user's performance using AR technology, allowing for intuitive understanding. Furthermore, the AI ​​regularly evaluates the user's progress and suggests new practice methods or topics based on their skill level. Thus, even beginners who have mastered basic skills can continue to receive advanced feedback and playing topics geared towards intermediate learners. Personalized performance coaches prevent beginners from giving up and support continued instrument playing by providing personalized, optimized guidance, readily available support, and efficient skill enhancement opportunities. Users can practice according to their own topics and pace, truly experiencing the joy of playing an instrument. Therefore, personalized performance coaches can provide optimal playing postures and fingering techniques based on the user's physical data and achieve personalized, optimized guidance through progress evaluation.

[0093] The personalized performance coach described in this embodiment includes an acquisition unit, an analysis unit, a generation unit, a provision unit, and an evaluation unit. The acquisition unit acquires the user's physical data. For example, the acquisition unit can use a smartphone camera to acquire the user's physical data. The acquisition unit can automatically analyze the user's height, limb length, joint position, and other physical characteristics. The analysis unit analyzes the physical data acquired by the acquisition unit. For example, the analysis unit can generate a user-specific profile based on the acquired physical data. The analysis unit analyzes the user's physical data in detail to generate basic data for providing personalized optimization guidance. The generation unit generates optimal playing posture and fingering techniques based on the data analyzed by the analysis unit. For example, the generation unit can analyze the user's performance video to generate ideal playing posture and fingering techniques. The generation unit uses a generative AI to analyze the user's performance video and generate optimal playing posture and fingering techniques. The provision unit provides feedback generated by the generation unit. For example, the provision unit can overlay the feedback generated by the generative AI onto the user's performance in an AR manner. The provision unit uses AR technology to display feedback, allowing the user to intuitively understand the feedback. The evaluation unit evaluates the user's progress based on the feedback provided by the provision unit and proposes new practice methods or topics. For example, the evaluation department can periodically assess the user's progress and propose new practice methods or topics based on their skill level. The evaluation department provides detailed assessments of the user's progress to offer appropriate practice methods or topics according to their skill level. Thus, the personalized performance coach described in this embodiment can provide optimal playing posture and fingering techniques based on the user's physical data, and achieve personalized optimization guidance through progress evaluation.

[0094] The acquisition unit is used to acquire users' physical data. For example, the acquisition unit can use a smartphone camera to acquire users' physical data. Specifically, it uses the smartphone camera to photograph the user's full body and automatically analyzes the user's height, limb length, joint position, and other physical characteristics through image processing technology. For this purpose, deep learning image recognition algorithms can be used to acquire users' physical data with high accuracy. For example, if a user stands in front of a smartphone camera in a specific pose, the acquisition unit can capture images from multiple angles and generate a 3D model. Based on this 3D model, the user's physical data can be analyzed in detail. In addition, the acquisition unit can also acquire data from wearable devices used by users daily. For example, it can collect data such as heart rate, steps, and calories burned from smartwatches or fitness trackers and integrate this data with the user's physical data to create a more comprehensive profile. Thus, the acquisition unit can collect users' physical data from multiple angles, providing detailed and accurate data.

[0095] The analysis unit is used to analyze the physical data acquired by the acquisition unit. For example, the analysis unit can generate a user-specific profile based on the acquired physical data. Specifically, the analysis unit analyzes the data from the acquisition unit to gain a detailed understanding of the user's physical characteristics. To this end, machine learning algorithms are used to classify the user's physical data and extract feature quantities. For example, based on data such as the user's height, limb length, and joint position, basic data is generated to determine the most suitable playing posture and fingering techniques for the user's physique. The analysis unit uses this data to create a personalized guidance profile suitable for the user's physique. This profile includes not only the user's physical characteristics but also past performance data and practice records. Thus, the analysis unit can analyze the user's physical data in detail to generate basic data for providing personalized optimization guidance. In addition, the analysis unit can continuously monitor the user's physical data and update the profile when necessary. Therefore, the analysis unit can always provide optimal guidance based on the latest data.

[0096] The generation department generates optimal playing posture and fingering techniques based on the data analyzed by the analysis department. For example, the generation department can analyze a user's performance video to generate ideal playing posture and fingering techniques. Specifically, the generation department uses generative AI to analyze the user's performance video and generate optimal playing posture and fingering techniques. The generative AI uses deep learning video analysis technology to analyze the user's playing movements in detail. For example, it analyzes the user's hand movements, finger positions, body posture, etc., when playing an instrument to determine the ideal playing posture and fingering techniques. Based on this data, the generation department generates playing posture and fingering techniques most suitable for the user's physique and playing style. In addition, the generation department also generates feedback to provide to the user, including specific playing instructions and practice method suggestions. For example, the generation department can generate practice methods to improve specific playing techniques or specifically point out areas for improvement in playing posture. Thus, the generation department can generate playing posture and fingering techniques most suitable for the user's physique and playing style, providing personalized optimization guidance.

[0097] The Provision Department provides feedback generated by the Generation Department. For example, it can overlay AI-generated feedback onto the user's performance using AR technology. Specifically, AR technology is used to overlay feedback onto the user's performance video in real time. This allows the user to intuitively understand areas for improvement while watching their own performance. For instance, when a user plays an instrument, real-time feedback on hand position and finger movements can be displayed, indicating correct playing posture and fingering techniques. The Provision Department can use visual guidance or animation to help users intuitively understand the feedback. This allows users to easily grasp specific areas for improvement while watching their own performance. Furthermore, the Provision Department can record the user's reactions to feedback and reflect them in subsequent feedback sessions. This allows the Provision Department to provide appropriate feedback based on the user's progress, achieving personalized optimization guidance. In addition, the Provision Department can flexibly adjust the display method of feedback based on the environment and device the user uses to receive it. For example, it can provide corresponding feedback displays for various devices such as smartphones, tablets, and AR glasses. This ensures that the Provision Department can provide optimal feedback to users in any environment.

[0098] Based on feedback provided by the provision department, the evaluation department assesses user progress and proposes new practice methods or topics. Specifically, the evaluation department analyzes user performance data and feedback history to evaluate the user's skill level and progress in detail. To this end, machine learning algorithms are used to analyze user performance data, evaluating skill improvement and topic achievement. For example, it evaluates the time spent by the user to master a specific playing technique and the achievable playing accuracy. Based on this data, the evaluation department proposes new practice methods or topics according to the user's skill level. For example, it specifically points out practice methods to improve a particular playing technique or the next topic to be mastered. Furthermore, the evaluation department regularly evaluates user progress and provides feedback to support continuous skill improvement. Thus, the evaluation department can provide appropriate practice methods or topics based on the user's skill level, achieving personalized optimization guidance. In addition, the evaluation department can record user reactions to feedback and reflect them in the next evaluation, thereby conducting more precise evaluations. Therefore, the evaluation department can comprehensively evaluate user progress and generate the foundational data for providing personalized optimization guidance.

[0099] The provisioning department can overlay AI-generated feedback onto the user's performance using AR technology. For example, the provisioning department can use AI to analyze the user's performance video, generate ideal playing posture and fingering techniques, and display this feedback in AR. The provisioning department utilizes AR technology to display feedback, allowing the user to intuitively understand it. For example, the provisioning department can visually overlay the ideal playing posture and fingering techniques onto the user's performance video. This allows the user to practice while comparing their own performance with the ideal playing posture. Some or all of the above processing in the provisioning department can be achieved through AI, or it can be done without AI. For example, the provisioning department can use a system that takes AI-generated feedback as input and AR display as output to display the feedback. This allows the user to understand the feedback more intuitively.

[0100] The evaluation department can periodically assess users' progress and propose new practice methods or topics based on their skill levels. For example, the evaluation department can periodically analyze users' performance videos and evaluate their progress. Based on the user's skill level, the evaluation department proposes appropriate practice methods or topics. For instance, the evaluation department can propose basic practice methods for beginners, or advanced practice methods or topics for intermediate learners. The evaluation department provides detailed evaluations of users' progress, offering feedback appropriate to their skill level. Thus, users can obtain corresponding practice methods or topics based on their own skill level, efficiently improving their skills. Some or all of the above processing in the evaluation department can be implemented using AI, or it can be done without AI. For example, the evaluation department can input users' performance videos into AI, which will then perform progress evaluation. This allows for the provision of appropriate practice methods or topics based on the user's skill level.

[0101] The acquisition unit can utilize a smartphone's camera to acquire a user's physical data. For example, the acquisition unit can obtain this data by having the user photograph their own physique using their smartphone camera. The acquisition unit then automatically analyzes the user's physical characteristics, such as height, limb length, and joint position, using the smartphone camera. To facilitate easy acquisition of physical data by users, the acquisition unit provides a simple method based on the smartphone camera. For example, the acquisition unit can analyze image data captured by the smartphone camera to extract physical data. Thus, users can easily acquire physical data without special equipment. Some or all of the above processing in the acquisition unit can be implemented using AI, or it can be done without AI. For example, the acquisition unit can input image data captured by the smartphone camera into AI, which will then perform physical data analysis. This allows users to easily acquire physical data.

[0102] The analysis unit can generate user-specific profiles based on the acquired physical data. For example, the analysis unit can analyze the acquired physical data in detail to generate user-specific profiles. Based on the user's physical data, the analysis unit generates foundational data for providing personalized optimization guidance. For example, the analysis unit analyzes the user's height, limb length, joint position, and other physical characteristics to generate user-specific profiles. Thus, the user can obtain the most suitable playing posture and fingering techniques for their own physique. Some or all of the above processing in the analysis unit can be implemented using AI, or it can be done without AI. For example, the analysis unit can input the acquired physical data into AI, which will then execute the generation of user-specific profiles. Thus, personalized optimization guidance is achieved by generating user-specific profiles.

[0103] The generation unit can analyze a user's performance video to generate ideal playing posture and fingering techniques. For example, the generation unit can input the user's performance video into the generation AI, which then generates the ideal playing posture and fingering techniques. The generation unit uses the generation AI to analyze the user's performance video and generate optimal playing posture and fingering techniques. For example, the generation unit can analyze the user's performance video in detail to generate ideal playing posture and fingering techniques. Thus, the user can practice while comparing their own performance with the ideal playing posture. Some or all of the above processing in the generation unit can be achieved through the generation AI, or it can be done without using the generation AI. For example, the generation unit can input the user's performance video into the generation AI, which then generates the ideal playing posture and fingering techniques. Thus, by analyzing the user's performance video, ideal playing posture and fingering techniques can be provided.

[0104] The acquisition unit can infer the user's emotions and adjust the timing of acquiring physical data based on these inferences. For example, when the user is relaxed, the unit encourages them to be photographed in a relaxed state to capture physical data of a natural posture. When the user is tense, guidance can be provided to alleviate tension, allowing the user to acquire physical data in a relaxed state. When the user is in a hurry, a simplified operating procedure can be provided to quickly acquire physical data. Thus, physical data can be acquired at the optimal time based on the user's emotions. Emotion inference can be achieved through emotion engines or generative AI, etc. The generative AI can be text-generated AI (such as LLM) or multimodal generative AI, but is not limited to these. Some or all of the above processing in the acquisition unit can be achieved through AI, or AI can be omitted. For example, the acquisition unit can input the user's facial expression data into the generative AI, which then performs emotion inference. Thus, physical data can be acquired at the optimal time based on the user's emotions.

[0105] The data acquisition department can analyze a user's past physical data and select the optimal acquisition method. For example, based on the user's past physical data, the department can choose the method that obtains the most accurate data. The department can also analyze changes in the user's past physical data and select a stable data acquisition method. Furthermore, the department can consider the frequency of past physical data acquisition and select the optimal acquisition time. Thus, physical data can be acquired in the optimal way based on past data. Some or all of the above processing in the acquisition department can be implemented using AI, or it can be done without AI. For example, the acquisition department can input the user's past physical data into AI, which can then select the optimal acquisition method. Thus, physical data can be acquired in the optimal way based on past data.

[0106] The data acquisition department can filter physical data based on the user's current health status and lifestyle habits. For example, it can consider the user's current health status and acquire physical data when they are in good physical condition. It can also analyze the user's lifestyle habits and acquire physical data at the most suitable time of day. Furthermore, the acquisition department can select the type of data to acquire based on the user's health status and lifestyle habits. Thus, data can be acquired according to the user's health status and lifestyle habits. Some or all of the above processing in the acquisition department can be implemented using AI, or it can be done without AI. For example, the acquisition department can input the user's health data into AI, which can then perform the filtering. Thus, data can be acquired according to the user's health status and lifestyle habits.

[0107] The acquisition unit can infer the user's emotions and prioritize the acquired physical data based on these inferred emotions. For example, when the user is relaxed, detailed physical data is prioritized. When the user is tense, basic physical data is prioritized. When the user is in a hurry, the most important physical data is prioritized. Thus, important data can be prioritized based on the user's emotions. Emotion inference can be achieved through emotion engines or generative AI, etc. The generative AI can be text-generating AI (such as LLM) or multimodal generative AI, but is not limited to these. Some or all of the above processing in the acquisition unit can be implemented by AI, or AI can be omitted. For example, the acquisition unit can input the user's facial expression data into the generative AI, which will then perform emotion inference. Thus, important data can be prioritized based on the user's emotions.

[0108] The acquisition department can consider the user's geographical location information when acquiring physical data, prioritizing the acquisition of highly relevant data. For example, when a user is at high altitude, data related to oxygen concentration and air pressure can be prioritized. When a user is in a city, data related to environmental noise and vibration can be prioritized. When a user is indoors, data related to indoor temperature and humidity can be prioritized. Thus, highly relevant data can be acquired based on the user's geographical location information. Some or all of the above processing in the acquisition department can be implemented using AI, or it can be done without AI. For example, the acquisition department can input the user's geographical location information into AI, which can then perform the acquisition of highly relevant data. Thus, highly relevant data can be acquired based on the user's geographical location information.

[0109] The acquisition department can analyze users' social media activities to obtain relevant data while acquiring physical data. For example, when a user posts sports-related content on social media, physical data related to that activity can be acquired. When a user shares health information on social media, physical data can also be acquired based on that information. When a user participates in a specific activity on social media, physical data related to that activity can also be acquired. Thus, relevant data can be acquired based on users' social media activities. Some or all of the above processing in the acquisition department can be implemented using AI, or it can be done without AI. For example, the acquisition department can input users' social media data into AI, which can then perform the acquisition of relevant data. Thus, relevant data can be acquired based on users' social media activities.

[0110] The analysis unit can infer the user's emotions and adjust the presentation of the analysis based on these inferences. For example, it provides detailed analysis results when the user is relaxed, concise results when the user is tense, and visual results when the user is in a hurry for easy understanding. Thus, the presentation of the analysis results can be adjusted according to the user's emotions. Emotion inference can be achieved through emotion engines or generative AI, etc. Generative AI can be text-generating AI (such as LLM) or multimodal generative AI, but is not limited to these. Some or all of the above processing in the analysis unit can be implemented using AI, or AI can be omitted. For example, the analysis unit can input the user's facial expression data into the generative AI, which then performs emotion inference. Therefore, the presentation of the analysis results can be adjusted according to the user's emotions.

[0111] The analysis unit can adjust the level of detail in the analysis based on the importance of the physical data. For example, it can perform detailed analysis on important physical data, while simplifying the analysis of basic physical data. It can also perform detailed analysis on physical data relevant to a specific purpose, depending on that purpose. Thus, important data can be analyzed in detail. Some or all of the above processing in the analysis unit can be implemented using AI, or it can be done without AI. For example, the analysis unit can input physical data into AI, which can then adjust the level of detail in the analysis based on its importance. This allows for detailed analysis of important data.

[0112] The parsing unit can apply different parsing algorithms based on the category of the physical data during parsing. For example, a skeletal parsing algorithm can be applied to skeletal data. A muscle parsing algorithm can also be applied to muscle data. A joint parsing algorithm can also be applied to joint data. Thus, appropriate parsing can be performed according to the data category. Some or all of the above processing in the parsing unit can be implemented by AI, or AI can be used without AI. For example, the parsing unit can input physical data into AI, and the AI ​​can execute the application of category-based parsing algorithms. Thus, appropriate parsing can be performed according to the data category.

[0113] The analysis unit can infer the user's emotions and adjust the length of the analysis based on the inferred emotions. For example, when the user is relaxed, a detailed analysis is performed, resulting in a longer report. When the user is tense, a concise analysis is performed, resulting in a shorter report. When the user is in a hurry, a brief analysis highlighting the key points is performed. Thus, the length of the analysis can be adjusted according to the user's emotions. Emotion inference can be achieved through emotion engines or generative AI, etc. The generative AI can be text-generating AI (such as LLM) or multimodal generative AI, but is not limited to these. Some or all of the above processing in the analysis unit can be achieved through AI, or AI can be omitted. For example, the analysis unit can input the user's facial expression data into the generative AI, which will then perform emotion inference. Thus, the length of the analysis can be adjusted according to the user's emotions.

[0114] The parsing unit can determine the parsing priority based on when the physical data was acquired. For example, it can prioritize parsing the most recent physical data. It can also refer to previous physical data when parsing the most recent data. Alternatively, it can prioritize parsing physical data acquired within a specific period. Thus, the most recent data can be parsed first. Some or all of the above processing in the parsing unit can be implemented using AI, or it can be done without AI. For example, the parsing unit can input physical data into AI, which will then execute a parsing priority based on the acquisition time. Thus, the most recent data can be parsed first.

[0115] The parsing unit can adjust the parsing order based on the relevance of the physical data during parsing. For example, it can prioritize parsing important physical data, highly relevant physical data, or physical data related to a specific purpose. Thus, highly relevant data can be parsed first. Some or all of the above processing in the parsing unit can be implemented using AI, or it can be done without AI. For example, the parsing unit can input physical data into AI, which can then perform a relevance-based adjustment of the parsing order. This allows highly relevant data to be parsed first.

[0116] The generation unit can infer the user's emotions and adjust the generated performance posture and fingering techniques accordingly. For example, when the user is relaxed, detailed performance postures and fingering techniques are provided. When the user is tense, concise performance postures and fingering techniques are provided. When the user is in a hurry, visual performance postures and fingering techniques are provided for quick understanding. Thus, the performance posture and fingering techniques can be adjusted according to the user's emotions. Emotion inference can be achieved through emotion engines or generative AI. The generative AI can be text-generated AI (such as LLM) or multimodal generative AI, but is not limited to these. Some or all of the above processing in the generation unit can be achieved through AI, or AI can be omitted. For example, the generation unit can input the user's facial expression data into the generative AI, which will then perform emotion inference. Thus, the performance posture and fingering techniques can be adjusted according to the user's emotions.

[0117] The generation unit can adjust the level of detail generated based on the importance of physical data during the generation process. For example, it can generate detailed playing postures and fingering techniques based on important physical data. It can also generate simplified playing postures and fingering techniques based on basic physical data. Furthermore, it can generate detailed playing postures and fingering techniques adapted to a specific purpose based on physical data relevant to that purpose. Thus, detailed playing postures and fingering techniques can be generated based on important data. Some or all of the above processing in the generation unit can be implemented using AI, or it can be done without AI. For example, the generation unit can input physical data into AI, which can then adjust the level of detail based on importance. Thus, detailed playing postures and fingering techniques can be generated based on important data.

[0118] The generation unit can apply different generation algorithms based on the category of physical data during generation. For example, it can generate suitable playing postures and fingering techniques based on skeletal data. It can also generate suitable playing postures and fingering techniques based on muscle data. Furthermore, it can generate suitable playing postures and fingering techniques based on joint data. Thus, appropriate playing postures and fingering techniques can be generated according to the data category. Some or all of the above processing in the generation unit can be implemented using AI, or it can be done without AI. For example, the generation unit can input physical data into AI, which will then execute a category-based generation algorithm. This allows for the generation of appropriate playing postures and fingering techniques based on the data category.

[0119] The generation unit can infer the user's emotions and adjust the length of the generated playing postures and fingering techniques accordingly. For example, when the user is relaxed, detailed playing postures and fingering techniques are provided. When the user is tense, concise playing postures and fingering techniques are provided. When the user is in a hurry, visual playing postures and fingering techniques are provided for quick understanding. Thus, the length of the playing postures and fingering techniques can be adjusted according to the user's emotions. Emotion inference can be achieved through emotion engines or generative AI, etc. The generative AI can be text-generated AI (such as LLM) or multimodal generative AI, but is not limited to these. Some or all of the above processing in the generation unit can be achieved through AI, or AI can be omitted. For example, the generation unit can input the user's facial expression data into the generative AI, which will then perform emotion inference. Thus, the length of the playing postures and fingering techniques can be adjusted according to the user's emotions.

[0120] The generation unit can determine the generation priority based on when the physical data is acquired. For example, it can generate playing postures and fingering techniques based on the latest physical data. It can also refer to previous physical data and generate playing postures and fingering techniques based on the latest data. Alternatively, it can generate playing postures and fingering techniques based on physical data acquired within a specific period. Thus, playing postures and fingering techniques can be generated based on the latest data. Some or all of the above processing in the generation unit can be implemented using AI, or AI can be omitted. For example, the generation unit can input physical data into AI, which will then determine the generation priority based on the acquisition time. Thus, playing postures and fingering techniques can be generated based on the latest data.

[0121] The generation unit can adjust the generation order based on the relevance of physical data during generation. For example, it can prioritize generating playing postures and fingering techniques based on important physical data. It can also prioritize generating playing postures and fingering techniques based on highly relevant physical data. Furthermore, it can prioritize generating playing postures and fingering techniques based on physical data relevant to a specific purpose. Thus, playing postures and fingering techniques can be generated based on highly relevant data. Some or all of the above processing in the generation unit can be implemented using AI, or AI can be omitted. For example, the generation unit can input physical data into AI, which will then perform a relevance-based generation order adjustment. Thus, playing postures and fingering techniques can be generated based on highly relevant data.

[0122] The feedback delivery unit can infer the user's emotions and adjust the display of feedback accordingly. For example, detailed feedback can be provided when the user is relaxed, concise feedback when the user is tense, and visual feedback for quick understanding when the user is in a hurry. Thus, the feedback display can be adjusted based on the user's emotions. Emotion inference can be achieved through emotion engines or generative AI, etc. Generative AI can be text-generated AI (such as LLM) or multimodal generative AI, but is not limited to these. Some or all of the above processing in the delivery unit can be implemented using AI, or AI can be omitted. For example, the delivery unit can input the user's facial expression data into the generative AI, which will then perform emotion inference. Thus, the feedback display can be adjusted based on the user's emotions.

[0123] The feedback department can refer to users' past feedback history to select the optimal display method when providing feedback. For example, the feedback department can select the most effective display method based on users' past feedback history. It can also analyze users' past feedback history to select an easy-to-understand display method. Furthermore, it can refer to users' past feedback history to adjust the display order of feedback. Thus, the optimal display method can be selected based on past feedback history. Some or all of the above processing in the feedback department can be implemented using AI, or it can be done without AI. For example, the feedback department can input users' feedback history into AI, which will then select the optimal display method. Thus, the optimal display method can be selected based on past feedback history.

[0124] The feedback provision department can customize the feedback content based on the user's current performance status. For example, the department can analyze the user's current performance and provide appropriate feedback. It can also adjust the level of detail in the feedback based on the user's performance status, and determine the priority of the feedback based on the user's performance status. Thus, appropriate feedback can be provided based on the current performance status. Some or all of the above processing in the provision department can be implemented using AI, or it can be done without AI. For example, the department can input the user's performance data into AI, which can then perform feedback customization. Thus, appropriate feedback can be provided based on the current performance status.

[0125] The feedback delivery unit can infer the user's emotions and prioritize feedback accordingly. For example, detailed feedback is prioritized when the user is relaxed. Concise feedback is prioritized when the user is tense. Visual feedback is prioritized when the user is in a hurry. Thus, important feedback can be prioritized based on the user's emotions. Emotion inference can be achieved through emotion engines or generative AI, etc. The generative AI can be text-generated AI (such as LLM) or multimodal generative AI, but is not limited to these. Some or all of the above processing in the feedback delivery unit can be achieved through AI, or AI can be omitted. For example, the feedback delivery unit can input the user's facial expression data into the generative AI, which will then perform emotion inference. Thus, important feedback can be prioritized based on the user's emotions.

[0126] The feedback delivery department can consider the user's geographic location information when providing feedback and select the optimal feedback method. For example, visual feedback can be prioritized when the user is outdoors. Detailed feedback can be provided when the user is indoors. Concise feedback can be provided when the user is moving. Thus, the optimal feedback method can be selected based on geographic location information. Some or all of the above processing in the delivery department can be implemented using AI, or it can be done without AI. For example, the delivery department can input the user's geographic location information into AI, which will then select the optimal feedback method. Thus, the optimal feedback method can be selected based on geographic location information.

[0127] The feedback provision department can analyze users' social media activities and propose feedback methods when providing feedback. For example, when a user posts sports-related content on social media, feedback related to that activity can be provided. When a user shares health information on social media, feedback can also be provided based on that information. When a user participates in a specific activity on social media, feedback related to that activity can also be provided. Thus, appropriate feedback methods can be proposed based on social media activities. Some or all of the above processing in the feedback provision department can be implemented using AI, or it can be done without AI. For example, the feedback provision department can input users' social media data into AI, which can then propose feedback methods. Thus, appropriate feedback methods can be proposed based on social media activities.

[0128] The evaluation department can infer users' emotions and adjust its evaluation methods accordingly. For example, when a user is relaxed, a detailed evaluation can be provided. When a user is tense, a concise evaluation can be provided. When a user is in a hurry, a visual evaluation can be provided for easy understanding. Thus, the evaluation method can be adjusted based on the user's emotions. Emotion inference can be achieved through emotion engines or generative AI, etc. Generative AI can be text-generated AI (such as LLM) or multimodal generative AI, but is not limited to these. Some or all of the above processing in the evaluation department can be achieved through AI, or AI can be omitted. For example, the evaluation department can input users' facial expression data into generative AI, which will then perform emotion inference. Thus, the evaluation method can be adjusted based on the user's emotions.

[0129] The evaluation department can analyze a user's past performance data during the evaluation process to select the optimal evaluation method. For example, the evaluation department can choose the most effective evaluation method based on the user's past performance data. It can also analyze past performance data to select an easy-to-understand evaluation method. Furthermore, it can refer to past performance data to adjust the display order of evaluations. Thus, the optimal evaluation method can be selected based on past performance data. Some or all of the above processing in the evaluation department can be implemented using AI, or it can be done without AI. For example, the evaluation department can input the user's performance data into AI, which will then select the optimal evaluation method. Thus, the optimal evaluation method can be selected based on past performance data.

[0130] The evaluation department can customize evaluation criteria based on the user's current skill level during the evaluation process. For example, the evaluation department can analyze the user's current skill level and set appropriate evaluation criteria. It can also adjust the level of detail in the evaluation based on the user's skill level. Furthermore, it can determine the priority of the evaluation based on the user's skill level. Thus, appropriate evaluation criteria can be set based on the current skill level. Some or all of the above processes in the evaluation department can be implemented using AI, or they can be performed without AI. For example, the evaluation department can input the user's skill data into AI, which will then execute the customization of the evaluation criteria. Thus, appropriate evaluation criteria can be set based on the current skill level.

[0131] The evaluation department can infer users' emotions and prioritize evaluations based on these inferences. For example, detailed evaluations are prioritized when a user is relaxed. Concise evaluations are prioritized when a user is stressed. Visual evaluations are prioritized when a user is in a hurry. Thus, important evaluations can be prioritized based on the user's emotions. Emotion inference can be achieved through emotion engines or generative AI. Generative AI can be text-generated AI (such as LLM) or multimodal generative AI, but is not limited to these. Some or all of the above processing in the evaluation department can be implemented using AI, or AI can be omitted. For example, the evaluation department can input users' facial expression data into generative AI, which will then perform emotion inference. Thus, important evaluations can be prioritized based on the user's emotions.

[0132] The evaluation department can consider the user's geographical location information during the evaluation process to select the optimal evaluation method. For example, when the user is outdoors, a visual evaluation can be prioritized. When the user is indoors, a detailed evaluation can also be provided. When the user is on the move, a concise evaluation can also be provided. Thus, the optimal evaluation method can be selected based on geographical location information. Some or all of the above processing in the evaluation department can be implemented using AI, or it can be done without AI. For example, the evaluation department can input the user's geographical location information into AI, which will then select the optimal evaluation method. Thus, the optimal evaluation method can be selected based on geographical location information.

[0133] The evaluation department can analyze users' social media activities during the evaluation process to propose evaluation methods. For example, when a user posts sports-related content on social media, evaluations related to that activity can be provided. When a user shares health information on social media, evaluations can also be provided based on that information. When a user participates in a specific activity on social media, evaluations related to that activity can also be provided. Thus, appropriate evaluation methods can be proposed based on social media activities. Some or all of the above processing in the evaluation department can be implemented using AI, or it can be done without AI. For example, the evaluation department can input users' social media data into AI, which can then propose evaluation methods. Thus, appropriate evaluation methods can be proposed based on social media activities.

[0134] The system involved in this embodiment is not limited to the above examples. For example, various modifications can be made as follows.

[0135] In addition to physical data, the acquisition department can also acquire user lifestyle data. For example, it can collect data on sleep patterns, diet, and exercise habits, and use this data to infer the user's physical condition and energy level. The analysis department can generate an optimal exercise plan that takes into account the user's physical condition and energy level based on the acquired lifestyle data. The generation department generates exercise content suitable for the user's physical condition and energy level based on the analysis results. The provision department provides the generated exercise content to the user, supporting continuous practice without pressure. Thus, personalized optimization guidance can be achieved based on the user's lifestyle.

[0136] The feedback unit can infer the user's emotions and adjust the feedback content accordingly. For example, detailed feedback can be provided when the user is relaxed, concise feedback when the user is tense, and visual feedback for quick understanding when the user is in a hurry. Thus, appropriate feedback can be provided based on the user's emotions. Emotion inference can be achieved through emotion engines or generative AI, etc. Generative AI can be text-generated AI (such as LLM) or multimodal generative AI, but is not limited to these. Some or all of the above processing in the feedback unit can be implemented using AI, or AI can be omitted. For example, the feedback unit can input the user's facial expression data into the generative AI, which will then perform emotion inference. Thus, appropriate feedback can be provided based on the user's emotions.

[0137] The evaluation department can analyze users' past practice data to select the optimal evaluation method. For example, it can choose the most effective evaluation method based on past practice data. It can also analyze past practice data to select an easy-to-understand evaluation method. Furthermore, it can adjust the display order of evaluations by referring to past practice data. Thus, the optimal evaluation method can be selected based on past practice data. Some or all of the above processing in the evaluation department can be implemented using AI, or it can be done without AI. For example, the evaluation department can input users' practice data into AI, which will then select the optimal evaluation method. Thus, the optimal evaluation method can be selected based on past practice data.

[0138] In addition to physical data, the acquisition department can also acquire user health data. For example, it can acquire data such as heart rate, blood pressure, and body temperature, and infer the user's health status based on this data. The analysis department can generate an optimal exercise plan that takes into account the user's health status based on the acquired health data. The generation department generates exercise content suitable for the user's health status based on the analysis results. The delivery department provides the generated exercise content to the user, supporting continuous practice without pressure. Thus, personalized optimization guidance can be achieved based on the user's health status.

[0139] The analysis unit can infer the user's emotions and adjust the presentation of the analysis based on these inferences. For example, it provides detailed analysis results when the user is relaxed, concise results when the user is tense, and visual results when the user is in a hurry for easy understanding. Thus, the presentation of the analysis results can be adjusted according to the user's emotions. Emotion inference can be achieved through emotion engines or generative AI, etc. Generative AI can be text-generating AI (such as LLM) or multimodal generative AI, but is not limited to these. Some or all of the above processing in the analysis unit can be implemented using AI, or AI can be omitted. For example, the analysis unit can input the user's facial expression data into the generative AI, which then performs emotion inference. Therefore, the presentation of the analysis results can be adjusted according to the user's emotions.

[0140] In addition to analyzing user performance videos, the generation department can also analyze user audio data to generate optimal playing posture and fingering techniques. For example, it acquires audio data from the user's performance, analyzing volume and rhythmic accuracy. Based on the analysis results, the generation department can adjust the user's playing posture and fingering techniques. The provision department then provides the user with the generated playing posture and fingering techniques, supporting the user in improving their musical expression. Thus, personalized optimization guidance can be achieved using audio data.

[0141] The acquisition unit can infer the user's emotions and adjust the timing of acquiring physical data based on these inferences. For example, when the user is relaxed, the unit encourages them to be photographed in a relaxed state to capture physical data of a natural posture. When the user is tense, guidance can be provided to alleviate tension, allowing the user to acquire physical data in a relaxed state. When the user is in a hurry, a simplified operating procedure can be provided to quickly acquire physical data. Thus, physical data can be acquired at the optimal time based on the user's emotions. Emotion inference can be achieved through emotion engines or generative AI, etc. The generative AI can be text-generated AI (such as LLM) or multimodal generative AI, but is not limited to these. Some or all of the above processing in the acquisition unit can be achieved through AI, or AI can be omitted. For example, the acquisition unit can input the user's facial expression data into the generative AI, which then performs emotion inference. Thus, physical data can be acquired at the optimal time based on the user's emotions.

[0142] In addition to user performance data, the evaluation department can also consider user musical preferences and goals when making evaluations. For example, it can obtain the user's preferred music genres and target performance styles, and set evaluation criteria based on this information. The evaluation department can adjust the level of detail and feedback content of the evaluation according to the user's musical preferences and goals. Thus, appropriate evaluations can be provided based on the user's individual objectives. Some or all of the above processing in the evaluation department can be implemented using AI, or it can be done without AI. For example, the evaluation department can input the user's musical preferences and target data into AI, which will then execute the setting of evaluation criteria. This allows for the provision of appropriate evaluations based on the user's musical preferences and goals.

[0143] The feedback delivery unit can infer the user's emotions and adjust the display of feedback accordingly. For example, detailed feedback can be provided when the user is relaxed, concise feedback when the user is tense, and visual feedback for quick understanding when the user is in a hurry. Thus, the feedback display can be adjusted based on the user's emotions. Emotion inference can be achieved through emotion engines or generative AI, etc. Generative AI can be text-generated AI (such as LLM) or multimodal generative AI, but is not limited to these. Some or all of the above processing in the delivery unit can be implemented using AI, or AI can be omitted. For example, the delivery unit can input the user's facial expression data into the generative AI, which will then perform emotion inference. Thus, the feedback display can be adjusted based on the user's emotions.

[0144] In addition to the user's physical data, the acquisition unit can also acquire the user's geographic location information. For example, when the user is at a high altitude, data related to oxygen concentration and air pressure can be acquired. When the user is in a city, data related to environmental noise and vibration can be acquired. When the user is indoors, data related to indoor temperature and humidity can be acquired. Thus, highly relevant data can be acquired based on the user's geographic location information. Some or all of the above processing in the acquisition unit can be implemented using AI, or it can be done without AI. For example, the acquisition unit can input the user's geographic location information into AI, which can then perform the acquisition of highly relevant data. Thus, highly relevant data can be acquired based on the user's geographic location information.

[0145] The following is a brief description of the processing flow of Implementation Method 2.

[0146] Step 1: The acquisition unit acquires the user's physical data. For example, the acquisition unit can use a smartphone camera to acquire the user's physical data. The acquisition unit automatically analyzes the user's height, limb length, joint position, and other physical characteristics.

[0147] Step 2: The analysis unit analyzes the physical data acquired by the acquisition unit. For example, the analysis unit can generate a user-specific profile based on the acquired physical data. The analysis unit analyzes the user's physical data in detail to generate the basic data used to provide personalized optimization guidance.

[0148] Step 3: The generation unit generates the optimal playing posture and fingering techniques based on the data analyzed by the analysis unit. For example, the generation unit can analyze the user's performance video to generate ideal playing posture and fingering techniques. The generation unit uses generation AI to analyze the user's performance video and generate the optimal playing posture and fingering techniques.

[0149] Step 4: The Provision Department provides feedback generated by the Generation Department. For example, the Provision Department can overlay the feedback generated by the AI ​​onto the user's performance using AR technology. The Provision Department uses AR technology to display the feedback, allowing the user to intuitively understand it.

[0150] Step 5: Based on the feedback provided by the supply department, the evaluation department assesses the user's progress and proposes new practice methods or topics. For example, the evaluation department may periodically evaluate the user's progress and propose new practice methods or topics based on the skill level. The evaluation department provides a detailed assessment of the user's progress in order to provide appropriate practice methods or topics based on the user's skill level.

[0151] The specific processing unit 290 sends the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires voice representing the user's input to the result of the specific processing. The control unit 46A sends the voice data representing the user's input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0152] Data generation model 58 is what is known as generative AI (Artificial Intelligence). An example of data generation model 58 includes ChatGPT (registered trademark) (Internet search).<URL:https: / / openai.com / blog / chatgpt> Generative AI, such as data generation model 58, is obtained by deep learning through a neural network. The data generation model 58 is input with a prompt containing instructions, and with inference data such as speech data representing speech, text data representing text, and image data representing images (e.g., data of still images or data of moving images). The data generation model 58 infers the input inference data according to the instructions shown in the prompt, and outputs the inference result in one or more data forms such as speech data, text data, and image data. The data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the above-described specific processing while using the data generation model 58. The data generation model 58 can also be a fine-tuned model to output inference results from prompts without instructions; in this case, the data generation model 58 can output inference results from prompts without instructions. The data processing device 12, etc., includes multiple data generation models 58, including AI other than generative AI. AI other than generative AI includes, but is not limited to, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or Naive Bayes. Furthermore, AI can also act as an AI agent. Additionally, when the processing described above is performed by AI, this processing may be partially or entirely performed by AI, but is not limited to this example. Moreover, processing performed by AI, including generative AI, can be replaced by rule-based processing, and vice versa.

[0153] Furthermore, the processing performed by the aforementioned data processing system 10 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart device 14, but it can also be executed jointly by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart device 14. Additionally, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the smart device 14 or external devices, and the smart device 14 acquires or collects information required for processing from the data processing device 12 or external devices.

[0154] Each of the elements, including the acquisition unit, analysis unit, generation unit, provision unit, and evaluation unit, can be implemented, for example, in at least one of the smart device 14 and the data processing device 12. For example, the acquisition unit uses the camera 42 of the smart device 14 to acquire the user's physical data. The analysis unit analyzes the acquired physical data using the specific processing unit 290 of the data processing device 12 to generate a user-specific profile. The generation unit generates optimal playing posture and fingering techniques based on the data analyzed by the specific processing unit 290 of the data processing device 12. The provision unit displays the generated feedback overlaid on the user's performance using AR technology via the control unit 46A of the smart device 14. The evaluation unit evaluates the user's progress using the specific processing unit 290 of the data processing device 12 and proposes new practice methods or topics. The correspondence between each unit and the device or control unit is not limited to the above example and can be modified in various ways.

[0155] Second Implementation Method

[0156] Figure 3 An example of the configuration of the data processing system 210 according to the second embodiment is shown.

[0157] like Figure 3 As shown, the data processing system 210 includes a data processing device 12 and smart glasses 214. One example of the data processing device 12 is a server.

[0158] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, RAM 30, and memory 32. The processor 28, RAM 30, and memory 32 are connected to a bus 34. Furthermore, the database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. An example of the network 54 includes a WAN and / or LAN, etc.

[0159] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, RAM 48, and memory 50. The processor 46, RAM 48, and memory 50 are connected to a bus 52. Furthermore, the microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0160] Microphone 238 receives user commands by receiving the user's voice. Microphone 238 captures the user's voice, converts the captured sound into speech data, and outputs it to processor 46. Speaker 240 outputs sound according to commands from processor 46.

[0161] Camera 42 is a small digital camera equipped with an optical system such as a lens, aperture and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, used to capture the user's surroundings (e.g., the shooting range defined by an angle of view equivalent to the field of vision of an average healthy person).

[0162] Communication I / F 44 is connected to network 54. Communication I / F 44 and 26 are responsible for the transmission and reception of various information between processor 46 and processor 28 via network 54. The transmission and reception of various information between processor 46 and processor 28 using communication I / F 44 and 26 is performed in a secure state.

[0163] Figure 4 An example of the main functions of the data processing device 12 and the smart glasses 214 is shown. Figure 4 As shown, in the data processing device 12, specific processing is performed by the processor 28. The memory 32 stores a specific processing program 56.

[0164] The processor 28 reads a specific processing program 56 from the memory 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is implemented by the processor 28 operating as a specific processing unit 290 based on the specific processing program 56 executed on the RAM 30.

[0165] The memory 32 stores a data generation model 58 and an emotion-specific model 59. The data generation model 58 and the emotion-specific model 59 are used by the specific processing unit 290. The specific processing unit 290 can use the emotion-specific model 59 to infer the user's emotions and perform specific processing based on the user's emotions. The emotion inference function using the emotion-specific model 59 (emotion-specific function) includes inferring and predicting the user's emotions, performing various inferences and predictions related to the user's emotions, but is not limited to this example. Furthermore, emotion inference and prediction also include, for example, emotion analysis (analysis).

[0166] In the smart glasses 214, specific processing is performed by the processor 46. A specific processing program 60 is stored in the memory 50. The processor 46 reads the specific processing program 60 from the memory 50 and executes the read specific processing program 60 on the RAM 48. Specific processing is implemented by the processor 46 operating as a control unit 46A based on the specific processing program 60 executed on the RAM 48. Furthermore, the smart glasses 214 may also have the same data generation model and emotion-specific model as the data generation model 58 and the emotion-specific model 59, and use these models to perform the same processing as the specific processing unit 290.

[0167] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains the processing results (prediction results, etc.) using the data generation model 58 by communicating with the server device that has the data generation model 58. In addition, the data processing device 12 may be a server device or a user-owned terminal device (e.g., a mobile phone, robot, home appliance, etc.).

[0168] The specific processing unit 290 sends the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires voice input representing the user's input to the specific processing result. The control unit 46A sends the voice data representing the user's input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0169] Data generation model 58 is a so-called generative AI. An example of data generation model 58 includes generative AIs such as ChatGPT. Data generation model 58 is obtained by performing deep learning on a neural network. A prompt containing instructions is input to data generation model 58, along with inference data such as speech data representing speech, text data representing text, and image data representing images (e.g., data from still images or moving images). Data generation model 58 infers the inference data based on the instructions shown in the prompt and outputs the inference result in one or more data forms such as speech data, text data, and image data. Data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. Specific processing unit 290 performs the aforementioned specific processing while using data generation model 58. Data generation model 58 can also be a fine-tuned model to output inference results from prompts without instructions; in this case, data generation model 58 can output inference results from prompts without instructions. The data processing device 12, etc., includes various data generation models 58, which include AI other than generative AI. These AIs include, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or Naive Bayes, and are capable of various processing methods, but are not limited to these examples. Furthermore, AI can also be an AI agent. Additionally, when the processing described above is performed by AI, this processing may be partially or entirely performed by AI, but is not limited to these examples. Moreover, processing performed by AI including generative AI can be replaced by rule-based processing, and vice versa.

[0170] The data processing system 210 of the second embodiment performs the same processing as the data processing system 10 of the first embodiment. The processing performed by the data processing system 210 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart glasses 214, but it can also be executed jointly by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart glasses 214. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the smart glasses 214 or external devices, and the smart glasses 214 acquires or collects information required for processing from the data processing device 12 or external devices.

[0171] Each of the aforementioned elements, including the acquisition unit, analysis unit, generation unit, provision unit, and evaluation unit, can be implemented, for example, in at least one of the smart glasses 214 and the data processing device 12. For instance, the acquisition unit acquires the user's physical data using the camera 42 of the smart glasses 214. The analysis unit analyzes the acquired physical data using the specific processing unit 290 of the data processing device 12 to generate a user-specific profile. The generation unit generates optimal playing posture and fingering techniques based on the data analyzed by the specific processing unit 290 of the data processing device 12. The provision unit displays the generated feedback overlaid on the user's performance using AR technology via the control unit 46A of the smart glasses 214. The evaluation unit evaluates the user's progress using the specific processing unit 290 of the data processing device 12 and proposes new practice methods or topics. The correspondence between each unit and the device or control unit is not limited to the above example and can be modified in various ways.

[0172] Third Implementation Method

[0173] Figure 5 An example of the configuration of the data processing system 310 according to the third embodiment is shown.

[0174] like Figure 5 As shown, the data processing system 310 includes a data processing device 12 and a head-mounted terminal 314. An example of the data processing device 12 is a server.

[0175] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, RAM 30, and memory 32. The processor 28, RAM 30, and memory 32 are connected to a bus 34. Furthermore, the database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. An example of the network 54 includes a WAN and / or LAN, etc.

[0176] The head-mounted terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and memory 50. The processor 46, RAM 48, and memory 50 are connected to a bus 52. Furthermore, the microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0177] Microphone 238 receives user commands by receiving the user's voice. Microphone 238 captures the user's voice, converts the captured sound into speech data, and outputs it to processor 46. Speaker 240 outputs sound according to commands from processor 46.

[0178] Camera 42 is a small digital camera equipped with an optical system such as a lens, aperture and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, used to capture the user's surroundings (e.g., the shooting range defined by an angle of view equivalent to the field of vision of an average healthy person).

[0179] Communication I / F 44 is connected to network 54. Communication I / F 44 and 26 are responsible for the transmission and reception of various information between processor 46 and processor 28 via network 54. The transmission and reception of various information between processor 46 and processor 28 using communication I / F 44 and 26 is performed in a secure state.

[0180] Figure 6 An example of the main functions of the data processing device 12 and the head-mounted terminal 314 is shown. Figure 6 As shown, in the data processing device 12, specific processing is performed by the processor 28. The memory 32 stores a specific processing program 56.

[0181] The processor 28 reads a specific processing program 56 from the memory 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is implemented by the processor 28 operating as a specific processing unit 290 based on the specific processing program 56 executed on the RAM 30.

[0182] The memory 32 stores a data generation model 58 and an emotion-specific model 59. The data generation model 58 and the emotion-specific model 59 are used by the specific processing unit 290. The specific processing unit 290 can use the emotion-specific model 59 to infer the user's emotions and perform specific processing based on the user's emotions. The emotion inference function using the emotion-specific model 59 (emotion-specific function) includes inferring and predicting the user's emotions, performing various inferences and predictions related to the user's emotions, but is not limited to this example. Furthermore, emotion inference and prediction also include, for example, emotion analysis (analysis).

[0183] In the head-mounted terminal 314, specific processing is performed by the processor 46. A specific program 60 is stored in the memory 50. The processor 46 reads the specific program 60 from the memory 50 and executes the read specific program 60 on the RAM 48. Specific processing is implemented by the processor 46 operating as a control unit 46A based on the specific program 60 executed on the RAM 48. Furthermore, the head-mounted terminal 314 may also have the same data generation model and emotion-specific model as the data generation model 58 and the emotion-specific model 59, and use these models to perform the same processing as the specific processing unit 290.

[0184] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains the processing results (prediction results, etc.) using the data generation model 58 by communicating with the server device that has the data generation model 58. In addition, the data processing device 12 may be a server device or a user-owned terminal device (e.g., a mobile phone, robot, home appliance, etc.).

[0185] The specific processing unit 290 sends the result of the specific processing to the head-mounted terminal 314. In the head-mounted terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires voice representing the user's input to the specific processing result. The control unit 46A sends the voice data representing the user's input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0186] Data generation model 58 is a so-called generative AI. An example of data generation model 58 includes generative AIs such as ChatGPT. Data generation model 58 is obtained by performing deep learning on a neural network. A prompt containing instructions is input to data generation model 58, along with inference data such as speech data representing speech, text data representing text, and image data representing images (e.g., data from still images or moving images). Data generation model 58 infers the inference data based on the instructions shown in the prompt and outputs the inference result in one or more data forms such as speech data, text data, and image data. Data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. Specific processing unit 290 performs the aforementioned specific processing while using data generation model 58. Data generation model 58 can also be a fine-tuned model to output inference results from prompts without instructions; in this case, data generation model 58 can output inference results from prompts without instructions. The data processing device 12, etc., includes various data generation models 58, which include AI other than generative AI. These AIs include, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or Naive Bayes, and are capable of various processing methods, but are not limited to these examples. Furthermore, AI can also be an AI agent. Additionally, when the processing described above is performed by AI, this processing may be partially or entirely performed by AI, but is not limited to these examples. Moreover, processing performed by AI including generative AI can be replaced by rule-based processing, and vice versa.

[0187] The data processing system 310 of the third embodiment performs the same processing as the data processing system 10 of the first embodiment. The processing performed by the data processing system 310 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the head-mounted terminal 314, but it can also be executed jointly by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the head-mounted terminal 314. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the head-mounted terminal 314 or external devices, and the head-mounted terminal 314 acquires or collects information required for processing from the data processing device 12 or external devices.

[0188] Each of the aforementioned elements, including the acquisition unit, analysis unit, generation unit, provision unit, and evaluation unit, can be implemented, for example, in at least one of the head-mounted terminal 314 and the data processing device 12. For example, the acquisition unit uses the camera 42 of the head-mounted terminal 314 to acquire the user's physical data. The analysis unit analyzes the acquired physical data using the specific processing unit 290 of the data processing device 12 to generate a user-specific profile. The generation unit generates optimal playing posture and fingering techniques based on the data analyzed by the specific processing unit 290 of the data processing device 12. The provision unit displays the generated feedback overlaid on the user's performance using AR overlay via the control unit 46A of the head-mounted terminal 314. The evaluation unit evaluates the user's progress using the specific processing unit 290 of the data processing device 12 and proposes new practice methods or topics. The correspondence between each unit and the device or control unit is not limited to the above examples and can be modified in various ways.

[0189] Fourth Implementation Method

[0190] Figure 7 An example of the configuration of the data processing system 410 according to the fourth embodiment is shown.

[0191] like Figure 7 As shown, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0192] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, RAM 30, and memory 32. The processor 28, RAM 30, and memory 32 are connected to a bus 34. Furthermore, the database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. An example of the network 54 includes a WAN and / or LAN, etc.

[0193] Robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control object 443. Computer 36 includes a processor 46, RAM 48, and memory 50. The processor 46, RAM 48, and memory 50 are connected to a bus 52. Furthermore, the microphone 238, speaker 240, camera 42, and control object 443 are also connected to the bus 52.

[0194] Microphone 238 receives user commands by receiving the user's voice. Microphone 238 captures the user's voice, converts the captured sound into speech data, and outputs it to processor 46. Speaker 240 outputs sound according to commands from processor 46.

[0195] The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture and shutter, and an imaging element such as a CMOS image sensor or a CCD image sensor, used to photograph the user's surroundings (e.g., the shooting range defined by an angle of view equivalent to the field of vision of an average healthy person).

[0196] Communication I / F 44 is connected to network 54. Communication I / F 44 and 26 are responsible for the transmission and reception of various information between processor 46 and processor 28 via network 54. The transmission and reception of various information between processor 46 and processor 28 using communication I / F 44 and 26 is performed in a secure state.

[0197] The controlled object 443 includes a display device, LEDs for the eyes, and motors for driving the arms, hands, and feet. The posture and movements of the robot 414 are controlled by controlling the motors for the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, facial expressions of the robot 414 can also be expressed by controlling the illumination state of the LEDs for the robot 414's eyes.

[0198] Figure 8 An example of the main functions of the data processing device 12 and the robot 414 is shown. For example... Figure 8 As shown, in the data processing device 12, specific processing is performed by the processor 28. The memory 32 stores a specific processing program 56.

[0199] The processor 28 reads a specific processing program 56 from the memory 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is implemented by the processor 28 operating as a specific processing unit 290 based on the specific processing program 56 executed on the RAM 30.

[0200] The memory 32 stores a data generation model 58 and an emotion-specific model 59. The data generation model 58 and the emotion-specific model 59 are used by the specific processing unit 290. The specific processing unit 290 can use the emotion-specific model 59 to infer the user's emotions and perform specific processing based on the user's emotions. The emotion inference function using the emotion-specific model 59 (emotion-specific function) includes inferring and predicting the user's emotions, performing various inferences and predictions related to the user's emotions, but is not limited to this example. Furthermore, emotion inference and prediction also include, for example, emotion analysis (analysis).

[0201] In robot 414, specific processing is performed by processor 46. A specific program 60 is stored in memory 50. Processor 46 reads the specific program 60 from memory 50 and executes the read specific program 60 on RAM 48. Specific processing is achieved by processor 46 acting as control unit 46A based on the specific program 60 executed on RAM 48. Furthermore, robot 414 may also have the same data generation model and emotion-specific model as data generation model 58 and emotion-specific model 59, and use these models to perform the same processing as specific processing unit 290.

[0202] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains the processing results (prediction results, etc.) using the data generation model 58 by communicating with the server device that has the data generation model 58. In addition, the data processing device 12 may be a server device or a user-owned terminal device (e.g., a mobile phone, robot, home appliance, etc.).

[0203] The specific processing unit 290 sends the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires voice representing the user's input regarding the result of the specific processing. The control unit 46A sends the voice data representing the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0204] Data generation model 58 is a so-called generative AI. An example of data generation model 58 includes generative AIs such as ChatGPT. Data generation model 58 is obtained by performing deep learning on a neural network. A prompt containing instructions is input to data generation model 58, along with inference data such as speech data representing speech, text data representing text, and image data representing images (e.g., data from still images or moving images). Data generation model 58 infers the inference data based on the instructions shown in the prompt and outputs the inference result in one or more data forms such as speech data, text data, and image data. Data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. Specific processing unit 290 performs the aforementioned specific processing while using data generation model 58. Data generation model 58 can also be a fine-tuned model to output inference results from prompts without instructions; in this case, data generation model 58 can output inference results from prompts without instructions. The data processing device 12, etc., includes various data generation models 58, which include AI other than generative AI. These AIs include, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or Naive Bayes, and are capable of various processing methods, but are not limited to these examples. Furthermore, AI can also be an AI agent. Additionally, when the processing described above is performed by AI, this processing may be partially or entirely performed by AI, but is not limited to these examples. Moreover, processing performed by AI including generative AI can be replaced by rule-based processing, and vice versa.

[0205] The data processing system 410 of the fourth embodiment performs the same processing as the data processing system 10 of the first embodiment. The processing performed by the data processing system 410 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the robot 414, but it can also be executed jointly by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the robot 414. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the robot 414 or external devices, and the robot 414 acquires or collects information required for processing from the data processing device 12 or external devices.

[0206] Each of the aforementioned elements, including the acquisition unit, analysis unit, generation unit, provision unit, and evaluation unit, can be implemented, for example, in at least one of the robot 414 and the data processing device 12. For instance, the acquisition unit uses the robot 414's camera 42 to acquire the user's physical data. The analysis unit analyzes the acquired physical data using the specific processing unit 290 of the data processing device 12 to generate a user-specific profile. The generation unit generates optimal playing posture and fingering techniques based on the data analyzed by the specific processing unit 290 of the data processing device 12. The provision unit, through the robot 414's control unit 46A, overlays the generated feedback onto the user's performance in an AR manner. The evaluation unit, through the specific processing unit 290 of the data processing device 12, evaluates the user's progress and proposes new practice methods or topics. The correspondence between each unit and the device or control unit is not limited to the above example and can be modified in various ways.

[0207] Furthermore, the emotion-specific model 59, serving as an emotion engine, can determine the user's emotion based on a specific mapping. Specifically, the emotion-specific model 59 can determine the user's emotion based on an emotion graph that serves as a specific mapping (see...). Figure 9 The robot's emotions can be determined by the emotion-specific model 59. In addition, the emotion-specific model 59 can also determine the robot's emotions in the same way, and the specific processing unit 290 can also perform specific processing using the robot's emotions.

[0208] Figure 9 This is a diagram representing an emotion map 400 that maps various emotions. In the emotion map 400, emotions are arranged radially from the center in concentric circles. The closer to the center of the concentric circles, the more primitive the emotion is. Further out on the concentric circles, emotions are arranged representing states or actions arising from mood. Emotion is a concept that includes both feelings and mental states. To the left of the concentric circles, emotions generated by reactions occurring in the brain are arranged roughly. To the right of the concentric circles, emotions guided by situational judgments are arranged roughly. Above and below the concentric circles, emotions generated by reactions occurring in the brain and guided by situational judgments are arranged roughly. Furthermore, the emotion of "pleasure" is arranged above the concentric circles, and the emotion of "unpleasantness" is arranged below. Thus, in the emotion map 400, various emotions are mapped according to the structure of emotion generation, while easily generated emotions are mapped nearby.

[0209] These emotions are distributed at the 3 o'clock position on the Emotion Chart 400, and usually fluctuate between peace and unease. In the right half of the Emotion Chart 400, because situational awareness is more dominant than internal feelings, it gives a sense of calm.

[0210] The inner side of the emotion diagram 400 represents the mind, and the outer side of the emotion diagram 400 represents actions. Therefore, the further you go to the outer side of the emotion diagram 400, the more the emotion can be seen (manifested in actions).

[0211] Here, human emotions are based on a balance of various factors such as posture and blood sugar levels. When these balances deviate from the ideal, it indicates unhappiness; when they approach the ideal, it indicates pleasure. In robots, cars, and motorcycles, emotions can also be created based on a balance of factors such as posture and remaining battery power. When these balances deviate from the ideal, it indicates unhappiness; when they approach the ideal, it indicates pleasure. Emotion maps can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Speech Emotion Recognition and Brain Physiological Signal Analysis Systems for Emotion, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). In the left half of the emotion map, emotions belonging to the "Reaction" domain, where sensation is dominant, are arranged. Furthermore, in the right half of the emotion map, emotions belonging to the "Situation" domain, where situational cognition is dominant, are arranged.

[0212] The emotion map defines two types of emotions that promote learning. One is a negative emotion located near the middle of "repentance" or "reflection" on the situation side. That is, when the robot experiences negative emotions such as "I never want to feel this way again" or "I never want to be scolded again." The other is a positive emotion located near "desire" on the response side. That is, when the robot experiences positive feelings such as "wanting more" or "wanting to know more."

[0213] The emotion-specific model 59 feeds user input into a pre-learned neural network to obtain emotion values ​​representing each emotion shown in the emotion graph 400, and determines the user's emotion. This neural network is pre-learned based on multiple learning data sets that combine user input with emotion values ​​representing each emotion shown in the emotion graph 400. Furthermore, this neural network is learned to... Figure 10 As shown in sentiment graph 900, sentiment values ​​in nearby configurations are similar to each other. Figure 10 Examples show that multiple emotions such as "peace of mind", "stability", and "reassurance" have similar emotional values.

[0214] In the above embodiments, a specific processing is described by a single computer 22, but the technology disclosed herein is not limited to this, and distributed processing by multiple computers, including computer 22, is also possible.

[0215] In the above embodiments, an example of storing a specific processing program 56 in memory 32 is illustrated, but the technology disclosed herein is not limited thereto. For example, the specific processing program 56 may also be stored in a portable computer-readable non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 performs specific processing according to the specific processing program 56.

[0216] Alternatively, the specific processing program 56 can be stored in a storage device such as a server connected to the data processing device 12 via a network 54, and the specific processing program 56 can be downloaded and installed into the computer 22 upon request from the data processing device 12.

[0217] Furthermore, it is not necessary to store the entire specific process 56 in a storage device such as a server connected to the data processing device 12 via the network 54, nor is it necessary to store the entire specific process 56 in the memory 32; a portion of the specific process 56 may also be stored.

[0218] As a hardware resource for performing specific processing, various processors can be used. For example, a CPU is a general-purpose processor that functions as a hardware resource for performing specific processing by executing software, i.e., programs. Additionally, a dedicated circuit can be listed as a processor; it is a processor with a circuit structure specifically designed for performing specific processing, such as a FPGA (Field-Programmable Gate Array), PLD (Programmable Logic Device), or ASIC (Application Specific Integrated Circuit). Every processor has built-in or connected memory, and every processor executes specific processing by using that memory.

[0219] The hardware resources for performing a specific process can consist of one of these various processors, or a combination of two or more processors of the same or different types (e.g., a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Furthermore, the hardware resources for performing a specific process can also be a single processor.

[0220] As an example of a single processor, the first type consists of a combination of one or more CPUs and software, which functions as a hardware resource to perform specific processing. The second type uses a processor, such as a System-on-a-chip (SoC), which implements the entire system functionality, including multiple hardware resources for performing specific processing, using a single IC chip. In this case, the specific processing is implemented using one or more of the aforementioned processors that serve as hardware resources.

[0221] Furthermore, as the hardware architecture of these various processors, more specifically, circuits combining semiconductor elements and other circuit components can be used. Moreover, the specific process described above is merely an example. Therefore, it goes without saying that, without departing from the main point, unnecessary steps can be removed, new steps can be added, or the processing order can be changed.

[0222] Furthermore, although the above examples have been described in terms of first to fourth embodiments, some or all of these embodiments can be combined. Additionally, the smart device 14, smart glasses 214, head-mounted terminal 314, and robot 414 are just examples and can be combined separately, or other devices may be used. Furthermore, although the above examples have been described in terms of morphological example 1 and morphological example 2, these can also be combined.

[0223] The foregoing descriptions and illustrations are detailed explanations of the parts covered by this disclosure and are merely one example of this disclosure. For instance, the descriptions of the above-described structure, function, role, and effect are just one example of the structure, function, role, and effect of the parts covered by this disclosure. Therefore, it goes without saying that, without departing from the spirit of this disclosure, unnecessary parts can be deleted, new elements can be added, or replacements can be made to the foregoing descriptions and illustrations. Furthermore, to avoid confusion and facilitate understanding of the parts covered by this disclosure, explanations of technical common sense that does not require special explanation for implementing this disclosure have been omitted from the foregoing descriptions and illustrations.

[0224] All documents, patent applications and technical standards described in this specification are incorporated herein by reference as if they were specifically and individually described by reference.

[0225] [Postscript 1]

[0226] A system, characterized in that it comprises:

[0227] The acquisition department is used to acquire physical data;

[0228] The analysis unit is used to analyze the physical data acquired by the acquisition unit;

[0229] The generation unit is used to generate the optimal playing posture and fingering techniques based on the data parsed by the analysis unit.

[0230] A providing unit is used to provide feedback generated by the generating unit;

[0231] The evaluation department is used to evaluate the user's progress based on the feedback provided by the providing department, and to propose new practice methods or topics.

[0232] [Postscript 2]

[0233] The system as described in Appendix 1 is characterized in that,

[0234] The providing unit displays the feedback generated by the AI ​​in an AR manner overlaid on the user's performance.

[0235] [Postscript 3]

[0236] The system as described in Appendix 1 is characterized in that,

[0237] The evaluation department regularly assesses users' progress and proposes new practice methods or topics based on their skill levels.

[0238] [Postscript 4]

[0239] The system as described in Appendix 1 is characterized in that,

[0240] The acquisition unit uses the smartphone's camera to acquire the user's physical data.

[0241] [Postscript 5]

[0242] The system as described in Appendix 1 is characterized in that,

[0243] The analysis unit generates a user-specific profile based on the acquired physical data.

[0244] [Postscript 6]

[0245] The system as described in Appendix 1 is characterized in that,

[0246] The generation unit analyzes the user's performance video and generates ideal playing postures and fingering techniques.

[0247] [Postscript 7]

[0248] The system as described in Appendix 1 is characterized in that,

[0249] The acquisition unit infers the user's emotions and adjusts the timing of acquiring physical data based on the inferred user emotions.

[0250] [Postscript 8]

[0251] The system as described in Appendix 1 is characterized in that,

[0252] The acquisition unit analyzes the user's past physical data and selects the optimal acquisition method.

[0253] [Postscript 9]

[0254] The system as described in Appendix 1 is characterized in that,

[0255] When acquiring physical data, the acquisition unit filters data based on the user's current health status and lifestyle habits.

[0256] [Postscript 10]

[0257] The system as described in Appendix 1 is characterized in that,

[0258] The acquisition unit infers the user's emotions and determines the priority of the acquired physical data based on the inferred user emotions.

[0259] [Postscript 11]

[0260] The system as described in Appendix 1 is characterized in that,

[0261] When acquiring physical data, the acquisition unit considers the user's geographical location information and prioritizes acquiring data with high relevance.

[0262] [Postscript 12]

[0263] The system as described in Appendix 1 is characterized in that,

[0264] When acquiring physical data, the acquisition unit analyzes the user's social media activities to obtain relevant data.

[0265] [Postscript 13]

[0266] The system as described in Appendix 1 is characterized in that,

[0267] The analysis unit infers the user's emotions and adjusts the analysis presentation based on the inferred user emotions.

[0268] [Postscript 14]

[0269] The system as described in Appendix 1 is characterized in that,

[0270] During the analysis process, the analysis unit adjusts the level of detail based on the importance of the physical data.

[0271] [Postscript 15]

[0272] The system as described in Appendix 1 is characterized in that,

[0273] During the parsing process, the parsing unit applies different parsing algorithms based on the category of the physical data.

[0274] [Postscript 16]

[0275] The system as described in Appendix 1 is characterized in that,

[0276] The parsing unit infers the user's emotions and adjusts the parsing length based on the inferred user emotions.

[0277] [Postscript 17]

[0278] The system as described in Appendix 1 is characterized in that,

[0279] During the analysis process, the analysis unit determines the analysis priority based on the timing of the acquisition of the physical data.

[0280] [Postscript 18]

[0281] The system as described in Appendix 1 is characterized in that,

[0282] During the analysis process, the analysis unit adjusts the analysis order based on the correlation of the physical data.

[0283] [Postscript 19]

[0284] The system as described in Appendix 1 is characterized in that,

[0285] The generation unit predicts the user's emotions and adjusts the generated performance posture and fingering techniques according to the predicted user emotions.

[0286] [Postscript 20]

[0287] The system as described in Appendix 1 is characterized in that,

[0288] During the generation process, the generation unit adjusts the level of detail based on the importance of the physical data.

[0289] [Postscript 21]

[0290] The system as described in Appendix 1 is characterized in that,

[0291] The generation unit applies different generation algorithms based on the category of physical data during the generation process.

[0292] [Postscript 22]

[0293] The system as described in Appendix 1 is characterized in that,

[0294] The generation unit predicts the user's emotions and adjusts the length of the generated playing posture and fingering techniques based on the predicted user emotions.

[0295] [Postscript 23]

[0296] The system as described in Appendix 1 is characterized in that,

[0297] The generation unit determines the generation priority based on the timing of acquiring the physical data during the generation process.

[0298] [Postscript 24]

[0299] The system as described in Appendix 1 is characterized in that,

[0300] The generation unit adjusts the generation order based on the correlation of the physical data during the generation process.

[0301] [Postscript 25]

[0302] The system as described in Appendix 1 is characterized in that,

[0303] The provider infers the user's emotions and adjusts the display method of the feedback based on the inferred user emotions.

[0304] [Postscript 26]

[0305] The system as described in Appendix 1 is characterized in that,

[0306] When providing feedback, the providing department refers to the user's previous feedback history and selects the optimal display method.

[0307] [Postscript 27]

[0308] The system as described in Appendix 1 is characterized in that,

[0309] When providing feedback, the providing department customizes the feedback content based on the user's current performance status.

[0310] [Postscript 28]

[0311] The system as described in Appendix 1 is characterized in that,

[0312] The providing unit infers the user's emotions and determines the priority of feedback based on the inferred user emotions.

[0313] [Postscript 29]

[0314] The system as described in Appendix 1 is characterized in that,

[0315] When providing feedback, the providing department considers the user's geographical location information and selects the optimal feedback method.

[0316] [Postscript 30]

[0317] The system as described in Appendix 1 is characterized in that,

[0318] When providing feedback, the providing department analyzes users' social media activities and proposes feedback methods.

[0319] [Postscript 31]

[0320] The system as described in Appendix 1 is characterized in that,

[0321] The evaluation department infers the user's emotions and adjusts the evaluation method based on the inferred user emotions.

[0322] [Postscript 32]

[0323] The system as described in Appendix 1 is characterized in that,

[0324] The evaluation department analyzes the user's past performance data and selects the optimal evaluation method during the evaluation process.

[0325] [Postscript 33]

[0326] The system as described in Appendix 1 is characterized in that,

[0327] When conducting evaluations, the evaluation department customizes evaluation criteria based on the user's current skill level.

[0328] [Postscript 34]

[0329] The system as described in Appendix 1 is characterized in that,

[0330] The evaluation department infers the user's emotions and determines the priority of the evaluation based on the inferred user emotions.

[0331] [Postscript 35]

[0332] The system as described in Appendix 1 is characterized in that,

[0333] When conducting evaluations, the evaluation department considers the user's geographical location information and selects the optimal evaluation method.

[0334] [Postscript 36]

[0335] The system as described in Appendix 1 is characterized in that,

[0336] The evaluation department analyzes users' social media activities and proposes evaluation methods during the evaluation process.

Claims

1. A system, characterized in that, include: The acquisition department is used to acquire physical data; The analysis unit is used to analyze the physical data acquired by the acquisition unit; The generation unit is used to generate the optimal playing posture and fingering techniques based on the data parsed by the analysis unit. A providing unit is used to provide feedback generated by the generating unit; The evaluation department is used to evaluate the user's progress based on the feedback provided by the providing department, and to propose new practice methods or topics.

2. The system as described in claim 1, characterized in that, The providing unit displays the feedback generated by the AI ​​in an AR manner overlaid on the user's performance.

3. The system as described in claim 1, characterized in that, The evaluation department regularly assesses users' progress and proposes new practice methods or topics based on their skill levels.

4. The system as described in claim 1, characterized in that, The acquisition unit uses the smartphone's camera to acquire the user's physical data.

5. The system as described in claim 1, characterized in that, The analysis unit generates a user-specific profile based on the acquired physical data.

6. The system as described in claim 1, characterized in that, The generation unit analyzes the user's performance video and generates ideal playing postures and fingering techniques.

7. The system as described in claim 1, characterized in that, The acquisition unit infers the user's emotions and adjusts the timing of acquiring physical data based on the inferred user emotions.

8. The system as described in claim 1, characterized in that, The acquisition unit analyzes the user's past physical data and selects the optimal acquisition method.

9. The system as described in claim 1, characterized in that, When acquiring physical data, the acquisition unit filters data based on the user's current health status and lifestyle habits.

10. The system as claimed in claim 1, characterized in that, The acquisition unit infers the user's emotions and determines the priority of the acquired physical data based on the inferred user emotions.

Images