system

Abstract

The system according to the embodiment aims to provide a customized learning plan based on the user's playing skill and match them with others who share the same passion for music. [Solution] A system according to an embodiment includes a collection unit, an analysis unit, a generation unit, a provision unit, and a matching unit. The collection unit collects performance data of a user. The analysis unit analyzes the data collected by the collection unit and evaluates the user's skill level. The generation unit generates a customized study plan based on the skill level evaluated by the analysis unit. The provision unit provides the study plan generated by the generation unit. The matching unit finds peers who share the same passion for music as the user and suggests online real-time sessions or band performances.

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional technologies do not adequately provide customized learning plans based on a user's playing skill or match users with others who share the same passion for music, so there is room for improvement.

[0005] The system according to the embodiment aims to provide a customized learning plan based on the user's playing skill and match them with others who share the same passion for music. [Means for solving the problem]

[0006] The system according to the embodiment includes a collection unit, an analysis unit, a generation unit, a provision unit, and a matching unit. The collection unit collects performance data of a user. The analysis unit analyzes the data collected by the collection unit and evaluates the user's skill level. The generation unit generates a customized learning plan based on the skill level evaluated by the analysis unit. The provision unit provides the learning plan generated by the generation unit. The matching unit finds peers who share the same passion for music as the user and suggests online real-time sessions or band performances. [Effects of the Invention]

[0007] The system according to the embodiment provides a customized learning plan based on the user's playing skill and can match users with others who share the same passion for music. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9]1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. DETAILED DESCRIPTION OF THE INVENTION

[0009] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0010] First, the terms used in the following description will be explained.

[0011] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, the processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (Tensor Processing Unit).

[0012] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0013] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0014] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), and Bluetooth (registered trademark).

[0015] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0016] [First embodiment] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0017] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0018] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0019] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0020] The reception device 38 includes a touch panel 38A and a microphone 38B, and receives user input. The touch panel 38A detects contact with a pointer (for example, a pen or a finger) to receive user input by the touch of the pointer. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 (see FIG. 2) acquires the data indicating the user input.

[0021] Output device 40 includes a display 40A and a speaker 40B, and presents data to a user by outputting the data in a form of expression that the user can perceive (e.g., audio and / or text). Display 40A displays visible information such as text and images in accordance with instructions from processor 46. Speaker 40B outputs audio in accordance with instructions from processor 46. Camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0022] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0023] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0024] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0025] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate a user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, but is not limited to these examples. Furthermore, the estimation and prediction of emotion also includes, for example, emotion analysis.

[0026] In the smart device 14, the specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The specific processing program 60 is used together with the specific processing program 56 by the data processing system 10. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific processing program 60 executed on the RAM 48. Note that the smart device 14 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the specific processing unit 290 using these models.

[0027] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device (e.g., a generation server) may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (prediction result, etc.) using the data generation model 58 by communicating with the server device having the data generation model 58. Furthermore, the data processing device 12 may be a server device, or may be a terminal device owned by a user (e.g., a mobile phone, a robot, a home appliance, etc.). Next, an example of processing by the data processing system 10 according to the first embodiment will be described.

[0028] (Example 1) A music learning system according to an embodiment of the present invention combines music and AI to provide a next-generation personalized learning experience. This system is more than just a musical instrument practice app; it allows users to experience the joy of carefully playing each chord and following the rhythm, while also providing the excitement of sharing their skills with other music lovers. First, the system uses AI to understand the user's performance skill level, preferences, and goals, and provides a customized learning plan just for that user. For example, a beginner user might be offered a plan that starts with practicing basic chords and gradually increases in difficulty. Meanwhile, an advanced user might be offered a plan aimed at more advanced techniques and playing complex pieces. Furthermore, the system can find peers who share the same passion for music as the user, allowing them to hone their individual performances and experience online real-time sessions and band performances. For example, if a user plays guitar, the system can find peers who play other instruments and suggest playing together as a band. In this way, the system accelerates the user's musical journey and provides services to further deepen their enjoyment of music. Users can learn at their own pace and enjoy a more fulfilling musical experience by playing with other music lovers. This allows the music learning system to collect and analyze a user's performance data, generate and provide a customized learning plan, and find peers who share the same passion for music as the user, suggesting online real-time sessions or band performances.

[0029] A music learning system according to an embodiment includes a collection unit, an analysis unit, a generation unit, a provision unit, and a matching unit. The collection unit collects performance data of a user. The performance data of a user includes, but is not limited to, audio data, video data, and MIDI data. For example, the collection unit records audio data of a user's performance using a microphone and saves the audio data as a digital audio file. The collection unit can also record video data of a user's performance using a camera and save the video data as a video file. The collection unit can also collect MIDI data of a user's performance and save the MIDI data in a digital format. The analysis unit analyzes the data collected by the collection unit to evaluate the user's skill level. Skill level evaluations include, but are not limited to, performance accuracy, sense of rhythm, and technical difficulty. For example, the analysis unit analyzes the collected audio data to evaluate performance accuracy. The analysis unit can also analyze the collected video data to evaluate rhythmic ability. The analysis unit can also analyze the collected MIDI data to evaluate technical difficulty. The generation unit generates a customized study plan based on the skill level evaluated by the analysis unit. The customized study plan may include, but is not limited to, the user's goals, current skill level, and learning progress. For example, the generation unit may generate a plan for a beginner user that starts with practicing basic chords and gradually increases in difficulty. For an advanced user, the generation unit may also generate a plan that aims to teach more advanced techniques or play complex pieces. The provision unit provides the study plan generated by the generation unit. Methods of provision include, but are not limited to, an online platform, email, app notification, etc. For example, the provision unit may provide the study plan through an online platform. Alternatively, the provision unit may provide the study plan via email. Furthermore, the provision unit may provide the study plan via app notification. The matching unit finds friends who share the same passion for music as the user and suggests online real-time sessions or band performances.Matching criteria include, but are not limited to, skill level, musical genre, and geographical proximity. For example, the matching unit finds peers with the same skill level based on the user's skill level. The matching unit can also find peers who share the same musical genre based on the user's musical genre. Furthermore, the matching unit can find peers who live nearby based on the user's geographical proximity. This allows the music learning system according to the embodiment to collect and analyze the user's performance data, generate and provide a customized learning plan, and find peers who share the same musical passion as the user, suggesting online real-time sessions or band performances.

[0030] The collection unit can collect audio data or video data of a user performing a musical piece. For example, the collection unit can use a microphone to record audio data of a user performing a musical piece and save it as a digital audio file. The collection unit can also use a camera to record video data of a user performing a musical piece and save it as a video file. For example, the collection unit can record audio data of a user performing a musical piece using a high-quality microphone and obtain clear audio data using noise reduction technology. The collection unit can also record video data of a user performing a musical piece using a high-resolution camera and obtain stable video data using video blur correction technology. The collection unit can also simultaneously collect audio data and video data of a user performing a musical piece, synchronize them, and save them. For example, the collection unit can simultaneously collect audio data and video data, assign timestamps, and synchronize them, enabling detailed analysis of the performance. This allows for detailed collection of the user's performance data. Some or all of the above-described processing by the collection unit can be performed using, for example, AI, or without AI. For example, the collection department can input audio and video data into the AI, which can then evaluate the quality of the data and select the optimal collection method.

[0031] The analysis unit can evaluate the user's skill level, sense of rhythm, performance accuracy, etc. based on the collected data. The analysis unit, for example, analyzes collected audio data to evaluate performance accuracy. For example, the analysis unit can analyze the pitch and rhythm accuracy of the audio data in detail to evaluate the skill level. The analysis unit can also analyze collected video data to evaluate rhythmic sense. For example, the analysis unit can analyze finger movements and facial expressions in the video data to evaluate rhythmic sense. The analysis unit can also analyze collected MIDI data to evaluate technical difficulty. For example, the analysis unit can analyze the tempo and dynamics of the MIDI data in detail to evaluate technical difficulty. This allows for a detailed evaluation of the user's skill level, sense of rhythm, and performance accuracy. Some or all of the above-described processing by the analysis unit may be performed using, for example, AI, or may be performed without AI. For example, the analysis unit can input audio data, video data, and MIDI data into AI, which analyzes the data and evaluates the skill level, sense of rhythm, and performance accuracy.

[0032] The generation unit can generate a learning plan suited to the user based on the data from the analysis unit. For example, for a beginner user, the generation unit generates a plan that starts with practicing basic chords and gradually increases in difficulty. For example, the generation unit provides a beginner user with a plan to learn how to hold basic chords and rhythm. The generation unit can also generate a plan for an advanced user to learn more advanced techniques or play complex pieces. For example, the generation unit provides an advanced user with a plan to learn advanced techniques such as fast playing and sweep picking. Furthermore, the generation unit can generate a customized learning plan based on the user's goals, current skill level, and learning progress. For example, if the user's goal is to be able to play a specific song, the generation unit provides a plan that includes the techniques and practice exercises required for that song. This allows the generation of an optimal learning plan for the user. Some or all of the above-described processing in the generation unit may be performed using, or without, AI. For example, the generation unit can input data from the analysis unit into AI, which then generates an optimal learning plan for the user.

[0033] The providing unit can provide the study plan generated by the generating unit. The providing unit can provide the study plan through, for example, an online platform. For example, the providing unit can provide an online platform that the user can access by logging in and display the study plan. The providing unit can also provide the study plan via email. For example, the providing unit can send the study plan to the user's email address so that the user can check the study plan via email. The providing unit can also provide the study plan through an app notification. For example, the providing unit can send an app notification to the user's smartphone to notify them of the study plan. This allows the generated study plan to be provided to the user. Some or all of the above-mentioned processing in the providing unit can be performed using, for example, AI, or can be performed without using AI. For example, the providing unit can input the study plan generated by the generating unit into AI, which can select the optimal presentation method.

[0034] The matching unit can find suitable friends based on the user's skill level or preferences and suggest online sessions. For example, the matching unit finds friends with the same skill level based on the user's skill level. For example, the matching unit analyzes the user's performance data and suggests friends with a similar skill level. The matching unit can also find friends who play the same music genre based on the user's music genre. For example, if the user likes rock, the matching unit suggests friends who play rock. Furthermore, the matching unit can find friends who live nearby based on the user's geographical proximity. For example, if the user lives in a specific area, the matching unit suggests friends who live in that area. This makes it possible to find the best friends for the user and suggest online sessions. Some or all of the above-mentioned processing in the matching unit may be performed using, or without, AI. For example, the matching unit can input data on the user's skill level and preferences into AI, which can then find the best friends.

[0035] The collection unit can analyze the user's past performance data and select the optimal collection method. The collection unit, for example, identifies from the past performance data a time period in which the user made the greatest improvement and collects data for that time period. For example, the collection unit analyzes the past performance data to identify a time period in which the user made the greatest improvement and collects data for that time period. The collection unit can also identify a time period in which the user is most focused based on the past performance data and collect data for that time period. For example, the collection unit analyzes the past performance data to identify a time period in which the user is most focused and collects data for that time period. The collection unit can also analyze the past performance data to identify a time period in which the user is most relaxed and collect data for that time period. For example, the collection unit analyzes the past performance data to identify a time period in which the user is most relaxed and collects data for that time period. This makes it possible to select the optimal collection method based on the past performance data. Some or all of the above-described processing in the collection unit may be performed using, for example, AI, or may be performed without using AI. For example, the collection unit can input the past performance data into AI, which then selects the optimal collection method.

[0036] When collecting performance data, the collection unit can filter the data based on the user's current practice status and goals. For example, if the user is practicing a specific song, the collection unit collects only data related to that song. For example, when the user is practicing a specific song, the collection unit collects only data related to that song and filters out other data. Furthermore, if the user is practicing a specific technique, the collection unit can collect only data related to that technique. For example, when the user is practicing a specific technique, the collection unit collects only data related to that technique and filters out other data. Furthermore, if the user has a specific goal, the collection unit can collect only data related to that goal. For example, when the user has a specific goal, the collection unit collects only data related to that goal and filters out other data. This allows data filtering based on the user's practice status and goals. Some or all of the above-described processing in the collection unit may be performed using, for example, AI, or may be performed without AI. For example, the collection unit can input data on the user's practice status and goals into AI, which can then determine the optimal filtering method.

[0037] When collecting performance data, the collection unit can prioritize collecting highly relevant data by taking into account the user's geographical location information. For example, if the user is performing in a specific area, the collection unit prioritizes collecting music data related to that area. For example, when the user is performing in a specific area, the collection unit prioritizes collecting music data related to that area and uses it for analysis. Furthermore, if the user is participating in a specific event, the collection unit can prioritize collecting data related to that event. For example, when the user is participating in a specific event, the collection unit prioritizes collecting data related to that event and uses it for analysis. Furthermore, if the user is practicing in a specific location, the collection unit can prioritize collecting data related to that location. For example, when the user is practicing in a specific location, the collection unit prioritizes collecting data related to that location and uses it for analysis. In this way, highly relevant data can be prioritized by taking into account the user's geographical location information. Some or all of the above-described processing by the collection unit may be performed using, for example, AI, or may be performed without using AI. For example, the collection unit can input the user's geographical location information to AI, which can then determine the optimal data prioritization.

[0038] When collecting performance data, the collection unit can analyze the user's social media activities and collect related data. The collection unit, for example, collects related data based on performance videos shared by the user on social media. For example, the collection unit analyzes performance videos shared by the user on social media and collects related data. The collection unit can also collect performance data of artists the user follows on social media. For example, the collection unit collects performance data of artists the user follows on social media and uses the data for analysis. The collection unit can also collect data of music communities the user participates in on social media. For example, the collection unit collects data of music communities the user participates in on social media and uses the data for analysis. This makes it possible to collect related data based on the user's social media activities. Some or all of the above-mentioned processing by the collection unit may be performed using, for example, AI, or may be performed without using AI. For example, the collection unit can input the user's social media activities into AI, which can determine the optimal data collection method.

[0039] The analysis unit can improve the accuracy of skill level evaluation by taking into account detailed parameters of the performance data during analysis. The analysis unit, for example, evaluates the skill level by analyzing in detail the pitch and rhythm accuracy of the audio data. For example, the analysis unit evaluates the skill level by evaluating the pitch accuracy and rhythm accuracy of the audio data. The analysis unit can also evaluate the skill level by analyzing finger movements and facial expressions of video data. For example, the analysis unit analyzes the accuracy of finger movements and changes in facial expressions of the video data to evaluate the skill level. The analysis unit can also evaluate the skill level by analyzing in detail the tempo and dynamics of the performance data. For example, the analysis unit analyzes the consistency of the tempo and changes in dynamics of the performance data to evaluate the skill level. This improves the accuracy of skill level evaluation by taking into account detailed parameters of the performance data. Some or all of the above-described processing in the analysis unit may be performed using, or without, AI. For example, the analysis unit inputs detailed parameters of the performance data into AI, which can improve the accuracy of skill level evaluation.

[0040] During analysis, the analysis unit can evaluate fluctuations in skill level by referring to the user's performance history. The analysis unit, for example, compares the performance data with past performance data to evaluate improvement in skill level. For example, the analysis unit compares past performance data with current performance data to evaluate improvement in skill level. The analysis unit can also evaluate decline in skill level by comparing the performance data with past performance data. For example, the analysis unit compares past performance data with current performance data to evaluate decline in skill level. Furthermore, the analysis unit can also evaluate stability of skill level by comparing the performance data with past performance data. For example, the analysis unit compares past performance data with current performance data to evaluate stability of skill level. This makes it possible to evaluate fluctuations in skill level based on the user's performance history. Some or all of the above-described processing in the analysis unit may be performed using, for example, AI, or may be performed without AI. For example, the analysis unit can input the user's performance history into AI, which then evaluates fluctuations in skill level.

[0041] During analysis, the analysis unit can evaluate the skill level by taking into account the geographical distribution of the performance data. For example, if a user is performing in a specific region, the analysis unit evaluates the skill level by taking into account the musical style of the region. For example, if a user is performing in a specific region, the analysis unit evaluates the skill level by taking into account the musical style of the region. Furthermore, if a user is performing in a specific country, the analysis unit can evaluate the skill level by taking into account the musical culture of the country. For example, if a user is performing in a specific country, the analysis unit evaluates the skill level by taking into account the musical culture of the country. Furthermore, if a user is performing in a specific city, the analysis unit can evaluate the skill level by taking into account the music scene of the city. For example, if a user is performing in a specific city, the analysis unit evaluates the skill level by taking into account the music scene of the city. In this way, the skill level can be evaluated by taking into account the geographical distribution of the performance data. Some or all of the above-described processing in the analysis unit may be performed using, for example, AI, or may be performed without using AI. For example, the analysis unit can input the geographical distribution of the performance data into AI, which can then evaluate the skill level.

[0042] The analysis unit can improve the accuracy of the skill level assessment by referring to related music theory and literature during analysis. The analysis unit, for example, evaluates the accuracy of the performance based on music theory. For example, the analysis unit evaluates the accuracy of pitch and rhythm based on music theory to assess the skill level. The analysis unit can also evaluate technical aspects of the performance by referring to music literature. For example, the analysis unit evaluates the technical aspects of the performance by referring to music literature to assess the skill level. Furthermore, the analysis unit can combine music theory and literature to assess the overall skill level. For example, the analysis unit combines music theory and literature to assess the overall skill level and provide detailed feedback. This can improve the accuracy of the skill level assessment by referring to related music theory and literature. Some or all of the above-described processing in the analysis unit may be performed using, or without, AI. For example, the analysis unit can input related music theory and literature into AI, which can improve the accuracy of the skill level assessment.

[0043] When generating a study plan, the generation unit can generate an optimal plan by referring to the user's past study history. The generation unit, for example, generates a plan that allows the user to study most effectively based on the user's past study history. For example, the generation unit analyzes the user's past study history and generates a plan that allows the user to study most effectively. The generation unit can also generate a plan that is most interesting to the user based on the user's past study history. For example, the generation unit analyzes the user's past study history and generates a plan that is most interesting to the user. The generation unit can also generate a plan that allows the user to make the most progress based on the user's past study history. For example, the generation unit analyzes the user's past study history and generates a plan that allows the user to make the most progress. This makes it possible to generate an optimal plan based on the user's past study history. Some or all of the above-described processing in the generation unit may be performed using, for example, AI, or may be performed without using AI. For example, the generation unit can input the user's past study history into AI, which then generates an optimal plan.

[0044] When generating a learning plan, the generation unit can adjust the difficulty of the plan based on the user's current goal. For example, if the user is a beginner, the generation unit provides a basic plan. For example, if the user is a beginner, the generation unit provides a plan for learning how to hold basic chords and rhythm. The generation unit can also provide a plan of medium difficulty if the user is an intermediate player. For example, if the user is an intermediate player, the generation unit provides a plan for learning medium-difficulty techniques or pieces of music. The generation unit can also provide an advanced plan if the user is an advanced player. For example, if the user is an advanced player, the generation unit provides a plan aimed at learning advanced techniques or playing complex pieces of music. This allows the difficulty of the plan to be adjusted based on the user's current goal. Some or all of the above-described processing in the generation unit may be performed using, or without, AI. For example, the generation unit can input the user's current goal into AI, which can then determine the optimal difficulty level of the plan.

[0045] When generating a study plan, the generation unit can generate an optimal plan by taking into account the user's geographical location information. For example, if the user is studying in a specific region, the generation unit generates a plan by taking into account the musical style of the region. For example, if the user is studying in a specific region, the generation unit generates a plan by taking into account the musical style of the region. Furthermore, if the user is studying in a specific country, the generation unit can generate a plan by taking into account the musical culture of the country. For example, if the user is studying in a specific country, the generation unit can generate a plan by taking into account the musical culture of the country. Furthermore, if the user is studying in a specific city, the generation unit can generate a plan by taking into account the music scene of the city. For example, if the user is studying in a specific city, the generation unit generates a plan by taking into account the music scene of the city. In this way, an optimal plan can be generated by taking into account the user's geographical location information. Some or all of the above-described processing in the generation unit may be performed using, for example, AI, or may be performed without using AI. For example, the generation unit can input the user's geographical location information into AI, which can then generate an optimal plan.

[0046] When generating a study plan, the generation unit can analyze the user's social media activity and propose a related plan. The generation unit can, for example, propose a related plan based on performance videos shared by the user on social media. For example, the generation unit can analyze performance videos shared by the user on social media and propose a related plan. The generation unit can also propose a plan taking into account the performance styles of artists the user follows on social media. For example, the generation unit can propose a plan taking into account the performance styles of artists the user follows on social media. The generation unit can also propose a related plan based on the activities of music communities the user participates in on social media. For example, the generation unit can analyze the activities of music communities the user participates in on social media and propose a related plan. This makes it possible to propose a related plan based on the user's social media activity. Some or all of the above-described processing by the generation unit can be performed using, or without, AI. For example, the generation unit can input the user's social media activity into AI, which can then propose an optimal plan.

[0047] When providing a study plan, the provision unit can select the optimal provision method by referring to the user's past study history. The provision unit, for example, selects the provision method that allows the user to study most effectively based on the past study history. For example, the provision unit analyzes the past study history and selects the provision method that allows the user to study most effectively. The provision unit can also select the provision method that the user is most interested in based on the past study history. For example, the provision unit analyzes the past study history and selects the provision method that the user is most interested in. The provision unit can also select the provision method that the user has made the most progress based on the past study history. For example, the provision unit analyzes the past study history and selects the provision method that the user has made the most progress. This makes it possible to select the optimal provision method based on the past study history. Some or all of the above-mentioned processing in the provision unit may be performed using, for example, AI, or may be performed without using AI. For example, the provision unit can input the past study history into AI, which then selects the optimal provision method.

[0048] When providing a study plan, the providing unit can adjust the timing of providing the study plan based on the user's current practice status. For example, when the user is practicing, the providing unit provides the study plan in real time. For example, when the user is practicing, the providing unit provides the study plan in real time and presents practice tasks suitable for the user. Furthermore, when the user is taking a break, the providing unit can also provide a study plan for the next practice. For example, when the user is taking a break, the providing unit provides a study plan for the next practice and presents practice tasks suitable for the user. Furthermore, when the user is practicing toward a specific goal, the providing unit can also provide a study plan at a timing that matches the goal. For example, when the user is practicing toward a specific goal, the providing unit provides a study plan at a timing that matches the goal and presents practice tasks suitable for the user. This makes it possible to adjust the timing of providing the study plan based on the user's current practice status. Some or all of the above-mentioned processing by the providing unit may be performed using, for example, AI, or may be performed without using AI. For example, the providing unit can input the user's practice status into AI, which can determine the optimal timing of providing the study plan.

[0049] When providing a study plan, the providing unit can select the optimal delivery method by taking into account the user's device information. For example, if the user is using a smartphone, the providing unit selects a delivery method that matches the screen size. For example, if the user is using a smartphone, the providing unit selects a delivery method that matches the screen size and provides a study plan suitable for the user. Furthermore, if the user is using a tablet, the providing unit can select a delivery method optimized for a large screen. For example, if the user is using a tablet, the providing unit selects a delivery method optimized for a large screen and provides a study plan suitable for the user. Furthermore, if the user is using a smartwatch, the providing unit can select a delivery method that is concise and highly visible. For example, if the user is using a smartwatch, the providing unit selects a delivery method that is concise and highly visible and provides a study plan suitable for the user. This makes it possible to select the optimal delivery method based on the user's device information. Some or all of the above-described processing by the providing unit may be performed using, for example, AI, or may be performed without using AI. For example, the providing unit can input the user's device information into AI, which then selects the optimal delivery method.

[0050] When providing a study plan, the providing unit can analyze the user's social media activity and provide a relevant plan. The providing unit can provide a relevant plan, for example, based on performance videos shared by the user on social media. For example, the providing unit can analyze performance videos shared by the user on social media and provide a relevant plan. The providing unit can also provide a plan taking into account the performance styles of artists the user follows on social media. For example, the providing unit can provide a plan taking into account the performance styles of artists the user follows on social media. Furthermore, the providing unit can also provide a relevant plan based on the activities of music communities the user participates in on social media. For example, the providing unit can analyze the activities of music communities the user participates in on social media and provide a relevant plan. This makes it possible to provide a relevant plan based on the user's social media activity. Some or all of the above-described processing by the providing unit can be performed using, or without, AI. For example, the providing unit can input the user's social media activity into AI, which can then provide an optimal plan.

[0051] During matching, the matching unit can find the most suitable peer by referring to the user's performance history. For example, the matching unit finds peers who play the same songs based on songs the user has played in the past. For example, the matching unit analyzes songs the user has played in the past and finds peers who play the same songs. The matching unit can also find peers who attended the same events based on events the user has participated in in the past. For example, the matching unit analyzes events the user has participated in in the past and finds peers who attended the same events. Furthermore, the matching unit can find peers who are practicing the same skills based on skills the user has practiced in the past. For example, the matching unit analyzes skills the user has practiced in the past and finds peers who are practicing the same skills. In this way, the most suitable peer can be found based on the user's performance history. Some or all of the above-described processing in the matching unit may be performed using, for example, AI, or may be performed without using AI. For example, the matching unit can input the user's performance history into AI, which then finds the most suitable peer.

[0052] The matching unit can improve the accuracy of matching based on the user's current goal during matching. For example, if a user has a goal of playing a specific song, the matching unit finds fellow players who play that song. For example, if a user has a goal of playing a specific song, the matching unit finds fellow players who play that song. Furthermore, if a user has a goal of mastering a specific skill, the matching unit can also find fellow players who are practicing that skill. For example, if a user has a goal of mastering a specific skill, the matching unit can also find fellow players who are practicing that skill. Furthermore, if a user has a goal of participating in a specific event, the matching unit can also find fellow players who will participate in that event. For example, if a user has a goal of participating in a specific event, the matching unit finds fellow players who will participate in that event. This can improve the accuracy of matching based on the user's current goal. Some or all of the above-described processing in the matching unit may be performed using, for example, AI, or may be performed without using AI. For example, the matching unit can input the user's current goal into AI, which can then find the optimal fellow player.

[0053] During matching, the matching unit can find the most suitable companion by taking into account the user's geographical location information. For example, if the user is performing in a specific region, the matching unit finds companions who live in that region. For example, if the user is performing in a specific region, the matching unit finds companions who live in that region. Furthermore, if the user is performing in a specific country, the matching unit can find companions who live in that country. For example, if the user is performing in a specific country, the matching unit finds companions who live in that country. Furthermore, if the user is performing in a specific city, the matching unit can find companions who live in that city. For example, if the user is performing in a specific city, the matching unit finds companions who live in that city. In this way, the most suitable companion can be found based on the user's geographical location information. Some or all of the above-described processing in the matching unit may be performed using, for example, AI, or may be performed without using AI. For example, the matching unit can input the user's geographical location information into AI, which can then find the most suitable companion.

[0054] The matching unit can analyze the user's social media activity during matching to suggest related peers. The matching unit can suggest related peers, for example, based on performance videos shared by the user on social media. For example, the matching unit can analyze performance videos shared by the user on social media and suggest related peers. The matching unit can also suggest related peers based on artists the user follows on social media. For example, the matching unit can suggest related peers based on artists the user follows on social media. The matching unit can also suggest related peers based on music communities the user participates in on social media. For example, the matching unit can analyze music communities the user participates in on social media and suggest related peers. This makes it possible to suggest related peers based on the user's social media activity. Some or all of the above-described processing in the matching unit can be performed using, or without, AI. For example, the matching unit can input the user's social media activity into AI, which can then suggest optimal peers.

[0055] The system according to the embodiment is not limited to the above-described example, and various modifications are possible, for example, as follows.

[0056] The analysis unit can identify the user's playing style based on the user's performance data and suggest a practice method optimal for that style. For example, if the user prefers jazz style playing, the analysis unit can suggest a practice method specialized for jazz. If the user prefers classical music, the analysis unit can also suggest a practice method specialized for classical music. Furthermore, if the user prefers rock style playing, the analysis unit can also suggest a practice method specialized for rock. This makes it possible to provide an optimal practice method according to the user's playing style. Some or all of the above-mentioned processing in the analysis unit may be performed using, for example, AI, or may be performed without using AI. For example, the analysis unit can input performance data into AI, which then suggests the optimal practice method.

[0057] When collecting the user's performance data, the collection unit can detect the user's physical movements and posture and provide feedback to improve performance efficiency. For example, the collection unit can detect the user's hand position and finger movements and suggest optimal positions. The collection unit can also detect the user's posture and provide advice on maintaining correct posture. Furthermore, the collection unit can detect the user's breathing pattern and suggest breathing techniques to perform in a relaxed state. This can provide feedback to improve the user's performance efficiency. Some or all of the above-described processing in the collection unit can be performed using, for example, AI, or can be performed without AI. For example, the collection unit can input data on physical movements and posture into AI, which can then provide optimal feedback.

[0058] When generating a study plan for a user, the generation unit can analyze the user's past study history and propose an optimal study plan. For example, the generation unit generates a plan that allows the user to study most effectively based on the user's past study history. For example, the generation unit analyzes the user's past study history and generates a plan that allows the user to study most effectively. The generation unit can also generate a plan that is most interesting to the user based on the user's past study history. For example, the generation unit analyzes the user's past study history and generates a plan that is most interesting to the user. Furthermore, the generation unit can also generate a plan that allows the user to make the most progress based on the user's past study history. For example, the generation unit analyzes the user's past study history and generates a plan that allows the user to make the most progress. This makes it possible to generate an optimal plan based on the user's past study history. Some or all of the above-described processing in the generation unit may be performed using, for example, AI, or may be performed without using AI. For example, the generation unit can input the user's past study history into AI, which then generates an optimal plan.

[0059] The matching unit can find friends who are compatible with the user based on the user's performance data and suggest online sessions. For example, the matching unit finds friends who are compatible with the user based on the user's performance style and skill level. For example, the matching unit analyzes the user's performance style and skill level and suggests friends who are compatible with the user. The matching unit can also find friends who like the same genre based on the user's musical genre preferences. For example, the matching unit analyzes the user's musical genre preferences and suggests friends who like the same genre. Furthermore, the matching unit can find friends who live nearby based on the user's geographical proximity. For example, the matching unit analyzes the user's geographical proximity and suggests friends who live nearby. This makes it possible to find friends who are compatible with the user and suggest online sessions. Some or all of the above-described processing in the matching unit may be performed using, for example, AI, or may be performed without using AI. For example, the matching unit can input performance data into AI, which then finds the optimal friends.

[0060] When analyzing the user's performance data, the analysis unit can evaluate fluctuations in skill level by referring to the user's performance history. For example, the analysis unit compares the user's performance data with past performance data to evaluate improvement in skill level. For example, the analysis unit compares past performance data with current performance data to evaluate improvement in skill level. The analysis unit can also evaluate decline in skill level by comparing the user's performance data with past performance data. For example, the analysis unit compares past performance data with current performance data to evaluate decline in skill level. Furthermore, the analysis unit can also evaluate stability of skill level by comparing the user's performance data with past performance data. For example, the analysis unit compares past performance data with current performance data to evaluate stability of skill level. This makes it possible to evaluate fluctuations in skill level based on the user's performance history. Some or all of the above-described processing in the analysis unit may be performed using, for example, AI, or may be performed without AI. For example, the analysis unit can input the user's performance history into AI, which then evaluates fluctuations in skill level.

[0061] The processing flow of the first embodiment will be briefly explained below.

[0062] Step 1: The collection unit collects the user's performance data. The user's performance data includes audio data, video data, MIDI data, etc. For example, the collection unit may use a microphone to record audio data and save it as a digital audio file. Alternatively, the collection unit may use a camera to record video data and save it as a video file. Furthermore, the collection unit may collect MIDI data and save it in a digital format. Step 2: The analysis unit analyzes the data collected by the collection unit and evaluates the user's skill level. The skill level evaluation includes performance accuracy, sense of rhythm, technical difficulty, etc. For example, the analysis unit analyzes audio data to evaluate performance accuracy, video data to evaluate sense of rhythm, and MIDI data to evaluate technical difficulty. Step 3: The generator generates a customized learning plan based on the skill level assessed by the analyzer. The customized learning plan includes the user's goals, current skill level, learning progress, etc. For example, a plan may be generated that starts beginners with practicing basic chords, while an advanced player may aim to learn advanced techniques and play complex pieces. Step 4: The providing unit provides the study plan generated by the generating unit. Methods of providing include an online platform, email, app notification, etc. For example, the study plan can be provided through an online platform, email, or app notification. Step 5: The matching unit finds people who share the same passion for music as the user and suggests online real-time sessions or band performances. Matching criteria include skill level, music genre, and geographic proximity. For example, skill level can be used to find people with the same skill level, music genre can be used to find people who share the same music genre, and geographic proximity can be used to find people who live nearby.

[0063] (Example 2) A music learning system according to an embodiment of the present invention combines music and AI to provide a next-generation personalized learning experience. This system is more than just a musical instrument practice app; it allows users to experience the joy of carefully playing each chord and following the rhythm, while also providing the excitement of sharing their skills with other music lovers. First, the system uses AI to understand the user's performance skill level, preferences, and goals, and provides a customized learning plan just for that user. For example, a beginner user might be offered a plan that starts with practicing basic chords and gradually increases in difficulty. Meanwhile, an advanced user might be offered a plan aimed at more advanced techniques and playing complex pieces. Furthermore, the system can find peers who share the same passion for music as the user, allowing them to hone their individual performances and experience online real-time sessions and band performances. For example, if a user plays guitar, the system can find peers who play other instruments and suggest playing together as a band. In this way, the system accelerates the user's musical journey and provides services to further deepen their enjoyment of music. Users can learn at their own pace and enjoy a more fulfilling musical experience by playing with other music lovers. This allows the music learning system to collect and analyze a user's performance data, generate and provide a customized learning plan, and find peers who share the same passion for music as the user, suggesting online real-time sessions or band performances.

[0064] A music learning system according to an embodiment includes a collection unit, an analysis unit, a generation unit, a provision unit, and a matching unit. The collection unit collects performance data of a user. The performance data of a user includes, but is not limited to, audio data, video data, and MIDI data. For example, the collection unit records audio data of a user's performance using a microphone and saves the audio data as a digital audio file. The collection unit can also record video data of a user's performance using a camera and save the video data as a video file. The collection unit can also collect MIDI data of a user's performance and save the MIDI data in a digital format. The analysis unit analyzes the data collected by the collection unit to evaluate the user's skill level. Skill level evaluations include, but are not limited to, performance accuracy, sense of rhythm, and technical difficulty. For example, the analysis unit analyzes the collected audio data to evaluate performance accuracy. The analysis unit can also analyze the collected video data to evaluate rhythmic ability. The analysis unit can also analyze the collected MIDI data to evaluate technical difficulty. The generation unit generates a customized study plan based on the skill level evaluated by the analysis unit. The customized study plan may include, but is not limited to, the user's goals, current skill level, and learning progress. For example, the generation unit may generate a plan for a beginner user that starts with practicing basic chords and gradually increases in difficulty. For an advanced user, the generation unit may also generate a plan that aims to teach more advanced techniques or play complex pieces. The provision unit provides the study plan generated by the generation unit. Methods of provision include, but are not limited to, an online platform, email, app notification, etc. For example, the provision unit may provide the study plan through an online platform. Alternatively, the provision unit may provide the study plan via email. Furthermore, the provision unit may provide the study plan via app notification. The matching unit finds friends who share the same passion for music as the user and suggests online real-time sessions or band performances.Matching criteria include, but are not limited to, skill level, musical genre, and geographical proximity. For example, the matching unit finds peers with the same skill level based on the user's skill level. The matching unit can also find peers who share the same musical genre based on the user's musical genre. Furthermore, the matching unit can find peers who live nearby based on the user's geographical proximity. This allows the music learning system according to the embodiment to collect and analyze the user's performance data, generate and provide a customized learning plan, and find peers who share the same musical passion as the user, suggesting online real-time sessions or band performances.

[0065] The collection unit can collect audio data or video data of a user performing a musical piece. For example, the collection unit can use a microphone to record audio data of a user performing a musical piece and save the data as a digital audio file. The collection unit can also use a camera to record video data of a user performing a musical piece and save the video data as a video file. For example, the collection unit can use a high-quality microphone to record audio data of a user performing a musical piece and use noise reduction technology to obtain clear audio data. The collection unit can also use a high-resolution camera to record video data of a user performing a musical piece and use video blur correction technology to obtain stable video data. The collection unit can also simultaneously collect audio data and video data of a user performing a musical piece and save them in synchronization. For example, the collection unit can simultaneously collect audio data and video data and synchronize them by adding a timestamp to enable detailed analysis of the performance. This allows for detailed collection of the user's performance data. Some or all of the above-described processing by the collection unit can be performed using, for example, AI, or without AI. For example, the collection department can input audio and video data into the AI, which can then evaluate the quality of the data and select the optimal collection method.

[0066] The analysis unit can evaluate the user's skill level, sense of rhythm, performance accuracy, etc. based on the collected data. The analysis unit, for example, analyzes collected audio data to evaluate performance accuracy. For example, the analysis unit can analyze the pitch and rhythm accuracy of the audio data in detail to evaluate the skill level. The analysis unit can also analyze collected video data to evaluate rhythmic sense. For example, the analysis unit can analyze finger movements and facial expressions in the video data to evaluate rhythmic sense. The analysis unit can also analyze collected MIDI data to evaluate technical difficulty. For example, the analysis unit can analyze the tempo and dynamics of the MIDI data in detail to evaluate technical difficulty. This allows for a detailed evaluation of the user's skill level, sense of rhythm, and performance accuracy. Some or all of the above-described processing by the analysis unit may be performed using, for example, AI, or may be performed without AI. For example, the analysis unit can input audio data, video data, and MIDI data into AI, which analyzes the data and evaluates the skill level, sense of rhythm, and performance accuracy.

[0067] The generation unit can generate a learning plan suited to the user based on the data from the analysis unit. For example, for a beginner user, the generation unit generates a plan that starts with practicing basic chords and gradually increases in difficulty. For example, the generation unit provides a beginner user with a plan to learn how to hold basic chords and rhythm. The generation unit can also generate a plan for an advanced user to learn more advanced techniques or play complex pieces. For example, the generation unit provides an advanced user with a plan to learn advanced techniques such as fast playing and sweep picking. Furthermore, the generation unit can generate a customized learning plan based on the user's goals, current skill level, and learning progress. For example, if the user's goal is to be able to play a specific song, the generation unit provides a plan that includes the techniques and practice exercises required for that song. This allows the generation of an optimal learning plan for the user. Some or all of the above-described processing in the generation unit may be performed using, or without, AI. For example, the generation unit can input data from the analysis unit into AI, which then generates an optimal learning plan for the user.

[0068] The providing unit can provide the study plan generated by the generating unit. The providing unit can provide the study plan through, for example, an online platform. For example, the providing unit can provide an online platform that the user can access by logging in and display the study plan. The providing unit can also provide the study plan via email. For example, the providing unit can send the study plan to the user's email address so that the user can check the study plan via email. The providing unit can also provide the study plan through an app notification. For example, the providing unit can send an app notification to the user's smartphone to notify them of the study plan. This allows the generated study plan to be provided to the user. Some or all of the above-mentioned processing in the providing unit can be performed using, for example, AI, or can be performed without using AI. For example, the providing unit can input the study plan generated by the generating unit into AI, which can select the optimal presentation method.

[0069] The matching unit can find suitable friends based on the user's skill level or preferences and suggest online sessions. For example, the matching unit finds friends with the same skill level based on the user's skill level. For example, the matching unit analyzes the user's performance data and suggests friends with a similar skill level. The matching unit can also find friends who play the same music genre based on the user's music genre. For example, if the user likes rock, the matching unit suggests friends who play rock. Furthermore, the matching unit can find friends who live nearby based on the user's geographical proximity. For example, if the user lives in a specific area, the matching unit suggests friends who live in that area. This makes it possible to find the best friends for the user and suggest online sessions. Some or all of the above-mentioned processing in the matching unit may be performed using, or without, AI. For example, the matching unit can input data on the user's skill level and preferences into AI, which can then find the best friends.

[0070] The collection unit can estimate the user's emotions and adjust the timing of collecting performance data based on the estimated user emotions. For example, when the user is relaxed, the collection unit collects performance data frequently to obtain detailed data. For example, when the user is relaxed, the collection unit collects data for each performance session and performs detailed analysis. Furthermore, when the user is feeling stressed, the collection unit can reduce the collection timing to reduce the burden on the user. For example, when the user is feeling stressed, the collection unit reduces the frequency of performance data collection to reduce the burden on the user. Furthermore, when the user is concentrating, the collection unit can collect data at the peak of the performance to capture the most important moment. For example, when the user is concentrating, the collection unit collects data at the peak of the performance to capture the most important moment. This allows the timing of collecting performance data to be adjusted according to the user's emotions. Emotion estimation is realized using an emotion estimation function, for example, using an emotion engine or a generation AI. The generation AI can be, but is not limited to, a text generation AI (e.g., LLM) or a multimodal generation AI. Some or all of the above-described processing in the collection unit may be performed using, for example, AI, or may be performed without using AI. For example, the collection unit may input the user's emotion data into AI, which may then determine the optimal collection timing.

[0071] The collection unit can analyze the user's past performance data and select the optimal collection method. The collection unit, for example, identifies from the past performance data a time period in which the user made the greatest improvement and collects data for that time period. For example, the collection unit analyzes the past performance data to identify a time period in which the user made the greatest improvement and collects data for that time period. The collection unit can also identify a time period in which the user is most focused based on the past performance data and collect data for that time period. For example, the collection unit analyzes the past performance data to identify a time period in which the user is most focused and collects data for that time period. The collection unit can also analyze the past performance data to identify a time period in which the user is most relaxed and collect data for that time period. For example, the collection unit analyzes the past performance data to identify a time period in which the user is most relaxed and collects data for that time period. This makes it possible to select the optimal collection method based on the past performance data. Some or all of the above-described processing in the collection unit may be performed using, for example, AI, or may be performed without using AI. For example, the collection unit can input the past performance data into AI, which then selects the optimal collection method.

[0072] When collecting performance data, the collection unit can filter the data based on the user's current practice status and goals. For example, if the user is practicing a specific song, the collection unit collects only data related to that song. For example, when the user is practicing a specific song, the collection unit collects only data related to that song and filters out other data. Furthermore, if the user is practicing a specific technique, the collection unit can collect only data related to that technique. For example, when the user is practicing a specific technique, the collection unit collects only data related to that technique and filters out other data. Furthermore, if the user has a specific goal, the collection unit can collect only data related to that goal. For example, when the user has a specific goal, the collection unit collects only data related to that goal and filters out other data. This allows data filtering based on the user's practice status and goals. Some or all of the above-described processing in the collection unit may be performed using, for example, AI, or may be performed without AI. For example, the collection unit can input data on the user's practice status and goals into AI, which can then determine the optimal filtering method.

[0073] The collection unit can estimate the user's emotions and determine the priority of data to be collected based on the estimated user's emotions. For example, when the user is relaxed, the collection unit prioritizes collecting detailed audio data. For example, when the user is relaxed, the collection unit prioritizes collecting detailed audio data and uses it for analysis. The collection unit can also prioritize collecting simple video data when the user is stressed. For example, when the user is stressed, the collection unit prioritizes collecting simple video data and uses it for analysis. Furthermore, when the user is concentrating, the collection unit can also prioritize collecting data related to the accuracy of the performance. For example, when the user is concentrating, the collection unit prioritizes collecting data related to the accuracy of the performance and uses it for analysis. This makes it possible to determine the priority of data to be collected based on the user's emotions. Emotion estimation is realized using an emotion estimation function, for example, an emotion engine or a generation AI. The generation AI can be a text generation AI (e.g., LLM) or a multimodal generation AI, but is not limited to such examples. Some or all of the above-mentioned processing in the collection unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the collection unit can input user emotional data into the AI, which can then determine the optimal data priorities.

[0074] When collecting performance data, the collection unit can prioritize collecting highly relevant data by taking into account the user's geographical location information. For example, if the user is performing in a specific area, the collection unit prioritizes collecting music data related to that area. For example, when the user is performing in a specific area, the collection unit prioritizes collecting music data related to that area and uses it for analysis. Furthermore, if the user is participating in a specific event, the collection unit can prioritize collecting data related to that event. For example, when the user is participating in a specific event, the collection unit prioritizes collecting data related to that event and uses it for analysis. Furthermore, if the user is practicing in a specific location, the collection unit can prioritize collecting data related to that location. For example, when the user is practicing in a specific location, the collection unit prioritizes collecting data related to that location and uses it for analysis. In this way, highly relevant data can be prioritized by taking into account the user's geographical location information. Some or all of the above-described processing by the collection unit may be performed using, for example, AI, or may be performed without using AI. For example, the collection unit can input the user's geographical location information to AI, which can then determine the optimal data prioritization.

[0075] When collecting performance data, the collection unit can analyze the user's social media activities and collect related data. The collection unit, for example, collects related data based on performance videos shared by the user on social media. For example, the collection unit analyzes performance videos shared by the user on social media and collects related data. The collection unit can also collect performance data of artists the user follows on social media. For example, the collection unit collects performance data of artists the user follows on social media and uses the data for analysis. The collection unit can also collect data of music communities the user participates in on social media. For example, the collection unit collects data of music communities the user participates in on social media and uses the data for analysis. This makes it possible to collect related data based on the user's social media activities. Some or all of the above-mentioned processing by the collection unit may be performed using, for example, AI, or may be performed without using AI. For example, the collection unit can input the user's social media activities into AI, which can determine the optimal data collection method.

[0076] The analysis unit can estimate the user's emotions and adjust the skill level evaluation criteria based on the estimated user emotions. For example, when the user is relaxed, the analysis unit evaluates the skill level using detailed evaluation criteria. For example, when the user is relaxed, the analysis unit evaluates the skill level using detailed evaluation criteria, resulting in an accurate evaluation. The analysis unit can also evaluate the skill level using simplified evaluation criteria when the user is stressed. For example, when the user is stressed, the analysis unit evaluates the skill level using simplified evaluation criteria, thereby reducing the user's burden. Furthermore, the analysis unit can also evaluate the skill level using strict evaluation criteria when the user is concentrating. For example, when the user is concentrating, the analysis unit evaluates the skill level using strict evaluation criteria and provides detailed feedback. This makes it possible to adjust the skill level evaluation criteria based on the user's emotions. Emotion estimation is realized using an emotion estimation function, for example, using an emotion engine or a generative AI. The generative AI can be, but is not limited to, a text generation AI (e.g., LLM) or a multimodal generation AI. Some or all of the above-described processing in the analysis unit may be performed using, for example, AI, or may be performed without using AI. For example, the analysis unit may input user emotion data into AI, which may then determine optimal evaluation criteria.

[0077] The analysis unit can improve the accuracy of skill level evaluation by taking into account detailed parameters of the performance data during analysis. The analysis unit, for example, evaluates the skill level by analyzing in detail the pitch and rhythm accuracy of the audio data. For example, the analysis unit evaluates the skill level by evaluating the pitch accuracy and rhythm accuracy of the audio data. The analysis unit can also evaluate the skill level by analyzing finger movements and facial expressions of video data. For example, the analysis unit analyzes the accuracy of finger movements and changes in facial expressions of the video data to evaluate the skill level. The analysis unit can also evaluate the skill level by analyzing in detail the tempo and dynamics of the performance data. For example, the analysis unit analyzes the consistency of the tempo and changes in dynamics of the performance data to evaluate the skill level. This improves the accuracy of skill level evaluation by taking into account detailed parameters of the performance data. Some or all of the above-described processing in the analysis unit may be performed using, or without, AI. For example, the analysis unit inputs detailed parameters of the performance data into AI, which can improve the accuracy of skill level evaluation.

[0078] During analysis, the analysis unit can evaluate fluctuations in skill level by referring to the user's performance history. The analysis unit, for example, compares the performance data with past performance data to evaluate improvement in skill level. For example, the analysis unit compares past performance data with current performance data to evaluate improvement in skill level. The analysis unit can also evaluate decline in skill level by comparing the performance data with past performance data. For example, the analysis unit compares past performance data with current performance data to evaluate decline in skill level. Furthermore, the analysis unit can also evaluate stability of skill level by comparing the performance data with past performance data. For example, the analysis unit compares past performance data with current performance data to evaluate stability of skill level. This makes it possible to evaluate fluctuations in skill level based on the user's performance history. Some or all of the above-described processing in the analysis unit may be performed using, for example, AI, or may be performed without AI. For example, the analysis unit can input the user's performance history into AI, which then evaluates fluctuations in skill level.

[0079] The analysis unit can estimate the user's emotions and adjust the display method of the evaluation results based on the estimated user emotions. For example, when the user is relaxed, the analysis unit displays detailed evaluation results. For example, when the user is relaxed, the analysis unit displays detailed evaluation results and provides feedback to the user. The analysis unit can also display simplified evaluation results when the user is stressed. For example, when the user is stressed, the analysis unit displays simplified evaluation results to reduce the user's burden. Furthermore, the analysis unit can also display strict evaluation results when the user is concentrating. For example, when the user is concentrating, the analysis unit displays strict evaluation results and provides detailed feedback. This makes it possible to adjust the display method of the evaluation results based on the user's emotions. Emotion estimation is realized using an emotion estimation function, for example, using an emotion engine or a generation AI. The generation AI can be a text generation AI (e.g., LLM) or a multimodal generation AI, but is not limited to such examples. Some or all of the above-mentioned processing in the analysis unit can be performed, for example, using AI or without AI. For example, the analysis unit can input the user's emotional data into the AI, which can then determine the optimal display method.

[0080] During analysis, the analysis unit can evaluate the skill level by taking into account the geographical distribution of the performance data. For example, if a user is performing in a specific region, the analysis unit evaluates the skill level by taking into account the musical style of the region. For example, if a user is performing in a specific region, the analysis unit evaluates the skill level by taking into account the musical style of the region. Furthermore, if a user is performing in a specific country, the analysis unit can evaluate the skill level by taking into account the musical culture of the country. For example, if a user is performing in a specific country, the analysis unit evaluates the skill level by taking into account the musical culture of the country. Furthermore, if a user is performing in a specific city, the analysis unit can evaluate the skill level by taking into account the music scene of the city. For example, if a user is performing in a specific city, the analysis unit evaluates the skill level by taking into account the music scene of the city. In this way, the skill level can be evaluated by taking into account the geographical distribution of the performance data. Some or all of the above-described processing in the analysis unit may be performed using, for example, AI, or may be performed without using AI. For example, the analysis unit can input the geographical distribution of the performance data into AI, which can then evaluate the skill level.

[0081] The analysis unit can improve the accuracy of the skill level assessment by referring to related music theory and literature during analysis. The analysis unit, for example, evaluates the accuracy of the performance based on music theory. For example, the analysis unit evaluates the accuracy of pitch and rhythm based on music theory to assess the skill level. The analysis unit can also evaluate technical aspects of the performance by referring to music literature. For example, the analysis unit evaluates the technical aspects of the performance by referring to music literature to assess the skill level. Furthermore, the analysis unit can combine music theory and literature to assess the overall skill level. For example, the analysis unit combines music theory and literature to assess the overall skill level and provide detailed feedback. This can improve the accuracy of the skill level assessment by referring to related music theory and literature. Some or all of the above-described processing in the analysis unit may be performed using, or without, AI. For example, the analysis unit can input related music theory and literature into AI, which can improve the accuracy of the skill level assessment.

[0082] The generation unit can estimate the user's emotions and adjust the content of the study plan based on the estimated user's emotions. For example, when the user is relaxed, the generation unit provides a detailed study plan. For example, when the user is relaxed, the generation unit provides a detailed study plan and presents exercises suitable for the user. The generation unit can also provide a simplified study plan when the user is stressed. For example, when the user is stressed, the generation unit provides a simplified study plan to reduce the user's burden. Furthermore, the generation unit can also provide a rigorous study plan when the user is concentrating. For example, when the user is concentrating, the generation unit provides a rigorous study plan and presents detailed exercises. This allows the content of the study plan to be adjusted based on the user's emotions. Emotion estimation is achieved using an emotion estimation function, for example, an emotion engine or a generation AI. The generation AI can be a text generation AI (e.g., LLM) or a multimodal generation AI, but is not limited to such examples. Some or all of the above-mentioned processing in the generation unit can be performed using, for example, AI, or without AI. For example, the generation unit can input the user's emotional data into the AI, which can then determine the content of the optimal learning plan.

[0083] When generating a study plan, the generation unit can generate an optimal plan by referring to the user's past study history. The generation unit, for example, generates a plan that allows the user to study most effectively based on the user's past study history. For example, the generation unit analyzes the user's past study history and generates a plan that allows the user to study most effectively. The generation unit can also generate a plan that is most interesting to the user based on the user's past study history. For example, the generation unit analyzes the user's past study history and generates a plan that is most interesting to the user. The generation unit can also generate a plan that allows the user to make the most progress based on the user's past study history. For example, the generation unit analyzes the user's past study history and generates a plan that allows the user to make the most progress. This makes it possible to generate an optimal plan based on the user's past study history. Some or all of the above-described processing in the generation unit may be performed using, for example, AI, or may be performed without using AI. For example, the generation unit can input the user's past study history into AI, which then generates an optimal plan.

[0084] When generating a learning plan, the generation unit can adjust the difficulty of the plan based on the user's current goal. For example, if the user is a beginner, the generation unit provides a basic plan. For example, if the user is a beginner, the generation unit provides a plan for learning how to hold basic chords and rhythm. The generation unit can also provide a plan of medium difficulty if the user is an intermediate player. For example, if the user is an intermediate player, the generation unit provides a plan for learning medium-difficulty techniques or pieces of music. The generation unit can also provide an advanced plan if the user is an advanced player. For example, if the user is an advanced player, the generation unit provides a plan aimed at learning advanced techniques or playing complex pieces of music. This allows the difficulty of the plan to be adjusted based on the user's current goal. Some or all of the above-described processing in the generation unit may be performed using, or without, AI. For example, the generation unit can input the user's current goal into AI, which can then determine the optimal difficulty level of the plan.

[0085] The generation unit can estimate the user's emotions and prioritize the study plans based on the estimated user emotions. For example, when the user is relaxed, the generation unit prioritizes providing a detailed study plan. For example, when the user is relaxed, the generation unit prioritizes providing a detailed study plan and presents practice tasks suitable for the user. The generation unit can also prioritize providing a simplified study plan when the user is stressed. For example, when the user is stressed, the generation unit prioritizes providing a simplified study plan to reduce the user's burden. Furthermore, the generation unit can also prioritize providing a rigorous study plan when the user is concentrating. For example, when the user is concentrating, the generation unit prioritizes providing a rigorous study plan and presents detailed practice tasks. This makes it possible to prioritize the study plans based on the user's emotions. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or generation AI. The generation AI can be, but is not limited to, a text generation AI (e.g., LLM) or a multimodal generation AI. Some or all of the above-described processing in the generation unit may be performed using, for example, AI, or may be performed without using AI. For example, the generation unit may input user emotion data into AI, which may then determine the priority of the optimal learning plan.

[0086] When generating a study plan, the generation unit can generate an optimal plan by taking into account the user's geographical location information. For example, if the user is studying in a specific region, the generation unit generates a plan by taking into account the musical style of the region. For example, if the user is studying in a specific region, the generation unit generates a plan by taking into account the musical style of the region. Furthermore, if the user is studying in a specific country, the generation unit can generate a plan by taking into account the musical culture of the country. For example, if the user is studying in a specific country, the generation unit can generate a plan by taking into account the musical culture of the country. Furthermore, if the user is studying in a specific city, the generation unit can generate a plan by taking into account the music scene of the city. For example, if the user is studying in a specific city, the generation unit generates a plan by taking into account the music scene of the city. In this way, an optimal plan can be generated by taking into account the user's geographical location information. Some or all of the above-described processing in the generation unit may be performed using, for example, AI, or may be performed without using AI. For example, the generation unit can input the user's geographical location information into AI, which can then generate an optimal plan.

[0087] When generating a study plan, the generation unit can analyze the user's social media activity and propose a related plan. The generation unit can, for example, propose a related plan based on performance videos shared by the user on social media. For example, the generation unit can analyze performance videos shared by the user on social media and propose a related plan. The generation unit can also propose a plan taking into account the performance styles of artists the user follows on social media. For example, the generation unit can propose a plan taking into account the performance styles of artists the user follows on social media. The generation unit can also propose a related plan based on the activities of music communities the user participates in on social media. For example, the generation unit can analyze the activities of music communities the user participates in on social media and propose a related plan. This makes it possible to propose a related plan based on the user's social media activity. Some or all of the above-described processing by the generation unit can be performed using, or without, AI. For example, the generation unit can input the user's social media activity into AI, which can then propose an optimal plan.

[0088] The providing unit can estimate the user's emotions and adjust the method of providing the study plan based on the estimated user's emotions. For example, when the user is relaxed, the providing unit selects a method of providing the study plan that includes detailed explanations. For example, when the user is relaxed, the providing unit selects a method of providing the study plan that includes detailed explanations and provides a study plan that is suitable for the user. The providing unit can also select a simplified method of providing the study plan when the user is stressed. For example, when the user is stressed, the providing unit selects a simplified method of providing the study plan to reduce the user's burden. The providing unit can also select a strict method of providing the study plan when the user is concentrating. For example, when the user is concentrating, the providing unit selects a strict method of providing the study plan to provide a detailed study plan. This makes it possible to adjust the method of providing the study plan based on the user's emotions. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or a generation AI. The generation AI can be a text generation AI (e.g., LLM) or a multimodal generation AI, but is not limited to such examples. Some or all of the above-mentioned processing in the providing unit can be performed, for example, using AI or without AI. For example, the providing unit can input the user's emotional data into the AI, which can then determine the optimal method of providing the data.

[0089] When providing a study plan, the provision unit can select the optimal provision method by referring to the user's past study history. The provision unit, for example, selects the provision method that allows the user to study most effectively based on the past study history. For example, the provision unit analyzes the past study history and selects the provision method that allows the user to study most effectively. The provision unit can also select the provision method that the user is most interested in based on the past study history. For example, the provision unit analyzes the past study history and selects the provision method that the user is most interested in. The provision unit can also select the provision method that the user has made the most progress based on the past study history. For example, the provision unit analyzes the past study history and selects the provision method that the user has made the most progress. This makes it possible to select the optimal provision method based on the past study history. Some or all of the above-mentioned processing in the provision unit may be performed using, for example, AI, or may be performed without using AI. For example, the provision unit can input the past study history into AI, which then selects the optimal provision method.

[0090] When providing a study plan, the providing unit can adjust the timing of providing the study plan based on the user's current practice status. For example, when the user is practicing, the providing unit provides the study plan in real time. For example, when the user is practicing, the providing unit provides the study plan in real time and presents practice tasks suitable for the user. Furthermore, when the user is taking a break, the providing unit can also provide a study plan for the next practice. For example, when the user is taking a break, the providing unit provides a study plan for the next practice and presents practice tasks suitable for the user. Furthermore, when the user is practicing toward a specific goal, the providing unit can also provide a study plan at a timing that matches the goal. For example, when the user is practicing toward a specific goal, the providing unit provides a study plan at a timing that matches the goal and presents practice tasks suitable for the user. This makes it possible to adjust the timing of providing the study plan based on the user's current practice status. Some or all of the above-mentioned processing by the providing unit may be performed using, for example, AI, or may be performed without using AI. For example, the providing unit can input the user's practice status into AI, which can determine the optimal timing of providing the study plan.

[0091] The providing unit can estimate the user's emotions and adjust the display method of the study plan based on the estimated user's emotions. For example, when the user is relaxed, the providing unit provides a display method including detailed explanations. For example, when the user is relaxed, the providing unit provides a display method including detailed explanations and displays a study plan suitable for the user. The providing unit can also provide a simplified display method when the user is stressed. For example, when the user is stressed, the providing unit provides a simplified display method to reduce the user's burden. Furthermore, the providing unit can also provide a strict display method when the user is concentrating. For example, when the user is concentrating, the providing unit provides a strict display method and displays a detailed study plan. This makes it possible to adjust the display method of the study plan based on the user's emotions. Emotion estimation is realized using an emotion estimation function, for example, using an emotion engine or generation AI. The generation AI can be a text generation AI (e.g., LLM) or a multimodal generation AI, but is not limited to such examples. Some or all of the above-mentioned processing in the providing unit can be performed, for example, using AI or without AI. For example, the providing unit can input the user's emotional data into the AI, which can then determine the optimal display method.

[0092] When providing a study plan, the providing unit can select the optimal delivery method by taking into account the user's device information. For example, if the user is using a smartphone, the providing unit selects a delivery method that matches the screen size. For example, if the user is using a smartphone, the providing unit selects a delivery method that matches the screen size and provides a study plan suitable for the user. Furthermore, if the user is using a tablet, the providing unit can select a delivery method optimized for a large screen. For example, if the user is using a tablet, the providing unit selects a delivery method optimized for a large screen and provides a study plan suitable for the user. Furthermore, if the user is using a smartwatch, the providing unit can select a delivery method that is concise and highly visible. For example, if the user is using a smartwatch, the providing unit selects a delivery method that is concise and highly visible and provides a study plan suitable for the user. This makes it possible to select the optimal delivery method based on the user's device information. Some or all of the above-described processing by the providing unit may be performed using, for example, AI, or may be performed without using AI. For example, the providing unit can input the user's device information into AI, which then selects the optimal delivery method.

[0093] When providing a study plan, the providing unit can analyze the user's social media activity and provide a relevant plan. The providing unit can provide a relevant plan, for example, based on performance videos shared by the user on social media. For example, the providing unit can analyze performance videos shared by the user on social media and provide a relevant plan. The providing unit can also provide a plan taking into account the performance styles of artists the user follows on social media. For example, the providing unit can provide a plan taking into account the performance styles of artists the user follows on social media. Furthermore, the providing unit can also provide a relevant plan based on the activities of music communities the user participates in on social media. For example, the providing unit can analyze the activities of music communities the user participates in on social media and provide a relevant plan. This makes it possible to provide a relevant plan based on the user's social media activity. Some or all of the above-described processing by the providing unit can be performed using, or without, AI. For example, the providing unit can input the user's social media activity into AI, which can then provide an optimal plan.

[0094] The matching unit can estimate the user's emotions and adjust matching criteria based on the estimated user emotions. For example, when the user is relaxed, the matching unit prioritizes matching with peers who are similarly relaxed. For example, when the user is relaxed, the matching unit prioritizes matching with peers who are similarly relaxed and suggests an online session. Furthermore, when the user is feeling stressed, the matching unit can also prioritize matching with peers who can help relieve stress. For example, when the user is feeling stressed, the matching unit prioritizes matching with peers who can help relieve stress and suggests an online session. Furthermore, when the user is concentrating, the matching unit can also prioritize matching with peers who are similarly concentrating. For example, when the user is concentrating, the matching unit prioritizes matching with peers who are similarly concentrating and suggests an online session. This makes it possible to adjust matching criteria based on the user's emotions. Emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI can be, but is not limited to, a text generation AI (e.g., LLM) or a multimodal generation AI. Some or all of the above-described processing in the matching unit may be performed using, for example, AI, or may be performed without using AI. For example, the matching unit may input user emotion data into AI, which may then determine optimal matching criteria.

[0095] During matching, the matching unit can find the most suitable peer by referring to the user's performance history. For example, the matching unit finds peers who play the same songs based on songs the user has played in the past. For example, the matching unit analyzes songs the user has played in the past and finds peers who play the same songs. The matching unit can also find peers who attended the same events based on events the user has participated in in the past. For example, the matching unit analyzes events the user has participated in in the past and finds peers who attended the same events. Furthermore, the matching unit can find peers who are practicing the same skills based on skills the user has practiced in the past. For example, the matching unit analyzes skills the user has practiced in the past and finds peers who are practicing the same skills. In this way, the most suitable peer can be found based on the user's performance history. Some or all of the above-described processing in the matching unit may be performed using, for example, AI, or may be performed without using AI. For example, the matching unit can input the user's performance history into AI, which then finds the most suitable peer.

[0096] The matching unit can improve the accuracy of matching based on the user's current goal during matching. For example, if a user has a goal of playing a specific song, the matching unit finds fellow players who play that song. For example, if a user has a goal of playing a specific song, the matching unit finds fellow players who play that song. Furthermore, if a user has a goal of mastering a specific skill, the matching unit can also find fellow players who are practicing that skill. For example, if a user has a goal of mastering a specific skill, the matching unit can also find fellow players who are practicing that skill. Furthermore, if a user has a goal of participating in a specific event, the matching unit can also find fellow players who will participate in that event. For example, if a user has a goal of participating in a specific event, the matching unit finds fellow players who will participate in that event. This can improve the accuracy of matching based on the user's current goal. Some or all of the above-described processing in the matching unit may be performed using, for example, AI, or may be performed without using AI. For example, the matching unit can input the user's current goal into AI, which can then find the optimal fellow player.

[0097] The matching unit can estimate the user's emotions and adjust the display method of the matching results based on the estimated user's emotions. For example, when the user is relaxed, the matching unit displays detailed matching results. For example, when the user is relaxed, the matching unit displays detailed matching results and provides feedback to the user. The matching unit can also display simplified matching results when the user is stressed. For example, when the user is stressed, the matching unit displays simplified matching results to reduce the user's burden. Furthermore, the matching unit can also display strict matching results when the user is concentrating. For example, when the user is concentrating, the matching unit displays strict matching results and provides detailed feedback. This makes it possible to adjust the display method of the matching results based on the user's emotions. Emotion estimation is realized using an emotion estimation function, for example, using an emotion engine or a generation AI. The generation AI can be a text generation AI (e.g., LLM) or a multimodal generation AI, but is not limited to such examples. Some or all of the above-mentioned processing in the matching unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the matching unit can input the user's emotional data into the AI, which can then determine the optimal display method.

[0098] During matching, the matching unit can find the most suitable companion by taking into account the user's geographical location information. For example, if the user is performing in a specific region, the matching unit finds companions who live in that region. For example, if the user is performing in a specific region, the matching unit finds companions who live in that region. Furthermore, if the user is performing in a specific country, the matching unit can find companions who live in that country. For example, if the user is performing in a specific country, the matching unit finds companions who live in that country. Furthermore, if the user is performing in a specific city, the matching unit can find companions who live in that city. For example, if the user is performing in a specific city, the matching unit finds companions who live in that city. In this way, the most suitable companion can be found based on the user's geographical location information. Some or all of the above-described processing in the matching unit may be performed using, for example, AI, or may be performed without using AI. For example, the matching unit can input the user's geographical location information into AI, which can then find the most suitable companion.

[0099] The matching unit can analyze the user's social media activity during matching to suggest related peers. The matching unit can suggest related peers, for example, based on performance videos shared by the user on social media. For example, the matching unit can analyze performance videos shared by the user on social media and suggest related peers. The matching unit can also suggest related peers based on artists the user follows on social media. For example, the matching unit can suggest related peers based on artists the user follows on social media. The matching unit can also suggest related peers based on music communities the user participates in on social media. For example, the matching unit can analyze music communities the user participates in on social media and suggest related peers. This makes it possible to suggest related peers based on the user's social media activity. Some or all of the above-described processing in the matching unit can be performed using, or without, AI. For example, the matching unit can input the user's social media activity into AI, which can then suggest optimal peers. === Hard Collateral 1-1 === Each of the multiple elements, including the collection unit, analysis unit, generation unit, provision unit, and matching unit, is realized, for example, by at least one of the smart device 14 and the data processing device 12. For example, the collection unit collects the user's performance data using the microphone 38B or camera 42 of the smart device 14, and the collected data is analyzed by the specific processing unit 290 of the data processing device 12. The analysis unit, for example, is realized by the specific processing unit 290 of the data processing device 12 and analyzes the collected data to evaluate the user's skill level. The generation unit, for example, is realized by the specific processing unit 290 of the data processing device 12 and generates a customized study plan based on the analysis results. The provision unit, for example, is realized by the control unit 46A of the smart device 14 and provides the generated study plan to the user. The matching unit, for example, is realized by the specific processing unit 290 of the data processing device 12 and finds friends who share the same passion for music as the user and suggests online real-time sessions or band performances. === Hard Collateral 1-2 === Each of the multiple elements, including the collection unit, analysis unit, generation unit, provision unit, and matching unit, is realized, for example, by at least one of the smart glasses 214 and the data processing device 12. For example, the collection unit collects the user's performance data using the microphone 238 and camera 42 of the smart glasses 214, and the data is analyzed by the specific processing unit 290 of the data processing device 12. The analysis unit, for example, is realized by the specific processing unit 290 of the data processing device 12 and analyzes the collected data to evaluate the user's skill level. The generation unit, for example, is realized by the specific processing unit 290 of the data processing device 12 and generates a customized study plan based on the analysis results. The provision unit, for example, is realized by the control unit 46A of the smart glasses 214 and provides the generated study plan to the user. The matching unit, for example, is realized by the specific processing unit 290 of the data processing device 12 and finds friends who share the same passion for music as the user and suggests online real-time sessions or band performances. === Hard Collateral 1-3 === Each of the multiple elements, including the collection unit, analysis unit, generation unit, provision unit, and matching unit, is realized, for example, by at least one of the headset-type terminal 314 and the data processing device 12. For example, the collection unit collects the user's performance data using the microphone 238 and camera 42 of the headset-type terminal 314, and the data is analyzed by the specific processing unit 290 of the data processing device 12. The analysis unit, realized, for example, by the specific processing unit 290 of the data processing device 12, analyzes the collected data and evaluates the user's skill level. The generation unit, realized, for example, by the specific processing unit 290 of the data processing device 12, generates a customized study plan based on the analysis results. The provision unit, realized, for example, by the control unit 46A of the headset-type terminal 314, provides the generated study plan to the user. The matching unit, realized, for example, by the specific processing unit 290 of the data processing device 12, finds friends who share the same passion for music as the user and suggests online real-time sessions or band performances. === Hard Collateral 1-4 === Each of the multiple elements, including the collection unit, analysis unit, generation unit, provision unit, and matching unit, is realized, for example, by at least one of the robot 414 and the data processing device 12. For example, the collection unit collects the user's performance data using the microphone 238 and camera 42 of the robot 414, and the data is analyzed by the specific processing unit 290 of the data processing device 12. The analysis unit, realized, for example, by the specific processing unit 290 of the data processing device 12, analyzes the collected data and evaluates the user's skill level. The generation unit, realized, for example, by the specific processing unit 290 of the data processing device 12, generates a customized study plan based on the analysis results. The provision unit, realized, for example, by the control unit 46A of the robot 414, provides the generated study plan to the user. The matching unit, realized, for example, by the specific processing unit 290 of the data processing device 12, finds friends who share the same passion for music as the user and suggests online real-time sessions or band performances.

[0100] The system according to the embodiment is not limited to the above-described example, and various modifications are possible, for example, as follows.

[0101] The analysis unit can identify the user's playing style based on the user's performance data and suggest a practice method optimal for that style. For example, if the user prefers jazz style playing, the analysis unit can suggest a practice method specialized for jazz. If the user prefers classical music, the analysis unit can also suggest a practice method specialized for classical music. Furthermore, if the user prefers rock style playing, the analysis unit can also suggest a practice method specialized for rock. This makes it possible to provide an optimal practice method according to the user's playing style. Some or all of the above-mentioned processing in the analysis unit may be performed using, for example, AI, or may be performed without using AI. For example, the analysis unit can input performance data into AI, which then suggests the optimal practice method.

[0102] When collecting the user's performance data, the collection unit can detect the user's physical movements and posture and provide feedback to improve performance efficiency. For example, the collection unit can detect the user's hand position and finger movements and suggest optimal positions. The collection unit can also detect the user's posture and provide advice on maintaining correct posture. Furthermore, the collection unit can detect the user's breathing pattern and suggest breathing techniques to perform in a relaxed state. This can provide feedback to improve the user's performance efficiency. Some or all of the above-described processing in the collection unit can be performed using, for example, AI, or can be performed without AI. For example, the collection unit can input data on physical movements and posture into AI, which can then provide optimal feedback.

[0103] When analyzing the user's performance data, the analysis unit can estimate the user's emotions and adjust the content of the feedback based on the estimated emotions. For example, the analysis unit can provide detailed feedback when the user is relaxed. For example, when the user is relaxed, the analysis unit can provide detailed feedback on the user's performance and point out areas for technical improvement. The analysis unit can also provide simplified feedback when the user is stressed. For example, when the user is stressed, the analysis unit can provide feedback focusing on basic areas for improvement, thereby reducing the user's burden. Furthermore, the analysis unit can provide strict feedback when the user is concentrating. For example, when the user is concentrating, the analysis unit can provide detailed feedback on the accuracy of the performance and sense of rhythm. This allows the content of the feedback to be adjusted based on the user's emotions. Emotion estimation is achieved using an emotion estimation function, for example, an emotion engine or a generation AI. The generation AI can be, but is not limited to, a text generation AI (e.g., LLM) or a multimodal generation AI. Some or all of the above-mentioned processing in the analysis unit can be performed using, for example, an AI, or without an AI. For example, the analysis unit can input the user's emotional data into the AI, which can then provide optimal feedback.

[0104] When generating a study plan for a user, the generation unit can analyze the user's past study history and propose an optimal study plan. For example, the generation unit generates a plan that allows the user to study most effectively based on the user's past study history. For example, the generation unit analyzes the user's past study history and generates a plan that allows the user to study most effectively. The generation unit can also generate a plan that is most interesting to the user based on the user's past study history. For example, the generation unit analyzes the user's past study history and generates a plan that is most interesting to the user. Furthermore, the generation unit can also generate a plan that allows the user to make the most progress based on the user's past study history. For example, the generation unit analyzes the user's past study history and generates a plan that allows the user to make the most progress. This makes it possible to generate an optimal plan based on the user's past study history. Some or all of the above-described processing in the generation unit may be performed using, for example, AI, or may be performed without using AI. For example, the generation unit can input the user's past study history into AI, which then generates an optimal plan.

[0105] The providing unit can estimate the user's emotions and adjust the method of providing the study plan based on the estimated emotions. For example, when the user is relaxed, the providing unit selects a method of providing the study plan that includes detailed explanations. For example, when the user is relaxed, the providing unit selects a method of providing the study plan that includes detailed explanations, and provides a study plan that is suitable for the user. The providing unit can also select a simplified method of providing the study plan when the user is stressed. For example, when the user is stressed, the providing unit selects a simplified method of providing the study plan to reduce the user's burden. The providing unit can also select a strict method of providing the study plan when the user is concentrating. For example, when the user is concentrating, the providing unit selects a strict method of providing the study plan to provide a detailed study plan. This makes it possible to adjust the method of providing the study plan based on the user's emotions. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or a generation AI. The generation AI can be a text generation AI (e.g., LLM) or a multimodal generation AI, but is not limited to such examples. Some or all of the above-mentioned processing in the providing unit can be performed, for example, using AI, or can be performed without using AI. For example, the providing unit can input the user's emotional data into the AI, which can then determine the optimal method of providing the data.

[0106] The matching unit can find friends who are compatible with the user based on the user's performance data and suggest online sessions. For example, the matching unit finds friends who are compatible with the user based on the user's performance style and skill level. For example, the matching unit analyzes the user's performance style and skill level and suggests friends who are compatible with the user. The matching unit can also find friends who like the same genre based on the user's musical genre preferences. For example, the matching unit analyzes the user's musical genre preferences and suggests friends who like the same genre. Furthermore, the matching unit can find friends who live nearby based on the user's geographical proximity. For example, the matching unit analyzes the user's geographical proximity and suggests friends who live nearby. This makes it possible to find friends who are compatible with the user and suggest online sessions. Some or all of the above-described processing in the matching unit may be performed using, for example, AI, or may be performed without using AI. For example, the matching unit can input performance data into AI, which then finds the optimal friends.

[0107] The collection unit can estimate the user's emotions and adjust the timing of collecting performance data based on the estimated emotions. For example, when the user is relaxed, the collection unit collects performance data frequently to obtain detailed data. For example, when the user is relaxed, the collection unit collects data for each performance session and performs detailed analysis. Furthermore, when the user is feeling stressed, the collection unit can reduce the collection timing to reduce the burden on the user. For example, when the user is feeling stressed, the collection unit reduces the frequency of collecting performance data to reduce the burden on the user. Furthermore, when the user is concentrating, the collection unit can collect data at the peak of the performance to capture the most important moment. For example, when the user is concentrating, the collection unit collects data at the peak of the performance to capture the most important moment. This allows the timing of collecting performance data to be adjusted according to the user's emotions. Emotion estimation is realized using an emotion estimation function, for example, using an emotion engine or a generation AI. The generation AI can be, but is not limited to, a text generation AI (e.g., LLM) or a multimodal generation AI. Some or all of the above-described processing in the collection unit may be performed using, for example, AI, or may be performed without using AI. For example, the collection unit may input the user's emotion data into AI, which may then determine the optimal collection timing.

[0108] When analyzing the user's performance data, the analysis unit can evaluate fluctuations in skill level by referring to the user's performance history. For example, the analysis unit compares the user's performance data with past performance data to evaluate improvement in skill level. For example, the analysis unit compares past performance data with current performance data to evaluate improvement in skill level. The analysis unit can also evaluate decline in skill level by comparing the user's performance data with past performance data. For example, the analysis unit compares past performance data with current performance data to evaluate decline in skill level. Furthermore, the analysis unit can also evaluate stability of skill level by comparing the user's performance data with past performance data. For example, the analysis unit compares past performance data with current performance data to evaluate stability of skill level. This makes it possible to evaluate fluctuations in skill level based on the user's performance history. Some or all of the above-described processing in the analysis unit may be performed using, for example, AI, or may be performed without AI. For example, the analysis unit can input the user's performance history into AI, which then evaluates fluctuations in skill level.

[0109] The providing unit can estimate the user's emotions and adjust the display method of the study plan based on the estimated emotions. For example, when the user is relaxed, the providing unit provides a display method including detailed explanations. For example, when the user is relaxed, the providing unit provides a display method including detailed explanations and displays a study plan suitable for the user. The providing unit can also provide a simplified display method when the user is stressed. For example, when the user is stressed, the providing unit provides a simplified display method to reduce the user's burden. Furthermore, the providing unit can also provide a strict display method when the user is concentrating. For example, when the user is concentrating, the providing unit provides a strict display method and displays a detailed study plan. This makes it possible to adjust the display method of the study plan based on the user's emotions. Emotion estimation is realized using an emotion estimation function, for example, an emotion engine or a generation AI. The generation AI can be a text generation AI (e.g., LLM) or a multimodal generation AI, but is not limited to such examples. Some or all of the above-mentioned processing in the providing unit can be performed using, for example, AI, or without AI. For example, the providing unit can input the user's emotional data into the AI, which can then determine the optimal display method.

[0110] The matching unit can estimate the user's emotions and adjust matching criteria based on the estimated emotions. For example, when the user is relaxed, the matching unit prioritizes matching with peers who are also relaxed. For example, when the user is relaxed, the matching unit prioritizes matching with peers who are also relaxed and suggests an online session. Furthermore, when the user is feeling stressed, the matching unit can also prioritize matching with peers who can help relieve stress. For example, when the user is feeling stressed, the matching unit prioritizes matching with peers who can help relieve stress and suggests an online session. Furthermore, when the user is concentrating, the matching unit can also prioritize matching with peers who are also concentrating. For example, when the user is concentrating, the matching unit prioritizes matching with peers who are also concentrating and suggests an online session. This makes it possible to adjust matching criteria based on the user's emotions. Emotion estimation is realized using an emotion estimation function, for example, using an emotion engine or a generation AI. The generation AI can be, but is not limited to, a text generation AI (e.g., LLM) or a multimodal generation AI. Some or all of the above-described processing in the matching unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the matching unit can input the user's emotional data into the AI, which can then determine the optimal matching criteria.

[0111] The processing flow of the second embodiment will be briefly explained below.

[0112] Step 1: The collection unit collects the user's performance data. The user's performance data includes audio data, video data, MIDI data, etc. For example, the collection unit may use a microphone to record audio data and save it as a digital audio file. Alternatively, the collection unit may use a camera to record video data and save it as a video file. Furthermore, the collection unit may collect MIDI data and save it in a digital format. Step 2: The analysis unit analyzes the data collected by the collection unit and evaluates the user's skill level. The skill level evaluation includes performance accuracy, sense of rhythm, technical difficulty, etc. For example, the analysis unit analyzes audio data to evaluate performance accuracy, video data to evaluate sense of rhythm, and MIDI data to evaluate technical difficulty. Step 3: The generator generates a customized learning plan based on the skill level assessed by the analyzer. The customized learning plan includes the user's goals, current skill level, learning progress, etc. For example, a plan may be generated that starts beginners with practicing basic chords, while an advanced player may aim to learn advanced techniques and play complex pieces. Step 4: The providing unit provides the study plan generated by the generating unit. Methods of providing include an online platform, email, app notification, etc. For example, the study plan can be provided through an online platform, email, or app notification. Step 5: The matching unit finds people who share the same passion for music as the user and suggests online real-time sessions or band performances. Matching criteria include skill level, music genre, and geographic proximity. For example, skill level can be used to find people with the same skill level, music genre can be used to find people who share the same music genre, and geographic proximity can be used to find people who live nearby.

[0113] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0114] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> Examples of the generative AI include a neural network (NN) and a neural network (NN). The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 receives a prompt containing an instruction, as well as inference data such as voice data representing speech, text data representing text, and image data representing an image (e.g., still image data or video data). The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in one or more data formats of voice data, text data, image data, etc. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction. In this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. The data processing device 12 and the like include multiple types of data generation models 58, and the data generation model 58 includes AIs other than the generative AI. The AI ​​other than the generative AI may be, for example, linear regression, logistic regression, decision tree, random forest, support vector machine (SVM), k-means clustering, convolutional neural network (CNN), recurrent neural network (RNN), generative adversarial network (GAN), or naive Bayes, and may perform various processes, but is not limited to these examples. The AI ​​may also be an AI agent. When the processing of each of the above-mentioned parts is performed by an AI, the processing may be performed in part or entirely by the AI, but is not limited to these examples. The processing performed by an AI including the generative AI may be replaced with rule-based processing, and rule-based processing may be replaced with processing performed by an AI including the generative AI.

[0115] Furthermore, the processing by the data processing system 10 described above is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart device 14, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart device 14. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the smart device 14 or an external device, and the smart device 14 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0116] The correspondence between each part and the device or control part is not limited to the example described above, and various modifications are possible.

[0117] [Second embodiment] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0118] 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0119] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0120] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0121] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to instructions from the processor 46.

[0122] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user's surroundings (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0123] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0124] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0125] The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0126] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate a user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, but is not limited to these examples. Furthermore, the estimation and prediction of emotion also includes, for example, emotion analysis.

[0127] In the smart glasses 214, the specific processing is performed by the processor 46. A specific processing program 60 is stored in the storage 50. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific processing program 60 executed on the RAM 48. The smart glasses 214 also have a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can perform processing similar to that of the specific processing unit 290 using these models.

[0128] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 communicates with the server device having the data generation model 58 to obtain a processing result (such as a prediction result) using the data generation model 58. Furthermore, the data processing device 12 may be a server device, or may be a terminal device (for example, a mobile phone, a robot, a home appliance, etc.) owned by a user.

[0129] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0130] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 receives a prompt including an instruction, as well as inference data such as audio data indicating speech, text data indicating text, and image data indicating an image (e.g., still image data or video data). The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in one or more data formats, such as audio data, text data, and image data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The identification processing unit 290 performs the above-mentioned identification processing using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction. In this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. The data processing device 12 and the like include multiple types of data generation models 58, and the data generation models 58 include AIs other than the generation AI. Examples of AIs other than the generation AI include, but are not limited to, linear regression, logistic regression, decision trees, random forests, support vector machines (SVMs), k-means clustering, convolutional neural networks (CNNs), recurrent neural networks (RNNs), generative adversarial networks (GANs), and naive Bayes. These AIs can perform various types of processing, but are not limited to these examples. The AI ​​may also be an AI agent. When the processing of each of the above-described parts is performed by an AI, the processing may be performed in part or entirely by the AI, but is not limited to these examples. Processing performed by an AI, including the generation AI, may be replaced with rule-based processing, and rule-based processing may be replaced with processing performed by an AI, including the generation AI.

[0131] The data processing system 210 according to the second embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 210 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart glasses 214, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart glasses 214. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the smart glasses 214 or an external device, etc., and the smart glasses 214 acquires or collects information required for processing from the data processing device 12 or an external device, etc.

[0132] The correspondence between each part and the device or control part is not limited to the example described above, and various modifications are possible.

[0133] [Third embodiment] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0134] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0135] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0136] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0137] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to instructions from the processor 46.

[0138] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user's surroundings (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0139] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0140] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0141] The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0142] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate a user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, but is not limited to these examples. Furthermore, the estimation and prediction of emotion also includes, for example, emotion analysis.

[0143] In the headset type terminal 314, the identification process is performed by the processor 46. A identification program 60 is stored in the storage 50. The processor 46 reads the identification program 60 from the storage 50 and executes the read identification program 60 on the RAM 48. The identification process is realized by the processor 46 operating as a control unit 46A in accordance with the identification program 60 executed on the RAM 48. Note that the headset type terminal 314 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the identification processing unit 290 using these models.

[0144] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 communicates with the server device having the data generation model 58 to obtain a processing result (such as a prediction result) using the data generation model 58. Furthermore, the data processing device 12 may be a server device, or may be a terminal device (for example, a mobile phone, a robot, a home appliance, etc.) owned by a user.

[0145] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0146] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 receives a prompt including an instruction, as well as inference data such as audio data indicating speech, text data indicating text, and image data indicating an image (e.g., still image data or video data). The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in one or more data formats, such as audio data, text data, and image data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The identification processing unit 290 performs the above-mentioned identification processing using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction. In this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. The data processing device 12 and the like include multiple types of data generation models 58, and the data generation models 58 include AIs other than the generation AI. Examples of AIs other than the generation AI include, but are not limited to, linear regression, logistic regression, decision trees, random forests, support vector machines (SVMs), k-means clustering, convolutional neural networks (CNNs), recurrent neural networks (RNNs), generative adversarial networks (GANs), and naive Bayes. These AIs can perform various types of processing, but are not limited to these examples. The AI ​​may also be an AI agent. When the processing of each of the above-described parts is performed by an AI, the processing may be performed in part or entirely by the AI, but is not limited to these examples. Processing performed by an AI, including the generation AI, may be replaced with rule-based processing, and rule-based processing may be replaced with processing performed by an AI, including the generation AI.

[0147] The data processing system 310 according to the third embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 310 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the headset type terminal 314, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the headset type terminal 314. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the headset type terminal 314 or an external device, etc., and the headset type terminal 314 acquires or collects information required for processing from the data processing device 12 or an external device, etc.

[0148] The correspondence between each part and the device or control part is not limited to the example described above, and various modifications are possible.

[0149] [Fourth embodiment] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[0150] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0151] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0152] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[0153] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to instructions from the processor 46.

[0154] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS image sensor or a CCD image sensor, and captures images of the user's surroundings (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0155] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0156] The control object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[0157] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0158] The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0159] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate a user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, but is not limited to these examples. Furthermore, the estimation and prediction of emotion also includes, for example, emotion analysis.

[0160] In the robot 414, the processor 46 performs the identification process. The storage 50 stores the identification program 60. The processor 46 reads the identification program 60 from the storage 50 and executes the read identification program 60 on the RAM 48. The identification process is realized by the processor 46 operating as the control unit 46A in accordance with the identification program 60 executed on the RAM 48. The robot 414 also has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can perform the same process as the identification processing unit 290 using these models.

[0161] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 communicates with the server device having the data generation model 58 to obtain a processing result (such as a prediction result) using the data generation model 58. Furthermore, the data processing device 12 may be a server device, or may be a terminal device (for example, a mobile phone, a robot, a home appliance, etc.) owned by a user.

[0162] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0163] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 receives a prompt including an instruction, as well as inference data such as audio data indicating speech, text data indicating text, and image data indicating an image (e.g., still image data or video data). The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in one or more data formats, such as audio data, text data, and image data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The identification processing unit 290 performs the above-mentioned identification processing using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction. In this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. The data processing device 12 and the like include multiple types of data generation models 58, and the data generation models 58 include AIs other than the generation AI. Examples of AIs other than the generation AI include, but are not limited to, linear regression, logistic regression, decision trees, random forests, support vector machines (SVMs), k-means clustering, convolutional neural networks (CNNs), recurrent neural networks (RNNs), generative adversarial networks (GANs), and naive Bayes. These AIs can perform various types of processing, but are not limited to these examples. The AI ​​may also be an AI agent. When the processing of each of the above-described parts is performed by an AI, the processing may be performed in part or entirely by the AI, but is not limited to these examples. Processing performed by an AI, including the generation AI, may be replaced with rule-based processing, and rule-based processing may be replaced with processing performed by an AI, including the generation AI.

[0164] The data processing system 410 according to the fourth embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 410 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the robot 414, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the robot 414. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the robot 414 or an external device, etc., and the robot 414 acquires or collects information required for processing from the data processing device 12 or an external device, etc.

[0165] The correspondence between each part and the device or control part is not limited to the example described above, and various modifications are possible.

[0166] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0167] FIG. 9 illustrates an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and behaviors arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion encompasses both emotions and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[0168] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[0169] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[0170] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is expressed, and when they approach the ideal, a state of pleasure is expressed. Emotions can also be created for robots, cars, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is expressed, and when they approach the ideal, a state of pleasure is expressed. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on speech emotion recognition and brain physiological signal analysis systems for emotions, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the area called "reaction," where sensation is dominant. The right half of the emotion map lists emotions belonging to the area called "situation," where situational awareness is dominant.

[0171] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[0172] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[0173] In the above embodiment, an example was given in which a specific process is performed by one computer 22, but the technology disclosed herein is not limited to this, and distributed processing of the specific process may be performed by multiple computers including computer 22.

[0174] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[0175] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0176] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[0177] The hardware resource for executing a specific process can be any of the following processors: A CPU is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. A dedicated electrical circuit, such as a field-programmable gate array (FPGA), a programmable logic device (PLD), or an application-specific integrated circuit (ASIC), is a processor with a circuit configuration specifically designed to execute a specific process. Each processor has built-in or connected memory, and uses the memory to execute the specific process.

[0178] The hardware resource that executes the specific process may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific process may be a single processor.

[0179] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[0180] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[0181] In the above example, the first to fourth embodiments have been described separately, but some or all of these embodiments may be combined. The smart device 14, smart glasses 214, headset terminal 314, and robot 414 are merely examples, and they may be combined, or other devices may be used. In the above example, the first and second embodiments have been described separately, but they may be combined.

[0182] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[0183] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[0184] [Explanation of symbols]

[0185] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot

Claims

1. a collection unit that collects performance data of users; an analysis unit that analyzes the data collected by the collection unit and evaluates the skill level of the user; a generation unit that generates a customized learning plan based on the skill level assessed by the analysis unit; a provision unit that provides the study plan generated by the generation unit; A matching unit that finds friends who share the same passion for music as the user and suggests online real-time sessions or band performances. A system characterized by:

2. The collecting unit Collect audio or video data as you play The system of claim 1 .

3. The analysis unit Based on the collected data, the user's skill level, sense of rhythm, playing accuracy, etc. are evaluated. The system of claim 1 .

4. The generation unit Based on the data from the analysis unit, a learning plan suited to the user is generated. The system of claim 1 .

5. The providing unit Providing a learning plan generated by the generation unit The system of claim 1 .

6. The matching unit Matching and suggesting online sessions with suitable buddies based on a user's skill level or preferences The system of claim 1 .

7. The collecting unit The user's emotions are estimated, and the timing of collecting performance data is adjusted based on the estimated user's emotions. The system of claim 1 .

8. The collecting unit Analyze the user's past performance data and select the optimal collection method The system of claim 1 .

Images