System

Abstract

To provide a system that analyzes a user's sports form in real time and provide feedback on improvement points.SOLUTION: A system comprises an imaging unit, a reading unit, an analysis unit, a comparison unit, and a feedback unit. The imaging unit captures a sports video of a user. The reading unit reads the video captured by the imaging unit. The analysis unit analyzes the video read by the reading unit to specify the body position or movement of the user. The comparison unit compares the data specified by the analysis unit with data on professional athletes. The feedback unit provides real-time feedback on the basis of the analysis result obtained by the comparison unit.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including a description and related instruction sentence regarding the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2022-180282 A Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional technology has the drawback of making it difficult to provide real-time feedback to improve sports form.

[0005] The system according to the embodiment aims to analyze a user's sports form in real time and provide feedback on areas for improvement. [Means for solving the problem]

[0006] The system according to the embodiment includes a shooting unit, a reading unit, an analysis unit, a comparison unit, and a feedback unit. The shooting unit shoots a sports video of a user. The reading unit reads the video shot by the shooting unit. The analysis unit analyzes the video read by the reading unit and identifies the position or movement of the user's body. The comparison unit compares the data identified by the analysis unit with data of a professional athlete. The feedback unit provides feedback in real time based on the analysis result obtained by the comparison unit. Effect of the Invention

[0007] The system according to the embodiment can analyze the user's sports form in real time and provide feedback on areas for improvement. [Brief description of the drawings]

[0008] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Diagram 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. FIG. [Diagram 3] FIG. 11 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Diagram 5] FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 13 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0010] First, the terms used in the following description will be explained.

[0011] In the following embodiments, a signed processor (hereinafter simply referred to as a "processor") may be one arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be one type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (Tensor Processing Unit).

[0012] In the following embodiments, a signed RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by the processor.

[0013] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0014] In the following embodiments, a communication I / F (Interface) with a code is an interface including a communication processor and an antenna. The communication I / F controls communication between multiple computers. An example of a communication standard applied to the communication I / F is a wireless communication standard including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0015] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. In addition, in this specification, the same idea as "A and / or B" is also applied when three or more things are expressed by connecting them with "and / or."

[0016] [First embodiment] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0017] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0018] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0019] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0020] The reception device 38 includes a touch panel 38A and a microphone 38B, and receives user input. The touch panel 38A detects contact with a pointer (e.g., a pen or a finger) to receive user input by the touch of the pointer. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 (see FIG. 2) acquires the data indicating the user input.

[0021] Output device 40 includes a display 40A and a speaker 40B, and presents data to a user by outputting the data in a form of expression that the user can perceive (e.g., voice and / or text). Display 40A displays visible information such as text and images according to instructions from processor 46. Speaker 40B outputs voice according to instructions from processor 46. Camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a Complementary Metal-Oxide-Semiconductor (CMOS) image sensor or a Charge Coupled Device (CCD) image sensor.

[0022] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54.

[0023] FIG. 2 shows an example of main functions of the data processing device 12 and the smart device 14.

[0024] As shown in Fig. 2, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32. The specific process program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific process program 56 from the storage 32, and executes the read specific process program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific process program 56 executed on the RAM 30.

[0025] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the emotion of the user using the emotion identification model 59 and perform identification processing using the user's emotion. In the emotion estimation function (emotion identification function) using the emotion identification model 59, various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, are performed, but are not limited to such examples. In addition, the estimation and prediction of the emotion also include, for example, analysis of the emotion.

[0026] In the smart device 14, the specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The specific processing program 60 is used together with the specific processing program 56 by the data processing system 10. The processor 46 reads the specific processing program 60 from the storage 50 and executes the specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific processing program 60 executed on the RAM 48. Note that the smart device 14 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the specific processing unit 290 using these models.

[0027] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device (e.g., a generation server) may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (prediction result, etc.) using the data generation model 58 by communicating with the server device having the data generation model 58. Also, the data processing device 12 may be a server device, or may be a terminal device (e.g., a mobile phone, a robot, a home appliance, etc.) owned by a user. Next, an example of processing by the data processing system 10 according to the first embodiment will be described.

[0028] (Example 1) The sports form improvement system according to the embodiment of the present invention is a system in which a user shoots a video of himself playing a sport, and an AI analyzes the video to suggest improvements to the form. In this system, a user shoots a video of himself playing a sport, an app loads the video, and an AI analyzes the target's body position, movement, technical elements, and the like in detail. Furthermore, the AI ​​compares and analyzes the data of people who are good at sports, such as professional athletes, with the target's data, and provides feedback and advice in real time based on the analysis results. For example, a user shoots a specific movement, such as a swing in a racket sport or a shot in a ball sport. This video is input into the app. Next, the app loads the video, and the AI ​​analyzes the target's body position, movement, technical elements, and the like in detail. The AI ​​analyzes each frame in the video and identifies the position and movement of each part of the user's body. For example, in the case of a swing in a racket sport, the position of the racket, the angle of the swing, the position of the foot, and the like are analyzed. Furthermore, the AI ​​compares and analyzes the data of people who are good at sports, such as professional athletes, with the target's data. The AI ​​refers to a database of professional athletes and compares it with the user's movement. For example, the swing of a professional athlete is compared with the user's swing to identify which parts are different. Finally, it provides feedback and advice in real time based on the analysis results. The app presents specific areas for improvement to the user and advises them on how to correct their form. For example, it provides advice such as "raise the angle of the racket a little more" or "widen the position of your feet a little more to make it more stable." This system allows users to objectively evaluate their own form and know specific areas for improvement, which is expected to lead to improvements in sports skills. This allows the sports form improvement system to efficiently analyze the user's sports form and provide feedback in real time.

[0029] The sports form improvement system according to the embodiment includes a shooting unit, a reading unit, an analysis unit, a comparison unit, and a feedback unit. The shooting unit shoots a video of a user playing a sport. The video of a user playing a sport includes, but is not limited to, a swing in a racket sport and a shot in a ball sport. The shooting unit can shoot a video using, for example, a camera of a smartphone or a tablet. The shooting unit can also shoot a high-resolution video using a dedicated camera. The reading unit reads the video shot by the shooting unit. The reading unit can, for example, import a video file into an app. The reading unit can also download and read a video from cloud storage. The analysis unit uses AI to analyze the video read by the reading unit and identify the position and movement of the user's body. The analysis unit, for example, analyzes each frame in the video and identifies the position and movement of each part of the user's body. For example, in the case of a swing in a racket sport, the position of the racket, the angle of the swing, the position of the foot, etc. are analyzed. The analysis unit can use, for example, deep learning technology to analyze each frame in the video and identify the position and movement of each part of the user's body. The comparison unit compares the data identified by the analysis unit with the data of professional players. The comparison unit, for example, refers to a database of professional players and compares it with the user's movements. For example, it compares the swing of the professional player with the swing of the user and identifies which parts are different. The comparison unit can use, for example, machine learning technology to compare the data of the professional player with the data of the user. The feedback unit provides feedback in real time based on the analysis results obtained by the comparison unit. For example, the feedback unit presents specific improvements to the user and advises how to correct the form. For example, advice such as "It would be better to raise the angle of the racket a little more" or "It will be more stable if you spread the position of your feet a little wider". The feedback unit can present specific improvements to the user using, for example, natural language processing technology.As a result, the sports form improving system according to the embodiment can efficiently analyze the sports form of the user and provide feedback in real time.

[0030] The filming unit shoots a video of the user playing a sport. Examples of videos of the user playing a sport include, but are not limited to, swings in racket sports and shots in ball sports. The filming unit can shoot the video using, for example, a smartphone or tablet camera. The filming unit can also shoot high-resolution video using a dedicated camera. Specifically, a smartphone or tablet camera can be easily carried around, allowing the user to easily record their own movements. This allows the user to film their form anywhere, such as at home or at a practice field. When a dedicated camera is used, high-resolution video can be shot, allowing for more detailed motion analysis. For example, a dedicated camera has a high frame rate and can smoothly record the user's high-speed movements. In addition, by using multiple cameras to shoot from different angles simultaneously, the user's movements can be captured three-dimensionally. This allows the filming unit to record the user's sports form from multiple angles and provide high-quality data for detailed analysis. Furthermore, the filming unit has a function of streaming the filmed video in real time, allowing the user to check their own movements on the spot. This allows the user to instantly correct their form and improve the efficiency of their practice.

[0031] The reading unit reads the video shot by the shooting unit. The reading unit can, for example, import a video file into an app. The reading unit can also download and read a video from cloud storage. Specifically, the user can directly import the video shot by the user into an app on a smartphone or tablet, and can easily prepare for analysis. When using cloud storage, the user can upload the video shot by the user to the cloud and access and read it from another device. This allows the user to share data between different devices and perform analysis flexibly. Furthermore, the reading unit can read videos in various formats regardless of the video format or resolution. For example, it supports common video formats such as MP4, AVI, and MOV, and the user can save the trouble of converting to a specific format. The reading unit also has a function of cutting out and analyzing a part of the video, and the user can focus on a specific action or scene to perform analysis. This allows the reading unit to efficiently import the video shot by the user and quickly prepare for analysis.

[0032] The analysis unit uses AI to analyze the video read by the reading unit and identify the position and movement of the user's body. The analysis unit, for example, analyzes each frame in the video and identifies the position and movement of each part of the user's body. For example, in the case of a swing in a racket sport, the position of the racket, the angle of the swing, the position of the foot, etc. The analysis unit can analyze each frame in the video and identify the position and movement of each part of the user's body using, for example, deep learning technology. Specifically, a deep learning model is used to extract the user's skeleton from each frame in the video and identify the position of each joint. This makes it possible to analyze the user's movements in detail and capture the characteristics of the form. Furthermore, the analysis unit can also analyze the speed and timing of the movements, and can evaluate the efficiency and performance of the user's movements. For example, the speed of the swing and the timing of the release of the ball can be analyzed to quantify the user's performance. In addition, the analysis unit has multiple deep learning models available to accommodate different sports and movements, and can select the optimal model depending on the sport the user plays. This makes it possible for the analysis unit to analyze the user's sports form with high accuracy and provide detailed data.

[0033] The comparison unit compares the data identified by the analysis unit with the data of the professional player. For example, the comparison unit refers to a database of professional players and compares it with the user's motion. For example, the comparison unit compares the swing of the professional player with the swing of the user to identify which parts are different. For example, the comparison unit can compare the data of the professional player with the data of the user using machine learning technology. Specifically, the motion database of the professional player stores motion data of top athletes in various sports, and by comparing it with the motion of the user, it is possible to identify areas for improvement in the form. The comparison unit can overlay the motion data of the user with the data of the professional player and visually display the differences in the positions and movements of each part. This allows the user to intuitively understand which parts of his or her form are different from those of the professional player. Furthermore, the comparison unit can statistically compare the motion data of the user with the data of the professional player using a machine learning algorithm and quantify specific areas for improvement. For example, it is possible to compare parameters such as the angle and speed of the swing, and the position of the foot, and clearly indicate which parts of the user's form need improvement. This allows the comparison unit to compare the sports form of the user with the data of the professional player and identify specific areas for improvement.

[0034] The feedback unit provides feedback in real time based on the analysis results obtained by the comparison unit. The feedback unit, for example, presents the user with specific improvements and advises how to correct the form. For example, advice such as "raise the angle of the racket a little more" or "widen the position of the feet a little more for stability" is provided. The feedback unit can present the user with specific improvements, for example, using natural language processing technology. Specifically, based on the analysis results, it identifies which parts of the user's movements need improvement and explains them in an easy-to-understand manner in natural language. Furthermore, the feedback unit can also provide feedback using audio and visuals. For example, the user can check the improvements while comparing his or her movements with those of a professional player on the screen of a smartphone or tablet. In addition, a voice assistant can be used to provide real-time audio feedback, allowing the user to correct the form on the spot. Furthermore, the feedback unit can record the user's progress and compare it with past data to evaluate the degree of improvement. This allows the user to realize his or her own growth and maintain motivation. This allows the feedback unit to provide the user with specific and easy-to-understand feedback and support the improvement of the sports form.

[0035] The analysis unit can identify the position and movement of the user's body. The analysis unit can, for example, use deep learning technology to analyze each frame in the video and identify the position and movement of each part of the user's body. For example, in the case of a swing in a racket sport, the position of the racket, the angle of the swing, the position of the foot, etc. The analysis unit can also identify the position and movement of the user's body using machine learning technology. For example, the position of the joints and the movement pattern are analyzed. Furthermore, the analysis unit can use multiple sensors to identify the position and movement of the user's body. For example, the position and movement of the user's body can be specified in detail using motion capture technology. This makes it possible to perform a detailed analysis by accurately specifying the position and movement of the user's body. Part or all of the above-mentioned processing in the analysis unit can be performed using, for example, AI, or can be performed without using AI. For example, the analysis unit can input video data to the generation AI and cause the generation AI to execute a process of specifying the position and movement of the user's body.

[0036] The comparison unit can refer to the database of professional players. The comparison unit, for example, refers to the database of professional players and compares it with the user's motion. For example, the comparison unit compares the swing of the professional player with the swing of the user to identify which parts are different. The comparison unit can also dynamically update the database of professional players and use the latest data. For example, the latest match data and training data of the professional players are dynamically updated and used for comparison. Furthermore, the comparison unit can refer to the database of professional players from different sports categories to make comparisons. For example, data of racket sports can be referenced and compared with the motion of ball sports. This makes it possible to identify areas for improvement in the user's form by comparing with the data of professional players. Part or all of the above-mentioned processing in the comparison unit may be performed, for example, using AI, or may be performed without using AI. For example, the comparison unit can input the database of professional players to the generation AI and cause the generation AI to compare the motion of the user with the motion of the professional player.

[0037] The feedback unit can present specific points for improvement. For example, the feedback unit presents specific points for improvement to the user and advises how to correct the form. For example, advice such as "raise the angle of the racket a little more" or "widen the position of the feet a little more for stability" is provided. The feedback unit can also provide optimal advice by referring to the user's past feedback history. For example, optimal advice is provided based on the user's past feedback history. Furthermore, the feedback unit can also specify in detail the points for improvement of the user's motion and present a specific method of improvement. For example, the feedback unit specifies the points for improvement in the angle of the user's swing and presents a specific method of correction. As a result, the specific points for improvement are presented, making it easier for the user to correct the form. A part or all of the above-mentioned processing in the feedback unit may be performed, for example, using AI, or may be performed without using AI. For example, the feedback unit can input the analysis results to the generation AI and cause the generation AI to execute the presentation of specific points for improvement.

[0038] The photographing unit can analyze the user's past photographing history at the time of photographing and automatically set the optimal photographing angle and distance. For example, the photographing unit automatically sets the photographing angle at which the user has obtained good results in the past. The photographing unit can also calculate the optimal distance from the user's past photographing history and automatically position the camera. Furthermore, the photographing unit can set the optimal lighting conditions based on the data of videos the user has photographed in the past. This enables efficient photographing by automatically setting the optimal photographing conditions based on the past photographing history. A part or all of the above-mentioned processing in the photographing unit may be performed, for example, using AI or may be performed without using AI. For example, the photographing unit can input past photographing history data to the generation AI and cause the generation AI to set the optimal photographing angle and distance.

[0039] The shooting unit can track the user's movements in real time during shooting and automatically select the optimal frame. For example, the shooting unit tracks the user's movements in real time and automatically selects a frame at the peak of the movement. The shooting unit can also automatically adjust the frame rate according to the speed of the user's movements. Furthermore, the shooting unit can automatically adjust the pan and zoom of the camera according to the direction of the user's movements. This makes it possible to track the user's movements in real time and select the optimal frame, thereby capturing important moments without missing them. Some or all of the above-mentioned processing in the shooting unit may be performed, for example, using AI, or may be performed without using AI. For example, the shooting unit can input the movement data acquired in real time to the generation AI and cause the generation AI to select the optimal frame.

[0040] The photographing unit can propose an optimal photographing environment based on the user's geographical location information at the time of photographing. For example, when the user photographs outdoors, the photographing unit proposes an optimal photographing location. In addition, when the user photographs indoors, the photographing unit can also propose optimal lighting conditions. Furthermore, when the user photographs at a specific sports facility, the photographing unit can also propose an optimal photographing point at the facility. This improves the quality of photographing by proposing an optimal photographing environment in consideration of the user's geographical location information. Part or all of the above-mentioned processing in the photographing unit may be performed, for example, using AI or may be performed without using AI. For example, the photographing unit can input geographical location information data to the generation AI and cause the generation AI to execute a proposal of an optimal photographing environment.

[0041] The photographing unit can analyze the user's social media activities at the time of photographing, and photograph related actions with priority. For example, the photographing unit preferentially photographs actions that the user wants to share on social media. The photographing unit can also analyze the user's past social media posts, and preferentially photograph popular actions. Furthermore, the photographing unit can analyze the reactions of the user's followers, and preferentially photograph actions that are of high interest. In this way, by analyzing the user's social media activities, it is possible to preferentially photograph actions that the user wants to share. A part or all of the above-mentioned processing in the photographing unit may be performed, for example, using AI, or may be performed without using AI. For example, the photographing unit can input social media activity data to the generation AI, and cause the generation AI to perform preferential photographing of related actions.

[0042] The reading unit can automatically evaluate the quality of the video at the time of reading and select the optimal analysis method. The reading unit, for example, evaluates the resolution of the video and selects a detailed analysis method in the case of high resolution. The reading unit can also evaluate the frame rate of the video and select a simple analysis method in the case of a low frame rate. Furthermore, the reading unit can evaluate the brightness and contrast of the video and select the optimal analysis method. In this way, the video quality is automatically evaluated to select the optimal analysis method and improve the accuracy of the analysis. A part or all of the above-mentioned processing in the reading unit may be performed, for example, using AI or may be performed without using AI. For example, the reading unit can input video quality data to the generation AI and cause the generation AI to select the optimal analysis method.

[0043] The reading unit can apply different reading algorithms depending on the content of the video when reading. The reading unit applies different reading algorithms depending on the type of sport, for example. The reading unit can also apply different reading algorithms depending on the speed of the movement. Furthermore, the reading unit can apply different reading algorithms depending on the length of the video. This allows efficient reading by applying an optimal reading algorithm depending on the content of the video. Some or all of the above-mentioned processing in the reading unit may be performed using, for example, AI, or may be performed without using AI. For example, the reading unit can input content data of the video to the generation AI and cause the generation AI to apply different reading algorithms.

[0044] The reading unit can perform optimal reading settings based on the location where the video was shot during reading. For example, in the case of a video shot outdoors, the reading unit optimizes brightness and contrast. In addition, in the case of a video shot indoors, the reading unit can also apply noise reduction. Furthermore, in the case of a video shot at a specific sports facility, the reading unit can perform optimal reading settings based on the characteristics of the facility. This improves the accuracy of reading by performing optimal reading settings based on the location where the video was shot. Part or all of the above-mentioned processing in the reading unit may be performed using, for example, AI, or may be performed without using AI. For example, the reading unit can input shooting location data to the generation AI and cause the generation AI to execute optimal reading settings.

[0045] The reading unit can improve the accuracy of reading by referring to the related metadata of the video at the time of reading. The reading unit can set the optimal reading setting by referring to, for example, the shooting date and time of the video. The reading unit can also set the optimal reading setting by referring to the shooting location of the video. Furthermore, the reading unit can also set the optimal reading setting by referring to information on the shooting device of the video. Thereby, the accuracy of reading can be improved by referring to the related metadata of the video. A part or all of the above-mentioned processing in the reading unit may be performed using, for example, AI, or may be performed without using AI. For example, the reading unit can input the related metadata to the generation AI and cause the generation AI to execute the improvement of the reading accuracy.

[0046] The analysis unit can optimize the analysis algorithm by referring to the user's past motion data during analysis. The analysis unit, for example, selects an optimal analysis algorithm based on the user's past motion data. The analysis unit can also analyze the user's past motion data, identify a motion pattern, and optimize the analysis algorithm. Furthermore, the analysis unit can improve the accuracy of the analysis by referring to the user's past motion data. As a result, the analysis algorithm is optimized by referring to the user's past motion data, and the accuracy of the analysis is improved. A part or all of the above-mentioned processing in the analysis unit may be performed, for example, using AI, or may be performed without using AI. For example, the analysis unit can input the past motion data to the generation AI and cause the generation AI to optimize the analysis algorithm.

[0047] The analysis unit can apply different analysis methods depending on the category of the movement during analysis. For example, in the case of racket sports, the analysis unit applies a swing analysis method. In addition, in the case of ball sports, the analysis unit can also apply a shoot analysis method. Furthermore, in the case of track and field, the analysis unit can also apply a running form analysis method. This improves the accuracy of the analysis by applying the optimal analysis method depending on the category of the movement. A part or all of the above-mentioned processing in the analysis unit may be performed using, for example, AI, or may be performed without using AI. For example, the analysis unit can input movement category data to the generation AI and cause the generation AI to apply different analysis methods.

[0048] The analysis unit can improve the accuracy of the analysis by taking into account the physical characteristics of the user during the analysis. The analysis unit can improve the accuracy of the analysis by taking into account, for example, the height and weight of the user. The analysis unit can also improve the accuracy of the analysis by taking into account the muscle strength and flexibility of the user. Furthermore, the analysis unit can improve the accuracy of the analysis by taking into account the age and gender of the user. In this way, the accuracy of the analysis can be improved by taking into account the physical characteristics of the user. A part or all of the above-mentioned processing in the analysis unit may be performed, for example, using AI, or may be performed without using AI. For example, the analysis unit can input the physical characteristic data to the generation AI and cause the generation AI to execute the improvement of the accuracy of the analysis.

[0049] The analysis unit can improve the accuracy of the analysis by referring to related sports science data during the analysis. The analysis unit can improve the accuracy of the analysis by referring to, for example, the latest research data in sports science. The analysis unit can also improve the accuracy of the analysis by referring to training data of professional athletes. Furthermore, the analysis unit can also improve the accuracy of the analysis by referring to sports medicine data. As a result, the accuracy of the analysis can be improved by referring to related sports science data. Some or all of the above-mentioned processing in the analysis unit can be performed, for example, using AI or can be performed without using AI. For example, the analysis unit can input sports science data to the generation AI and cause the generation AI to perform the improvement of the accuracy of the analysis.

[0050] The comparison unit can dynamically update the database of professional players and use the latest data at the time of comparison. The comparison unit can dynamically update, for example, the latest match data of professional players and use it for comparison. The comparison unit can also dynamically update the latest training data of professional players and use it for comparison. Furthermore, the comparison unit can dynamically update the latest technical data of professional players and use it for comparison. In this way, the accuracy of the comparison can be improved by using the latest data of professional players. A part or all of the above-mentioned processing in the comparison unit can be performed, for example, using AI or can be performed without using AI. For example, the comparison unit can input the database of professional players to the generation AI and cause the generation AI to perform dynamic updating.

[0051] The comparison unit can specify the difference between the user's motion and the professional player's motion in detail during the comparison. The comparison unit specifies, for example, the difference in the angle between the user's swing and the professional player's swing. The comparison unit can also specify the difference between the user's foot position and the professional player's foot position. Furthermore, the comparison unit can specify the difference between the user's motion speed and the professional player's motion speed. This allows specific improvements to be presented by specifying the difference between the user's motion and the professional player's motion in detail. A part or all of the above-mentioned processing in the comparison unit may be performed, for example, using AI, or may be performed without using AI. For example, the comparison unit can input the user's motion data and the professional player's motion data to the generation AI and cause the generation AI to specify the difference.

[0052] The comparison unit can improve the accuracy of the comparison by taking into account the sports experience of the user at the time of comparison. The comparison unit can improve the accuracy of the comparison by taking into account, for example, the number of years of sports experience of the user. The comparison unit can also improve the accuracy of the comparison by taking into account the past match results of the user. Furthermore, the comparison unit can also improve the accuracy of the comparison by taking into account the training history of the user. In this way, the accuracy of the comparison can be improved by taking into account the sports experience of the user. A part or all of the above-mentioned processing in the comparison unit may be performed, for example, using AI, or may be performed without using AI. For example, the comparison unit can input the sports experience data of the user to the generation AI and cause the generation AI to execute the improvement of the accuracy of the comparison.

[0053] The comparison unit can compare the motion data of professional athletes by referring to different sports categories. For example, the comparison unit refers to data of racket sports and compares it with the motion of ball sports. The comparison unit can also refer to data of track and field sports and compare it with the motion of field sports. Furthermore, the comparison unit can also refer to data of swimming and compare it with the motion of track and field sports. This allows for a more multifaceted comparison by referring to data of different sports categories. Some or all of the above-mentioned processing in the comparison unit may be performed, for example, using AI, or may be performed without using AI. For example, the comparison unit can input data of different sports categories to the generation AI and cause the generation AI to perform the comparison.

[0054] The feedback unit can provide optimal advice by referring to the user's past feedback history at the time of feedback. The feedback unit can provide optimal advice based on the user's past feedback history, for example. The feedback unit can also analyze the user's past feedback history, identify a pattern of the action, and provide advice. Furthermore, the feedback unit can also improve the accuracy of the feedback by referring to the user's past feedback history. As a result, optimal advice can be provided by referring to the user's past feedback history. A part or all of the above-mentioned processing in the feedback unit can be performed, for example, using AI, or can be performed without using AI. For example, the feedback unit can input past feedback history data to the generation AI and cause the generation AI to provide optimal advice.

[0055] The feedback unit can specify the improvement points of the user's motion in detail at the time of feedback and present a specific improvement method. The feedback unit can specify the improvement points of the user's swing angle, for example, and present a specific correction method. The feedback unit can also specify the improvement points of the user's foot position, and present a specific correction method. Furthermore, the feedback unit can specify the improvement points of the user's motion speed, and present a specific correction method. As a result, a specific improvement method can be presented by specifying the improvement points of the user's motion in detail. A part or all of the above-mentioned processing in the feedback unit can be performed, for example, using AI, or can be performed without using AI. For example, the feedback unit can input motion data to the generation AI and cause the generation AI to execute the identification of the improvement points and the presentation of a specific correction method.

[0056] The feedback unit can provide optimal advice by taking into account the geographical location information of the user at the time of feedback. For example, when the user is training outdoors, the feedback unit can suggest an optimal training method. In addition, when the user is training indoors, the feedback unit can also suggest an optimal training method. Furthermore, when the user is training at a specific sports facility, the feedback unit can also provide advice based on the characteristics of the facility. In this way, optimal advice can be provided by taking into account the geographical location information of the user. Part or all of the above-mentioned processing in the feedback unit may be performed, for example, using AI or may be performed without using AI. For example, the feedback unit can input geographical location information data to the generation AI and cause the generation AI to provide optimal advice.

[0057] The feedback unit can provide relevant advice by analyzing the user's social media activities at the time of feedback. The feedback unit can provide advice on actions that the user wants to share on social media, for example. The feedback unit can also analyze the user's past social media posts and provide advice on popular actions. Furthermore, the feedback unit can analyze the reactions of the user's followers and provide advice on actions of high interest. In this way, relevant advice can be provided by analyzing the user's social media activities. Some or all of the above-mentioned processing in the feedback unit can be performed, for example, using AI or can be performed without using AI. For example, the feedback unit can input social media activity data to the generation AI and cause the generation AI to provide relevant advice.

[0058] The system according to the embodiment is not limited to the above-described example, and various modifications are possible, for example, as follows.

[0059] The sports form improvement system can further acquire health data of the user and use it for analysis in the analysis unit. For example, the user's heart rate and calorie consumption can be acquired, and improvements to the form can be identified based on this data. The system can also acquire the user's sleep data and provide advice that takes into account the user's fatigue level. Furthermore, the system can acquire the user's dietary data and provide training advice based on the user's nutritional state. This makes it possible to provide feedback that takes into account the user's overall health condition.

[0060] The analysis unit can take weather data into account when analyzing the user's movements. For example, in the case of outdoor sports, data such as wind speed, temperature, and humidity is acquired, and the impact of these factors on movements is reflected in the analysis. It can also perform analysis that takes into account slipperiness in rainy weather. Furthermore, it can also suggest appropriate training times by taking into account the hours of sunlight and the amount of ultraviolet light. This allows for more accurate analysis that takes environmental factors into account.

[0061] The comparison unit can take the user's individual goals into consideration when analyzing the user's motion data. For example, if the user is aiming to win a particular tournament, the comparison unit can suggest the optimal training method for that goal. Also, if the user is aiming to maintain health, the comparison unit can suggest a training method that avoids excessive strain. Furthermore, if the user wants to master a particular skill, the comparison unit can provide advice specialized for that skill. This makes it possible to provide optimal feedback according to the user's individual goals.

[0062] The feedback unit can refer to the user's training history and suggest long-term training plans. For example, it can plan the next training content based on past training data. It can also evaluate the user's progress and present specific steps to achieve the goal. It can also suggest rest periods and timing for nutritional supplementation to maximize the effect of training. This makes it possible to support the user in conducting effective training in the long term.

[0063] The imaging unit can suggest an optimal imaging schedule based on the user's past imaging data. For example, it can suggest the next imaging time based on the time period in which the user got good results in the past. It can also suggest the optimal imaging date taking into account the user's daily rhythm. Furthermore, it can adjust the imaging frequency to match the user's training plan. This allows for efficient imaging that matches the user's individual schedule.

[0064] The shooting unit can track the user's movements in real time and automatically select the optimal shooting mode. For example, if the user is moving at high speed, a high-speed shutter mode can be selected. Also, if the user is performing fine movements, a macro mode can be selected. Furthermore, if the user is performing movements over a wide area, a panorama mode can be selected. In this way, by selecting the optimal shooting mode according to the user's movements, important moments can be captured without missing them.

[0065] The photography unit can suggest optimal photography equipment based on the user's geographical location information. For example, in the case of outdoor photography, a waterproof camera can be suggested. In addition, in the case of indoor photography, lighting equipment can be suggested. Furthermore, in the case of photography at a specific sports facility, lenses and tripods suitable for the facility can be suggested. In this way, the quality of photography can be improved by suggesting optimal equipment according to the user's photography environment.

[0066] The filming unit can analyze the user's social media activity and suggest ways to edit the filmed video. For example, it can suggest an editing method that emphasizes the actions that the user wants to share on social media. It can also analyze the user's past social media posts and suggest popular editing styles. It can also analyze the reactions of the user's followers and suggest editing points that are of high interest. In this way, it is possible to create effective videos by suggesting the optimal editing method that takes into account the user's social media activity.

[0067] The process flow of the first embodiment will be briefly described below.

[0068] Step 1: The imaging unit captures a video of the user playing a sport. Videos of the user playing a sport include, but are not limited to, swings in racket sports and shots in ball sports. The imaging unit can capture the video using, for example, a camera on a smartphone or tablet. The imaging unit can also capture high-resolution video using a dedicated camera. Step 2: The reading unit reads the video captured by the shooting unit. The reading unit can, for example, import a video file into the app. The reading unit can also download and read a video from cloud storage. Step 3: The analysis unit uses AI to analyze the video loaded by the loading unit and identify the position and movement of the user's body. The analysis unit, for example, analyzes each frame in the video and identifies the position and movement of each part of the user's body. For example, in the case of a swing in a racket sport, the position of the racket, the angle of the swing, the position of the feet, etc. The analysis unit can analyze each frame in the video and identify the position and movement of each part of the user's body using, for example, deep learning technology. Step 4: The comparison unit compares the data identified by the analysis unit with the data of the professional player. For example, the comparison unit refers to a database of professional players and compares it with the user's motion. For example, the comparison unit compares the swing of the professional player with the swing of the user and identifies which parts are different. For example, the comparison unit can use machine learning technology to compare the data of the professional player with the data of the user. Step 5: The feedback unit provides real-time feedback based on the analysis results obtained by the comparison unit. For example, the feedback unit presents the user with specific improvements and advises how to correct their form. For example, advice such as "You should raise the angle of the racket a little more" or "You will be more stable if you spread your feet out a little more" is provided. The feedback unit can present the user with specific improvements using, for example, natural language processing technology.

[0069] (Example 2) The sports form improvement system according to the embodiment of the present invention is a system in which a user shoots a video of himself playing a sport, and an AI analyzes the video to suggest improvements to the form. In this system, a user shoots a video of himself playing a sport, an app loads the video, and an AI analyzes the target's body position, movement, technical elements, and the like in detail. Furthermore, the AI ​​compares and analyzes the data of people who are good at sports, such as professional athletes, with the target's data, and provides feedback and advice in real time based on the analysis results. For example, a user shoots a specific movement, such as a swing in a racket sport or a shot in a ball sport. This video is input into the app. Next, the app loads the video, and the AI ​​analyzes the target's body position, movement, technical elements, and the like in detail. The AI ​​analyzes each frame in the video and identifies the position and movement of each part of the user's body. For example, in the case of a swing in a racket sport, the position of the racket, the angle of the swing, the position of the foot, and the like are analyzed. Furthermore, the AI ​​compares and analyzes the data of people who are good at sports, such as professional athletes, with the target's data. The AI ​​refers to a database of professional athletes and compares it with the user's movement. For example, the swing of a professional athlete is compared with the user's swing to identify which parts are different. Finally, it provides feedback and advice in real time based on the analysis results. The app presents specific areas for improvement to the user and advises them on how to correct their form. For example, it provides advice such as "raise the angle of the racket a little more" or "widen the position of your feet a little more to make it more stable." This system allows users to objectively evaluate their own form and know specific areas for improvement, which is expected to lead to improvements in sports skills. This allows the sports form improvement system to efficiently analyze the user's sports form and provide feedback in real time.

[0070] The sports form improvement system according to the embodiment includes a shooting unit, a reading unit, an analysis unit, a comparison unit, and a feedback unit. The shooting unit shoots a video of a user playing a sport. The video of a user playing a sport includes, but is not limited to, a swing in a racket sport and a shot in a ball sport. The shooting unit can shoot a video using, for example, a camera of a smartphone or a tablet. The shooting unit can also shoot a high-resolution video using a dedicated camera. The reading unit reads the video shot by the shooting unit. The reading unit can, for example, import a video file into an app. The reading unit can also download and read a video from cloud storage. The analysis unit uses AI to analyze the video read by the reading unit and identify the position and movement of the user's body. The analysis unit, for example, analyzes each frame in the video and identifies the position and movement of each part of the user's body. For example, in the case of a swing in a racket sport, the position of the racket, the angle of the swing, the position of the foot, etc. are analyzed. The analysis unit can use, for example, deep learning technology to analyze each frame in the video and identify the position and movement of each part of the user's body. The comparison unit compares the data identified by the analysis unit with the data of professional players. The comparison unit, for example, refers to a database of professional players and compares it with the user's movements. For example, it compares the swing of the professional player with the swing of the user and identifies which parts are different. The comparison unit can use, for example, machine learning technology to compare the data of the professional player with the data of the user. The feedback unit provides feedback in real time based on the analysis results obtained by the comparison unit. For example, the feedback unit presents specific improvements to the user and advises how to correct the form. For example, advice such as "It would be better to raise the angle of the racket a little more" or "It will be more stable if you spread the position of your feet a little wider". The feedback unit can present specific improvements to the user using, for example, natural language processing technology.As a result, the sports form improving system according to the embodiment can efficiently analyze the sports form of the user and provide feedback in real time.

[0071] The filming unit shoots a video of the user playing a sport. Examples of videos of the user playing a sport include, but are not limited to, swings in racket sports and shots in ball sports. The filming unit can shoot the video using, for example, a smartphone or tablet camera. The filming unit can also shoot high-resolution video using a dedicated camera. Specifically, a smartphone or tablet camera can be easily carried around, allowing the user to easily record their own movements. This allows the user to film their form anywhere, such as at home or at a practice field. When a dedicated camera is used, high-resolution video can be shot, allowing for more detailed motion analysis. For example, a dedicated camera has a high frame rate and can smoothly record the user's high-speed movements. In addition, by using multiple cameras to shoot from different angles simultaneously, the user's movements can be captured three-dimensionally. This allows the filming unit to record the user's sports form from multiple angles and provide high-quality data for detailed analysis. Furthermore, the filming unit has a function of streaming the filmed video in real time, allowing the user to check their own movements on the spot. This allows the user to instantly correct their form and improve the efficiency of their practice.

[0072] The reading unit reads the video shot by the shooting unit. The reading unit can, for example, import a video file into an app. The reading unit can also download and read a video from cloud storage. Specifically, the user can directly import the video shot by the user into an app on a smartphone or tablet, and can easily prepare for analysis. When using cloud storage, the user can upload the video shot by the user to the cloud and access and read it from another device. This allows the user to share data between different devices and perform analysis flexibly. Furthermore, the reading unit can read videos in various formats regardless of the video format or resolution. For example, it supports common video formats such as MP4, AVI, and MOV, and the user can save the trouble of converting to a specific format. The reading unit also has a function of cutting out and analyzing a part of the video, and the user can focus on a specific action or scene to perform analysis. This allows the reading unit to efficiently import the video shot by the user and quickly prepare for analysis.

[0073] The analysis unit uses AI to analyze the video read by the reading unit and identify the position and movement of the user's body. The analysis unit, for example, analyzes each frame in the video and identifies the position and movement of each part of the user's body. For example, in the case of a swing in a racket sport, the position of the racket, the angle of the swing, the position of the foot, etc. The analysis unit can analyze each frame in the video and identify the position and movement of each part of the user's body using, for example, deep learning technology. Specifically, a deep learning model is used to extract the user's skeleton from each frame in the video and identify the position of each joint. This makes it possible to analyze the user's movements in detail and capture the characteristics of the form. Furthermore, the analysis unit can also analyze the speed and timing of the movements, and can evaluate the efficiency and performance of the user's movements. For example, the speed of the swing and the timing of the release of the ball can be analyzed to quantify the user's performance. In addition, the analysis unit has multiple deep learning models available to accommodate different sports and movements, and can select the optimal model depending on the sport the user plays. This makes it possible for the analysis unit to analyze the user's sports form with high accuracy and provide detailed data.

[0074] The comparison unit compares the data identified by the analysis unit with the data of the professional player. For example, the comparison unit refers to a database of professional players and compares it with the user's motion. For example, the comparison unit compares the swing of the professional player with the swing of the user to identify which parts are different. For example, the comparison unit can compare the data of the professional player with the data of the user using machine learning technology. Specifically, the motion database of the professional player stores motion data of top athletes in various sports, and by comparing it with the motion of the user, it is possible to identify areas for improvement in the form. The comparison unit can overlay the motion data of the user with the data of the professional player and visually display the differences in the positions and movements of each part. This allows the user to intuitively understand which parts of his or her form are different from those of the professional player. Furthermore, the comparison unit can statistically compare the motion data of the user with the data of the professional player using a machine learning algorithm and quantify specific areas for improvement. For example, it is possible to compare parameters such as the angle and speed of the swing, and the position of the foot, and clearly indicate which parts of the user's form need improvement. This allows the comparison unit to compare the sports form of the user with the data of the professional player and identify specific areas for improvement.

[0075] The feedback unit provides feedback in real time based on the analysis results obtained by the comparison unit. The feedback unit, for example, presents the user with specific improvements and advises how to correct the form. For example, advice such as "raise the angle of the racket a little more" or "widen the position of the feet a little more for stability" is provided. The feedback unit can present the user with specific improvements, for example, using natural language processing technology. Specifically, based on the analysis results, it identifies which parts of the user's movements need improvement and explains them in an easy-to-understand manner in natural language. Furthermore, the feedback unit can also provide feedback using audio and visuals. For example, the user can check the improvements while comparing his or her movements with those of a professional player on the screen of a smartphone or tablet. In addition, a voice assistant can be used to provide real-time audio feedback, allowing the user to correct the form on the spot. Furthermore, the feedback unit can record the user's progress and compare it with past data to evaluate the degree of improvement. This allows the user to realize his or her own growth and maintain motivation. This allows the feedback unit to provide the user with specific and easy-to-understand feedback and support the improvement of the sports form.

[0076] The analysis unit can identify the position and movement of the user's body. The analysis unit can, for example, use deep learning technology to analyze each frame in the video and identify the position and movement of each part of the user's body. For example, in the case of a swing in a racket sport, the position of the racket, the angle of the swing, the position of the foot, etc. The analysis unit can also identify the position and movement of the user's body using machine learning technology. For example, the position of the joints and the movement pattern are analyzed. Furthermore, the analysis unit can use multiple sensors to identify the position and movement of the user's body. For example, the position and movement of the user's body can be specified in detail using motion capture technology. This makes it possible to perform a detailed analysis by accurately specifying the position and movement of the user's body. Part or all of the above-mentioned processing in the analysis unit can be performed using, for example, AI, or can be performed without using AI. For example, the analysis unit can input video data to the generation AI and cause the generation AI to execute a process of specifying the position and movement of the user's body.

[0077] The comparison unit can refer to the database of professional players. The comparison unit, for example, refers to the database of professional players and compares it with the user's motion. For example, the comparison unit compares the swing of the professional player with the swing of the user to identify which parts are different. The comparison unit can also dynamically update the database of professional players and use the latest data. For example, the latest match data and training data of the professional players are dynamically updated and used for comparison. Furthermore, the comparison unit can refer to the database of professional players from different sports categories to make comparisons. For example, data of racket sports can be referenced and compared with the motion of ball sports. This makes it possible to identify areas for improvement in the user's form by comparing with the data of professional players. Part or all of the above-mentioned processing in the comparison unit may be performed, for example, using AI, or may be performed without using AI. For example, the comparison unit can input the database of professional players to the generation AI and cause the generation AI to compare the motion of the user with the motion of the professional player.

[0078] The feedback unit can present specific points for improvement. For example, the feedback unit presents specific points for improvement to the user and advises how to correct the form. For example, advice such as "raise the angle of the racket a little more" or "widen the position of the feet a little more for stability" is provided. The feedback unit can also provide optimal advice by referring to the user's past feedback history. For example, optimal advice is provided based on the user's past feedback history. Furthermore, the feedback unit can also specify in detail the points for improvement of the user's motion and present a specific method of improvement. For example, the feedback unit specifies the points for improvement in the angle of the user's swing and presents a specific method of correction. As a result, the specific points for improvement are presented, making it easier for the user to correct the form. A part or all of the above-mentioned processing in the feedback unit may be performed, for example, using AI, or may be performed without using AI. For example, the feedback unit can input the analysis results to the generation AI and cause the generation AI to execute the presentation of specific points for improvement.

[0079] The shooting unit can estimate the user's emotion and adjust the timing of shooting based on the estimated user's emotion. For example, when the user is nervous, the shooting unit waits until the user relaxes before starting shooting. Also, when the user is concentrating, the shooting unit can immediately start shooting to maintain the user's concentration. Furthermore, when the user is tired, the shooting unit can resume shooting after a break. In this way, the timing of shooting can be adjusted according to the user's emotion, thereby making it possible to shoot an optimal video. The estimation of emotion is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the shooting unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the shooting unit can input the user's emotion data to the generation AI and cause the generation AI to adjust the timing of shooting.

[0080] The photographing unit can analyze the user's past photographing history at the time of photographing and automatically set the optimal photographing angle and distance. For example, the photographing unit automatically sets the photographing angle at which the user has obtained good results in the past. The photographing unit can also calculate the optimal distance from the user's past photographing history and automatically position the camera. Furthermore, the photographing unit can set the optimal lighting conditions based on the data of videos the user has photographed in the past. This enables efficient photographing by automatically setting the optimal photographing conditions based on the past photographing history. A part or all of the above-mentioned processing in the photographing unit may be performed, for example, using AI or may be performed without using AI. For example, the photographing unit can input past photographing history data to the generation AI and cause the generation AI to set the optimal photographing angle and distance.

[0081] The shooting unit can track the user's movements in real time during shooting and automatically select the optimal frame. For example, the shooting unit tracks the user's movements in real time and automatically selects a frame at the peak of the movement. The shooting unit can also automatically adjust the frame rate according to the speed of the user's movements. Furthermore, the shooting unit can automatically adjust the pan and zoom of the camera according to the direction of the user's movements. This makes it possible to track the user's movements in real time and select the optimal frame, thereby capturing important moments without missing them. Some or all of the above-mentioned processing in the shooting unit may be performed, for example, using AI, or may be performed without using AI. For example, the shooting unit can input the movement data acquired in real time to the generation AI and cause the generation AI to select the optimal frame.

[0082] The photographing unit can estimate the user's emotion and determine the priority of the actions to be photographed based on the estimated user's emotion. For example, when the user is excited, the photographing unit preferentially photographs the highlight scenes of the actions. Also, when the user is relaxed, the photographing unit can evenly photograph the entire actions. Furthermore, when the user is tired, the photographing unit can preferentially photograph only important actions. In this way, the priority of the actions to be photographed can be determined according to the user's emotion, so that important actions can be preferentially photographed. The estimation of the emotion is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the photographing unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the photographing unit can input the user's emotion data to the generation AI and cause the generation AI to execute the determination of the priority of the actions to be photographed.

[0083] The photographing unit can propose an optimal photographing environment based on the user's geographical location information at the time of photographing. For example, when the user photographs outdoors, the photographing unit proposes an optimal photographing location. In addition, when the user photographs indoors, the photographing unit can also propose optimal lighting conditions. Furthermore, when the user photographs at a specific sports facility, the photographing unit can also propose an optimal photographing point at the facility. This improves the quality of photographing by proposing an optimal photographing environment in consideration of the user's geographical location information. Part or all of the above-mentioned processing in the photographing unit may be performed, for example, using AI or may be performed without using AI. For example, the photographing unit can input geographical location information data to the generation AI and cause the generation AI to execute a proposal of an optimal photographing environment.

[0084] The photographing unit can analyze the user's social media activities at the time of photographing, and photograph related actions with priority. For example, the photographing unit preferentially photographs actions that the user wants to share on social media. The photographing unit can also analyze the user's past social media posts, and preferentially photograph popular actions. Furthermore, the photographing unit can analyze the reactions of the user's followers, and preferentially photograph actions that are of high interest. In this way, by analyzing the user's social media activities, it is possible to preferentially photograph actions that the user wants to share. A part or all of the above-mentioned processing in the photographing unit may be performed, for example, using AI, or may be performed without using AI. For example, the photographing unit can input social media activity data to the generation AI, and cause the generation AI to perform preferential photographing of related actions.

[0085] The reading unit can estimate the user's emotion and adjust the video loading speed based on the estimated user's emotion. For example, when the user is in a hurry, the reading unit sets the video loading speed to high. Also, when the user is relaxed, the reading unit can set the video loading speed to normal. Furthermore, when the user is stressed, the reading unit can optimize the video loading speed and play the video smoothly. This allows the video to be played according to the user's needs by adjusting the video loading speed according to the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the reading unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the reading unit can input the user's emotion data to the generation AI and cause the generation AI to adjust the video loading speed.

[0086] The reading unit can automatically evaluate the quality of the video at the time of reading and select the optimal analysis method. The reading unit, for example, evaluates the resolution of the video and selects a detailed analysis method in the case of high resolution. The reading unit can also evaluate the frame rate of the video and select a simple analysis method in the case of a low frame rate. Furthermore, the reading unit can evaluate the brightness and contrast of the video and select the optimal analysis method. In this way, the video quality is automatically evaluated to select the optimal analysis method and improve the accuracy of the analysis. A part or all of the above-mentioned processing in the reading unit may be performed, for example, using AI or may be performed without using AI. For example, the reading unit can input video quality data to the generation AI and cause the generation AI to select the optimal analysis method.

[0087] The reading unit can apply different reading algorithms depending on the content of the video when reading. The reading unit applies different reading algorithms depending on the type of sport, for example. The reading unit can also apply different reading algorithms depending on the speed of the movement. Furthermore, the reading unit can apply different reading algorithms depending on the length of the video. This allows efficient reading by applying an optimal reading algorithm depending on the content of the video. Some or all of the above-mentioned processing in the reading unit may be performed using, for example, AI, or may be performed without using AI. For example, the reading unit can input content data of the video to the generation AI and cause the generation AI to apply different reading algorithms.

[0088] The reading unit can estimate the user's emotion and determine the priority of the videos to be read based on the estimated user's emotion. For example, when the user is excited, the reading unit preferentially reads important scenes. Also, when the user is relaxed, the reading unit can evenly read the entire video. Furthermore, when the user is in a hurry, the reading unit can preferentially read scenes that highlight the main points. In this way, important scenes can be preferentially read by determining the priority of the videos according to the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the reading unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the reading unit can input the user's emotion data to the generation AI and cause the generation AI to execute the determination of the priority of the videos to be read.

[0089] The reading unit can perform optimal reading settings based on the location where the video was shot during reading. For example, in the case of a video shot outdoors, the reading unit optimizes brightness and contrast. In addition, in the case of a video shot indoors, the reading unit can also apply noise reduction. Furthermore, in the case of a video shot at a specific sports facility, the reading unit can perform optimal reading settings based on the characteristics of the facility. This improves the accuracy of reading by performing optimal reading settings based on the location where the video was shot. Part or all of the above-mentioned processing in the reading unit may be performed using, for example, AI, or may be performed without using AI. For example, the reading unit can input shooting location data to the generation AI and cause the generation AI to execute optimal reading settings.

[0090] The reading unit can improve the accuracy of reading by referring to the related metadata of the video at the time of reading. The reading unit can set the optimal reading setting by referring to, for example, the shooting date and time of the video. The reading unit can also set the optimal reading setting by referring to the shooting location of the video. Furthermore, the reading unit can also set the optimal reading setting by referring to information on the shooting device of the video. Thereby, the accuracy of reading can be improved by referring to the related metadata of the video. A part or all of the above-mentioned processing in the reading unit may be performed using, for example, AI, or may be performed without using AI. For example, the reading unit can input the related metadata to the generation AI and cause the generation AI to execute the improvement of the reading accuracy.

[0091] The analysis unit can estimate the user's emotion and adjust the level of detail of the analysis based on the estimated user's emotion. For example, when the user is relaxed, the analysis unit performs a detailed analysis. Furthermore, when the user is in a hurry, the analysis unit can also perform a simple analysis. Furthermore, when the user is excited, the analysis unit can perform a visually easy-to-understand analysis. This allows the analysis result suitable for the user to be provided by adjusting the level of detail of the analysis according to the user's emotion. The estimation of the emotion is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the analysis unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the analysis unit can input the user's emotion data to the generation AI and cause the generation AI to adjust the level of detail of the analysis.

[0092] The analysis unit can optimize the analysis algorithm by referring to the user's past motion data during analysis. The analysis unit, for example, selects an optimal analysis algorithm based on the user's past motion data. The analysis unit can also analyze the user's past motion data, identify a motion pattern, and optimize the analysis algorithm. Furthermore, the analysis unit can improve the accuracy of the analysis by referring to the user's past motion data. As a result, the analysis algorithm is optimized by referring to the user's past motion data, and the accuracy of the analysis is improved. A part or all of the above-mentioned processing in the analysis unit may be performed, for example, using AI, or may be performed without using AI. For example, the analysis unit can input the past motion data to the generation AI and cause the generation AI to optimize the analysis algorithm.

[0093] The analysis unit can apply different analysis methods depending on the category of the movement during analysis. For example, in the case of racket sports, the analysis unit applies a swing analysis method. In addition, in the case of ball sports, the analysis unit can also apply a shoot analysis method. Furthermore, in the case of track and field, the analysis unit can also apply a running form analysis method. This improves the accuracy of the analysis by applying the optimal analysis method depending on the category of the movement. A part or all of the above-mentioned processing in the analysis unit may be performed using, for example, AI, or may be performed without using AI. For example, the analysis unit can input movement category data to the generation AI and cause the generation AI to apply different analysis methods.

[0094] The analysis unit can estimate the user's emotion and adjust the display method of the analysis result based on the estimated user's emotion. For example, when the user is nervous, the analysis unit provides a simple and highly visible display method. Furthermore, when the user is relaxed, the analysis unit can also provide a display method including detailed information. Furthermore, when the user is in a hurry, the analysis unit can also provide a display method that focuses on the main points. This allows for a display suitable for the user by adjusting the display method of the analysis result according to the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the analysis unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the analysis unit can input the user's emotion data to the generation AI and cause the generation AI to adjust the display method of the analysis result.

[0095] The analysis unit can improve the accuracy of the analysis by taking into account the physical characteristics of the user during the analysis. The analysis unit can improve the accuracy of the analysis by taking into account, for example, the height and weight of the user. The analysis unit can also improve the accuracy of the analysis by taking into account the muscle strength and flexibility of the user. Furthermore, the analysis unit can improve the accuracy of the analysis by taking into account the age and gender of the user. In this way, the accuracy of the analysis can be improved by taking into account the physical characteristics of the user. A part or all of the above-mentioned processing in the analysis unit may be performed, for example, using AI, or may be performed without using AI. For example, the analysis unit can input the physical characteristic data to the generation AI and cause the generation AI to execute the improvement of the accuracy of the analysis.

[0096] The analysis unit can improve the accuracy of the analysis by referring to related sports science data during the analysis. The analysis unit can improve the accuracy of the analysis by referring to, for example, the latest research data in sports science. The analysis unit can also improve the accuracy of the analysis by referring to training data of professional athletes. Furthermore, the analysis unit can also improve the accuracy of the analysis by referring to sports medicine data. As a result, the accuracy of the analysis can be improved by referring to related sports science data. Some or all of the above-mentioned processing in the analysis unit can be performed, for example, using AI or can be performed without using AI. For example, the analysis unit can input sports science data to the generation AI and cause the generation AI to perform the improvement of the accuracy of the analysis.

[0097] The comparison unit can estimate the user's emotion and adjust the comparison criteria based on the estimated user's emotion. For example, when the user is relaxed, the comparison unit applies detailed comparison criteria. Also, when the user is in a hurry, the comparison unit can apply simple comparison criteria. Furthermore, when the user is excited, the comparison unit can apply visually easy-to-understand comparison criteria. This allows a comparison suitable for the user to be made by adjusting the comparison criteria according to the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the comparison unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the comparison unit can input the user's emotion data to the generation AI and cause the generation AI to adjust the comparison criteria.

[0098] The comparison unit can dynamically update the database of professional players and use the latest data at the time of comparison. The comparison unit can dynamically update, for example, the latest match data of professional players and use it for comparison. The comparison unit can also dynamically update the latest training data of professional players and use it for comparison. Furthermore, the comparison unit can dynamically update the latest technical data of professional players and use it for comparison. In this way, the accuracy of the comparison can be improved by using the latest data of professional players. A part or all of the above-mentioned processing in the comparison unit can be performed, for example, using AI or can be performed without using AI. For example, the comparison unit can input the database of professional players to the generation AI and cause the generation AI to perform dynamic updating.

[0099] The comparison unit can specify the difference between the user's motion and the professional player's motion in detail during the comparison. The comparison unit specifies, for example, the difference in the angle between the user's swing and the professional player's swing. The comparison unit can also specify the difference between the user's foot position and the professional player's foot position. Furthermore, the comparison unit can specify the difference between the user's motion speed and the professional player's motion speed. This allows specific improvements to be presented by specifying the difference between the user's motion and the professional player's motion in detail. A part or all of the above-mentioned processing in the comparison unit may be performed, for example, using AI, or may be performed without using AI. For example, the comparison unit can input the user's motion data and the professional player's motion data to the generation AI and cause the generation AI to specify the difference.

[0100] The comparison unit can estimate the user's emotion and adjust the display method of the comparison result based on the estimated user's emotion. For example, when the user is nervous, the comparison unit provides a simple and highly visible display method. Furthermore, when the user is relaxed, the comparison unit can also provide a display method including detailed information. Furthermore, when the user is in a hurry, the comparison unit can also provide a display method that focuses on the main points. This allows a display suitable for the user to be achieved by adjusting the display method of the comparison result according to the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the comparison unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the comparison unit can input the user's emotion data to the generation AI and cause the generation AI to adjust the display method of the comparison result.

[0101] The comparison unit can improve the accuracy of the comparison by taking into account the sports experience of the user at the time of comparison. The comparison unit can improve the accuracy of the comparison by taking into account, for example, the number of years of sports experience of the user. The comparison unit can also improve the accuracy of the comparison by taking into account the past match results of the user. Furthermore, the comparison unit can also improve the accuracy of the comparison by taking into account the training history of the user. In this way, the accuracy of the comparison can be improved by taking into account the sports experience of the user. A part or all of the above-mentioned processing in the comparison unit may be performed, for example, using AI, or may be performed without using AI. For example, the comparison unit can input the sports experience data of the user to the generation AI and cause the generation AI to execute the improvement of the accuracy of the comparison.

[0102] The comparison unit can compare the motion data of professional athletes by referring to different sports categories. For example, the comparison unit refers to data of racket sports and compares it with the motion of ball sports. The comparison unit can also refer to data of track and field sports and compare it with the motion of field sports. Furthermore, the comparison unit can also refer to data of swimming and compare it with the motion of track and field sports. This allows for a more multifaceted comparison by referring to data of different sports categories. Some or all of the above-mentioned processing in the comparison unit may be performed, for example, using AI, or may be performed without using AI. For example, the comparison unit can input data of different sports categories to the generation AI and cause the generation AI to perform the comparison.

[0103] The feedback unit can estimate the user's emotion and adjust the content of the feedback based on the estimated user's emotion. For example, the feedback unit provides detailed feedback when the user is relaxed. Also, the feedback unit can provide brief feedback when the user is in a hurry. Furthermore, the feedback unit can provide visually easy-to-understand feedback when the user is excited. This allows feedback suitable for the user to be provided by adjusting the content of the feedback according to the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the feedback unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the feedback unit can input the user's emotion data to the generation AI and cause the generation AI to adjust the content of the feedback.

[0104] The feedback unit can provide optimal advice by referring to the user's past feedback history at the time of feedback. The feedback unit can provide optimal advice based on the user's past feedback history, for example. The feedback unit can also analyze the user's past feedback history, identify a pattern of the action, and provide advice. Furthermore, the feedback unit can also improve the accuracy of the feedback by referring to the user's past feedback history. As a result, optimal advice can be provided by referring to the user's past feedback history. A part or all of the above-mentioned processing in the feedback unit can be performed, for example, using AI, or can be performed without using AI. For example, the feedback unit can input past feedback history data to the generation AI and cause the generation AI to provide optimal advice.

[0105] The feedback unit can specify the improvement points of the user's motion in detail at the time of feedback and present a specific improvement method. The feedback unit can specify the improvement points of the user's swing angle, for example, and present a specific correction method. The feedback unit can also specify the improvement points of the user's foot position, and present a specific correction method. Furthermore, the feedback unit can specify the improvement points of the user's motion speed, and present a specific correction method. As a result, a specific improvement method can be presented by specifying the improvement points of the user's motion in detail. A part or all of the above-mentioned processing in the feedback unit can be performed, for example, using AI, or can be performed without using AI. For example, the feedback unit can input motion data to the generation AI and cause the generation AI to execute the identification of the improvement points and the presentation of a specific correction method.

[0106] The feedback unit can estimate the user's emotion and determine the priority of the feedback based on the estimated user's emotion. For example, when the user is excited, the feedback unit can provide important feedback with priority. Also, when the user is relaxed, the feedback unit can provide the entire feedback evenly. Furthermore, when the user is in a hurry, the feedback unit can provide feedback with a focus on the main points with priority. In this way, by determining the priority of the feedback according to the user's emotion, it is possible to provide important feedback with priority. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the feedback unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the feedback unit can input the user's emotion data to the generation AI and cause the generation AI to execute the determination of the priority of the feedback.

[0107] The feedback unit can provide optimal advice by taking into account the geographical location information of the user at the time of feedback. For example, when the user is training outdoors, the feedback unit can suggest an optimal training method. In addition, when the user is training indoors, the feedback unit can also suggest an optimal training method. Furthermore, when the user is training at a specific sports facility, the feedback unit can also provide advice based on the characteristics of the facility. In this way, optimal advice can be provided by taking into account the geographical location information of the user. Part or all of the above-mentioned processing in the feedback unit may be performed, for example, using AI or may be performed without using AI. For example, the feedback unit can input geographical location information data to the generation AI and cause the generation AI to provide optimal advice.

[0108] The feedback unit can provide relevant advice by analyzing the user's social media activities at the time of feedback. The feedback unit can provide advice on actions that the user wants to share on social media, for example. The feedback unit can also analyze the user's past social media posts and provide advice on popular actions. Furthermore, the feedback unit can analyze the reactions of the user's followers and provide advice on actions of high interest. In this way, relevant advice can be provided by analyzing the user's social media activities. Some or all of the above-mentioned processing in the feedback unit can be performed, for example, using AI or can be performed without using AI. For example, the feedback unit can input social media activity data to the generation AI and cause the generation AI to provide relevant advice.

[0109] The system according to the embodiment is not limited to the above-described example, and various modifications are possible, for example, as follows.

[0110] The sports form improvement system can further acquire health data of the user and use it for analysis in the analysis unit. For example, the user's heart rate and calorie consumption can be acquired, and improvements to the form can be identified based on this data. The system can also acquire the user's sleep data and provide advice that takes into account the user's fatigue level. Furthermore, the system can acquire the user's dietary data and provide training advice based on the user's nutritional state. This makes it possible to provide feedback that takes into account the user's overall health condition.

[0111] The analysis unit can take weather data into account when analyzing the user's movements. For example, in the case of outdoor sports, data such as wind speed, temperature, and humidity is acquired, and the impact of these factors on movements is reflected in the analysis. It can also perform analysis that takes into account slipperiness in rainy weather. Furthermore, it can also suggest appropriate training times by taking into account the hours of sunlight and the amount of ultraviolet light. This allows for more accurate analysis that takes environmental factors into account.

[0112] The comparison unit can take the user's individual goals into consideration when analyzing the user's motion data. For example, if the user is aiming to win a particular tournament, the comparison unit can suggest the optimal training method for that goal. Also, if the user is aiming to maintain health, the comparison unit can suggest a training method that avoids excessive strain. Furthermore, if the user wants to master a particular skill, the comparison unit can provide advice specialized for that skill. This makes it possible to provide optimal feedback according to the user's individual goals.

[0113] The feedback unit can refer to the user's training history and suggest long-term training plans. For example, it can plan the next training content based on past training data. It can also evaluate the user's progress and present specific steps to achieve the goal. It can also suggest rest periods and timing for nutritional supplementation to maximize the effect of training. This makes it possible to support the user in conducting effective training in the long term.

[0114] The shooting unit can estimate the user's emotions and adjust the shooting environment settings based on the estimated user's emotions. For example, if the user is nervous, relaxing music can be played. Also, if the user is concentrating, a quiet environment can be provided to help the user maintain their concentration. Furthermore, if the user is tired, lighting can be adjusted to create an environment that is easy on the eyes. In this way, better videos can be shot by providing an optimal shooting environment according to the user's emotions.

[0115] The imaging unit can suggest an optimal imaging schedule based on the user's past imaging data. For example, it can suggest the next imaging time based on the time period in which the user got good results in the past. It can also suggest the optimal imaging date taking into account the user's daily rhythm. Furthermore, it can adjust the imaging frequency to match the user's training plan. This allows for efficient imaging that matches the user's individual schedule.

[0116] The shooting unit can track the user's movements in real time and automatically select the optimal shooting mode. For example, if the user is moving at high speed, a high-speed shutter mode can be selected. Also, if the user is performing fine movements, a macro mode can be selected. Furthermore, if the user is performing movements over a wide area, a panorama mode can be selected. In this way, by selecting the optimal shooting mode according to the user's movements, important moments can be captured without missing them.

[0117] The shooting unit can estimate the user's emotion and determine the order of scenes to be shot based on the estimated user's emotion. For example, if the user is excited, the highlight scenes of the user's actions are shot first. If the user is relaxed, the entire action can be shot evenly. Furthermore, if the user is tired, only important scenes can be shot preferentially. In this way, by determining the order of scenes to be shot according to the user's emotion, important scenes can be shot without missing them.

[0118] The photography unit can suggest optimal photography equipment based on the user's geographical location information. For example, in the case of outdoor photography, a waterproof camera can be suggested. In addition, in the case of indoor photography, lighting equipment can be suggested. Furthermore, in the case of photography at a specific sports facility, lenses and tripods suitable for the facility can be suggested. In this way, the quality of photography can be improved by suggesting optimal equipment according to the user's photography environment.

[0119] The filming unit can analyze the user's social media activity and suggest ways to edit the filmed video. For example, it can suggest an editing method that emphasizes the actions that the user wants to share on social media. It can also analyze the user's past social media posts and suggest popular editing styles. It can also analyze the reactions of the user's followers and suggest editing points that are of high interest. In this way, it is possible to create effective videos by suggesting the optimal editing method that takes into account the user's social media activity.

[0120] The process flow of the second embodiment will be briefly described below.

[0121] Step 1: The imaging unit captures a video of the user playing a sport. Videos of the user playing a sport include, but are not limited to, swings in racket sports and shots in ball sports. The imaging unit can capture the video using, for example, a camera on a smartphone or tablet. The imaging unit can also capture high-resolution video using a dedicated camera. Step 2: The reading unit reads the video captured by the shooting unit. The reading unit can, for example, import a video file into the app. The reading unit can also download and read a video from cloud storage. Step 3: The analysis unit uses AI to analyze the video loaded by the loading unit and identify the position and movement of the user's body. The analysis unit, for example, analyzes each frame in the video and identifies the position and movement of each part of the user's body. For example, in the case of a swing in a racket sport, the position of the racket, the angle of the swing, the position of the feet, etc. The analysis unit can analyze each frame in the video and identify the position and movement of each part of the user's body using, for example, deep learning technology. Step 4: The comparison unit compares the data identified by the analysis unit with the data of the professional player. For example, the comparison unit refers to a database of professional players and compares it with the user's motion. For example, the comparison unit compares the swing of the professional player with the swing of the user and identifies which parts are different. For example, the comparison unit can use machine learning technology to compare the data of the professional player with the data of the user. Step 5: The feedback unit provides real-time feedback based on the analysis results obtained by the comparison unit. For example, the feedback unit presents the user with specific improvements and advises how to correct their form. For example, advice such as "You should raise the angle of the racket a little more" or "You will be more stable if you spread your feet out a little more" is provided. The feedback unit can present the user with specific improvements using, for example, natural language processing technology.

[0122] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0123] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image (e.g., still image data or video data) are input. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt, and outputs the inference result in one or more data formats of voice data, text data, and image data. The data generation model 58 includes, for example, a text generation AI, an image generation AI, and a multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing while using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction, and in this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. In the data processing device 12 and the like, the data generation model 58 includes a plurality of types, and the data generation model 58 includes an AI other than the generation AI. The AI ​​other than the generation AI is, for example, a linear regression, a logistic regression, a decision tree, a random forest, a support vector machine (SVM), a k-means clustering, a convolutional neural network (CNN), a recurrent neural network (RNN), a generative adversarial network (GAN), or a naive Bayes, and can perform various processes, but is not limited to such examples. The AI ​​may also be an AI agent. In addition, when the processes of each of the above-mentioned parts are performed by an AI, the processes are performed in part or in whole by the AI, but are not limited to such examples.In addition, processing performed by AI including the generation AI may be replaced with rule-based processing, and rule-based processing may be replaced with processing performed by AI including the generation AI.

[0124] Furthermore, the above-described processing by the data processing system 10 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart device 14, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart device 14. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the smart device 14 or an external device, and the smart device 14 acquires or collects information required for processing from the data processing device 12 or an external device.

[0125] Each of the multiple elements including the above-mentioned photographing unit, reading unit, analysis unit, comparison unit, and feedback unit is realized, for example, in at least one of the smart device 14 and the data processing device 12. For example, the photographing unit photographs a sports video of the user using the camera 42 of the smart device 14. The reading unit imports the video into the application by the control unit 46A of the smart device 14. The analysis unit analyzes the video using AI by the specific processing unit 290 of the data processing device 12 to identify the position and movement of the user's body. The comparison unit compares the data of the professional player with the data of the user by the specific processing unit 290 of the data processing device 12. The feedback unit provides feedback in real time based on the analysis result by the specific processing unit 290 of the data processing device 12. The correspondence between each unit and the device or control unit is not limited to the above-mentioned example, and various changes are possible.

[0126] [Second embodiment] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0127] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0128] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0129] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0130] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0131] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user's surroundings (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[0132] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0133] Fig. 4 shows an example of main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0134] The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0135] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the emotion of the user using the emotion identification model 59 and perform identification processing using the user's emotion. In the emotion estimation function (emotion identification function) using the emotion identification model 59, various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, are performed, but are not limited to such examples. In addition, the estimation and prediction of the emotion also include, for example, analysis of the emotion.

[0136] In the smart glasses 214, the specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific processing program 60 executed on the RAM 48. Note that the smart glasses 214 have a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the specific processing unit 290 using these models.

[0137] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (such as a prediction result) using the data generation model 58 by communicating with the server device having the data generation model 58. In addition, the data processing device 12 may be a server device, or may be a terminal device (such as a mobile phone, a robot, or a home appliance) owned by a user.

[0138] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0139] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image (e.g., still image data or video data) is input. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt, and outputs the inference result in one or more data formats of voice data, text data, and image data. The data generation model 58 includes, for example, a text generation AI, an image generation AI, and a multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing while using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction, and in this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. In the data processing device 12 and the like, the data generation model 58 includes a plurality of types, and the data generation model 58 includes an AI other than the generation AI. The AI ​​other than the generation AI is, for example, a linear regression, a logistic regression, a decision tree, a random forest, a support vector machine (SVM), a k-means clustering, a convolutional neural network (CNN), a recurrent neural network (RNN), a generative adversarial network (GAN), or a naive Bayes, and can perform various processes, but is not limited to such examples. The AI ​​may be an AI agent. In addition, when the processes of each part described above are performed by an AI, the processes are performed in part or in whole by the AI, but are not limited to such examples. In addition, the processes performed by the AI ​​including the generation AI may be replaced with rule-based processes, and the rule-based processes may be replaced with processes performed by the AI ​​including the generation AI.

[0140] The data processing system 210 according to the second embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 210 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart glasses 214, but may be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart glasses 214. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the smart glasses 214 or an external device, and the smart glasses 214 acquires or collects information required for processing from the data processing device 12 or an external device.

[0141] Each of the multiple elements including the above-mentioned photographing unit, reading unit, analysis unit, comparison unit, and feedback unit is realized, for example, by at least one of the smart glasses 214 and the data processing device 12. For example, the photographing unit photographs a sports video of the user using the camera 42 of the smart glasses 214. The reading unit imports the video into the application by the control unit 46A of the smart glasses 214. The analysis unit analyzes the video using AI by the specific processing unit 290 of the data processing device 12 to identify the position and movement of the user's body. The comparison unit compares the data of the professional player with the data of the user by the specific processing unit 290 of the data processing device 12. The feedback unit provides feedback in real time based on the analysis result by the specific processing unit 290 of the data processing device 12. The correspondence between each unit and the device or control unit is not limited to the above-mentioned example, and various changes are possible.

[0142] [Third embodiment] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0143] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0144] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0145] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0146] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0147] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user's surroundings (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[0148] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0149] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0150] The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0151] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the emotion of the user using the emotion identification model 59 and perform identification processing using the user's emotion. In the emotion estimation function (emotion identification function) using the emotion identification model 59, various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, are performed, but are not limited to such examples. In addition, the estimation and prediction of the emotion also include, for example, analysis of the emotion.

[0152] In the headset type terminal 314, the specific processing is performed by the processor 46. The storage 50 stores a specific program 60. The processor 46 reads the specific program 60 from the storage 50, and executes the read specific program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific program 60 executed on the RAM 48. Note that the headset type terminal 314 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the identification processing unit 290 using these models.

[0153] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (such as a prediction result) using the data generation model 58 by communicating with the server device having the data generation model 58. In addition, the data processing device 12 may be a server device, or may be a terminal device (such as a mobile phone, a robot, or a home appliance) owned by a user.

[0154] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input for the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0155] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image (e.g., still image data or video data) is input. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt, and outputs the inference result in one or more data formats of voice data, text data, and image data. The data generation model 58 includes, for example, a text generation AI, an image generation AI, and a multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing while using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction, and in this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. In the data processing device 12 and the like, the data generation model 58 includes a plurality of types, and the data generation model 58 includes an AI other than the generation AI. The AI ​​other than the generation AI is, for example, a linear regression, a logistic regression, a decision tree, a random forest, a support vector machine (SVM), a k-means clustering, a convolutional neural network (CNN), a recurrent neural network (RNN), a generative adversarial network (GAN), or a naive Bayes, and can perform various processes, but is not limited to such examples. The AI ​​may be an AI agent. In addition, when the processes of each part described above are performed by an AI, the processes are performed in part or in whole by the AI, but are not limited to such examples. In addition, the processes performed by the AI ​​including the generation AI may be replaced with rule-based processes, and the rule-based processes may be replaced with processes performed by the AI ​​including the generation AI.

[0156] The data processing system 310 according to the third embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 310 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the headset type terminal 314, but may be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the headset type terminal 314. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the headset type terminal 314 or an external device, and the headset type terminal 314 acquires or collects information required for processing from the data processing device 12 or an external device.

[0157] Each of the multiple elements including the above-mentioned photographing unit, reading unit, analysis unit, comparison unit, and feedback unit is realized, for example, in at least one of the headset type terminal 314 and the data processing device 12. For example, the photographing unit photographs a sports video of the user using the camera 42 of the headset type terminal 314. The reading unit imports the video into the application by the control unit 46A of the headset type terminal 314. The analysis unit analyzes the video using AI by the specific processing unit 290 of the data processing device 12 to identify the position and movement of the user's body. The comparison unit compares the data of the professional player with the data of the user by the specific processing unit 290 of the data processing device 12. The feedback unit provides feedback in real time based on the analysis result by the specific processing unit 290 of the data processing device 12. The correspondence between each unit and the device or control unit is not limited to the above-mentioned example, and various changes are possible.

[0158] [Fourth embodiment] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[0159] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0160] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0161] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[0162] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0163] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS image sensor or a CCD image sensor, and captures images of the user's surroundings (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of a typical healthy person).

[0164] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0165] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and legs. The posture and behavior of the robot 414 are controlled by controlling the motors of the arms, hands, and legs. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[0166] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0167] The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0168] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the emotion of the user using the emotion identification model 59 and perform identification processing using the user's emotion. In the emotion estimation function (emotion identification function) using the emotion identification model 59, various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, are performed, but are not limited to such examples. In addition, the estimation and prediction of the emotion also include, for example, analysis of the emotion.

[0169] In the robot 414, the specific processing is performed by the processor 46. The storage 50 stores a specific program 60. The processor 46 reads out the specific program 60 from the storage 50, and executes the specific program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific program 60 executed on the RAM 48. The robot 414 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the specific processing unit 290 using these models.

[0170] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (such as a prediction result) using the data generation model 58 by communicating with the server device having the data generation model 58. In addition, the data processing device 12 may be a server device, or may be a terminal device (such as a mobile phone, a robot, or a home appliance) owned by a user.

[0171] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0172] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image (e.g., still image data or video data) is input. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt, and outputs the inference result in one or more data formats of voice data, text data, and image data. The data generation model 58 includes, for example, a text generation AI, an image generation AI, and a multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing while using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction, and in this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. In the data processing device 12 and the like, the data generation model 58 includes a plurality of types, and the data generation model 58 includes an AI other than the generation AI. The AI ​​other than the generation AI is, for example, a linear regression, a logistic regression, a decision tree, a random forest, a support vector machine (SVM), a k-means clustering, a convolutional neural network (CNN), a recurrent neural network (RNN), a generative adversarial network (GAN), or a naive Bayes, and can perform various processes, but is not limited to such examples. The AI ​​may be an AI agent. In addition, when the processes of each part described above are performed by an AI, the processes are performed in part or in whole by the AI, but are not limited to such examples. In addition, the processes performed by the AI ​​including the generation AI may be replaced with rule-based processes, and the rule-based processes may be replaced with processes performed by the AI ​​including the generation AI.

[0173] The data processing system 410 according to the fourth embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 410 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the robot 414, but may be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the robot 414. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the robot 414 or an external device, etc., and the robot 414 acquires or collects information required for processing from the data processing device 12 or an external device, etc.

[0174] Each of the multiple elements including the above-mentioned photographing unit, reading unit, analysis unit, comparison unit, and feedback unit is realized, for example, by at least one of the robot 414 and the data processing device 12. For example, the photographing unit photographs a sports video of the user using the camera 42 of the robot 414. The reading unit imports the video into the application by the control unit 46A of the robot 414. The analysis unit analyzes the video using AI by the specific processing unit 290 of the data processing device 12 to identify the position and movement of the user's body. The comparison unit compares the data of the professional player with the data of the user by the specific processing unit 290 of the data processing device 12. The feedback unit provides feedback in real time based on the analysis result by the specific processing unit 290 of the data processing device 12. The correspondence between each unit and the device or control unit is not limited to the above-mentioned example, and various changes are possible.

[0175] The emotion identification model 59 as an emotion engine may determine the emotion of the user according to a specific mapping. Specifically, the emotion identification model 59 may determine the emotion of the user according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the emotion of the robot, and the identification processing unit 290 may perform identification processing using the emotion of the robot.

[0176] FIG. 9 is a diagram showing an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive emotions are arranged. The more outside the concentric circles, the more emotions that represent states and actions that arise from a state of mind are arranged. Emotions are a concept that includes emotions and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions that occur in the brain are arranged. On the right side of the concentric circles, emotions that are generally induced by situational judgment are arranged. On the upper and lower sides of the concentric circles, emotions that are generally generated from reactions that occur in the brain and are induced by situational judgment are arranged. In addition, on the upper side of the concentric circles, emotions of "pleasure" are arranged, and on the lower side, emotions of "discomfort" are arranged. In this way, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[0177] These emotions are distributed in the 3 o'clock direction of emotion map 400 and usually fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[0178] The inside of emotion map 400 represents what is going on inside one's mind, and the outside of emotion map 400 represents behavior, so the further out you go on emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[0179] Here, human emotions are based on various balances such as posture and blood sugar level, and when these balances are far from the ideal, it indicates an unpleasant state, and when they are close to the ideal, it indicates a pleasant state. Emotions can also be created for robots, cars, motorcycles, etc., based on various balances such as posture and battery level, so that when these balances are far from the ideal, it indicates an unpleasant state, and when they are close to the ideal, it indicates a pleasant state. The emotion map may be generated, for example, based on the emotion map of Dr. Mitsuyoshi (Research on speech emotion recognition and emotion brain physiological signal analysis system, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). On the left half of the emotion map, emotions belonging to an area called "reaction" where sensation is dominant are lined up. On the right half of the emotion map, emotions belonging to an area called "situation" where situation recognition is dominant are lined up.

[0180] The emotion map defines two emotions that promote learning. The first is the negative emotion around the middle of "repentance" or "remorse" on the situation side. In other words, this is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the positive emotion around "desire" on the response side. In other words, this is when the robot has positive feelings such as "I want more" or "I want to know more."

[0181] The emotion identification model 59 inputs the user input to a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the emotion of the user. This neural network is pre-trained based on multiple learning data that are combinations of the user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in Fig. 10. Fig. 10 shows an example in which multiple emotions, "relief," "calm," and "encouraging," have similar emotion values.

[0182] In the above embodiment, an example was given in which a specific process is performed by one computer 22, but the technology disclosed herein is not limited to this, and distributed processing of the specific process may be performed by multiple computers, including computer 22.

[0183] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable computer-readable non-transitory storage medium such as a Universal Serial Bus (USB) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[0184] In addition, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 upon request from the data processing device 12.

[0185] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[0186] As the hardware resource for executing the specific process, various processors as shown below can be used. An example of the processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing the specific process by executing software, i.e., a program. Another example of the processor is a dedicated electric circuit, which is a processor having a circuit configuration designed exclusively for executing a specific process, such as a Field-Programmable Gate Array (FPGA), a Programmable Logic Device (PLD), or an Application Specific Integrated Circuit (ASIC). Each processor has a built-in or connected memory, and each processor executes the specific process by using the memory.

[0187] The hardware resource that executes the specific process may be one of these various processors, or may be a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific process may be a single processor.

[0188] As an example of a configuration using one processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a configuration using a processor that realizes the functions of the entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[0189] Furthermore, more specifically, the hardware structure of these various processors can be an electric circuit that combines circuit elements such as semiconductor elements. The specific processes described above are merely examples. It goes without saying that unnecessary steps may be deleted, new steps may be added, or the order of processes may be changed without departing from the spirit of the invention.

[0190] In the above example, the first to fourth embodiments are described separately, but some or all of these embodiments may be combined. The smart device 14, the smart glasses 214, the headset terminal 314, and the robot 414 are only examples, and they may be combined with each other, or may be other devices. In the above example, the first and second embodiments are described separately, but they may be combined.

[0191] The above description and illustrations are detailed descriptions of the parts related to the technology of the present disclosure, and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, function, action, and effect is an example of the configuration, function, action, and effect of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replaced with respect to the description and illustrations shown above, within the scope of the gist of the technology of the present disclosure. In addition, in order to avoid confusion and to facilitate understanding of the parts related to the technology of the present disclosure, the description and illustrations shown above omit explanations of technical common sense that do not require particular explanation in enabling the implementation of the technology of the present disclosure.

[0192] All publications, patent applications, and standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, and standard was specifically and individually indicated to be incorporated by reference.

[0193] (Appendix 1) A shooting unit that shoots a sports video of a user; A reading unit that reads the video captured by the imaging unit; an analysis unit that analyzes the video read by the reading unit and identifies a body position or movement of a user; A comparison unit that compares the data identified by the analysis unit with data of professional athletes; A feedback unit that provides feedback in real time based on the analysis result obtained by the comparison unit. A system characterized by: (Appendix 2) The analysis unit is Identifying the user's body position and movements 2. The system of claim 1, (Appendix 3) The comparison unit is Browse our database of professional players 2. The system of claim 1, (Appendix 4) The feedback unit is Present specific improvements 2. The system of claim 1, (Appendix 5) The imaging unit includes: 2. The system of claim 1, characterized by a method for estimating a user's emotion and adjusting the timing of photographing based on the estimated user's emotion. (Appendix 6) The imaging unit includes: When taking a photo, the system analyzes the user's past shooting history and automatically sets the optimal shooting angle and distance. 2. The system of claim 1, (Appendix 7) The imaging unit includes: When shooting, the system tracks the user's movements in real time and automatically selects the optimal frame. 2. The system of claim 1, (Appendix 8) The imaging unit includes: A user's emotion is estimated, and a priority order for photographing actions is determined based on the estimated user's emotion. 2. The system of claim 1, (Appendix 9) The imaging unit includes: When taking a photo, the system suggests the optimal shooting environment based on the user's geographic location information. 2. The system of claim 1, (Appendix 10) The imaging unit includes: When taking a photo, the system analyzes the user's social media activity and prioritizes capturing relevant actions. 2. The system of claim 1, (Appendix 11) The reading unit is Infer user emotions and adjust video loading speed based on the estimated user emotions 2. The system of claim 1, (Appendix 12) The reading unit is Automatically evaluates video quality upon loading and chooses the best analysis method 2. The system of claim 1, (Appendix 13) The reading unit is During loading, apply different loading algorithms depending on the video content 2. The system of claim 1, (Appendix 14) The reading unit is Infer user emotions and prioritize videos to load based on the estimated user emotions 2. The system of claim 1, (Appendix 15) The reading unit is During import, we use the best import settings based on where your video was shot 2. The system of claim 1, (Appendix 16) The reading unit is At load time, we look at the video's associated metadata to improve loading accuracy. 2. The system of claim 1, (Appendix 17) The analysis unit is Infer user emotions and adjust the level of analysis detail based on the estimated user emotions 2. The system of claim 1, (Appendix 18) The analysis unit is During analysis, the analysis algorithm is optimized by referring to the user's past behavior data. 2. The system of claim 1, (Appendix 19) The analysis unit is During analysis, different analysis methods are applied depending on the category of behavior 2. The system of claim 1, (Appendix 20) The analysis unit is Infer user emotions and adjust the way analysis results are displayed based on the inferred user emotions 2. The system of claim 1, (Appendix 21) The analysis unit is During analysis, the accuracy of the analysis is improved by taking into account the user's physical characteristics. 2. The system of claim 1, (Appendix 22) The analysis unit is During analysis, relevant sports science data is referenced to improve the accuracy of the analysis. 2. The system of claim 1, (Appendix 23) The comparison unit is Inferring a user's emotions and adjusting the comparison criteria based on the estimated user's emotions 2. The system of claim 1, (Appendix 24) The comparison unit is Dynamically update your professional player database to use the latest data when making comparisons 2. The system of claim 1, (Appendix 25) The comparison unit is When comparing, pinpoint the differences between your movements and those of professional athletes 2. The system of claim 1, (Appendix 26) The comparison unit is Inferring user emotion and adjusting the way comparison results are displayed based on the estimated user emotion 2. The system of claim 1, (Appendix 27) The comparison unit is When comparing, the accuracy of the comparison is improved by taking into account the user's sports experience. 2. The system of claim 1, (Appendix 28) The comparison unit is When comparing, we also refer to the motion data of professional athletes from different sports categories to compare. 2. The system of claim 1, (Appendix 29) The feedback unit is Inferring user emotions and adjusting feedback based on the estimated user emotions 2. The system of claim 1, (Appendix 30) The feedback unit is When providing feedback, provide optimal advice by referring to the user's past feedback history 2. The system of claim 1, (Appendix 31) The feedback unit is When providing feedback, identify areas for improvement in user behavior in detail and provide specific methods for improvement 2. The system of claim 1, (Appendix 32) The feedback unit is Estimate user emotions and prioritize feedback based on the estimated user emotions 2. The system of claim 1, (Appendix 33) The feedback unit is When providing feedback, the geographic location of the user is taken into account to provide optimal advice 2. The system of claim 1, (Appendix 34) The feedback unit is Analyze users' social media activity to provide relevant advice upon feedback 2. The system of claim 1, [Explanation of symbols]

[0194] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot

Claims

1. A shooting unit that shoots a sports video of a user; A reading unit that reads the video captured by the imaging unit; an analysis unit that analyzes the video read by the reading unit and identifies a body position or movement of a user; A comparison unit that compares the data identified by the analysis unit with data of professional athletes; A feedback unit that provides feedback in real time based on the analysis result obtained by the comparison unit. A system characterized by:

2. The analysis unit is Identifying the user's body position and movements 2. The system of claim 1 .

3. The comparison unit is Browse our database of professional players 2. The system of claim 1 .

4. The feedback unit is Present specific improvements 2. The system of claim 1 .

5. The imaging unit includes:

2. The system of claim 1, further comprising a method for estimating a user's emotion and adjusting timing of capture based on the estimated user's emotion.

6. The imaging unit includes: When taking a photo, the system analyzes the user's past shooting history and automatically sets the optimal shooting angle and distance.

2. The system of claim 1 .

7. The imaging unit includes: When shooting, the system tracks the user's movements in real time and automatically selects the optimal frame.

2. The system of claim 1 .

8. The imaging unit includes: A user's emotion is estimated, and a priority order for photographing actions is determined based on the estimated user's emotion.

2. The system of claim 1 .

Images