Information processing system

Abstract

The present invention provides an information processing system comprising: an input means for inputting a training target and a motion type of a user; the generating device is used for receiving information input by the input device and generating a training plan; the shooting and uploading device is used for shooting a training video by a user and uploading the training video; the analysis device is used for analyzing the uploaded training video and determining an action error and a correction key point; and the feedback providing device is used for providing the determined correction key points and the specific improvement method to the user.

Description

Technical Field

[0001] The technology disclosed herein relates to an information processing system. Background Technology

[0002] Japanese Patent Application Publication No. 2022-180282 discloses a method for controlling a role-based chatbot executed by at least one processor. The method includes the following steps: receiving a user's speech; adding the user's speech to a prompt word, the prompt word containing instruction statements associated with an explanation of the chatbot's role; encoding the prompt word; and inputting the encoded prompt word into a language model to generate a chatbot response to the user's speech.

[0003] Existing sports training methods often fail to provide users with personalized and targeted training guidance, especially in the absence of professional coaches. This makes it difficult to identify and effectively improve problems in their movements. Consequently, improvements in athletic performance are limited, training efficiency and results are not maximized, and the limited feedback channels during training hinder continuous optimization of training outcomes. Summary of the Invention

[0004] To address the aforementioned problems, this invention provides an information processing system comprising: an input device for inputting a user's training goals and exercise type; a generation device for receiving the input information and generating a training plan; a shooting and uploading device for enabling the user to shoot and upload training videos; an analysis device for analyzing the uploaded training videos, identifying movement errors and key correction points; and a feedback providing device for providing feedback on correction points and specific improvement methods to the user. Through these means, automated and personalized analysis and guidance of the user's training process can be achieved, thereby improving training effectiveness and athletic performance.

[0005] "Input device" refers to an information collection device or module used to receive user input of training objectives and types of exercise.

[0006] "Generation device" refers to a device or module that generates a targeted training plan based on information input by the user.

[0007] "Shooting and uploading device" refers to a device or module used to shoot user training videos and upload the videos to the system server.

[0008] "Analysis device" refers to a device or module that analyzes uploaded training videos to identify user action errors and correction points.

[0009] "Feedback provider" refers to a device or module that outputs the key points of correction and specific improvement methods obtained from the analysis to the user.

[0010] "Training goal" refers to the specific athletic performance or skill level that a user hopes to achieve through training.

[0011] "Sports type" refers to the specific sports activity that the user chooses to participate in training.

[0012] A "training plan" refers to a customized program for a user, based on user input and professional reference materials, which includes training content, frequency, and objectives.

[0013] "Training video" refers to a dynamic video file that records the user's movements during the training process.

[0014] "Motion error" refers to the deviation or deficiency between the user's trained movements and the professional standard movements.

[0015] "Correction points" refer to the specific action details that need to be prioritized and adjusted in order to compensate for action errors.

[0016] "Specific improvement methods" refers to actionable adjustment suggestions or guidance provided to users based on motion errors and key correction points. Attached Figure Description

[0017] Figure 1 This is a conceptual diagram illustrating an example of the configuration of the data processing system according to the first embodiment.

[0018] Figure 2 This is a conceptual diagram illustrating an example of the main functions of the data processing apparatus and smart device according to the first embodiment.

[0019] Figure 3 This is a conceptual diagram illustrating an example of the configuration of the data processing system according to the second embodiment.

[0020] Figure 4 This is a conceptual diagram illustrating an example of the main functions of the data processing device and smart glasses according to the second embodiment.

[0021] Figure 5 This is a conceptual diagram illustrating an example of the configuration of the data processing system according to the third embodiment.

[0022] Figure 6 This is a conceptual diagram illustrating an example of the main functions of the data processing apparatus and head-mounted terminal according to the third embodiment.

[0023] Figure 7 This is a conceptual diagram illustrating an example of the configuration of the data processing system according to the fourth embodiment.

[0024] Figure 8 This is a conceptual diagram illustrating an example of the main functions of the data processing device and robot according to the fourth embodiment.

[0025] Figure 9 This represents an emotion map that maps multiple emotions.

[0026] Figure 10 This represents an emotion map that maps multiple emotions.

[0027] Figure 11 This is a sequence diagram illustrating the processing flow of the data processing system of the first embodiment.

[0028] Figure 12 This is a sequence diagram illustrating the processing flow of the data processing system in Application Example 1.

[0029] Figure 13 This is a sequence diagram illustrating the processing flow of the data processing system of the second embodiment.

[0030] Figure 14 This is a sequence diagram illustrating the processing flow of the data processing system in Application Example 2. Detailed Implementation

[0031] Hereinafter, an example of an implementation of the system according to the present disclosure will be described with reference to the accompanying drawings.

[0032] First, let me explain the terminology used in the following instructions.

[0033] In the following embodiments, the processor (hereinafter referred to as "processor") with reference numerals may be a single computing device or a combination of multiple computing devices. Furthermore, the processor may be a single computing device or a combination of multiple computing devices. Examples of computing devices include CPU (Central Processing Unit), GPU (Graphics Processing Unit), GPGPU (General-Purpose computing on Graphics Processing Units), APU (Accelerated Processing Unit), etc.

[0034] In the following embodiments, RAM (Random Access Memory), as indicated in the figures, is a memory that temporarily stores information and is used as working memory by the processor.

[0035] In the following embodiments, the memory, as indicated by the reference numerals, is one or more non-volatile storage devices that store various programs and parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), disks (e.g., hard disks), or magnetic tapes.

[0036] In the following embodiments, the communication I / F (Interface) with reference numerals is an interface that includes a communication processor and an antenna, etc. The communication I / F is responsible for communication between multiple computers. As an example of a communication specification applicable to the communication I / F, wireless communication specifications such as 5G (5th Generation Mobile Communication System), Wi-Fi (wireless fidelity) (registered trademark), or Bluetooth (registered trademark) can be listed.

[0037] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B". That is, "A and / or B" means that it can be only A, only B, or a combination of A and B. Furthermore, in this specification, when "and / or" connects to express more than three items, the same interpretation as "A and / or B" applies.

[0038] First Implementation Method

[0039] Figure 1 An example of the configuration of the data processing system 10 according to the first embodiment is shown.

[0040] like Figure 1 As shown, the data processing system 10 includes a data processing device 12 and an intelligent device 14. A server can be cited as an example of the data processing device 12.

[0041] The data processing apparatus 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" as understood in this disclosure. The computer 22 includes a processor 28, RAM 30, and memory 32. The processor 28, RAM 30, and memory 32 are connected to a bus 34. Furthermore, the database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0042] The smart device 14 includes a computer 36, a receiving device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, RAM 48, and memory 50. The processor 46, RAM 48, and memory 50 are connected to a bus 52. In addition, the receiving device 38, output device 40, camera 42, and communication I / F 44 are also connected to the bus 52.

[0043] The receiving device 38 includes a touchscreen 38A and a microphone 38B, and receives user input. The touchscreen 38A receives user input via touch by detecting contact with an indicator (e.g., a pen or finger). The microphone 38B receives user input via sound by detecting the user's voice. The control unit 46A in the processor 46 sends data representing the user input received by the touchscreen 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data representing the user input.

[0044] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting data in a form perceptible to the user 20 (e.g., sound and / or text). The display 40A displays visual information such as text and images according to instructions from the processor 46. The speaker 40B outputs sound according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0045] Communication I / F44 is connected to network 54. Communication I / F44 and 26 are responsible for sending and receiving various information between processor 46 and processor 28 via network 54.

[0046] Figure 2 The diagram shows an example of the main functions of the data processing device 12 and the smart device 14.

[0047] like Figure 2 As shown, in the data processing apparatus 12, specific processing is performed by the processor 28. A specific processing program 56 is stored in the memory 32. The specific processing program 56 is an example of a "program" as understood in this disclosure. The processor 28 reads the specific processing program 56 from the memory 32 and executes the read specific processing program 56 on the RAM 30. Specific processing is implemented by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0048] The memory 32 stores a data generation model 58 and an emotion-specific model 59. The data generation model 58 and the emotion-specific model 59 are used by the specific processing unit 290. The specific processing unit 290 can use the emotion-specific model 59 to infer the user's emotions and perform specific processing based on the user's emotions. In the emotion inference function (emotion-specific function) using the emotion-specific model 59, various inferences and predictions related to the user's emotions are performed, including inferences and predictions of the user's emotions, but this is not limited to this example. Furthermore, emotion inference and prediction may also include, for example, emotion analysis (parsing).

[0049] In the smart device 14, the processor 46 performs the acceptance output processing. The memory 50 stores the acceptance output program 60. The acceptance output program 60 is used in conjunction with the data processing system 10 and the specific processing program 56. The processor 46 reads the acceptance output program 60 from the memory 50 and executes the read acceptance output program 60 on the RAM 48. The acceptance output processing is implemented by the processor 46 acting as the control unit 46A according to the acceptance output program 60 executed on the RAM 48. Furthermore, the smart device 14 has the same data generation model and emotion-specific model as the data generation model 58 and the emotion-specific model 59, and these models can also be used to perform the same processing as the specific processing unit 290. The acceptance output processing is implemented by the processor 46 acting as the control unit 46A according to the acceptance output program 60 executed on the RAM 48.

[0050] Alternatively, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device (e.g., a generation server) may have the data generation model 58. In this case, the data processing device 12 obtains the processing results (prediction results, etc.) using the data generation model 58 by communicating with the server device that has the data generation model 58. Furthermore, the data processing device 12 may be a server device or a user-held terminal device (e.g., a mobile phone, robot, home appliance, etc.). Next, an example of the processing of the data processing system 10 of the first embodiment will be described.

[0051] Example 1

[0052] The flow of a specific process in Example 1 will be described. Each part of the system described below is implemented by the data processing device 12 and the smart device 14. Furthermore, the data processing device 12 is referred to as the "server," and the smart device 14 is referred to as the "terminal."

[0053] Existing sports training assistance systems suffer from insufficient accuracy and relevance in analyzing user training movements and providing personalized training plans. Traditional systems struggle to intelligently generate optimal training plans based on each user's goals and sport type, and they cannot efficiently analyze user-uploaded training videos to accurately identify movement errors and provide targeted improvement suggestions, making it difficult for users to continuously and effectively improve their athletic performance.

[0054] The specific processing performed by the specific processing unit 290 of the data processing apparatus 12 in Embodiment 1 is achieved by the following means.

[0055] In this invention, the server includes an information acquisition device for obtaining the user's training goals and type of exercise; a plan generation device for generating instructive text input to a generative artificial intelligence model based on the aforementioned information and thereby generating a personalized training plan; a recording and sending device and a parsing device for receiving and analyzing the user's training videos; and a prompting device for pushing correction points and specific improvement methods to the user. This allows for the automatic generation of scientifically sound training plans based on the user's individual needs. Furthermore, by analyzing the user's training movements using artificial intelligence, the system accurately locates errors and correction points in the movements, providing the user with highly accurate and personalized feedback, thereby significantly improving the efficiency and effectiveness of sports training.

[0056] "Information acquisition device" refers to a device used to collect or receive user input of training objectives and types of exercise, including but not limited to electronic devices with human-computer interaction interfaces.

[0057] A “plan generation device” refers to a device that automatically generates utterance instructions based on information provided by the user, which are then input into a generative artificial intelligence model, and the generative artificial intelligence model generates a personalized training plan accordingly.

[0058] "Generative AI models" refer to AI models that can automatically generate personalized training plans and related suggestions based on input instructional text, including language models, knowledge bases, etc.

[0059] "Recording and transmitting device" refers to a device used to record a user's training process video and upload the video file to an information processing device, including the camera component and communication component of the terminal device.

[0060] A “analysis device” is a device that performs frame-by-frame analysis on a received video file and compares the analysis results with standard motion information to determine motion errors and correction points.

[0061] "Standard movement information" refers to the parametric description of professional sports movements stored in a database for comparison with user movement data.

[0062] A "prompt device" is a device used to output or present feedback information such as the corrected parts and specific improvement methods to the user. It can be a software module or display component of a terminal device.

[0063] "Personalized training plans" refer to customized training schedules and content arrangements automatically generated by artificial intelligence models based on each user's individual needs, such as exercise goals and physical conditions.

[0064] "Motion error" refers to the deviation between the user's actual action and the standard action information, including incorrect posture, unreasonable timing, and other phenomena.

[0065] "Correction points" refer to specific parts or key elements in user actions that require targeted improvement, used to guide users in optimizing training performance.

[0066] This invention relates to an information processing system that utilizes generative artificial intelligence models and prompts to provide personalized guidance for sports training. The system involves collaboration between a server, a terminal, and the user, combining modern artificial intelligence technology to achieve efficient, automated, and precise assistance in sports training.

[0067] The server can utilize general-purpose computer hardware with high-performance computing capabilities, such as physical servers based on x86 or ARM architectures, and the operating system can be Linux. The software environment includes a Python runtime environment, deep learning frameworks (such as TensorFlow and PyTorch), open-source video analysis tools (such as OpenCV and MediaPipe), and generative artificial intelligence models (such as APIs based on large language models or self-built models). The server is deployed with relational or non-relational databases (such as PostgreSQL and MongoDB) to store user data, standard action parameters, professional knowledge data, etc.

[0068] The terminal includes internet-connected mobile smart devices such as smartphones and tablets, with operating systems such as Android or iOS. A dedicated app is installed on the terminal, providing functions such as data entry, video recording, information display, and communication. The hardware integrates a camera, display screen, and data communication module.

[0069] Users first input the type of sport and their personal training goals through the terminal's application interface. The terminal then sends the collected input information to the server in real time via secure protocols such as HTTPS. The server receives the data through an information acquisition device and, based on a professional database and sports theory, automatically generates prompts for the input-generative artificial intelligence model using algorithms. An example of a prompt is as follows:

[0070] Please generate a training plan for a football player to improve free-kick accuracy. The user's goal is to achieve high-quality free-kick shots. Please output the following: 1. Daily training schedule; 2. Specific training content and technical points; 3. Target indicators for each stage.

[0071] The server inputs the aforementioned prompts into the deployed generative AI model, which then outputs a personalized training plan, including a schedule, training content, and key technical points. This training plan is transmitted back to the terminal via the server and displayed to the user on the screen in a clear card or list format.

[0072] Users follow a training plan and use the terminal's camera to record videos of their training process (e.g., the entire process of practicing a soccer free kick). The terminal automatically compresses and converts the recorded video files before uploading them to a server. The server uses posture detection software such as OpenPose or MediaPipe to perform frame-level analysis, identifying and parameterizing key nodes of the user's movements (e.g., supporting leg, swinging leg, body center of gravity). By comparing the user's movements with standard movement information in a database, the server automatically determines the user's movement errors and key correction points.

[0073] Based on the analysis results and combined with professional knowledge, the server automatically generates targeted improvement suggestions, such as "Move your supporting foot closer to the ball" and "Lean forward when shooting." This text information is pushed back to the terminal via the server, and users can view the feedback results in the app interface and continuously optimize their training movements.

[0074] Through the above methods, this system can automatically complete integrated functions such as personalized training plan generation, intelligent analysis of the training process, and real-time corrective feedback push, significantly improving the accuracy of sports training and user experience.

[0075] use Figure 11 The processing flow is explained.

[0076] Step 1:

[0077] The user enters their training goals and type of exercise in the dedicated application on the terminal and clicks confirm. The terminal displays an input interface, prompting the user to fill in the goals and items. The input is text data, and the output is the information entered by the user. The terminal performs preliminary formatting processing on the input data (such as removing extra spaces) and packages it into a data packet for uploading to the server.

[0078] Step 2:

[0079] The server receives user training goals and exercise types from the terminal. Combining this with a sports knowledge database, the server uses a data processing module to transform the user's goals and activities into prompts for generative artificial intelligence models. The input is structured user information, and the output is instructional text. During this process, the server extracts and concatenates keywords to generate specific instruction text for subsequent AI plan generation.

[0080] Step 3:

[0081] The server inputs the generated prompts into the deployed generative AI model via a network API. Based on the input instructional text, the AI ​​model leverages its large-scale inference capabilities to generate a detailed training plan that includes a training schedule, practice content, and phased sub-goals. The input is the prompt text, and the output is a personalized training plan. The server performs structured and complete verification on the AI ​​output before returning it to the terminal.

[0082] Step 4:

[0083] The terminal receives the training plan data returned by the server. It then parses and renders the training plan data into a visual interface, displaying information such as a calendar, practice content, and key points for the user to view. The input is training plan text or structured data, and the output is the interface content. The terminal also allows users to click on various training items to display detailed explanations or tutorial links.

[0084] Step 5:

[0085] Users conduct actual training according to the training plan. Simultaneously, users record their training process using the terminal's camera. The terminal automatically utilizes the camera hardware to record a high-definition video file. The input is the user's actual movement scene, and the output is a video file. After video recording is complete, the terminal prompts the user to select the upload option via the interface.

[0086] Step 6:

[0087] The terminal converts and compresses the video file selected by the user, and uploads it to the designated storage location on the server via a secure protocol. The input is the original video file, and the output is a video data packet that the server can parse. During the upload process, the terminal checks the video integrity and network conditions to ensure successful upload.

[0088] Step 7:

[0089] After receiving the training video, the server calls a video analysis module (such as OpenPose or MediaPipe) to parse the video content frame by frame and extract the user's key body point data. The server compares the user's motion parameters with a standard motion database to detect motion differences and errors. The input is a video file, and the output is motion analysis data, motion errors, and correction points. The server performs data extraction, comparison, error determination, and other data operations.

[0090] Step 8:

[0091] Based on motion errors and correction points, and combined with a motion knowledge base, the server uses a text generation module to output personalized improvement suggestions, such as "bring the supporting foot closer to the ball" or "lean the upper body forward." The input is the analysis results and standard knowledge; the output is text feedback. The server packages the feedback content and sends it to the user's terminal.

[0092] Step 9:

[0093] The terminal receives video analysis feedback for this training session and displays it clearly on the application interface, prompting the user to pay attention to relevant details during the next training session. The input is text suggestions pushed by the server, and the output is an interactive interface display and user-readable feedback. The terminal allows users to view historical feedback and track their progress.

[0094] Application Example 1

[0095] The process flow corresponding to the specific processing in Use Case 1 will be described below. The various parts of the system described below are implemented by the data processing device 12 and the intelligent device 14. Furthermore, the data processing device 12 is referred to as the "server" and the intelligent device 14 is referred to as the "terminal".

[0096] Existing training and skill enhancement systems generally suffer from the following problems: they require operators to possess high levels of professional knowledge and skills; the training process relies on manual guidance, resulting in low efficiency, delayed feedback, and difficulty in accurately identifying errors and areas for improvement. Especially in multi-domain skill training scenarios such as robot control and sports skill enhancement, there is a lack of intelligent systems capable of providing personalized, efficient, and continuously optimized feedback incorporating emotional states. Therefore, how to provide an intelligent system that can automatically acquire user goals and training content, automatically analyze movements based on standard examples, intelligently generate training plans and provide personalized dynamic feedback, while simultaneously optimizing feedback based on the user's emotional state, thereby improving training efficiency and skill advancement, has become an urgent technical challenge.

[0097] The specific processing performed by the specific processing unit 290 of the data processing apparatus 12 in Application Example 1 is achieved by the following means.

[0098] In this invention, the server includes a data acquisition device for obtaining user goal achievement and activity content, a plan generation device for generating guidance plans based on a generative artificial intelligence model, an image recording and transmission device for collecting and uploading work image data, a motion analysis device for frame-by-frame analysis and comparison with standard behavior, an information provision device for optimizing personalized feedback by combining an emotion inference model, and an update control device for continuously updating guidance content. This enables automatic data acquisition throughout the training process, automatic generation of training plans, intelligent motion analysis and error correction, precise delivery of personalized feedback, and optimized feedback based on emotion perception, thereby effectively improving the user's skill level and training efficiency, and meeting the needs of intelligent training in multiple fields.

[0099] "Data information acquisition device" refers to an information input device used to collect and receive target, activity content or related basic data input by users, which may include hardware terminals, user interfaces and their management software systems.

[0100] "Plan generation device" refers to an information processing and output module that automatically generates training plans or guidance schemes based on user data by utilizing stored data, professional knowledge information, and generative artificial intelligence models.

[0101] "Image recording and transmission device" refers to a terminal device and related interface that can collect image data of the operation process through a camera device and upload the data to a server through a network.

[0102] "Motion analysis device" refers to an analysis and processing unit that analyzes received image data frame by frame, extracts motion features, combines them with standard motion data, and performs comparison and error determination.

[0103] "Emotion inference model" refers to an algorithm model or system that uses artificial intelligence technology to infer and judge the emotional state of users based on various information such as images, voice, and behavior.

[0104] "Information providing device" refers to a human-computer interaction module that outputs and presents the training feedback, improvement suggestions, and personalized guidance results generated by the system in a way that is suitable for the user.

[0105] "Update control device" refers to a system control unit that dynamically adjusts, automatically optimizes, and continuously refreshes guidance content based on historical data and newly collected data.

[0106] "Generative artificial intelligence models" refer to artificial intelligence algorithms or neural network models that can automatically generate text, suggestions, plans, or analytical conclusions based on input prompts.

[0107] "Benchmark movements" refer to standardized movement data that are pre-set based on professional knowledge or industry standards in a relevant skill field and used as a reference for movement judgment.

[0108] Personalized feedback refers to suggestions or guidance that adaptively adjust the content and expression of feedback based on different user attributes, behaviors, and emotional states.

[0109] The embodiments of the present invention are described below.

[0110] This system consists of three collaborative components: a server, a terminal, and a user. The server is responsible for data processing, intelligent analysis, and plan generation; the terminal is mainly responsible for data input, image acquisition, and result display; and the user operates the system through the terminal interface and participates in actual training.

[0111] The server is equipped with a high-performance processor, storage devices, and deep learning computing units, and can connect to databases (such as MySQL, NoSQL, etc.) to store user historical data and knowledge bases. The server runs generative artificial intelligence models (such as general large-scale language models, action recognition AI, and emotion recognition neural networks) for natural language understanding, training plan generation, video content parsing, and sentiment analysis. The main software includes a server operating system (such as Linux), an application server (such as Nginx), AI inference frameworks (such as TensorFlow, PyTorch), and a task scheduling and management module.

[0112] The terminal can be a smartphone, tablet, or portable hardware equipped with a camera, running user-interactive applications (such as mobile apps or web-based front-end systems). The terminal supports inputting training targets and acquiring operational images through a UI, and communicates with the server via encrypted network protocols.

[0113] Users first fill in the training objectives and activity content on the terminal application, such as "improve the welding accuracy of robots" or "improve the free kick accuracy in soccer". After the terminal collects this information, it sends it to the server through an encrypted API interface.

[0114] Based on the received user goals and attributes, the server queries the database for relevant standard action data and domain knowledge, and sends a request containing prompts to the generative AI model to automatically generate a personalized training plan. A typical training plan includes daily tasks, key techniques, and achievement standards for each stage.

[0115] Users perform actual operations or training under the guidance of the terminal, while simultaneously recording the entire action process through the terminal's camera. After video acquisition is complete, the terminal application uploads the video file to the server. The server decodes the uploaded video frame by frame, extracts key action features using deep learning action analysis models (such as pose estimation algorithms and motion trajectory recognition algorithms), automatically compares it with a benchmark action, identifies action differences, detects errors, and identifies areas for improvement.

[0116] The server processes the analysis of key improvement points, specific suggestions, and typical comparison screenshots into personalized feedback. If the system is equipped with an emotion recognition module, it simultaneously analyzes the user's facial expressions, voice, and other data in the uploaded images to determine emotional states such as high pressure, frustration, and confidence, and dynamically optimizes the feedback content (such as using encouraging tones and customized suggestions) to promote active user participation and continuous skill improvement.

[0117] After receiving feedback from the server, the terminal displays step-by-step suggestions, improvement examples, dynamic push notifications, and video comparison results in a clear and intuitive manner on the user's training dashboard page. Users adjust their training methods according to the suggestions, repeatedly try new methods, and continuously upload new data, forming an efficient, automated, personalized, and emotion-driven closed-loop training process.

[0118] Specific examples:

[0119] User A wanted to improve the welding accuracy of their robot. After A entered "improve welding accuracy" into the mobile app, the server automatically generated a training plan that included key points of welding torch operation, phased practice tasks, and daily progress tracking. A practiced according to the plan, filmed the process with their phone, and uploaded the video to the server. The server identified issues in A's operation, such as "upper arm movement speed slower than standard" and "significant deviation in starting point," and prompted A to "increase upper arm movement speed and pay attention to wrist initial positioning." If the server detected a tense expression in A's video, it would also send encouraging feedback such as "stay relaxed and complete the movement confidently." By practicing and uploading again, A continuously received targeted suggestions and emotional support, gradually improving their welding skills.

[0120] Here are some examples of prompts for generative artificial intelligence models:

[0121] "Please analyze the robot welding process videos uploaded by users, identify the differences between each step and the standard, and provide detailed and clear feedback to the users."

[0122] "Please identify the user's technical errors during free ball training, provide specific solutions for each area of ​​improvement, and adjust the encouragement method based on the user's facial expressions."

[0123] "Based on the motion characteristics and emotional expression of the user in the uploaded video, we recommend suitable training tasks and positive encouragement statements for the next step."

[0124] Through the above structure and processing flow, the system can automatically, accurately, and continuously optimize users' operational skills and athletic abilities in multiple fields without requiring highly skilled human guidance.

[0125] use Figure 12 The processing flow is explained.

[0126] Step 1:

[0127] Users launch the dedicated application on the terminal, click on the new training entry, and enter the training goal and activity content, such as "improving the welding accuracy of robots" or "improving the accuracy of free kicks in soccer".

[0128] The input consists of text information entered by the user, and the output is a data structure generated locally on the terminal. The terminal performs format validation and standardization on the input information, and then packages the data into JSON format, ready for upload.

[0129] Step 2:

[0130] The terminal uploads the user-input training objectives and activity content as a request body to the server's designated API interface via an encrypted network protocol (such as HTTPS).

[0131] The input is user data in JSON format, and the output is the request packet received by the server, which performs permission verification, data parsing, and database storage. The terminal displays a "Uploading Information" status on the interface.

[0132] Step 3:

[0133] The server parses the received user data, extracts the target, activity type, and user attributes, and retrieves relevant standard action data and professional knowledge information from the database.

[0134] The input consists of the parsed user goal and activity content, and the output consists of standard action data and knowledge base information matching the user request. The server performs data matching and preprocessing to prepare for subsequent AI calls.

[0135] Step 4:

[0136] The server invokes the generative artificial intelligence model, inputs a prompt to generate a training plan into the model, and submits standard action data and user goals.

[0137] The server invokes the natural language generation or action suggestion module on the inference engine. The input is the user's goals and standard data, and the output is personalized training plan text (such as daily task assignments and action point tables). The server writes the results to the database and prepares to send them to the terminal.

[0138] Step 5:

[0139] The server sends the training plan data to the terminal via a push notification API. Upon receiving the data, the terminal displays the training plan content visually on the training dashboard or task page.

[0140] The input is the training plan JSON data pushed by the server, and the output is a visual plan table on the terminal page. The terminal renders the plan locally and displays a "Training plan is ready" message.

[0141] Step 6:

[0142] Users perform exercises in a real-world environment according to the training plan, such as robotic welding or athletic training. Simultaneously, they click "Start Recording" within the terminal application, which uses the camera module to record the user's entire training process. After completing the training, the user clicks "Stop Recording," confirms within the application, and selects to upload the video.

[0143] The input is a video stream captured by the camera, and the output is a video file saved on the terminal. The terminal automatically detects the video size; if it is too large, it compresses it and prepares for uploading.

[0144] Step 7:

[0145] The terminal uploads the video file to the server via a segmented concurrent upload API, displaying a progress bar during the upload process. The server receives all video segments, reassembles them, and stores the file on a dedicated storage server, while also recording the video index and metadata.

[0146] The input is a video file captured by the user, and the output is a complete video file and corresponding metadata stored on the server. The server records information such as the user, video, and upload time.

[0147] Step 8:

[0148] The server decodes the received video files and extracts motion features frame by frame using motion recognition and pose estimation algorithms (such as deep learning models). The server automatically compares these motion features with standard motion data to identify abnormal movements, incorrect positions, speed differences, and other issues.

[0149] The input is the user's training video, and the output is a data structure annotated with action differences and error points. The server generates a detailed analysis report in the background.

[0150] Step 9:

[0151] The server invokes a generative artificial intelligence model to express the action analysis results in natural language, automatically generating detailed improvement suggestions, including specific action correction instructions and example explanations. If an emotion recognition module is available, it further analyzes the user's facial and vocal data to identify the current emotional state and adjusts the feedback tone accordingly (e.g., adding encouraging expressions).

[0152] The input consists of an analysis report and sentiment analysis results; the output is personalized natural language feedback text and, if necessary, illustrations. The server processes this information into a feedback message.

[0153] Step 10:

[0154] The server pushes improvement suggestions, key comparison frame screenshots, personalized feedback, and emotional support messages to the terminal via API. The terminal displays all feedback details in a pop-up window on the training dashboard, allowing users to view action comparisons, suggestion lists, and emotional suggestions.

[0155] The input is feedback information generated by the server, and the output is a mixed text and image feedback page on the terminal. The terminal allows users to click to view details of each suggestion or compare video frames.

[0156] Step 11:

[0157] Users read the feedback carefully, adjust their movements according to the suggestions, and practice them in the next training session, continuously cycling through the training and uploading videos.

[0158] The input is personalized feedback displayed on the terminal, and the output is the user's adjustments and improvements during actual training. Users can decide whether to immediately re-shoot and upload the video based on the suggestions, thus achieving a closed loop for continuous skill improvement.

[0159] Alternatively, an emotion engine for inferring user emotions can be combined. That is, the specific processing unit 290 can also use the emotion-specific model 59 to infer user emotions and perform specific processing using user emotions.

[0160] Example 2

[0161] The flow of a specific process in Example 2 will be described. Each part of the system described below is implemented by the data processing device 12 and the smart device 14. The data processing device 12 will be referred to as the "server," and the smart device 14 as the "terminal."

[0162] The specific processing performed by the specific processing unit 290 of the data processing apparatus 12 in Embodiment 2 is achieved by the following means.

[0163] In this invention, the server includes: an information acquisition device for acquiring user target information and movement type information; an automatic generation device for automatically generating training plans based on the acquired information; an image recording device for recording and uploading user training images; an image analysis device for performing frame-by-frame analysis and standard movement comparison on the training images; an improvement point extraction device and a guidance prompting device for extracting movement errors and generating improvement suggestions; and an emotion recognition and adjustment device for intelligently adjusting the feedback content based on real-time recognition of the user's emotional state using image or voice data. The automatic generation device or guidance prompting device employs a generative artificial intelligence model, controlling the model's interactive content through prompt statements. This enables intelligent training feedback combining motion technique analysis and emotion recognition, dynamically generating personalized training guidance best suited to the user's current skills and psychological state, thereby significantly improving the effectiveness and sustainability of sports training.

[0164] "Information acquisition device" refers to a hardware or software module used to receive or collect training target information and exercise type information input by the user.

[0165] "Automatic generation device" refers to a processing unit that automatically generates personalized training plans based on acquired information and using relevant algorithms or artificial intelligence models.

[0166] "Video recording device" refers to a device used to record and capture images of a user's exercise training process, including hardware or applications with shooting and storage functions.

[0167] "Image analysis device" refers to a hardware or software module that performs frame-by-frame analysis, motion recognition, and comparison with baseline motions on uploaded sports training image data to extract motion feature information.

[0168] The "improvement point extraction device" refers to a processing unit that identifies and extracts motion errors and improvement suggestions from image analysis results.

[0169] "Guidance and prompting device" refers to a hardware or software module that generates and outputs personalized exercise guidance and suggestions to users based on improvement points.

[0170] "Emotion recognition and adjustment device" refers to a processing module that identifies a user's emotional state based on user image or audio data and dynamically adjusts the feedback content according to emotional fluctuations.

[0171] "Generative artificial intelligence models" refer to artificial intelligence algorithms or systems that automatically generate natural language text or other output content based on input prompts.

[0172] A "prompt processing device" refers to a module that takes specific statements as input information and passes them to a generative artificial intelligence model to control its output interactive content.

[0173] "Communication channel" refers to the network channel that enables the transmission of data or images between the terminal and the server, including wired and wireless methods.

[0174] "Benchmark movement pattern" refers to pre-set movement data or models that represent standard or professional levels, used for comparative analysis with user movements.

[0175] This invention provides an intelligent sports training system based on the collaborative operation of a server and a terminal, to achieve personalized and emotionally-oriented sports guidance and feedback for users. The following will describe in detail the specific implementation methods, hardware and software configurations, data processing flow, main technologies used, and practical application examples of this invention.

[0176] This system typically includes a server, terminal devices, and a data communication network connecting the two. Terminal devices, such as smartphones and tablets, use dedicated sports training apps to achieve functions like information acquisition, image recording, and result display. The server is equipped with a high-performance central processing unit, a database management module, generative artificial intelligence models (such as the open-source GPT model or a self-built Chinese sports-specific generative model), emotion recognition algorithms (such as facial expression recognition APIs and deep learning models for speech emotion analysis), image processing hardware (GPU servers), and related software (such as OpenPose, YOLOv7, TensorFlow, etc.).

[0177] In the implementation process, the terminal collects the user's training goals and exercise type, and sends this information to the server via the app. Upon detecting the input, the server first retrieves relevant training data and theoretical knowledge from its database. Then, using a generative artificial intelligence model, the server processes the prompts regarding exercise type and goals, automatically generating a personalized training plan and feeding it back to the terminal. The generative model can dynamically adjust the training plan based on the prompts. For example, the user's input such as "football" or "improve free-kick accuracy" will form the model's input prompts, and the server will automatically output a corresponding three-week training schedule with detailed precautions.

[0178] Users exercise according to the training plan and capture training video using the terminal's camera. The terminal formats the captured video (e.g., compressing it into MP4 or H.264) before uploading it to the system server. Upon receiving the video, the server uses an image analysis device and AI algorithms such as OpenPose to calculate each frame of the video and identify key movement points. The server then compares the video with a standard movement model (pre-intelligently extracted expert demonstration movements) to identify deviations and areas for improvement, automatically recording these in a technical analysis report.

[0179] The server also uses emotion recognition algorithms to analyze image and sound data to determine the user's psychological state during training, such as tension or frustration. The emotion recognition and adjustment device combines the determined emotions to adaptively adjust the final system feedback. For example, it can automatically add encouraging statements or relaxation suggestions to prompts, making the training more humane.

[0180] All analysis and feedback are processed by the server and then returned to the terminal via the app. The terminal app clearly displays training technique analysis and emotional feedback through a UI interface, helping users better understand common mistakes and improvement plans. Users refer to the feedback, correct their movements, and train repeatedly, forming a self-optimizing closed loop.

[0181] Application examples:

[0182] Users select "Football" on the mobile app, fill in their goal of "improving free-kick accuracy," and submit. The server generates a three-week training plan including daily practice details. After each day's training, users record and upload videos of their free-kick movements. The server automatically analyzes and detects "the supporting foot being too far from the ball when taking the shot" and detects facial tension in the user's expression, providing feedback such as "adjust the supporting foot, maintain a distance of 20cm, relax, take a deep breath, and shoot confidently." Users then correct their movements accordingly.

[0183] Typical prompts for generative artificial intelligence models include:

[0184] "Please analyze the free-kick training video I uploaded and point out the main technical problems in my movements. If I am feeling nervous, please add suggestions on how to help the athlete relax, in addition to technical advice."

[0185] "I need a three-week training plan to improve the accuracy of free kicks in football. Please specify the daily tasks and explain the purpose of each training exercise."

[0186] "Please analyze my current emotions based on my facial expressions and voice, and provide personalized technical and psychological support suggestions."

[0187] This invention achieves motion analysis, customized training plans, emotion recognition, personalized feedback, and human-machine integration optimization through intelligent data processing on the server and efficient interaction on the terminal, providing athletes with a more scientific and comprehensive training guidance platform.

[0188] use Figure 13 The processing flow is explained.

[0189] Step 1:

[0190] Users open a dedicated application on the terminal, select or enter the sport type and training goal, such as "soccer" or "improve free-kick accuracy," and submit the information. The terminal receives the user input and packages the information into a standardized JSON-formatted data packet. The input is manually entered data by the user, and the output is a formatted data packet, which is prepared for uploading to the server via encrypted communication.

[0191] Step 2:

[0192] The terminal sends formatted data packets to the server via the network. Upon receiving the data packets, the server first parses the motion type and target, then retrieves relevant training cases and theoretical knowledge from the database. The input is the JSON data packet uploaded by the terminal; the server extracts and queries the data, outputting relevant background data and model parameters.

[0193] Step 3:

[0194] The server utilizes a generative artificial intelligence model, combining training knowledge from a database with the input target requirements, to automatically generate personalized training plans. The input consists of user target information and historical data; the server uses generative models and algorithms to perform inference and generation, and the output is a detailed training plan for the user (such as daily training content and key guidance points).

[0195] Step 4:

[0196] The server sends the generated training plan to the terminal via the network. Upon receiving the plan, the terminal parses it and displays it through the app interface as a list, calendar, or chart. The input is the plan content generated by the server; the terminal performs format conversion and interface rendering on this content, outputting a visual training plan interface.

[0197] Step 5:

[0198] Users begin actual training according to the training plan and record training videos using the terminal device's camera. After the video files are automatically saved, the terminal compresses and formats them, adding timestamps and training category information. The input is the newly recorded video file, which, after local transcoding and other processing, is output as a standardized video data packet.

[0199] Step 6:

[0200] The terminal uploads video data packets to the server via the network. Upon receiving the data, the server uses image analysis algorithms (such as OpenPose or YoloV7) to perform motion analysis frame by frame. The input is the uploaded video data packet; the server performs frame decomposition and motion recognition, and the output is structured motion data, technical indicators, and defect analysis results.

[0201] Step 7:

[0202] The server compares the user's action data with a standard action model to identify errors and areas for improvement, while also detecting emotional features in the audio or images. The server uses emotion recognition algorithms to extract features from facial expressions, voice, etc., and combines this with action analysis to provide a preliminary judgment. The input consists of action metrics and multimodal data; the output is a comprehensive report including technical analysis and emotional state.

[0203] Step 8:

[0204] Based on the analysis report, the server uses a generative artificial intelligence model to generate personalized feedback suggestions for action errors and the user's emotional state. If user tension is detected, appropriate encouragement and relaxation suggestions are added. The inputs are the analysis report and emotional data, and the outputs are the adjusted feedback text and suggestions.

[0205] Step 9:

[0206] The server sends the final feedback suggestions back to the terminal via the network. Upon receiving the feedback, the terminal integrates technical improvements and personalized suggestions into its interface, allowing users to view the feedback in real time. The input is the comprehensive feedback sent by the server, and the output is interactive text, charts, and multimedia feedback information on the terminal's app interface.

[0207] Step 10:

[0208] Based on the feedback displayed on the terminal, users modify their training movements or make mental adjustments, and then enter a new round of training, video capture, and uploading; the system process repeats cyclically. The input is the specific suggestions from the system feedback, and the output is the user's actual training behavior and subsequent data collection.

[0209] Application Example 2

[0210] The process flow corresponding to the specific processing in Use Case 2 will be described below. The various parts of the system described below are implemented by the data processing device 12 and the intelligent device 14. In addition, the data processing device 12 is referred to as the "server" and the intelligent device 14 is referred to as the "terminal".

[0211] Existing training support systems often focus solely on analyzing and responding to user actions, neglecting the user's emotional state. They fail to dynamically adjust feedback based on individual emotional differences, resulting in insufficient user motivation and low training continuity and efficiency. This is particularly problematic in scenarios like factory automation, where traditional systems cannot effectively integrate operators' real-time emotions and personalized needs, hindering the optimization of operational efficiency and skill improvement.

[0212] The specific processing performed by the specific processing unit 290 of the data processing apparatus 12 in Application Example 2 is achieved by the following means.

[0213] In this invention, the server includes an information input device for inputting user training goals and classifications, a plan generation device for receiving information and generating training plans using a generative artificial intelligence model, a recording and transmission device for recording and uploading training process data, an information parsing device for analyzing uploaded image data and locating action errors and correction points, an information prompting device for providing users with correction suggestions and personalized feedback, an emotion recognition device for recognizing user emotional states and adjusting feedback methods, and an information management device for managing and recording training and feedback data. This enables the dynamic generation of personalized training plans and feedback based on individual user goals and real-time emotional states, enhancing user motivation, improving training continuity, and effectively promoting operational efficiency and skill improvement.

[0214] "Information input device" refers to an input interface or device used to receive and record user input of training objectives, classifications, or related parameters.

[0215] A “plan generation device” refers to a software or hardware unit that can automatically generate training plans based on input information and using generative artificial intelligence models.

[0216] "Recording and transmission device" refers to a data acquisition and communication module used to collect image, audio and other data during user training and upload them to a server.

[0217] "Information analysis device" refers to a data analysis unit that analyzes uploaded image data or other sensor data to identify motion errors and parts that need correction.

[0218] "Information prompting device" refers to an output module that provides personalized suggestions and feedback generated by the system to the user in the form of text, voice, interface, etc.

[0219] "Emotion recognition device" refers to an analysis and decision-making module that can determine a user's current emotional state in real time by analyzing images, voice or biosignals, and dynamically adjust the feedback content.

[0220] "Information management device" refers to a management system or database used to comprehensively manage and record user training processes, analysis results, feedback history, and related data.

[0221] "Generative artificial intelligence models" refer to artificial intelligence algorithms or systems that can automatically generate content based on input prompts and be used to support training plan development, personalized feedback, and other tasks.

[0222] "Benchmark action" refers to the standardized action or idealized operating procedure used in the system as a standard for action comparison.

[0223] "Personalized feedback" refers to personalized improvement suggestions and encouraging content generated based on each user's goals, actions, and real-time emotional state.

[0224] This invention can be implemented in the following specific ways.

[0225] The server is equipped with a high-performance central processing unit, graphics processing unit, memory modules, and large-capacity storage devices. It runs an operating system such as Linux and is equipped with a database management system (such as a relational database and a non-relational database). The terminal can be an intelligent device with a camera, microphone, display, and network communication module, such as a smartphone, tablet, or personal computer. The server deploys software systems such as generative artificial intelligence models (such as general-purpose large model systems), action recognition algorithms (such as pose estimation algorithms), and emotion recognition algorithms (such as deep learning-based facial expression and speech emotion analysis algorithms).

[0226] Users first operate the system through a dedicated application on the terminal. In the application interface, users input their training objectives and categories, such as "machine operation" or "efficiency improvement." The terminal then sends the user data to the server via a secure network communication protocol (such as HTTPS).

[0227] After receiving user information, the server invokes a generative artificial intelligence model. This model can be a language generation model, which flexibly generates targeted training plans using prompts. For example, the server could invoke the AI ​​model with the following prompt: "Based on the user's goal of 'mechanical operation,' please develop a seven-day training plan, specifying the tasks to be completed each day."

[0228] The terminal receives and displays the training plan, which users can refer to for gradual practice. During each training session, users record their own movements and voice using the terminal's camera and microphone. The terminal then compresses the collected image and audio data and uploads it to the server.

[0229] The server uses motion analysis software (such as pose estimation algorithms and skeleton recognition modules) to analyze user-uploaded data frame by frame, comparing the user's actions with the system's built-in benchmark motion samples. For example, the server identifies deviations in the user's actions during the "button operation" phase and pinpoints the specific error location based on the task requirements. Subsequently, the server calls upon a generative artificial intelligence model to generate personalized correction suggestions based on the motion analysis results, such as through a prompt like: "The user raised their elbow too high at the 5th second; please generate specific improvement suggestions using encouraging language."

[0230] The server simultaneously performs emotion recognition processing on the user's video or audio data, adjusting the tone and content of feedback based on real-time detected emotions (such as anxiety, fatigue, confidence, etc.). For example, if the user exhibits stress or frustration, the system feedback will automatically include comforting and encouraging statements.

[0231] The terminal displays the feedback information generated by the server to the user in the form of an interface, text, animation, or voice, helping the user understand their shortcomings and obtain targeted improvement suggestions. Users can refer to these suggestions in their next practice session to continuously optimize their behavior.

[0232] All training processes, analysis results, feedback content, and information such as users' historical emotional changes are automatically archived and visualized by the server's information management module, providing users and operators with personalized and dynamic self-improvement paths and management basis.

[0233] A concrete example: A factory operator inputs "robotic arm assembly" and "efficiency improvement" through a tablet application. The server generates a personalized training schedule based on these prompts. The worker records and uploads video of their movements using the tablet's camera. The server analyzes the worker's errors using a pose estimation algorithm and detects anxiety signals in their facial expressions, then returns personalized feedback such as, "We suggest you slow down, focus on standard procedures; your progress is commendable!" The worker repeatedly practices and receives feedback based on these suggestions, gradually improving their skills and confidence.

[0234] Examples of available prompt statements:

[0235] Based on the user's goal of "robot operation", generate a personalized training plan and provide daily step-by-step suggestions.

[0236] Analyze the following user action videos, compare them with standard actions, identify existing problems, and generate improvement suggestions. If you detect that the user is experiencing low spirits, please add encouraging feedback.

[0237] To address the need for improved operator efficiency, training plans are generated, and appropriate incentive suggestions are provided based on the user's real-time emotional state.

[0238] use Figure 14 The processing flow is explained.

[0239] Step 1:

[0240] Users open the application on their devices, enter the training objective and classification (such as "mechanical operation" or "efficiency improvement"), and click submit.

[0241] Input: Training target and classification text

[0242] The terminal collects input data based on user operations and sends it to the server API interface via a secure network protocol.

[0243] Output: A data packet containing the user's target and classification (sent to the server).

[0244] Step 2:

[0245] The server receives user goals and categories uploaded by the terminal, calls the generative artificial intelligence model, and automatically generates a personalized training plan using the set prompts.

[0246] Input: User goals and classification data

[0247] The server processes information through data retrieval, prompt statement construction, and AI model invocation to obtain a structured training plan.

[0248] Output: A training plan including daily tasks, content, and assessment points (sent to the terminal).

[0249] Step 3:

[0250] The terminal receives the training plan, parses the content into a user-friendly interface, and displays it on the main application interface in stages according to schedule and tasks.

[0251] Input: Training plan data

[0252] The terminal performs interface data parsing and visualization processing to display tasks, goals, completion times, etc.

[0253] Output: Visualized training schedule and daily tasks (displayed to the user)

[0254] Step 4:

[0255] Users begin training according to the interface plan, recording the process using the terminal's camera / microphone and saving the video. After completing the exercise, users click "upload."

[0256] Input: User-generated image and audio data

[0257] The terminal records and encodes multimedia data such as video and audio, and uploads it to a designated interface on the server via the Internet.

[0258] Output: A video file containing the user's training process (uploaded to the server).

[0259] Step 5:

[0260] The server receives and stores the uploaded video data, and calls action recognition algorithms (such as pose estimation algorithms) to analyze the video frame by frame.

[0261] Input: User-uploaded video data

[0262] The server extracts action features, compares them with standard action samples, and uses classification algorithms to locate action errors and identify areas that need correction.

[0263] Output: Analysis results of motion error and correction points (saved and used for feedback generation)

[0264] Step 6:

[0265] The server uses a generative artificial intelligence model to automatically generate personalized improvement suggestions and next steps based on action analysis results. The generated content is then adjusted in conjunction with prompts.

[0266] Input: Action error and correction point data

[0267] The server processes the above data into natural language text and generates feedback information.

[0268] Output: Personalized feedback text and suggested corrections (sent to the terminal)

[0269] Step 7:

[0270] The server performs emotion recognition on the image and audio data uploaded by users, analyzing facial expressions, tone of voice, etc., to determine the user's current emotional state.

[0271] Input: User video and audio data

[0272] The server outputs specific emotion categories through sentiment analysis algorithms and adjusts the way feedback is expressed accordingly.

[0273] Output: Emotionally adapted feedback content (integrated into the overall feedback).

[0274] Step 8:

[0275] The terminal receives feedback from the server (including suggestions for action improvement and emotional support information), which is then displayed on the application in the form of text, charts, animations, or voice, prompting the user to make the next adjustment.

[0276] Input: Server feedback data

[0277] The terminal updates the interface and displays content, while also providing key information via audio and video.

[0278] Output: Intuitive feedback and suggestions presented to the user.

[0279] Step 9:

[0280] Users adjust their practice based on the received suggestions, practice again, record videos, and continue to upload them, achieving continuous iterative improvement in training and feedback.

[0281] Input: Feedback, suggestions, and personal training experiences

[0282] Users optimize actions based on feedback, and the process of collecting new data at the terminal is repeated.

[0283] Output: Improved training images (uploaded to the server, process loop)

[0284] The specific processing unit 290 sends the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires sound representing user input regarding the result of the specific processing. The control unit 46A sends the sound data representing user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the sound data.

[0285] Data generation model 58 is a so-called generative AI (Artificial Intelligence). Examples of data generation models 58 include ChatGPT (registered trademark) (accessible via the internet (URL: https: / / openai.com / blog / chatgpt)). Data generation model 58 is obtained through deep learning on a neural network. Input to data generation model 58 are prompt words containing instructions, and inference data such as sound data representing sound, text data representing text, and image data representing images (e.g., still image data or animation data). Data generation model 58 infers from the input inference data based on the instructions represented by the prompt words and outputs the inference result in one or more data forms, such as sound data, text data, and image data. Data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or induction. The specific processing unit 290 performs the aforementioned specific processing while using data generation model 58. The data generation model 58 can also be a model finely tuned to output inference results from prompts that do not contain instructions. In this case, the data generation model 58 can output inference results based on prompts that do not contain instructions. The data processing apparatus 12, etc., includes various data generation models 58, including AI other than the generation AI. AI other than the generation AI can be, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or Naive Bayes, and can perform various processes, but is not limited to this example. Furthermore, the AI ​​can also be an AI agent. Furthermore, when the processing of the above-mentioned parts is performed by AI, the processing can be partially or entirely performed by AI, but is not limited to this example. Furthermore, the processing performed by the AI ​​including the generation AI can be replaced by processing in the rule base, and the processing in the rule base can also be replaced by processing performed by the AI ​​including the generation AI.

[0286] Furthermore, the processing of the aforementioned data processing system 10 is performed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart device 14, but it can also be performed by both the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart device 14. Additionally, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the smart device 14 or external devices, and the smart device 14 acquires or collects information required for processing from the data processing device 12 or external devices.

[0287] For example, the collection unit is implemented by the control unit 46A of the smart device 14 or the specific processing unit 290 of the data processing device 12. For example, the acquisition unit uses the camera 42 or communication I / F 44 of the smart device 14 to acquire step data, which is then processed by the specific processing unit 290 of the data processing device 12. For example, the analysis unit is implemented by the specific processing unit 290 of the data processing device 12, which analyzes the data from the collection unit and the acquisition unit. For example, the generation unit is implemented by the specific processing unit 290 of the data processing device 12, which uses a generation AI to generate a menu. For example, the serving unit is implemented by the output device 40 of the smart device 14 or the specific processing unit 290 of the data processing device 12, which provides the generated menu to the user. The correspondence between each unit and the device or control unit is not limited to the above examples and various changes can be made.

[0288] In the above embodiments, examples of specific processing by the data processing device 12 are given, but the technology disclosed herein is not limited to this, and specific processing may also be performed by the smart device 14.

[0289] Second Implementation Method

[0290] Figure 3 An example of the configuration of the data processing system 210 according to the second embodiment is shown.

[0291] like Figure 3 As shown, the data processing system 210 includes a data processing device 12 and smart glasses 214. A server can be cited as an example of the data processing device 12.

[0292] The data processing apparatus 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" as understood in this disclosure. The computer 22 includes a processor 28, RAM 30, and memory 32. The processor 28, RAM 30, and memory 32 are connected to a bus 34. Furthermore, the database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0293] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, RAM 48, and memory 50. The processor 46, RAM 48, and memory 50 are connected to a bus 52. Furthermore, the microphone 238, speaker 240, camera 42, and communication I / F 44 are also connected to the bus 52.

[0294] Microphone 238 receives instructions from user 20 by receiving sounds emitted by user 20. Microphone 238 captures sounds emitted by user 20 and converts the captured sounds into sound data, which is then output to processor 46. Speaker 240 outputs sound according to instructions from processor 46.

[0295] Camera 42 is a small digital camera equipped with an optical system such as a lens, aperture and shutter, and imaging elements such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, to capture images of the user 20's surroundings (e.g., the field of view defined by an angle equivalent to the field of vision of an average healthy person).

[0296] Communication I / F44 is connected to network 54. Communication I / F44 and 26 are responsible for the transmission and reception of various information between processor 46 and processor 28 via network 54. The transmission and reception of various information between processor 46 and processor 28 using communication I / F44 and 26 is performed in a secure state.

[0297] Figure 4 This illustrates an example of the main functions of the data processing device 12 and the smart glasses 214. For example... Figure 4 As shown, in the data processing device 12, specific processing is performed by the processor 28. The specific processing program 56 is stored in the memory 32.

[0298] The specific processing program 56 is an example of a "program" involved in the technology of this disclosure. The processor 28 reads the specific processing program 56 from the memory 32 and executes the read specific processing program 56 on the RAM 30. Specific processing is implemented by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0299] The memory 32 stores a data generation model 58 and an emotion-specific model 59. The data generation model 58 and the emotion-specific model 59 are used by the specific processing unit 290. The specific processing unit 290 can use the emotion-specific model 59 to infer the user's emotions and perform specific processing based on the user's emotions. In the emotion inference function (emotion-specific function) using the emotion-specific model 59, various inferences and predictions related to the user's emotions are performed, including inferences and predictions of the user's emotions, but this is not limited to this example. Furthermore, emotion inference and prediction may also include, for example, emotion analysis (parsing).

[0300] In the smart glasses 214, the processor 46 performs reception and output processing. The memory 50 stores the reception and output program 60. The processor 46 reads the reception and output program 60 from the memory 50 and executes the read reception and output program 60 on the RAM 48. The reception and output processing is implemented by the processor 46 operating as a control unit 46A according to the reception and output program 60 executed on the RAM 48. Furthermore, the smart glasses 214 has the same data generation model and emotion-specific model as the data generation model 58 and the emotion-specific model 59, and these models can also be used to perform the same processing as the specific processing unit 290.

[0301] Next, the specific processing of the specific processing unit 290 of the data processing device 12 will be described. Each part of the system described below is implemented by the data processing device 12 and the smart glasses 214. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0302] Example 1

[0303] The process is the same as that of the specific process described in Embodiment 1 in the first embodiment above, so the description is omitted.

[0304] Application Example 1

[0305] The process is the same as that in the specific processing described in Application Example 1 of the first embodiment above, so the description is omitted.

[0306] Example 2

[0307] The process is the same as that of the specific process in Embodiment 2 described in the first embodiment above, so the description is omitted.

[0308] Application Example 2

[0309] The process is the same as that in the specific processing described in Application Example 2 of the first embodiment above, so the description is omitted.

[0310] The specific processing unit 290 sends the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A outputs the result of the specific processing to the speaker 240. The microphone 238 acquires sound input representing the user's input regarding the result of the specific processing. The control unit 46A sends the sound data representing the user's input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the sound data.

[0311] Data generation model 58 is a so-called generative AI (Artificial Intelligence). Examples of data generation models 58 include ChatGPT (registered trademark) (accessible via the internet (URL: https: / / openai.com / blog / chatgpt)). Data generation model 58 is obtained through deep learning on a neural network. Input to data generation model 58 are prompt words containing instructions, and inference data such as sound data representing sound, text data representing text, and image data representing images (e.g., still image data or animation data). Data generation model 58 infers from the input inference data based on the instructions represented by the prompt words and outputs the inference result in one or more data forms, such as sound data, text data, and image data. Data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or induction. The specific processing unit 290 performs the aforementioned specific processing while using data generation model 58. The data generation model 58 can also be a model finely tuned to output inference results from prompts that do not contain instructions. In this case, the data generation model 58 can output inference results based on prompts that do not contain instructions. The data processing apparatus 12, etc., includes various data generation models 58, including AI other than the generation AI. AI other than the generation AI can be, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or Naive Bayes, and can perform various processes, but is not limited to this example. Furthermore, the AI ​​can also be an AI agent. Furthermore, when the processing of the above-mentioned parts is performed by AI, the processing can be partially or entirely performed by AI, but is not limited to this example. Furthermore, the processing performed by the AI ​​including the generation AI can be replaced by processing in the rule base, and the processing in the rule base can also be replaced by processing performed by the AI ​​including the generation AI.

[0312] Furthermore, the processing of the aforementioned data processing system 10 is performed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart glasses 214, but it can also be performed by both the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart glasses 214. Additionally, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the smart glasses 214 or external devices, and the smart glasses 214 acquires or collects information required for processing from the data processing device 12 or external devices.

[0313] For example, the collection unit is implemented by the control unit 46A of the smart glasses 214 or the specific processing unit 290 of the data processing device 12. For example, the acquisition unit uses the camera 42 or communication I / F 44 of the smart glasses 214 to acquire step data, which is then processed by the specific processing unit 290 of the data processing device 12. For example, the analysis unit is implemented by the specific processing unit 290 of the data processing device 12, which analyzes the data from the collection unit and the acquisition unit. For example, the generation unit is implemented by the specific processing unit 290 of the data processing device 12, which uses a generation AI to generate a menu. For example, the serving unit is implemented by the speaker 240 of the smart glasses 214 or the specific processing unit 290 of the data processing device 12, which provides the generated menu to the user. The correspondence between each unit and the device or control unit is not limited to the above examples and various changes can be made.

[0314] In the above embodiments, examples of specific processing by the data processing device 12 are given, but the technology disclosed herein is not limited to this, and specific processing may also be performed by the smart glasses 214.

[0315] Third Implementation Method

[0316] Figure 5 An example of the configuration of the data processing system 310 according to the third embodiment is shown.

[0317] like Figure 5 As shown, the data processing system 310 includes a data processing device 12 and a head-mounted terminal 314. A server can be cited as an example of the data processing device 12.

[0318] The data processing apparatus 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" as understood in this disclosure. The computer 22 includes a processor 28, RAM 30, and memory 32. The processor 28, RAM 30, and memory 32 are connected to a bus 34. Furthermore, the database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0319] The head-mounted terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and memory 50. The processor 46, RAM 48, and memory 50 are connected to a bus 52. Furthermore, the microphone 238, speaker 240, camera 42, display 343, and communication I / F 44 are also connected to the bus 52.

[0320] Microphone 238 receives instructions from user 20 by receiving sounds emitted by user 20. Microphone 238 captures sounds emitted by user 20 and converts the captured sounds into sound data, which is then output to processor 46. Speaker 240 outputs sound according to instructions from processor 46.

[0321] Camera 42 is a small digital camera equipped with an optical system such as a lens, aperture and shutter, and imaging elements such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, to capture images of the user 20's surroundings (e.g., the field of view defined by an angle equivalent to the field of vision of an average healthy person).

[0322] Communication I / F44 is connected to network 54. Communication I / F44 and 26 are responsible for the transmission and reception of various information between processor 46 and processor 28 via network 54. The transmission and reception of various information between processor 46 and processor 28 using communication I / F44 and 26 is performed in a secure state.

[0323] Figure 6 This illustrates an example of the main functions of the data processing device 12 and the head-mounted terminal 314. For example... Figure 6 As shown, in the data processing device 12, specific processing is performed by the processor 28. The specific processing program 56 is stored in the memory 32.

[0324] The specific processing program 56 is an example of a "program" involved in the technology of this disclosure. The processor 28 reads the specific processing program 56 from the memory 32 and executes the read specific processing program 56 on the RAM 30. Specific processing is implemented by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0325] The memory 32 stores the data generation model 58 and the emotion-specific model 59. The data generation model 58 and the emotion-specific model 59 are used by the specific processing unit 290.

[0326] In the head-mounted terminal 314, the processor 46 performs the acceptance / output processing. The memory 50 stores the acceptance / output program 60. The processor 46 reads the acceptance / output program 60 from the memory 50 and executes the read acceptance / output program 60 on the RAM 48. The acceptance / output processing is implemented by the processor 46 operating as a control unit 46A according to the acceptance / output program 60 executed on the RAM 48.

[0327] Next, the specific processing of the specific processing unit 290 of the data processing device 12 will be described. Each part of the system described below is implemented by the data processing device 12 and the head-mounted terminal 314. In the following description, the data processing device 12 will be referred to as the "server" and the head-mounted terminal 314 will be referred to as the "terminal".

[0328] Example 1

[0329] The process is the same as that of the specific process described in Embodiment 1 in the first embodiment above, so the description is omitted.

[0330] Application Example 1

[0331] The process is the same as that in the specific processing described in Application Example 1 of the first embodiment above, so the description is omitted.

[0332] Example 2

[0333] The process is the same as that of the specific process in Embodiment 2 described in the first embodiment above, so the description is omitted.

[0334] Application Example 2

[0335] The process is the same as that in the specific processing described in Application Example 2 of the first embodiment above, so the description is omitted.

[0336] The specific processing unit 290 sends the result of the specific processing to the head-mounted terminal 314. In the head-mounted terminal 314, the control unit 46A outputs the result of the specific processing to the speaker 240 and the display 343. The microphone 238 acquires sound input representing the user's input regarding the result of the specific processing. The control unit 46A sends the sound data representing the user's input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the sound data.

[0337] Data generation model 58 is a so-called generative AI (Artificial Intelligence). Examples of data generation models 58 include ChatGPT (registered trademark) (accessible via the internet (URL: https: / / openai.com / blog / chatgpt)). Data generation model 58 is obtained through deep learning on a neural network. Input to data generation model 58 are prompt words containing instructions, and inference data such as sound data representing sound, text data representing text, and image data representing images (e.g., still image data or animation data). Data generation model 58 infers the input inference data based on the instructions represented by the prompt words and outputs the inference result in one or more data forms such as sound data, text data, and image data. Data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or induction. The specific processing unit 290 performs the aforementioned specific processing while using data generation model 58. The data generation model 58 can also be a model finely tuned to output inference results from prompts that do not contain instructions. In this case, the data generation model 58 can output inference results based on prompts that do not contain instructions. The data processing apparatus 12, etc., includes various data generation models 58, including AI other than the generation AI. AI other than the generation AI can be, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or Naive Bayes, and can perform various processes, but is not limited to this example. Furthermore, the AI ​​can also be an AI agent. Furthermore, when the processing of the above-mentioned parts is performed by AI, the processing can be partially or entirely performed by AI, but is not limited to this example. Furthermore, the processing performed by the AI ​​including the generation AI can be replaced by processing in the rule base, and the processing in the rule base can also be replaced by processing performed by the AI ​​including the generation AI.

[0338] Furthermore, the processing of the aforementioned data processing system 10 is performed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the head-mounted terminal 314, but it can also be performed by both the specific processing unit 290 of the data processing device 12 and the control unit 46A of the head-mounted terminal 314. Additionally, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the head-mounted terminal 314 or external devices, and the head-mounted terminal 314 acquires or collects information required for processing from the data processing device 12 or external devices.

[0339] For example, the collection unit is implemented by the control unit 46A of the head-mounted terminal 314 or the specific processing unit 290 of the data processing device 12. For example, the acquisition unit uses the camera 42 or communication I / F 44 of the head-mounted terminal 314 to acquire step data, which is then processed by the specific processing unit 290 of the data processing device 12. For example, the analysis unit is implemented by the specific processing unit 290 of the data processing device 12 to analyze the data from the collection unit and the acquisition unit. For example, the generation unit is implemented by the specific processing unit 290 of the data processing device 12 to generate a menu using a generation AI. For example, the serving unit is implemented by the speaker 240 and display 343 of the head-mounted terminal 314 or the specific processing unit 290 of the data processing device 12 to provide the generated menu to the user. The correspondence between each unit and the device or control unit is not limited to the above examples and various changes can be made.

[0340] In the above embodiments, examples of specific processing by the data processing device 12 are given, but the technology disclosed herein is not limited to this, and specific processing may also be performed by the head-mounted terminal 314.

[0341] Fourth Implementation Method

[0342] Figure 7 An example of the configuration of the data processing system 410 according to the fourth embodiment is shown.

[0343] like Figure 7 As shown, the data processing system 410 includes a data processing device 12 and a robot 414. A server can be cited as an example of the data processing device 12.

[0344] The data processing apparatus 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" as understood in this disclosure. The computer 22 includes a processor 28, RAM 30, and memory 32. The processor 28, RAM 30, and memory 32 are connected to a bus 34. Furthermore, the database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0345] Robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and memory 50. The processor 46, RAM 48, and memory 50 are connected to a bus 52. Furthermore, the microphone 238, speaker 240, camera 42, controlled object 443, and communication I / F 44 are also connected to the bus 52.

[0346] Microphone 238 receives instructions from user 20 by receiving sounds emitted by user 20. Microphone 238 captures sounds emitted by user 20 and converts the captured sounds into sound data, which is then output to processor 46. Speaker 240 outputs sound according to instructions from processor 46.

[0347] Camera 42 is a small digital camera equipped with an optical system such as a lens, aperture and shutter, and imaging elements such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, to photograph the area around robot 414 (e.g., the field of view defined by a perspective equivalent to the field of vision of an average healthy person).

[0348] Communication I / F44 is connected to network 54. Communication I / F44 and 26 are responsible for the transmission and reception of various information between processor 46 and processor 28 via network 54. The transmission and reception of various information between processor 46 and processor 28 using communication I / F44 and 26 is performed in a secure state.

[0349] The controlled object 443 includes a display device, LEDs (light-emitting diodes) for the eyes, and motors for driving the arms, hands, and feet. The posture or movement of the robot 414 is controlled by controlling the motors in the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. In addition, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0350] Figure 8 This illustrates an example of the main functions of the data processing device 12 and the robot 414. For example... Figure 8 As shown, in the data processing device 12, specific processing is performed by the processor 28. The specific processing program 56 is stored in the memory 32.

[0351] The specific processing program 56 is an example of a "program" involved in the technology of this disclosure. The processor 28 reads the specific processing program 56 from the memory 32 and executes the read specific processing program 56 on the RAM 30. Specific processing is implemented by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0352] The memory 32 stores the data generation model 58 and the emotion-specific model 59. The data generation model 58 and the emotion-specific model 59 are used by the specific processing unit 290.

[0353] In robot 414, the processor 46 performs the acceptance and output processing. The memory 50 stores the acceptance and output program 60. The processor 46 reads the acceptance and output program 60 from the memory 50 and executes the read acceptance and output program 60 on RAM 48. The acceptance and output processing is implemented by the processor 46 acting as the control unit 46A according to the acceptance and output program 60 executed on RAM 48.

[0354] Next, the specific processing of the specific processing unit 290 of the data processing device 12 will be described. Each part of the system described below is implemented by the data processing device 12 and the robot 414. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 will be referred to as the "terminal".

[0355] Example 1

[0356] The process is the same as that of the specific process described in Embodiment 1 in the first embodiment above, so the description is omitted.

[0357] Application Example 1

[0358] The process is the same as that in the specific processing described in Application Example 1 of the first embodiment above, so the description is omitted.

[0359] Example 2

[0360] The process is the same as that of the specific process in Embodiment 2 described in the first embodiment above, so the description is omitted.

[0361] Application Example 2

[0362] The process is the same as that in the specific processing described in Application Example 2 of the first embodiment above, so the description is omitted.

[0363] The specific processing unit 290 sends the result of the specific processing to the robot 414. In the robot 414, the control unit 46A outputs the result of the specific processing to the speaker 240 and the controlled object 443. The microphone 238 acquires sound input representing the result of the specific processing. The control unit 46A sends the sound data representing the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the sound data.

[0364] Data generation model 58 is a so-called generative AI (Artificial Intelligence). Examples of data generation models 58 include ChatGPT (registered trademark) (accessible via the internet (URL: https: / / openai.com / blog / chatgpt)). Data generation model 58 is obtained through deep learning on a neural network. Input to data generation model 58 are prompt words containing instructions, and inference data such as sound data representing sound, text data representing text, and image data representing images (e.g., still image data or animation data). Data generation model 58 infers from the input inference data based on the instructions represented by the prompt words and outputs the inference result in one or more data forms, such as sound data, text data, and image data. Data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or induction. The specific processing unit 290 performs the aforementioned specific processing while using data generation model 58. The data generation model 58 can also be a model finely tuned to output inference results from prompts that do not contain instructions. In this case, the data generation model 58 can output inference results based on prompts that do not contain instructions. The data processing apparatus 12, etc., includes various data generation models 58, including AI other than the generation AI. AI other than the generation AI can be, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or Naive Bayes, and can perform various processes, but is not limited to this example. Furthermore, the AI ​​can also be an AI agent. Furthermore, when the processing of the above-mentioned parts is performed by AI, the processing can be partially or entirely performed by AI, but is not limited to this example. Furthermore, the processing performed by the AI ​​including the generation AI can be replaced by processing in the rule base, and the processing in the rule base can also be replaced by processing performed by the AI ​​including the generation AI.

[0365] Furthermore, the processing of the aforementioned data processing system 10 is performed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the robot 414, but it can also be performed by both the specific processing unit 290 of the data processing device 12 and the control unit 46A of the robot 414. Additionally, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the robot 414 or external devices, and the robot 414 acquires or collects information required for processing from the data processing device 12 or external devices.

[0366] For example, the collection unit is implemented by the control unit 46A of the robot 414 or the specific processing unit 290 of the data processing device 12. For example, the acquisition unit uses the camera 42 or communication I / F 44 of the robot 414 to acquire step data, which is then processed by the specific processing unit 290 of the data processing device 12. For example, the analysis unit is implemented by the specific processing unit 290 of the data processing device 12, which analyzes the data from the collection unit and the acquisition unit. For example, the generation unit is implemented by the specific processing unit 290 of the data processing device 12, which uses a generation AI to generate a menu. For example, the serving unit is implemented by the speaker 240 of the robot 414 and the control object 443 or the specific processing unit 290 of the data processing device 12, which provides the generated menu to the user. The correspondence between each unit and the device or control unit is not limited to the above examples and various changes can be made.

[0367] In the above embodiments, examples of specific processing by the data processing device 12 are given, but the technology disclosed herein is not limited to this, and specific processing may also be performed by the robot 414.

[0368] Furthermore, the emotion-specific model 59, acting as an emotion engine, can determine a user's emotion based on a specific mapping. Specifically, the emotion-specific model 59 can determine a user's emotion based on an emotion graph that serves as a specific mapping (see [reference]). Figure 9 The system determines the user's emotions. Furthermore, the emotion-specific model 59 can similarly determine the robot's emotions, and the specific processing unit 290 performs specific processing based on the robot's emotions.

[0369] Figure 9 This is a diagram representing an emotion map 400 that maps multiple emotions. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotion is. On the outer side of the concentric circles, emotions representing states or behaviors arising from mood are arranged. Emotions are concepts that include feelings and mental states. Emotions generated by reactions occurring in the brain are arranged roughly to the left of the concentric circles. Emotions derived from situational judgments are arranged roughly to the right of the concentric circles. Emotions generated by reactions occurring in the brain and derived from situational judgments are arranged roughly above and below the concentric circles. Furthermore, "pleasant" emotions are arranged above the concentric circles, and "unpleasant" emotions are arranged below them. Thus, in the emotion map 400, multiple emotions are mapped based on the structure that generates emotions, and emotions that are likely to occur simultaneously are mapped close to each other.

[0370] These emotions are distributed at the three o'clock position of the emotion map 400, typically fluctuating between peace and anxiety. In the right half of the emotion map 400, situational awareness dominates over internal sensation, thus resulting in an impression of calm.

[0371] The inner side of the emotion map 400 represents the inner state, while the outer side represents behavior. Therefore, the further outward you are from the emotion map 400, the more visible the emotion becomes (manifested in behavior).

[0372] Here, human emotions are based on various balances such as posture and blood sugar levels. When these balances deviate from an ideal state, it indicates an unpleasant state; when they approach the ideal state, it indicates a pleasant state. Emotions in robots, cars, motorcycles, etc., can also be created in the following way: based on various balances such as posture and remaining battery power, when these balances deviate from an ideal state, it indicates an unpleasant state; when they approach the ideal state, it indicates a pleasant state. Emotion maps can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a Brain Physiological Signal Analysis System for Voice Emotion Recognition and Emotion, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). In the left half of the emotion map, emotions belonging to the sensory-dominated region, called "response," are arranged. Furthermore, in the right half of the emotion map, emotions belonging to the situational cognition-dominated region, called "situation," are arranged.

[0373] In the emotion map, two types of emotions that promote learning are defined. One is a negative emotion on the situational side, in the middle or peripheral region of "repentance" or "reflection." This occurs when the robot experiences negative emotions such as "I don't want to experience this feeling again" or "I don't want to be blamed again." The other is a positive emotion on the response side, near the "desire" region. This occurs when there are positive feelings such as "wanting more" or "wanting to know more."

[0374] The emotion-specific model 59 inputs user input into a pre-trained neural network to obtain emotion values ​​representing each emotion shown in the emotion map 400, thereby determining the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network... Figure 10 As shown in the sentiment graph 900, it was trained in a way that sentiments that are configured close to each other have similar values. Figure 10 The text shows examples of emotions such as "peace of mind", "stability", and "reassurance" that have similar emotion values.

[0375] The above description focuses on the functions of the data processing device 12, but the system of this disclosure is not necessarily installed on a server. The system of this disclosure can also be installed as a general information processing system. This disclosure can also be installed, for example, as a software program running on a personal computer, an application running on a smartphone, etc. The method of this disclosure can also be provided to users in the form of SaaS (Software as a Service).

[0376] In the above embodiments, an example of a specific process being performed by a single computer 22 is given. However, the technology disclosed herein is not limited to this, and the specific process can also be distributed among multiple computers, including computer 22. For example, the data generation model 58 can be located on an external device of the data processing apparatus 12, where data is generated based on the input data.

[0377] In the above embodiments, examples of storing a specific processing program 56 in the memory 32 have been described, but the technology disclosed herein is not limited thereto. For example, the specific processing program 56 may also be stored in a portable computer-readable non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed into the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0378] Alternatively, a specific processing program 56 may be pre-stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 according to the requirements of the data processing device 12.

[0379] In addition, it is not necessary to store all the specific processing program 56 in the storage device such as the server connected to the data processing device 12 via the network 54 or in the memory 32; a portion of the specific processing program 56 may be stored in advance.

[0380] As hardware resources for performing specific processes, various processors, as shown below, can be used. For example, a CPU can be listed as a processor, which functions as a general-purpose processor that performs specific processes by executing software, i.e., a program. Furthermore, processors can be listed as special-purpose circuits such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application-Specific Integrated Circuits), which are processors with circuitry specifically designed to perform specific processes. Each processor has built-in or connected memory, and each processor executes specific processes using that memory.

[0381] The hardware resources for performing a specific process can consist of one of these various processors, or a combination of two or more processors of the same or different types (e.g., a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resources for performing a specific process can be a single processor.

[0382] As an example of a single processor, there are two approaches: First, a processor is composed of a combination of one or more CPUs and software, which functions as a hardware resource to perform a specific process; second, as represented by a SoC (System-on-a-chip), a processor is used to implement the functionality of the entire system, which includes multiple hardware resources for performing a specific process, using a single IC (Integrated Circuit) chip. In this way, the specific process is implemented by using one or more of the aforementioned processors as hardware resources.

[0383] Furthermore, the hardware architecture of these various processors, more specifically, can utilize circuits that combine semiconductor elements and other circuit components. Moreover, the specific process described above is just one example. Therefore, without departing from the main point, unnecessary steps can certainly be deleted, new steps added, or the processing order changed.

[0384] The descriptions and illustrations above are detailed explanations of a portion of the technology disclosed herein, and are merely one example of the technology disclosed herein. For example, the above descriptions of the structure, function, effect, and results are just one example of the structure, function, effect, and results of a portion of the technology disclosed herein. Therefore, without departing from the spirit of the technology disclosed herein, unnecessary parts may be deleted, new elements added, or replacements may be made to the descriptions and illustrations above. Furthermore, to avoid confusion and facilitate understanding of a portion of the technology disclosed herein, explanations of common technical knowledge that do not require special explanation under the premise of being able to implement the technology disclosed herein have been omitted from the descriptions and illustrations above.

[0385] All documents, patent applications and technical specifications set forth in this specification are incorporated herein by reference to the same extent that each document, patent application and technical specification is specifically and individually described therein and referenced by reference.

[0386] In addition, the following notes are provided in response to the above explanation.

[0387] Example 1

[0388] (Note 1)

[0389] An information processing system includes: an information acquisition device for acquiring a user's training goals and exercise types; a plan generation device for receiving information acquired by the information acquisition device and generating instructive text input to a generative artificial intelligence model, and generating a personalized training plan through the generative artificial intelligence model; a recording and sending device for capturing video files of the user's training process and sending the video files to the information processing device; an analysis device for analyzing the received video files according to a time sequence and comparing them with standard movement information to determine movement errors and correction areas; and a prompting device for providing the user with information on the correction areas and improvement methods.

[0390] (Note 2)

[0391] According to the information processing system described in Appendix 1, the parsing device segments and parses the motion frame by frame and compares it with standard motion performance information, thereby realizing motion accuracy assessment and correction point extraction.

[0392] (Note 3)

[0393] According to the information processing system described in Appendix 1, the plan generation device generates personalized optimized training plans based on a generative artificial intelligence model, with reference to a professional knowledge database and a motion theory database.

[0394] Application Example 1

[0395] (Note 1)

[0396] An information processing system includes: a data information acquisition device for acquiring data on user goals and activity content; a plan generation device for receiving data from the data information acquisition device and automatically generating guidance plans using a generative artificial intelligence model, referencing behavioral data and professional knowledge information stored in a storage device; an image recording and transmission device for enabling users to operate observation devices to record operational image data and transmitting the operational image data to a storage device via a communication line; an action analysis device for decomposing the transmitted operational image data, extracting action features frame by frame, comparing and evaluating them with benchmark actions, and performing error detection and improvement point suggestion analysis; an information provision device for optimizing feedback content and prompting methods based on the results of the action analysis device using an emotion inference model and outputting personalized guidance information; and an update control device for repeatedly using image data and analysis data for learning and continuously updating guidance content.

[0397] (Note 2)

[0398] According to the information processing system described in Appendix 1, the motion analysis device extracts motion data for each step and compares and evaluates it with a standard motion benchmark. Based on professional knowledge information applicable to various activity fields, it automatically determines errors and extracts improvement points.

[0399] (Note 3)

[0400] According to the information processing system described in Appendix 1, the plan generation device and the information providing device apply natural language processing of a generative artificial intelligence model that includes prompt statements to generate personalized optimized feedback based on user attributes, historical information and emotional state.

[0401] Example 2

[0402] (Note 1)

[0403] An information processing system includes: an information acquisition device for acquiring user target information and movement type information; an automatic generation device for automatically generating a training plan based on the information acquired by the information acquisition device; an image recording device for recording the user's movement; an image analysis device for transmitting the recorded movement images through a predetermined communication channel and analyzing the received image data; an improvement point extraction device for extracting movement errors and improvement points from the analysis results of the image analysis device; a guidance prompting device for generating and providing suggestions and guidance to the user based on the improvement points; and an emotion recognition and adjustment device for determining the user's emotional state through the image data or additional voice data, and adjusting the feedback content according to the emotional state; wherein the automatic generation device or the guidance prompting device includes a generation device employing a generative artificial intelligence model and is provided with a prompt processing device that provides prompt statements as input information to the generative artificial intelligence model.

[0404] (Note 2)

[0405] According to the information processing system described in Appendix 1, the image analysis device analyzes the image data as a single frame and compares it based on a reference action mode to determine the error and feature quantity.

[0406] (Note 3)

[0407] According to the information processing system described in Appendix 1, the automatic generation device optimizes and generates adaptive training plans for each user based on training-related knowledge, theoretical models, and generative artificial intelligence models.

[0408] Application Example 2

[0409] (Note 1)

[0410] An information processing system includes: an information input device for inputting user training objectives and classifications; a plan generation device for receiving information provided by the information input device and generating a training plan using a generative artificial intelligence model; a recording and transmission device for recording the user training process and uploading relevant data; an information analysis device for analyzing the uploaded image data and identifying errors and correction points based on benchmark movements; an information prompting device for providing the user with the identified correction points and personalized improvement methods; an emotion recognition device for identifying the user's emotional state by analyzing images, voice, or other biometric data and adjusting feedback content and prompting methods accordingly; and an information management device for managing and recording training implementation, analysis results, and feedback history.

[0411] (Note 2)

[0412] According to the information processing system described in Appendix 1, the information parsing device performs frame-by-frame analysis of human movements in the image data and extracts movement errors by comparing them with reference movements.

[0413] (Note 3)

[0414] According to the information processing system described in Appendix 1, the plan generation device optimizes the generative artificial intelligence model based on professional technical information in the knowledge database and the latest training theories, so as to personalize the training plan.

Claims

1. An information processing system, characterized in that, include: An input device used to input the user's training goals and types of exercise; A generating device for receiving information input from the input device and generating a training plan; A device for capturing and uploading training videos for users; Analysis device for analyzing the uploaded training video and determining motion errors and correction points; and Feedback provisioning device for providing users with the identified correction points and specific improvement methods.

2. The information processing system according to claim 1, characterized in that, The analysis device analyzes the user's actions frame by frame and compares them with professional performance.

3. The information processing system according to claim 1, characterized in that, The generating device optimizes training plans based on professional experience and the latest sports theories.

Images