System

Abstract

To provide a system which offers appropriate advice about a body movement and an attitude of a user in order to maintain health of the user.SOLUTION: A system includes means of: acquiring information about a body movement and an attitude of a user via a wearable device; transmitting the acquired information to a server directly or via a terminal; generating advice about the body movement and the attitude of the user in motion from the acquired information by using a generative model; and providing the generated advice to the user.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including a description and related instruction sentence regarding the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2022-180282 A Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional chatbots have had difficulty providing users with appropriate advice regarding their body movements and posture in order to maintain their health. [Means for solving the problem]

[0005] The present disclosure solves the conventional problems by acquiring information on the user's body movements and posture, generating advice on the user's body movements and posture during activities using a generative model, and providing the advice to the user. Specifically, information on the user's body movements and posture is acquired via a terminal carried by the user or a terminal owned by the user, and the server inputs the received information into a generative model and performs analysis. The terminal is preferably a wearable device worn by the user. At this time, information on the user's body movements and posture is collected and acquired using sensors such as an acceleration sensor and a gyro sensor mounted on the wearable device. The generative model is a neural network using a machine learning algorithm, and generates advice on the user's body movements and posture during activities. The generated advice is provided to the user, for example, in real time via the wearable device, thereby preventing injuries in the user's daily activities and contributing to maintaining the user's physical health. In addition, using a wearable device as described above makes it possible to provide advice for a wider variety of activities.

[0006] Here, a "wearable device" refers to an electronic device that a user can wear, such as a wristwatch or smart glasses. In addition, "information on body movement and posture" refers to data on the user's body movement and posture acquired by the wearable device through a sensor. A "generative model" refers to a model that may be a neural network trained using a machine learning algorithm and is used to generate advice on the user's body movement and posture during activity. An "advice" refers to providing specific instructions or advice to the user on improving their behavior or posture. "In real time" means that information is exchanged and processed instantly, and advice is provided while the user is active. A "user" refers to an individual or user who wears a wearable device and uses the system. [Brief description of the drawings]

[0007] [Figure 1]1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Diagram 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. FIG. [Diagram 3] FIG. 11 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Diagram 5] FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 13 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] 4 is a sequence diagram showing a process flow of the data processing system according to the first embodiment. FIG. [Figure 12] 11 is a sequence diagram showing a process flow of the data processing system in application example 1. FIG. [Figure 13] FIG. 11 is a sequence diagram showing the flow of processing of the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 11 is a sequence diagram showing the flow of processing in the data processing system in application example 2 when combined with an emotion engine. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0009] First, the terms used in the following description will be explained.

[0010] In the following embodiments, a signed processor (hereinafter simply referred to as a "processor") may be one arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be one type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), etc.

[0011] In the following embodiments, a signed RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by the processor.

[0012] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0013] In the following embodiments, a communication I / F (Interface) with a code is an interface including a communication processor and an antenna. The communication I / F controls communication between multiple computers. An example of a communication standard applied to the communication I / F is a wireless communication standard including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0014] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. In addition, in this specification, the same idea as "A and / or B" is also applied when three or more things are expressed by connecting them with "and / or."

[0015] [First embodiment] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0016] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0017] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[0018] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0019] The reception device 38 includes a touch panel 38A and a microphone 38B, and receives user input. The touch panel 38A detects contact with a pointer (e.g., a pen or a finger) to receive user input by the touch of the pointer. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0020] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (e.g., voice and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs voice according to instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a Complementary Metal-Oxide-Semiconductor (CMOS) image sensor or a Charge Coupled Device (CCD) image sensor.

[0021] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54.

[0022] FIG. 2 shows an example of main functions of the data processing device 12 and the smart device 14.

[0023] As shown in Fig. 2, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32. The specific process program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific process program 56 from the storage 32, and executes the read specific process program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific process program 56 executed on the RAM 30.

[0024] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0025] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores a reception output program 60. The reception output program 60 is used together with the specific processing program 56 by the data processing system 10. The processor 46 reads out the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0026] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0027] This embodiment is a system that is composed of the following elements.

[0028] 1. Wearable devices: A wearable device is an electronic device that can be worn by a user, such as a wristwatch, and is equipped with various sensors. The wearable device also acquires information about the user's body movements and posture and transmits the information to a terminal or a server.

[0029] 2. Server: The server inputs the received information into a generative model for analysis. The generative model is a neural network that uses a machine learning algorithm to generate advice on the user's body movements and posture during activities.

[0030] 3. Terminal: The terminal receives the generated advice from the server and provides it to the user. The advice is provided to the wearable device in real time. Note that the terminal may be the same as the wearable device.

[0031] An example of the advice providing process by the above-mentioned system will be described below. For example, consider a case where a user is jogging while wearing a wearable device. The wearable device constantly monitors the user's movements and posture using built-in sensors such as an acceleration sensor and a gyro sensor, and transmits the information obtained from the sensors to a server via a terminal or directly. The server inputs the received information into a generative model and analyzes the user's running style using a machine learning algorithm. If it is detected that the user's posture is incorrect, the generated advice is transmitted to the terminal as a specific instruction such as "run with your back straight for more effective training." The user can jog more effectively by checking the advice displayed on the terminal or on the wearable device received from the terminal and correcting their posture.

[0032] The process flow will be explained below.

[0033] Step 1: Acquire body movement and posture information using a wearable device. To acquire the information, the user wears a wearable device. The wearable device uses built-in sensors to collect information about the user's body movement and posture. The collected information is then sent to a server via the terminal or directly.

[0034] Step 2: Analyze using the generative model on the server. The server inputs the information received from the wearable device or terminal into the generative model. The generative model is a neural network that uses a machine learning algorithm and generates advice regarding the user's body movements and posture during activities. The server sends the advice generated by the generative model to the terminal.

[0035] Step 3: Display the advice on the terminal. The terminal displays the received advice on the terminal or on the wearable device by transmitting the advice to the wearable device. This allows the user to check the advice.

[0036] Example 1 Next, a description will be given of Example 1. In the following description, the data processing device 12 is referred to as a "server" and the smart device 14 is referred to as a "terminal."

[0037] Conventional exercise support systems have had difficulty obtaining information about the user's body movements and posture in real time and providing specific and effective advice based on that information. In addition, the wearable device worn by the user has limited analytical capabilities, which limits the accuracy and quality of the information obtained. Furthermore, the advice generated was general and abstract, making it difficult for the user to immediately understand and implement specific improvement methods.

[0038] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0039] In this invention, the server includes means for acquiring information about the user's body movements and posture via the wearable device, means for transmitting the acquired information to the server via the terminal or directly, means for generating advice about the user's body movements and posture during activity from the acquired information using a generative model, means for transmitting the generated advice to the terminal and providing it to the user, and means for providing the advice to the user in real time. This makes it possible to analyze information about the user's body movements and posture with high accuracy and in real time, and provide specific and effective advice.

[0040] A "wearable device" is an electronic device that can be worn by a user and is equipped with various sensors for obtaining information about body movements and posture.

[0041] A "terminal" is a device that transmits and receives information between a wearable device and a server, and has a function of providing generated advice to a user.

[0042] A "server" is a computer system that receives data transmitted from a terminal via a network, analyzes the data using a generative model, and generates advice.

[0043] A "generative model" is a neural network that uses a machine learning algorithm and is a model that analyzes data related to the user's body movements and posture to generate advice.

[0044] "Advice" is specific and effective instructions for correcting or improving the user's behavior based on the user's body movements and posture analyzed by the generative model.

[0045] "Information acquisition means" is a general term for methods and apparatuses that use various sensors mounted on wearable devices to collect data regarding the user's body movements and posture.

[0046] "Data transmission means" refers to a communication function or means for transmitting collected data to a terminal or a server.

[0047] The "analysis means" is a means for analyzing collected data using a generative model and generating appropriate advice for the user.

[0048] "Advice providing means" is a general term for a method or device for conveying generated advice to a user via a terminal.

[0049] "Real-time" refers to processing and information being provided immediately, without delay.

[0050] The embodiments of the present invention will be specifically described below.

[0051] First, the system consists of three main components: a wearable device, a terminal, and a server.

[0052] Wearable Devices A user puts on a wearable device. This device may be a wristwatch or have another shape, and is equipped with multiple sensors. Specifically, these sensors include an acceleration sensor and a gyro sensor. These sensors acquire data on the user's body movements and posture in real time. The acquired data is temporarily stored in the device and then transmitted to a terminal or a server.

[0053] Terminal The terminal is an intermediary device that transmits acquired data to the server. Terminals include smartphones and tablets. They have the function of receiving data from the wearable device via Bluetooth or Wi-Fi and transmitting that data to the server in real time. They also have the role of providing advice received from the server to the user.

[0054] server The server is the central analysis device of the system. The server receives data sent from the wearable device or terminal and analyzes it using a generative model. The generative model is a neural network using a machine learning algorithm, and generates specific and effective advice from information about the user's body movements and posture. The generated advice is sent back to the terminal and provided to the user.

[0055] Examples For example, consider the case where a user is jogging while wearing a wearable device. In this case, the following actions are performed:

[0056] 1. Wearable devices constantly monitor the user's movements and posture using acceleration sensors and gyro sensors.

[0057] 2. The acquired data is sent to the device via Bluetooth.

[0058] 3. The device transmits the received data to the server in real time via Wi-Fi.

[0059] 4. The server inputs the received data into the generative model and analyzes the user's movements and posture.

[0060] 5. The generative model generates specific advice, such as "Run with your back straight for a more effective workout."

[0061] 6. The generated advice is sent to the terminal, which displays it to the user.

[0062] Examples of prompt statements "If users are hunching their back while jogging, provide advice on effective posture correction methods."

[0063] Additional example prompts: "Generate appropriate advice when a user's running style is detected as poor." "Provide posture advice to help joggers get a more effective workout."

[0064] In this way, the user can receive specific and effective advice in real time, enabling the user to train more effectively.

[0065] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0066] Step 1: The user puts on the wearable device. This device is equipped with multiple sensors, such as an acceleration sensor and a gyro sensor, and collects data on the user's body movements and posture in real time.

[0067] Input: User's body movements and posture

[0068] Data processing: Data acquisition by sensors (accelerometer, gyro sensor)

[0069] Output: Acquired movement and posture data (e.g. XYZ axis acceleration data, angular velocity data)

[0070] Specific operation: The acceleration sensor captures the up and down movement while jogging, and the gyro sensor detects the rotation of the body.

[0071] Step 2: The wearable device transmits the acquired data to the terminal using a wireless communication method such as Bluetooth.

[0072] Input: Acquired movement and posture data

[0073] Data processing: Wireless data transmission (Bluetooth)

[0074] Output: Data sent to the terminal

[0075] Specific operation: The wearable device uses Bluetooth to transmit the acquired acceleration and angular velocity data to the terminal.

[0076] Step 3: The terminal transmits the data received from the wearable device to the server. The terminal uploads the data in real time via Wi-Fi.

[0077] Input: Data received from a wearable device

[0078] Data processing: Wireless data transmission (Wi-Fi)

[0079] Output: Data sent to the server

[0080] Specific operation: The sensor data received by the device is sent to the server via Wi-Fi.

[0081] Step 4: The server inputs the received data into the generative model, which is a neural network that uses machine learning algorithms to analyze the data.

[0082] Input: Data sent from the terminal

[0083] Data processing: Data analysis using generative models (neural networks)

[0084] Output: Analysis results (e.g. advice on how to improve the user's posture)

[0085] Specific operation: The server inputs sensor data into a generative AI model for analysis, analyzing the user's posture and movements.

[0086] Step 5: The server sends the advice generated by the generative model to the terminal, which includes specific instructions for the user to modify their behavior.

[0087] Input: Analysis results

[0088] Data processing: generating and sending advice (over the network)

[0089] Output: Advice sent to terminal

[0090] Specific operation: The server generates advice such as "If you run with your back straight, you will get a more effective workout" and sends it to the terminal.

[0091] Step 6: The terminal displays the received advice to the user. The display method may be a pop-up display on the smartphone screen, a voice notification, or a message displayed on the screen of the wearable device.

[0092] Input: Advice sent by the server

[0093] Data processing: Display of advice

[0094] Output:Notification to the user

[0095] Specific action: The device will display a notification on the smartphone screen or wearable device display, such as "Running with your back straight will give you a more effective workout."

[0096] Through the above steps, the user can receive specific advice in real time.

[0097] (Application example 1) Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0098] In conventional factory work, it is difficult to manage the movements and postures of workers, which has led to problems of reduced work efficiency and safety. In particular, poor posture and incorrect movements during work are cited as factors that can lead to industrial accidents and reduced production efficiency. To solve this, a system that can monitor the movements and postures of workers in real time and provide appropriate feedback is needed.

[0099] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0100] In this invention, the server includes a means for acquiring information on the user's body movements and posture, a means for generating advice on the user's body movements and posture during activity using a generative model, a means for providing the advice to the user, and a means for analyzing data on the movements and posture of factory workers and providing advice aimed at improving safety and efficiency. This makes it possible to improve safety and production efficiency by monitoring the movements and posture of workers in real time and providing specific advice.

[0101] A "user" is a person who uses the system or performs tasks.

[0102] "Information regarding body movement and posture" refers to data and measurement results regarding the user's body movement and posture.

[0103] A "generative model" is a neural network trained with machine learning algorithms to generate advice regarding a user's body movements and posture.

[0104] "Advice" refers to instructions or suggestions provided to a user that are intended to improve their body movement or posture.

[0105] A "wearable device" is an electronic device that can be worn by a user and is equipped with sensors related to body movements and posture.

[0106] A "factory worker" is a worker who performs work in a factory.

[0107] "Data analysis" refers to using a generative model to analyze the acquired information about body movements and postures.

[0108] "Safety" refers to reducing the risk of accidents and injuries when workers perform their work.

[0109] "Improving efficiency" refers to increasing work productivity and promoting optimal use of time and resources.

[0110] "Real-time" refers to information being obtained and advice being given almost instantly.

[0111] The present invention provides a system for monitoring the movements and postures of factory workers in real time to improve safety and work efficiency. The system is configured as follows.

[0112] Hardware configuration:

[0113] Wearable device: An electronic device with built-in sensors (accelerometer, gyro sensor, etc.) that is worn by the user to obtain information about the user's body movements and posture.

[0114] Server: Processes the received data and performs analysis using generative AI models. This server is a computer system that implements Python and machine learning libraries (e.g., TENSORFLOW (registered trademark), PyTorch).

[0115] Terminal: A device such as a smartphone or tablet that is a display device for providing advice from the server to the user. This terminal works in conjunction with the wearable device to send and receive data.

[0116] Software configuration:

[0117] 1. Data acquisition and transmission process: The wearable device collects information about the user's body movements and posture in real time through sensors. This data is then transmitted to a server via wireless communication.

[0118] 2. Data analysis process: The server inputs the received data into the generative AI model and uses machine learning algorithms to analyze the user's behavior. TensorFlow or PyTorch is used for the generative AI model.

[0119] 3. Advice generation process: The generative AI model generates specific advice for the user's movements and posture based on the analyzed data. For example, it may generate instructions such as "It is safer to work with your waist bent."

[0120] 4. Advice Providing Process: The server sends the generated advice to the terminal and displays it. The user can check the advice in real time on the terminal and perform safe and efficient work.

[0121] Examples: Consider a case where a worker is lifting heavy parts in a factory. The wearable device monitors the worker's body movements in real time and sends the data to a server. The server analyzes the received data with a generative AI model, and if it detects that the worker's posture is inappropriate, it generates advice such as "You should bend your waist more." This advice is sent to the terminal and can be checked by the worker in real time.

[0122] Example prompt: "If the user's acceleration is 0.01, 0.02, -0.98, and the gyroscope values ​​are 0.001, 0.002, 0.003, what body movement and posture corrections are required?"

[0123] This makes it possible to constantly optimize the movements of factory workers and support safe and efficient work.

[0124] The flow of the specific process in the application example 1 will be described with reference to FIG.

[0125] Step 1: The wearable device acquires information about the user's body movements and posture. Specifically, it uses an acceleration sensor and a gyro sensor to detect the user's body movements (e.g., 0.01, 0.02, -0.98) and posture (e.g., 0.001, 0.002, 0.003) and compiles them into a data packet.

[0126] Input: User's body movements and posture

[0127] Output: Sensor data packets (e.g. acceleration data, gyroscope data)

[0128] Step 2: The sensor data acquired by the wearable device is sent to the server via wireless communication. Specifically, data packets are transferred to the server using Bluetooth or Wi-Fi.

[0129] Input: Sensor data packet

[0130] Output: Data packets forwarded to the server

[0131] Step 3: The server inputs the received sensor data into the generative AI model for analysis. Specifically, the received data is preprocessed (e.g., noise filtering, normalization) and input into the generative AI model (using TensorFlow or PyTorch) to generate analysis results on the user's movements and postures.

[0132] Input: Data packet forwarded to the server

[0133] Output: Analysis results regarding the user's movements and postures

[0134] Step 4: The generative AI model generates advice based on the results of its analysis. Specifically, it generates specific feedback such as "bend your waist more" or "straighten your back" based on the analysis results.

[0135] Input: Analysis results regarding the user's movements and posture

[0136] Output: Specific advice message

[0137] Step 5: The server sends the generated advice to the terminal (e.g., a smartphone or tablet). Specifically, a protocol (e.g., HTTP, WebSocket) is used to send the generated advice message to the terminal application.

[0138] Input: A specific advice message

[0139] Output: Advice messages printed to the terminal.

[0140] Step 6: The device provides the received advice to the user. Specifically, the advice is displayed on the device display in real time. For example, a message such as "It would be safer if you bend your waist more" is displayed.

[0141] Input: Advice message displayed on terminal

[0142] Output: User-visible advice display

[0143] Furthermore, an emotion engine that estimates the emotion of the user may be combined. That is, the identification processing unit 290 may estimate the emotion of the user using the emotion identification model 59, and perform identification processing using the emotion of the user.

[0144] This embodiment is a system that is composed of the following elements.

[0145] 1. Wearable devices: A wearable device is an electronic device that can be worn by a user and is equipped with various sensors. The wearable device also acquires information about the user's body movements and posture and transmits the information to a terminal or a server.

[0146] 2. Server: The server inputs the received information into a generative model for analysis. The generative model is a neural network that uses a machine learning algorithm and generates advice regarding the user's body movements and posture during activities. The server is also combined with an emotion engine that recognizes the user's emotions and generates appropriate advice according to those emotions.

[0147] 3. Terminal: The terminal receives the generated advice from the server and provides it to the user. The advice is provided to the user in real time. Note that the terminal may be the same as the wearable device.

[0148] An example of the advice providing process by the above-mentioned system will be described below. For example, consider a case where a user is jogging while wearing a wearable device. The wearable device constantly monitors the user's movements and posture using sensors such as an acceleration sensor and a gyro sensor built into it, and transmits the information obtained from the sensors to the server via the terminal or directly. The server inputs the received information into a generative model and analyzes the user's running style using a machine learning algorithm. If it is detected that the user's posture is incorrect, the generated advice is transmitted to the terminal as a specific instruction such as "run with your back straight, you can train more effectively." In addition, the server recognizes the user's emotions using an emotion engine, and generates encouraging advice if the user is tired. For example, if the user feels tired, the generated advice is transmitted to the terminal as a specific encouraging word such as "You're doing a great job! Let's run to the end!" The user can check the advice displayed on the terminal or on the wearable device received from the terminal and correct their posture to jog more effectively. In addition, if the advice includes encouraging words, it is expected that the user's motivation to continue jogging will increase.

[0149] The process flow will be explained below.

[0150] Step 1: Acquire information on body movements and posture using a wearable device. To acquire the information, the user wears the wearable device. The wearable device uses built-in sensors to collect information on the user's body movements and posture. The collected information is then sent to a server via a terminal or directly. The wearable device or the terminal also transmits information for recognizing the user's emotions. The information for recognizing the user's emotions may include images of the user's facial expressions, pulse rate, etc.

[0151] Step 2: Analyze using the generative model on the server. The server inputs the information received from the wearable device or terminal into the generative model. The generative model is a neural network using a machine learning algorithm, and generates advice regarding the user's body movements and posture during activity. The server sends the advice generated by the generative model to the terminal. The emotion engine equipped in the server is also a neural network using a machine learning algorithm for recognizing the user's emotions. The server uses the emotion engine to recognize the user's emotions from the received information for recognizing the user's emotions, and determines whether the user is tired.

[0152] Step 3: The server generates appropriate advice based on the emotion. The server combines the analysis results with the emotion recognition results to generate appropriate advice for the user. If the user is tired, the generated advice includes specific instructions as words of encouragement, such as "You're doing great! Let's run to the end!"

[0153] Step 4: Display the advice on the terminal. The terminal displays the received advice on the terminal or on the wearable device by transmitting it to the wearable device. This allows the user to confirm the advice. After confirming the advice, the user can continue jogging while correcting their posture and being encouraged by the encouraging words.

[0154] Example 2 Next, a description will be given of Example 2. In the following description, the data processing device 12 is referred to as a "server" and the smart device 14 is referred to as a "terminal."

[0155] In recent years, there has been a demand for a system that monitors a user's body movements and posture and provides appropriate advice in training and daily life. However, conventional technology cannot provide advice that takes into account the user's emotional state, and there are limitations in terms of maintaining the user's motivation and providing effective training support. Therefore, the present invention aims to provide a system that improves the effectiveness of a user's training and increases the user's motivation by providing advice that takes into account not only the user's body movements and posture information but also the user's emotional state.

[0156] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0157] In this invention, the server includes a means for acquiring information on the user's body movements and posture, a means for generating advice on the user's body movements and posture during activity using a generative model, a means for providing the advice to the user, a means for analyzing the acquired information and including an emotion engine for recognizing the user's emotion, and a means for generating appropriate advice according to the user's emotion. This makes it possible to provide specific and appropriate advice in real time according to the user's body movements, posture, and emotional state.

[0158] "Means for acquiring information regarding the user's body movements and posture" refers to an apparatus or method for collecting data regarding the user's body movements and posture, and specifically includes a wearable device with a built-in sensor.

[0159] "Means for generating advice regarding a user's body movements and posture during an activity using a generative model" refers to a system for analyzing a user's body movements and posture data and generating advice based thereon, and includes a neural network trained with a machine learning algorithm.

[0160] The "means for providing the advice to the user" refers to a device or method for transmitting the generated advice to the user in real time, and uses a terminal or a display.

[0161] "Means including an emotion engine for analyzing acquired information and recognizing the user's emotions" refers to technology for analyzing the user's body movements and physiological data and recognizing the user's emotional state, including algorithms and software.

[0162] The "means for generating appropriate advice in accordance with the user's emotions" refers to a system for generating advice or messages suited to the user based on the recognized emotional state of the user, and includes an emotion analysis engine.

[0163] This system is designed to monitor the user's body movements and posture and provide appropriate advice. Specifically, the wearable device, terminal, and server work together to collect data, analyze it, and generate advice.

[0164] A wearable device is an electronic device that can be worn by a user and is equipped with various sensors such as an acceleration sensor and a gyro sensor. This makes it possible to obtain information about the user's body movements and posture in real time. For example, when a user is jogging, the wearable device records the movement of the user's legs and the angle of the back muscles. This information is transmitted to a terminal via wireless technologies such as Bluetooth and Wi-Fi.

[0165] The terminal is responsible for sending the received information to the server. The terminal is a device that the user can carry around, such as a smartphone or tablet. For example, the user's smartphone receives data from the wearable device and transfers the data to the server via the Internet.

[0166] The server inputs the received data into a generative AI model and analyzes the user's body movements and posture. This generative model is a neural network that uses machine learning algorithms to analyze the user's movement patterns and generate optimal advice. For example, the generated advice might be, "You can train more effectively if you run with your back straight."

[0167] In addition, the server also uses an emotion engine to analyze the user's emotional state. The emotion engine analyzes physiological data such as the user's heart rate and breathing patterns to recognize the user's emotional state, such as tiredness or concentration. For example, if the user is recognized as tired, an encouraging message such as "You're almost there, keep going!" is generated.

[0168] The generated advice is then provided to the user via the device. The user can check the advice in real time on their smartphone or tablet, correct their posture as necessary, and stay motivated. This allows the user to train more effectively, and is expected to increase their motivation.

[0169] Examples of prompts include: "Generate advice for when the user's posture is slouching." "Generate an encouraging message if you detect that the user is tired."

[0170] By using this system, users can receive specific and appropriate advice in real time based on their body movements, posture, and emotional state, enabling them to train more effectively.

[0171] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0172] Program processing flow

[0173] Step 1: The user puts on the wearable device.

[0174] Description: Before starting a jog or daily activity, the user puts on a wearable device on their arm or wrist. The device has built-in acceleration and gyro sensors, and collects data on the user's body movements and posture in real time.

[0175] Input: Sensor information from wearable devices

[0176] Output: User movement and posture data

[0177] Step 2: The wearable device monitors the user's body movements and posture.

[0178] Description: The wearable device continuously monitors the user's body movements and posture using built-in acceleration and gyro sensors. For example, while the user is jogging, it collects data such as foot movements, back angle, and hand movements.

[0179] Input: User movement and posture data

[0180] Output: Raw data as monitoring results

[0181] Step 3: The wearable device transmits the acquired data to the terminal or directly to the server.

[0182] Description: Wearable devices use wireless technologies such as Bluetooth or Wi-Fi to transmit the acquired data to a device such as a smartphone, or directly to a server.

[0183] Input: Raw data as monitoring results

[0184] Output: The data sent.

[0185] Step 4: The server inputs the received data into the generative AI model.

[0186] Description: The server receives the user's movement and posture data sent from the wearable device and inputs it into a generative AI model, which is a neural network using machine learning algorithms to analyze the data and generate appropriate advice.

[0187] Input: Data sent

[0188] Output: Input data for a generative AI model

[0189] Step 5: The generative model analyzes the data and generates appropriate advice.

[0190] Description: The generative AI model analyzes the data it receives. For example, if it detects that the user is slouching, it generates advice like "Stand up straight."

[0191] Input: Input data for the generative AI model

[0192] Output: Advice statement

[0193] Step 6: The server analyzes the user's emotions using the emotion engine.

[0194] Description: The server uses the emotion engine to analyze the user's heart rate, breathing patterns, etc., and recognizes the user's fatigue state and emotions. For example, if the user is recognized as tired, data is output to generate an encouraging message.

[0195] Input: User's physiological data

[0196] Output: Emotional state data

[0197] Step 7: The server generates advice according to the emotion.

[0198] Description: The server generates appropriate advice and encouraging messages for the user based on the emotional state data obtained from the emotion engine. For example, it generates an encouraging message such as "You're almost there, keep trying!"

[0199] Input: Emotional state data

[0200] Output: Advice sentences according to emotions

[0201] Step 8: The server sends the generated advice to the terminal.

[0202] Description: The server sends the generated advice to the user's device in real time, allowing the user to receive immediate feedback.

[0203] Input: Generated advice statement

[0204] Output: Advice text sent to terminal.

[0205] Step 9: The terminal provides advice to the user.

[0206] Description: The device provides the user with advice received from the server. For example, the device display shows "Please stand up straight" or "You're almost there, let's do our best!". The user can see this and correct their posture, which motivates them to continue training.

[0207] Input: Advice text sent to the terminal

[0208] Output: Provide advice to the user

[0209] By having each step work together in this way, users can receive appropriate advice in real time, enabling them to conduct more effective training and activities.

[0210] (Application example 2) Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0211] Conventional fitness instruction systems using wearable devices have the problem that they can only obtain information about the user's body movements and posture, and are unable to provide advice that takes into account the user's emotional state. This can make it difficult for users to maintain their training effectiveness and motivation.

[0212] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means. In this invention, the server includes a means for acquiring information on the user's body movements and posture, a means for generating advice on the user's body movements and posture during activity using a generative model, a means for providing the advice to the user, and a means for recognizing the user's emotions and generating appropriate advice according to the emotions. This makes it possible not only to provide the user with real-time posture and form correction guidance during training, but also to provide encouragement and advice for improving motivation that takes into account the user's emotional state. This improves the training effect of the user and makes it possible to maintain motivation.

[0213] "Information regarding the user's body movements and posture" refers to motion data such as the position, direction, speed, and acceleration of the user's body, as well as posture information during a specific activity, obtained via a wearable device.

[0214] A "generative model" is a neural network trained using a machine learning algorithm, and is a model that analyzes a user's body movements and postures to generate advice.

[0215] "Means for providing advice to a user" refers to a mechanism for notifying a user of the generated advice, and specifically refers to a terminal such as a smartphone, smart glasses, or a head-mounted display.

[0216] "Means for recognizing the user's emotions and generating appropriate advice based on those emotions" refers to a mechanism for analyzing the user's motion data and vital signs to estimate their emotional state and generate advice or encouraging messages that are adapted to that state.

[0217] The present invention is a system that tracks the user's body movements and postures, analyzes the data, and provides appropriate advice. It also includes a function to recognize the user's emotional state and generate encouraging messages according to the emotion. The system is composed of a wearable device, a server, and a terminal.

[0218] A wearable device is an electronic device worn by a user and is equipped with multiple sensors, such as an acceleration sensor and a gyro sensor, to obtain information about the user's body movements and posture. The wearable device transmits the obtained data to a terminal or directly to a server.

[0219] The server inputs the received data into a generative model and analyzes the user's body movements and posture. The generative model is a neural network trained using machine learning algorithms. The server generates advice to provide to the user based on the analysis results. It also has an emotion engine that recognizes the user's emotional state. Depending on the user's emotional state, it generates appropriate advice or encouraging messages.

[0220] The terminal is a device that receives the advice sent from the server and provides it to the user. Examples of such devices include smartphones, smart glasses, and head-mounted displays. The user can check the advice provided on the terminal and correct their own movements and posture. This advice is provided in real time and is expected to improve the effectiveness of the user's training.

[0221] As a concrete example, consider the case where a user is doing squats at a fitness club. The wearable device detects the user's movements and sends them to the server. The server analyzes the received data and generates specific advice such as "Bend your knees a bit more." If the emotion engine recognizes that the user is tired, it also sends an encouraging message such as "You're doing great! Not much longer!" The user can train while checking these instructions and encouragement on the device.

[0222] Examples of prompts to be input to a generative AI model include the following:

[0223] "This shows posture data when the user is wearing the wearable device. Based on the following data, determine whether the user's current squat posture is correct and generate advice if necessary. Also, consider the user's emotions and include words of encouragement if the user seems tired. Sensor data: {Accelerometer value}, {Gyro sensor value}."

[0224] This system allows users to receive real-time guidance on how to correct their form while training, and also provides encouraging messages that take into account their emotional state, improving the effectiveness of their training and helping them maintain their motivation.

[0225] The flow of the specific process in the application example 2 will be described with reference to FIG.

[0226] Step 1: The wearable device acquires information about the user's body movements and posture. This information is data obtained from acceleration and gyro sensors, and specifically includes the user's position, direction, speed, acceleration, etc. The input is the user's motion data, and the output is compiled as sensor data.

[0227] Step 2: The wearable device transmits the acquired sensor data to the terminal or directly to the server. This data is transmitted in real time and communicated using an appropriate protocol. The input is the sensor data and the output is the data to be sent to the server.

[0228] Step 3: The server inputs the received sensor data into a generative model to analyze the user's body movements and posture. The generative model is a neural network trained with machine learning algorithms and generates appropriate advice based on the analyzed data. The input is the sensor data and the output is advice as a result of the analysis.

[0229] Step 4: The server generates advice based on the analysis results and sends it to the terminal. The advice includes specific posture correction instructions. The input is the advice as the analysis result, and the output is the data to be sent to the terminal.

[0230] Step 5: At the same time, the server recognizes the user's emotional state through the emotion engine. This uses the motion data and vital sign data. It estimates the emotional state and generates advice or encouraging messages accordingly. The input is the motion data and vital sign data, and the output is advice based on the emotion.

[0231] Step 6: The server sends appropriate advice or encouraging messages to the terminal based on the emotion recognition result. The input is the advice as the emotion recognition result, and the output is the data to be sent to the terminal.

[0232] Step 7: The terminal provides the advice received from the server to the user. This can be done using a smartphone, smart glasses, a head-mounted display, etc., and the user can view the advice and correct their movements and posture. The input is the advice data from the server, and the output is the display on the terminal.

[0233] Step 8: The user checks the display on the device and corrects their movements and posture in real time to improve the training effect. In addition, motivation is maintained by encouraging messages that take into account the user's emotional state. The input is the display data on the device, and the output is the user's actions.

[0234] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0235] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by making a neural network perform deep learning. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image is input. The data generation model 58 performs inference on the input inference data according to the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0236] In the above embodiment, an example was given in which the specific process was performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0237] [Second embodiment] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0238] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214 as an example of a wearable device. An example of the data processing device 12 is a server.

[0239] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[0240] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0241] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0242] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[0243] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54. The transmission and reception of various types of information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0244] Fig. 4 shows an example of main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0245] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0246] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0247] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0248] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[0249] This embodiment is a system that is composed of the following elements.

[0250] 1. Server: The server inputs the received information into a generative model for analysis. The generative model is a neural network that uses a machine learning algorithm to generate advice on the user's body movements and posture during activities.

[0251] 3. Terminal: The terminal can be worn by the user and is equipped with various sensors. The terminal also acquires information on the user's body movements and posture and transmits it to the server. Furthermore, the terminal receives advice generated by the server and provides it to the user. The advice acquired from the server is provided in real time.

[0252] An example of the advice providing process by the above-mentioned system will be described below. For example, consider a case where a user is jogging while wearing a terminal. The terminal constantly monitors the user's movements and posture using built-in sensors such as an acceleration sensor and a gyro sensor, and transmits the information obtained from the sensors to a server. The server inputs the received information into a generative model and analyzes the user's running style using a machine learning algorithm. If it is detected that the user's posture is incorrect, the generated advice is transmitted to the terminal as specific instructions such as "run with your back straight for more effective training." The user can check the advice displayed on the terminal and correct their posture to jog more effectively.

[0253] The process flow will be explained below.

[0254] Step 1: The device acquires information on body movements and posture. To acquire the information, the user wears the device. The device uses built-in sensors to collect information on the user's body movements and posture. The collected information is then sent to the server.

[0255] Step 2: Analyze using the generative model on the server. The server inputs the information received from the device into the generative model. The generative model is a neural network that uses a machine learning algorithm and generates advice regarding the user's body movements and posture during activities. The server sends the advice generated by the generative model to the device.

[0256] Step 3: The received advice is displayed on the terminal, so that the user can confirm the advice.

[0257] Example 1 Next, a description will be given of Example 1. In the following description, the data processing device 12 is referred to as a "server" and the smart glasses 214 are referred to as a "terminal".

[0258] Conventional exercise support systems have had difficulty obtaining information about the user's body movements and posture in real time and providing specific and effective advice based on that information. In addition, the wearable device worn by the user has limited analytical capabilities, which limits the accuracy and quality of the information obtained. Furthermore, the advice generated was general and abstract, making it difficult for the user to immediately understand and implement specific improvement methods.

[0259] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0260] In this invention, the server includes means for acquiring information about the user's body movements and posture via the wearable device, means for transmitting the acquired information to the server via the terminal or directly, means for generating advice about the user's body movements and posture during activity from the acquired information using a generative model, means for transmitting the generated advice to the terminal and providing it to the user, and means for providing the advice to the user in real time. This makes it possible to analyze information about the user's body movements and posture with high accuracy and in real time, and provide specific and effective advice.

[0261] A "wearable device" is an electronic device that can be worn by a user and is equipped with various sensors for obtaining information about body movements and posture.

[0262] A "terminal" is a device that transmits and receives information between a wearable device and a server, and has a function of providing generated advice to a user.

[0263] A "server" is a computer system that receives data transmitted from a terminal via a network, analyzes the data using a generative model, and generates advice.

[0264] A "generative model" is a neural network that uses a machine learning algorithm and is a model that analyzes data related to the user's body movements and posture to generate advice.

[0265] "Advice" is specific and effective instructions for correcting or improving the user's behavior based on the user's body movements and posture analyzed by the generative model.

[0266] "Information acquisition means" is a general term for methods and apparatuses that use various sensors mounted on wearable devices to collect data regarding the user's body movements and posture.

[0267] "Data transmission means" refers to a communication function or means for transmitting collected data to a terminal or a server.

[0268] The "analysis means" is a means for analyzing collected data using a generative model and generating appropriate advice for the user.

[0269] "Advice providing means" is a general term for a method or device for conveying generated advice to a user via a terminal.

[0270] "Real-time" refers to processing and information being provided immediately, without delay.

[0271] The embodiments of the present invention will be specifically described below.

[0272] First, the system consists of three main components: a wearable device, a terminal, and a server.

[0273] Wearable Devices A user puts on a wearable device. This device may be a wristwatch or have another shape, and is equipped with multiple sensors. Specifically, these sensors include an acceleration sensor and a gyro sensor. These sensors acquire data on the user's body movements and posture in real time. The acquired data is temporarily stored in the device and then transmitted to a terminal or a server.

[0274] Terminal The terminal is an intermediary device that transmits acquired data to the server. Terminals include smartphones and tablets. They have the function of receiving data from the wearable device via Bluetooth or Wi-Fi and transmitting that data to the server in real time. They also have the role of providing advice received from the server to the user.

[0275] server The server is the central analysis device of the system. The server receives data sent from the wearable device or terminal and analyzes it using a generative model. The generative model is a neural network using a machine learning algorithm, and generates specific and effective advice from information about the user's body movements and posture. The generated advice is sent back to the terminal and provided to the user.

[0276] Examples For example, consider the case where a user is jogging while wearing a wearable device. In this case, the following actions are performed:

[0277] 1. Wearable devices constantly monitor the user's movements and posture using acceleration sensors and gyro sensors.

[0278] 2. The acquired data is sent to the device via Bluetooth.

[0279] 3. The device transmits the received data to the server in real time via Wi-Fi.

[0280] 4. The server inputs the received data into the generative model and analyzes the user's movements and posture.

[0281] 5. The generative model generates specific advice, such as "Run with your back straight for a more effective workout."

[0282] 6. The generated advice is sent to the terminal, which displays it to the user.

[0283] Examples of prompt statements "If users are hunching their back while jogging, provide advice on effective posture correction methods."

[0284] Additional example prompts: "Generate appropriate advice when a user's running style is detected as poor." "Provide posture advice to help joggers get a more effective workout."

[0285] In this way, the user can receive specific and effective advice in real time, enabling the user to train more effectively.

[0286] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0287] Step 1: The user puts on the wearable device. This device is equipped with multiple sensors, such as an acceleration sensor and a gyro sensor, and collects data on the user's body movements and posture in real time.

[0288] Input: User's body movements and posture

[0289] Data processing: Data acquisition by sensors (accelerometer, gyro sensor)

[0290] Output: Acquired movement and posture data (e.g. XYZ axis acceleration data, angular velocity data)

[0291] Specific operation: The acceleration sensor captures the up and down movement while jogging, and the gyro sensor detects the rotation of the body.

[0292] Step 2: The wearable device transmits the acquired data to the terminal using a wireless communication method such as Bluetooth.

[0293] Input: Acquired movement and posture data

[0294] Data processing: Wireless data transmission (Bluetooth)

[0295] Output: Data sent to the terminal

[0296] Specific operation: The wearable device uses Bluetooth to transmit the acquired acceleration and angular velocity data to the terminal.

[0297] Step 3: The terminal transmits the data received from the wearable device to the server. The terminal uploads the data in real time via Wi-Fi.

[0298] Input: Data received from a wearable device

[0299] Data processing: Wireless data transmission (Wi-Fi)

[0300] Output: Data sent to the server

[0301] Specific operation: The sensor data received by the device is sent to the server via Wi-Fi.

[0302] Step 4: The server inputs the received data into the generative model, which is a neural network that uses machine learning algorithms to analyze the data.

[0303] Input: Data sent from the terminal

[0304] Data processing: Data analysis using generative models (neural networks)

[0305] Output: Analysis results (e.g. advice on how to improve the user's posture)

[0306] Specific operation: The server inputs sensor data into a generative AI model for analysis, analyzing the user's posture and movements.

[0307] Step 5: The server sends the advice generated by the generative model to the terminal, which includes specific instructions for the user to modify their behavior.

[0308] Input: Analysis results

[0309] Data processing: generating and sending advice (over the network)

[0310] Output: Advice sent to terminal

[0311] Specific operation: The server generates advice such as "If you run with your back straight, you will get a more effective workout" and sends it to the terminal.

[0312] Step 6: The terminal displays the received advice to the user. The display method may be a pop-up display on the smartphone screen, a voice notification, or a message displayed on the screen of the wearable device.

[0313] Input: Advice sent by the server

[0314] Data processing: Display of advice

[0315] Output:Notification to the user

[0316] Specific action: The device will display a notification on the smartphone screen or wearable device display, such as "Running with your back straight will give you a more effective workout."

[0317] Through the above steps, the user can receive specific advice in real time.

[0318] (Application example 1) Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[0319] In conventional factory work, it is difficult to manage the movements and postures of workers, which has led to problems of reduced work efficiency and safety. In particular, poor posture and incorrect movements during work are cited as factors that can lead to industrial accidents and reduced production efficiency. To solve this, a system that can monitor the movements and postures of workers in real time and provide appropriate feedback is needed.

[0320] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0321] In this invention, the server includes a means for acquiring information on the user's body movements and posture, a means for generating advice on the user's body movements and posture during activity using a generative model, a means for providing the advice to the user, and a means for analyzing data on the movements and posture of factory workers and providing advice aimed at improving safety and efficiency. This makes it possible to improve safety and production efficiency by monitoring the movements and posture of workers in real time and providing specific advice.

[0322] A "user" is a person who uses the system or performs tasks.

[0323] "Information regarding body movement and posture" refers to data and measurement results regarding the user's body movement and posture.

[0324] A "generative model" is a neural network trained with machine learning algorithms to generate advice regarding a user's body movements and posture.

[0325] "Advice" refers to instructions or suggestions provided to a user that are intended to improve their body movement or posture.

[0326] A "wearable device" is an electronic device that can be worn by a user and is equipped with sensors related to body movements and posture.

[0327] A "factory worker" is a worker who performs work in a factory.

[0328] "Data analysis" refers to using a generative model to analyze the acquired information about body movements and postures.

[0329] "Safety" refers to reducing the risk of accidents and injuries when workers perform their work.

[0330] "Improving efficiency" refers to increasing work productivity and promoting optimal use of time and resources.

[0331] "Real-time" refers to information being obtained and advice being given almost instantly.

[0332] The present invention provides a system for monitoring the movements and postures of factory workers in real time to improve safety and work efficiency. The system is configured as follows.

[0333] Hardware configuration:

[0334] Wearable device: An electronic device with built-in sensors (accelerometer, gyro sensor, etc.) that is worn by the user to obtain information about the user's body movements and posture.

[0335] Server: Processes the received data and performs analysis using generative AI models. This server is a computer system that implements Python and machine learning libraries (e.g. TensorFlow, PyTorch).

[0336] Terminal: A device such as a smartphone or tablet that is a display device for providing advice from the server to the user. This terminal works in conjunction with the wearable device to send and receive data.

[0337] Software configuration:

[0338] 1. Data acquisition and transmission process: The wearable device collects information about the user's body movements and posture in real time through sensors. This data is then transmitted to a server via wireless communication.

[0339] 2. Data analysis process: The server inputs the received data into the generative AI model and uses machine learning algorithms to analyze the user's behavior. TensorFlow or PyTorch is used for the generative AI model.

[0340] 3. Advice generation process: The generative AI model generates specific advice for the user's movements and posture based on the analyzed data. For example, it may generate instructions such as "It is safer to work with your waist bent."

[0341] 4. Advice Providing Process: The server sends the generated advice to the terminal and displays it. The user can check the advice in real time on the terminal and perform safe and efficient work.

[0342] Examples: Consider a case where a worker is lifting heavy parts in a factory. The wearable device monitors the worker's body movements in real time and sends the data to a server. The server analyzes the received data with a generative AI model, and if it detects that the worker's posture is inappropriate, it generates advice such as "You should bend your waist more." This advice is sent to the terminal and can be checked by the worker in real time.

[0343] Example prompt: "If the user's acceleration is 0.01, 0.02, -0.98, and the gyroscope values ​​are 0.001, 0.002, 0.003, what body movement and posture corrections are required?"

[0344] This makes it possible to constantly optimize the movements of factory workers and support safe and efficient work.

[0345] The flow of the specific process in the application example 1 will be described with reference to FIG.

[0346] Step 1: The wearable device acquires information about the user's body movements and posture. Specifically, it uses an acceleration sensor and a gyro sensor to detect the user's body movements (e.g., 0.01, 0.02, -0.98) and posture (e.g., 0.001, 0.002, 0.003) and compiles them into a data packet.

[0347] Input: User's body movements and posture

[0348] Output: Sensor data packets (e.g. acceleration data, gyroscope data)

[0349] Step 2: The sensor data acquired by the wearable device is sent to the server via wireless communication. Specifically, data packets are transferred to the server using Bluetooth or Wi-Fi.

[0350] Input: Sensor data packet

[0351] Output: Data packets forwarded to the server

[0352] Step 3: The server inputs the received sensor data into the generative AI model for analysis. Specifically, the received data is preprocessed (e.g., noise filtering, normalization) and input into the generative AI model (using TensorFlow or PyTorch) to generate analysis results on the user's movements and postures.

[0353] Input: Data packet forwarded to the server

[0354] Output: Analysis results regarding the user's movements and postures

[0355] Step 4: The generative AI model generates advice based on the results of its analysis. Specifically, it generates specific feedback such as "bend your waist more" or "straighten your back" based on the analysis results.

[0356] Input: Analysis results regarding the user's movements and posture

[0357] Output: Specific advice message

[0358] Step 5: The server sends the generated advice to the terminal (e.g., a smartphone or tablet). Specifically, a protocol (e.g., HTTP, WebSocket) is used to send the generated advice message to the terminal application.

[0359] Input: A specific advice message

[0360] Output: Advice messages printed to the terminal.

[0361] Step 6: The device provides the received advice to the user. Specifically, the advice is displayed on the device display in real time. For example, a message such as "It would be safer if you bend your waist more" is displayed.

[0362] Input: Advice message displayed on terminal

[0363] Output: User-visible advice display

[0364] In addition, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0365] This embodiment is a system that is composed of the following elements.

[0366] 1. Server: The server inputs the received information into a generative model for analysis. The generative model is a neural network that uses a machine learning algorithm and generates advice regarding the user's body movements and posture during activities. The server is also combined with an emotion engine that recognizes the user's emotions and generates appropriate advice according to those emotions.

[0367] 2. Terminal: The terminal can be worn by the user and is equipped with various sensors. The terminal also acquires information on the user's body movements and posture and transmits it to the server. Furthermore, the terminal receives advice generated by the server and provides it to the user. The advice is provided in real time.

[0368] An example of the advice providing process by the above-mentioned system will be described below. For example, consider a case where a user is jogging while wearing a terminal. The terminal constantly monitors the user's movement and posture using sensors such as an acceleration sensor and a gyro sensor built in, and transmits the information obtained from the sensors to the server. The server inputs the received information into a generative model and analyzes the user's running style using a machine learning algorithm. If it is detected that the user's posture is incorrect, the generated advice is transmitted to the terminal as a specific instruction such as "run with your back straight, you can train more effectively." In addition, the server recognizes the user's emotions using an emotion engine, and generates encouraging advice if the user is tired. For example, if the user feels tired, the generated advice is transmitted to the terminal as a specific encouraging word such as "You're doing a great job! Let's run to the end!" The user can jog more effectively by checking the advice displayed on the terminal and correcting their posture. In addition, if the advice includes encouraging words, it is expected that the user's motivation to continue jogging will increase.

[0369] The process flow will be explained below.

[0370] Step 1: The terminal acquires information on body movement and posture. To acquire the information, the user wears the terminal. The terminal uses a built-in sensor to collect information on the user's body movement and posture. The collected information is then transmitted to a server. The terminal also transmits information for recognizing the user's emotions. The information for recognizing the user's emotions may include an image of the user's facial expression, pulse rate, etc.

[0371] Step 2: Analyze using the generative model on the server. The server inputs the information received from the device into the generative model. The generative model is a neural network that uses a machine learning algorithm, and generates advice regarding the user's body movements and posture during activity. The server sends the advice generated by the generative model to the device. The emotion engine equipped on the server is also a neural network that uses a machine learning algorithm to recognize the user's emotions. The server uses the emotion engine to recognize the user's emotions from the received information for recognizing the user's emotions, and determines whether the user is tired.

[0372] Step 3: The server generates appropriate advice based on the emotion. The server combines the analysis results with the emotion recognition results to generate appropriate advice for the user. If the user is tired, the generated advice includes specific instructions as words of encouragement, such as "You're doing great! Let's run to the end!"

[0373] Step 4: Display the advice on the terminal. The terminal displays the received advice on the terminal. This allows the user to confirm the advice. Then, the user who has confirmed the advice can continue jogging while correcting their posture and being encouraged by the encouraging words.

[0374] Example 2 Next, a description will be given of Example 2. In the following description, the data processing device 12 is referred to as a "server" and the smart glasses 214 are referred to as a "terminal".

[0375] In recent years, there has been a demand for a system that monitors a user's body movements and posture and provides appropriate advice in training and daily life. However, conventional technology cannot provide advice that takes into account the user's emotional state, and there are limitations in terms of maintaining the user's motivation and providing effective training support. Therefore, the present invention aims to provide a system that improves the effectiveness of a user's training and increases the user's motivation by providing advice that takes into account not only the user's body movements and posture information but also the user's emotional state.

[0376] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0377] In this invention, the server includes a means for acquiring information on the user's body movements and posture, a means for generating advice on the user's body movements and posture during activity using a generative model, a means for providing the advice to the user, a means for analyzing the acquired information and including an emotion engine for recognizing the user's emotion, and a means for generating appropriate advice according to the user's emotion. This makes it possible to provide specific and appropriate advice in real time according to the user's body movements, posture, and emotional state.

[0378] "Means for acquiring information regarding the user's body movements and posture" refers to an apparatus or method for collecting data regarding the user's body movements and posture, and specifically includes a wearable device with a built-in sensor.

[0379] "Means for generating advice regarding a user's body movements and posture during an activity using a generative model" refers to a system for analyzing a user's body movements and posture data and generating advice based thereon, and includes a neural network trained with a machine learning algorithm.

[0380] The "means for providing the advice to the user" refers to a device or method for transmitting the generated advice to the user in real time, and uses a terminal or a display.

[0381] "Means including an emotion engine for analyzing acquired information and recognizing the user's emotions" refers to technology for analyzing the user's body movements and physiological data and recognizing the user's emotional state, including algorithms and software.

[0382] The "means for generating appropriate advice in accordance with the user's emotions" refers to a system for generating advice or messages suited to the user based on the recognized emotional state of the user, and includes an emotion analysis engine.

[0383] This system is designed to monitor the user's body movements and posture and provide appropriate advice. Specifically, the wearable device, terminal, and server work together to collect data, analyze it, and generate advice.

[0384] A wearable device is an electronic device that can be worn by a user and is equipped with various sensors such as an acceleration sensor and a gyro sensor. This makes it possible to obtain information about the user's body movements and posture in real time. For example, when a user is jogging, the wearable device records the movement of the user's legs and the angle of the back muscles. This information is transmitted to a terminal via wireless technologies such as Bluetooth and Wi-Fi.

[0385] The terminal is responsible for sending the received information to the server. The terminal is a device that the user can carry around, such as a smartphone or tablet. For example, the user's smartphone receives data from the wearable device and transfers the data to the server via the Internet.

[0386] The server inputs the received data into a generative AI model and analyzes the user's body movements and posture. This generative model is a neural network that uses machine learning algorithms to analyze the user's movement patterns and generate optimal advice. For example, the generated advice might be, "You can train more effectively if you run with your back straight."

[0387] In addition, the server also uses an emotion engine to analyze the user's emotional state. The emotion engine analyzes physiological data such as the user's heart rate and breathing patterns to recognize the user's emotional state, such as tiredness or concentration. For example, if the user is recognized as tired, an encouraging message such as "You're almost there, keep going!" is generated.

[0388] The generated advice is then provided to the user via the device. The user can check the advice in real time on their smartphone or tablet, correct their posture as necessary, and stay motivated. This allows the user to train more effectively, and is expected to increase their motivation.

[0389] Examples of prompts include: "Generate advice for when the user's posture is slouching." "Generate an encouraging message if you detect that the user is tired."

[0390] By using this system, users can receive specific and appropriate advice in real time based on their body movements, posture, and emotional state, enabling them to train more effectively.

[0391] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0392] Program processing flow

[0393] Step 1: The user puts on the wearable device.

[0394] Description: Before starting a jog or daily activity, the user puts on a wearable device on their arm or wrist. The device has built-in acceleration and gyro sensors, and collects data on the user's body movements and posture in real time.

[0395] Input: Sensor information from wearable devices

[0396] Output: User movement and posture data

[0397] Step 2: The wearable device monitors the user's body movements and posture.

[0398] Description: The wearable device continuously monitors the user's body movements and posture using built-in acceleration and gyro sensors. For example, while the user is jogging, it collects data such as foot movements, back angle, and hand movements.

[0399] Input: User movement and posture data

[0400] Output: Raw data as monitoring results

[0401] Step 3: The wearable device transmits the acquired data to the terminal or directly to the server.

[0402] Description: Wearable devices use wireless technologies such as Bluetooth or Wi-Fi to transmit the acquired data to a device such as a smartphone, or directly to a server.

[0403] Input: Raw data as monitoring results

[0404] Output: The data sent.

[0405] Step 4: The server inputs the received data into the generative AI model.

[0406] Description: The server receives the user's movement and posture data sent from the wearable device and inputs it into a generative AI model, which is a neural network using machine learning algorithms to analyze the data and generate appropriate advice.

[0407] Input: Data sent

[0408] Output: Input data for a generative AI model

[0409] Step 5: The generative model analyzes the data and generates appropriate advice.

[0410] Description: The generative AI model analyzes the data it receives. For example, if it detects that the user is slouching, it generates advice like "Stand up straight."

[0411] Input: Input data for the generative AI model

[0412] Output: Advice statement

[0413] Step 6: The server analyzes the user's emotions using the emotion engine.

[0414] Description: The server uses the emotion engine to analyze the user's heart rate, breathing patterns, etc., and recognizes the user's fatigue state and emotions. For example, if the user is recognized as tired, data is output to generate an encouraging message.

[0415] Input: User's physiological data

[0416] Output: Emotional state data

[0417] Step 7: The server generates advice according to the emotion.

[0418] Description: The server generates appropriate advice and encouraging messages for the user based on the emotional state data obtained from the emotion engine. For example, it generates an encouraging message such as "You're almost there, keep trying!"

[0419] Input: Emotional state data

[0420] Output: Advice sentences according to emotions

[0421] Step 8: The server sends the generated advice to the terminal.

[0422] Description: The server sends the generated advice to the user's device in real time, allowing the user to receive immediate feedback.

[0423] Input: Generated advice statement

[0424] Output: Advice text sent to terminal.

[0425] Step 9: The terminal provides advice to the user.

[0426] Description: The device provides the user with advice received from the server. For example, the device display shows "Please stand up straight" or "You're almost there, let's do our best!". The user can see this and correct their posture, which motivates them to continue training.

[0427] Input: Advice text sent to the terminal

[0428] Output: Provide advice to the user

[0429] By having each step work together in this way, users can receive appropriate advice in real time, enabling them to conduct more effective training and activities.

[0430] (Application example 2) Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[0431] Conventional fitness instruction systems using wearable devices have the problem that they can only obtain information about the user's body movements and posture, and are unable to provide advice that takes into account the user's emotional state. This can make it difficult for users to maintain their training effectiveness and motivation.

[0432] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means. In this invention, the server includes a means for acquiring information on the user's body movements and posture, a means for generating advice on the user's body movements and posture during activity using a generative model, a means for providing the advice to the user, and a means for recognizing the user's emotions and generating appropriate advice according to the emotions. This makes it possible not only to provide the user with real-time posture and form correction guidance during training, but also to provide encouragement and advice for improving motivation that takes into account the user's emotional state. This improves the training effect of the user and makes it possible to maintain motivation.

[0433] "Information regarding the user's body movements and posture" refers to motion data such as the position, direction, speed, and acceleration of the user's body, as well as posture information during a specific activity, obtained via a wearable device.

[0434] A "generative model" is a neural network trained using a machine learning algorithm, and is a model that analyzes a user's body movements and postures to generate advice.

[0435] "Means for providing advice to a user" refers to a mechanism for notifying a user of the generated advice, and specifically refers to a terminal such as a smartphone, smart glasses, or a head-mounted display.

[0436] "Means for recognizing the user's emotions and generating appropriate advice based on those emotions" refers to a mechanism for analyzing the user's motion data and vital signs to estimate their emotional state and generate advice or encouraging messages that are adapted to that state.

[0437] The present invention is a system that tracks the user's body movements and postures, analyzes the data, and provides appropriate advice. It also includes a function to recognize the user's emotional state and generate encouraging messages according to the emotion. The system is composed of a wearable device, a server, and a terminal.

[0438] A wearable device is an electronic device worn by a user and is equipped with multiple sensors, such as an acceleration sensor and a gyro sensor, to obtain information about the user's body movements and posture. The wearable device transmits the obtained data to a terminal or directly to a server.

[0439] The server inputs the received data into a generative model and analyzes the user's body movements and posture. The generative model is a neural network trained using machine learning algorithms. The server generates advice to provide to the user based on the analysis results. It also has an emotion engine that recognizes the user's emotional state. Depending on the user's emotional state, it generates appropriate advice or encouraging messages.

[0440] The terminal is a device that receives the advice sent from the server and provides it to the user. Examples of such devices include smartphones, smart glasses, and head-mounted displays. The user can check the advice provided on the terminal and correct their own movements and posture. This advice is provided in real time and is expected to improve the effectiveness of the user's training.

[0441] As a concrete example, consider the case where a user is doing squats at a fitness club. The wearable device detects the user's movements and sends them to the server. The server analyzes the received data and generates specific advice such as "Bend your knees a bit more." If the emotion engine recognizes that the user is tired, it also sends an encouraging message such as "You're doing great! Not much longer!" The user can train while checking these instructions and encouragement on the device.

[0442] Examples of prompts to be input to a generative AI model include the following:

[0443] "This shows posture data when the user is wearing the wearable device. Based on the following data, determine whether the user's current squat posture is correct and generate advice if necessary. Also, consider the user's emotions and include words of encouragement if the user seems tired. Sensor data: {Accelerometer value}, {Gyro sensor value}."

[0444] This system allows users to receive real-time guidance on how to correct their form while training, and also provides encouraging messages that take into account their emotional state, improving the effectiveness of their training and helping them maintain their motivation.

[0445] The flow of the specific process in the application example 2 will be described with reference to FIG.

[0446] Step 1: The wearable device acquires information about the user's body movements and posture. This information is data obtained from acceleration and gyro sensors, and specifically includes the user's position, direction, speed, acceleration, etc. The input is the user's motion data, and the output is compiled as sensor data.

[0447] Step 2: The wearable device transmits the acquired sensor data to the terminal or directly to the server. This data is transmitted in real time and communicated using an appropriate protocol. The input is the sensor data and the output is the data to be sent to the server.

[0448] Step 3: The server inputs the received sensor data into a generative model to analyze the user's body movements and posture. The generative model is a neural network trained with machine learning algorithms and generates appropriate advice based on the analyzed data. The input is the sensor data and the output is advice as a result of the analysis.

[0449] Step 4: The server generates advice based on the analysis results and sends it to the terminal. The advice includes specific posture correction instructions. The input is the advice as the analysis result, and the output is the data to be sent to the terminal.

[0450] Step 5: At the same time, the server recognizes the user's emotional state through the emotion engine. This uses the motion data and vital sign data. It estimates the emotional state and generates advice or encouraging messages accordingly. The input is the motion data and vital sign data, and the output is advice based on the emotion.

[0451] Step 6: The server sends appropriate advice or encouraging messages to the terminal based on the emotion recognition result. The input is the advice as the emotion recognition result, and the output is the data to be sent to the terminal.

[0452] Step 7: The terminal provides the advice received from the server to the user. This can be done using a smartphone, smart glasses, a head-mounted display, etc., and the user can view the advice and correct their movements and posture. The input is the advice data from the server, and the output is the display on the terminal.

[0453] Step 8: The user checks the display on the device and corrects their movements and posture in real time to improve the training effect. In addition, motivation is maintained by encouraging messages that take into account the user's emotional state. The input is the display data on the device, and the output is the user's actions.

[0454] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0455] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by making a neural network perform deep learning. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image is input. The data generation model 58 performs inference on the input inference data according to the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0456] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0457] [Third embodiment] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0458] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0459] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[0460] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0461] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0462] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[0463] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54. The transmission and reception of various types of information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0464] Fig. 6 shows an example of main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0465] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0466] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0467] In the headset type terminal 314, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0468] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server", and the headset type terminal 314 will be referred to as the "terminal".

[0469] This embodiment is a system that is composed of the following elements.

[0470] 1. Server: The server inputs the received information into a generative model for analysis. The generative model is a neural network that uses a machine learning algorithm to generate advice on the user's body movements and posture during activities.

[0471] 3. Terminal: The terminal can be worn by the user and is equipped with various sensors. The terminal also acquires information on the user's body movements and posture and transmits it to the server. Furthermore, the terminal receives advice generated by the server and provides it to the user. The advice acquired from the server is provided in real time.

[0472] An example of the advice providing process by the above-mentioned system will be described below. For example, consider a case where a user is jogging while wearing a terminal. The terminal constantly monitors the user's movements and posture using built-in sensors such as an acceleration sensor and a gyro sensor, and transmits the information obtained from the sensors to a server. The server inputs the received information into a generative model and analyzes the user's running style using a machine learning algorithm. If it is detected that the user's posture is incorrect, the generated advice is transmitted to the terminal as specific instructions such as "run with your back straight for more effective training." The user can check the advice displayed on the terminal and correct their posture to jog more effectively.

[0473] The process flow will be explained below.

[0474] Step 1: The device acquires information on body movements and posture. To acquire the information, the user wears the device. The device uses built-in sensors to collect information on the user's body movements and posture. The collected information is then sent to the server.

[0475] Step 2: Analyze using the generative model on the server. The server inputs the information received from the device into the generative model. The generative model is a neural network that uses a machine learning algorithm and generates advice regarding the user's body movements and posture during activities. The server sends the advice generated by the generative model to the device.

[0476] Step 3: The received advice is displayed on the terminal, so that the user can confirm the advice.

[0477] Example 1 Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal".

[0478] Conventional exercise support systems have had difficulty obtaining information about the user's body movements and posture in real time and providing specific and effective advice based on that information. In addition, the wearable device worn by the user has limited analytical capabilities, which limits the accuracy and quality of the information obtained. Furthermore, the advice generated was general and abstract, making it difficult for the user to immediately understand and implement specific improvement methods.

[0479] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0480] In this invention, the server includes means for acquiring information about the user's body movements and posture via the wearable device, means for transmitting the acquired information to the server via the terminal or directly, means for generating advice about the user's body movements and posture during activity from the acquired information using a generative model, means for transmitting the generated advice to the terminal and providing it to the user, and means for providing the advice to the user in real time. This makes it possible to analyze information about the user's body movements and posture with high accuracy and in real time, and provide specific and effective advice.

[0481] A "wearable device" is an electronic device that can be worn by a user and is equipped with various sensors for obtaining information about body movements and posture.

[0482] A "terminal" is a device that transmits and receives information between a wearable device and a server, and has a function of providing generated advice to a user.

[0483] A "server" is a computer system that receives data transmitted from a terminal via a network, analyzes the data using a generative model, and generates advice.

[0484] A "generative model" is a neural network that uses a machine learning algorithm and is a model that analyzes data related to the user's body movements and posture to generate advice.

[0485] "Advice" is specific and effective instructions for correcting or improving the user's behavior based on the user's body movements and posture analyzed by the generative model.

[0486] "Information acquisition means" is a general term for methods and apparatuses that use various sensors mounted on wearable devices to collect data regarding the user's body movements and posture.

[0487] "Data transmission means" refers to a communication function or means for transmitting collected data to a terminal or a server.

[0488] The "analysis means" is a means for analyzing collected data using a generative model and generating appropriate advice for the user.

[0489] "Advice providing means" is a general term for a method or device for conveying generated advice to a user via a terminal.

[0490] "Real-time" refers to processing and information being provided immediately, without delay.

[0491] The embodiments of the present invention will be specifically described below.

[0492] First, the system consists of three main components: a wearable device, a terminal, and a server.

[0493] Wearable Devices A user puts on a wearable device. This device may be a wristwatch or have another shape, and is equipped with multiple sensors. Specifically, these sensors include an acceleration sensor and a gyro sensor. These sensors acquire data on the user's body movements and posture in real time. The acquired data is temporarily stored in the device and then transmitted to a terminal or a server.

[0494] Terminal The terminal is an intermediary device that transmits acquired data to the server. Terminals include smartphones and tablets. They have the function of receiving data from the wearable device via Bluetooth or Wi-Fi and transmitting that data to the server in real time. They also have the role of providing advice received from the server to the user.

[0495] server The server is the central analysis device of the system. The server receives data sent from the wearable device or terminal and analyzes it using a generative model. The generative model is a neural network using a machine learning algorithm, and generates specific and effective advice from information about the user's body movements and posture. The generated advice is sent back to the terminal and provided to the user.

[0496] Examples For example, consider the case where a user is jogging while wearing a wearable device. In this case, the following actions are performed:

[0497] 1. Wearable devices constantly monitor the user's movements and posture using acceleration sensors and gyro sensors.

[0498] 2. The acquired data is sent to the device via Bluetooth.

[0499] 3. The device transmits the received data to the server in real time via Wi-Fi.

[0500] 4. The server inputs the received data into the generative model and analyzes the user's movements and posture.

[0501] 5. The generative model generates specific advice, such as "Run with your back straight for a more effective workout."

[0502] 6. The generated advice is sent to the terminal, which displays it to the user.

[0503] Examples of prompt statements "If users are hunching their back while jogging, provide advice on effective posture correction methods."

[0504] Additional example prompts: "Generate appropriate advice when a user's running style is detected as poor." "Provide posture advice to help joggers get a more effective workout."

[0505] In this way, the user can receive specific and effective advice in real time, enabling the user to train more effectively.

[0506] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0507] Step 1: The user puts on the wearable device. This device is equipped with multiple sensors, such as an acceleration sensor and a gyro sensor, and collects data on the user's body movements and posture in real time.

[0508] Input: User's body movements and posture

[0509] Data processing: Data acquisition by sensors (accelerometer, gyro sensor)

[0510] Output: Acquired movement and posture data (e.g. XYZ axis acceleration data, angular velocity data)

[0511] Specific operation: The acceleration sensor captures the up and down movement while jogging, and the gyro sensor detects the rotation of the body.

[0512] Step 2: The wearable device transmits the acquired data to the terminal using a wireless communication method such as Bluetooth.

[0513] Input: Acquired movement and posture data

[0514] Data processing: Wireless data transmission (Bluetooth)

[0515] Output: Data sent to the terminal

[0516] Specific operation: The wearable device uses Bluetooth to transmit the acquired acceleration and angular velocity data to the terminal.

[0517] Step 3: The terminal transmits the data received from the wearable device to the server. The terminal uploads the data in real time via Wi-Fi.

[0518] Input: Data received from a wearable device

[0519] Data processing: Wireless data transmission (Wi-Fi)

[0520] Output: Data sent to the server

[0521] Specific operation: The sensor data received by the device is sent to the server via Wi-Fi.

[0522] Step 4: The server inputs the received data into the generative model, which is a neural network that uses machine learning algorithms to analyze the data.

[0523] Input: Data sent from the terminal

[0524] Data processing: Data analysis using generative models (neural networks)

[0525] Output: Analysis results (e.g. advice on how to improve the user's posture)

[0526] Specific operation: The server inputs sensor data into a generative AI model for analysis, analyzing the user's posture and movements.

[0527] Step 5: The server sends the advice generated by the generative model to the terminal, which includes specific instructions for the user to modify their behavior.

[0528] Input: Analysis results

[0529] Data processing: generating and sending advice (over the network)

[0530] Output: Advice sent to terminal

[0531] Specific operation: The server generates advice such as "If you run with your back straight, you will get a more effective workout" and sends it to the terminal.

[0532] Step 6: The terminal displays the received advice to the user. The display method may be a pop-up display on the smartphone screen, a voice notification, or a message displayed on the screen of the wearable device.

[0533] Input: Advice sent by the server

[0534] Data processing: Display of advice

[0535] Output:Notification to the user

[0536] Specific action: The device will display a notification on the smartphone screen or wearable device display, such as "Running with your back straight will give you a more effective workout."

[0537] Through the above steps, the user can receive specific advice in real time.

[0538] (Application example 1) Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0539] In conventional factory work, it is difficult to manage the movements and postures of workers, which has led to problems of reduced work efficiency and safety. In particular, poor posture and incorrect movements during work are cited as factors that can lead to industrial accidents and reduced production efficiency. To solve this, a system that can monitor the movements and postures of workers in real time and provide appropriate feedback is needed.

[0540] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0541] In this invention, the server includes a means for acquiring information on the user's body movements and posture, a means for generating advice on the user's body movements and posture during activity using a generative model, a means for providing the advice to the user, and a means for analyzing data on the movements and posture of factory workers and providing advice aimed at improving safety and efficiency. This makes it possible to improve safety and production efficiency by monitoring the movements and posture of workers in real time and providing specific advice.

[0542] A "user" is a person who uses the system or performs tasks.

[0543] "Information regarding body movement and posture" refers to data and measurement results regarding the user's body movement and posture.

[0544] A "generative model" is a neural network trained with machine learning algorithms to generate advice regarding a user's body movements and posture.

[0545] "Advice" refers to instructions or suggestions provided to a user that are intended to improve their body movement or posture.

[0546] A "wearable device" is an electronic device that can be worn by a user and is equipped with sensors related to body movements and posture.

[0547] A "factory worker" is a worker who performs work in a factory.

[0548] "Data analysis" refers to using a generative model to analyze the acquired information about body movements and postures.

[0549] "Safety" refers to reducing the risk of accidents and injuries when workers perform their work.

[0550] "Improving efficiency" refers to increasing work productivity and promoting optimal use of time and resources.

[0551] "Real-time" refers to information being obtained and advice being given almost instantly.

[0552] The present invention provides a system for monitoring the movements and postures of factory workers in real time to improve safety and work efficiency. The system is configured as follows.

[0553] Hardware configuration:

[0554] Wearable device: An electronic device with built-in sensors (accelerometer, gyro sensor, etc.) that is worn by the user to obtain information about the user's body movements and posture.

[0555] Server: Processes the received data and performs analysis using generative AI models. This server is a computer system that implements Python and machine learning libraries (e.g. TensorFlow, PyTorch).

[0556] Terminal: A device such as a smartphone or tablet that is a display device for providing advice from the server to the user. This terminal works in conjunction with the wearable device to send and receive data.

[0557] Software configuration:

[0558] 1. Data acquisition and transmission process: The wearable device collects information about the user's body movements and posture in real time through sensors. This data is then transmitted to a server via wireless communication.

[0559] 2. Data analysis process: The server inputs the received data into the generative AI model and uses machine learning algorithms to analyze the user's behavior. TensorFlow or PyTorch is used for the generative AI model.

[0560] 3. Advice generation process: The generative AI model generates specific advice for the user's movements and posture based on the analyzed data. For example, it may generate instructions such as "It is safer to work with your waist bent."

[0561] 4. Advice Providing Process: The server sends the generated advice to the terminal and displays it. The user can check the advice in real time on the terminal and perform safe and efficient work.

[0562] Examples: Consider a case where a worker is lifting heavy parts in a factory. The wearable device monitors the worker's body movements in real time and sends the data to a server. The server analyzes the received data with a generative AI model, and if it detects that the worker's posture is inappropriate, it generates advice such as "You should bend your waist more." This advice is sent to the terminal and can be checked by the worker in real time.

[0563] Example prompt: "If the user's acceleration is 0.01, 0.02, -0.98, and the gyroscope values ​​are 0.001, 0.002, 0.003, what body movement and posture corrections are required?"

[0564] This makes it possible to constantly optimize the movements of factory workers and support safe and efficient work.

[0565] The flow of the specific process in the application example 1 will be described with reference to FIG.

[0566] Step 1: The wearable device acquires information about the user's body movements and posture. Specifically, it uses an acceleration sensor and a gyro sensor to detect the user's body movements (e.g., 0.01, 0.02, -0.98) and posture (e.g., 0.001, 0.002, 0.003) and compiles them into a data packet.

[0567] Input: User's body movements and posture

[0568] Output: Sensor data packets (e.g. acceleration data, gyroscope data)

[0569] Step 2: The sensor data acquired by the wearable device is sent to the server via wireless communication. Specifically, data packets are transferred to the server using Bluetooth or Wi-Fi.

[0570] Input: Sensor data packet

[0571] Output: Data packets forwarded to the server

[0572] Step 3: The server inputs the received sensor data into the generative AI model for analysis. Specifically, the received data is preprocessed (e.g., noise filtering, normalization) and input into the generative AI model (using TensorFlow or PyTorch) to generate analysis results on the user's movements and postures.

[0573] Input: Data packet forwarded to the server

[0574] Output: Analysis results regarding the user's movements and postures

[0575] Step 4: The generative AI model generates advice based on the results of its analysis. Specifically, it generates specific feedback such as "bend your waist more" or "straighten your back" based on the analysis results.

[0576] Input: Analysis results regarding the user's movements and posture

[0577] Output: Specific advice message

[0578] Step 5: The server sends the generated advice to the terminal (e.g., a smartphone or tablet). Specifically, a protocol (e.g., HTTP, WebSocket) is used to send the generated advice message to the terminal application.

[0579] Input: A specific advice message

[0580] Output: Advice messages printed to the terminal.

[0581] Step 6: The device provides the received advice to the user. Specifically, the advice is displayed on the device display in real time. For example, a message such as "It would be safer if you bend your waist more" is displayed.

[0582] Input: Advice message displayed on terminal

[0583] Output: User-visible advice display

[0584] In addition, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0585] This embodiment is a system that is composed of the following elements.

[0586] 1. Server: The server inputs the received information into a generative model for analysis. The generative model is a neural network that uses a machine learning algorithm and generates advice regarding the user's body movements and posture during activities. The server is also combined with an emotion engine that recognizes the user's emotions and generates appropriate advice according to those emotions.

[0587] 2. Terminal: The terminal can be worn by the user and is equipped with various sensors. The terminal also acquires information on the user's body movements and posture and transmits it to the server. Furthermore, the terminal receives advice generated by the server and provides it to the user. The advice is provided in real time.

[0588] An example of the advice providing process by the above-mentioned system will be described below. For example, consider a case where a user is jogging while wearing a terminal. The terminal constantly monitors the user's movement and posture using sensors such as an acceleration sensor and a gyro sensor built in, and transmits the information obtained from the sensors to the server. The server inputs the received information into a generative model and analyzes the user's running style using a machine learning algorithm. If it is detected that the user's posture is incorrect, the generated advice is transmitted to the terminal as a specific instruction such as "run with your back straight, you can train more effectively." In addition, the server recognizes the user's emotions using an emotion engine, and generates encouraging advice if the user is tired. For example, if the user feels tired, the generated advice is transmitted to the terminal as a specific encouraging word such as "You're doing a great job! Let's run to the end!" The user can jog more effectively by checking the advice displayed on the terminal and correcting their posture. In addition, if the advice includes encouraging words, it is expected that the user's motivation to continue jogging will increase.

[0589] The process flow will be explained below.

[0590] Step 1: The terminal acquires information on body movement and posture. To acquire the information, the user wears the terminal. The terminal uses a built-in sensor to collect information on the user's body movement and posture. The collected information is then transmitted to a server. The terminal also transmits information for recognizing the user's emotions. The information for recognizing the user's emotions may include an image of the user's facial expression, pulse rate, etc.

[0591] Step 2: Analyze using the generative model on the server. The server inputs the information received from the device into the generative model. The generative model is a neural network that uses a machine learning algorithm, and generates advice regarding the user's body movements and posture during activity. The server sends the advice generated by the generative model to the device. The emotion engine equipped on the server is also a neural network that uses a machine learning algorithm to recognize the user's emotions. The server uses the emotion engine to recognize the user's emotions from the received information for recognizing the user's emotions, and determines whether the user is tired.

[0592] Step 3: The server generates appropriate advice based on the emotion. The server combines the analysis results with the emotion recognition results to generate appropriate advice for the user. If the user is tired, the generated advice includes specific instructions as words of encouragement, such as "You're doing great! Let's run to the end!"

[0593] Step 4: Display the advice on the terminal. The terminal displays the received advice on the terminal. This allows the user to confirm the advice. Then, the user who has confirmed the advice can continue jogging while correcting their posture and being encouraged by the encouraging words.

[0594] Example 2 Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal".

[0595] In recent years, there has been a demand for a system that monitors a user's body movements and posture and provides appropriate advice in training and daily life. However, conventional technology cannot provide advice that takes into account the user's emotional state, and there are limitations in terms of maintaining the user's motivation and providing effective training support. Therefore, the present invention aims to provide a system that improves the effectiveness of a user's training and increases the user's motivation by providing advice that takes into account not only the user's body movements and posture information but also the user's emotional state.

[0596] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0597] In this invention, the server includes a means for acquiring information on the user's body movements and posture, a means for generating advice on the user's body movements and posture during activity using a generative model, a means for providing the advice to the user, a means for analyzing the acquired information and including an emotion engine for recognizing the user's emotion, and a means for generating appropriate advice according to the user's emotion. This makes it possible to provide specific and appropriate advice in real time according to the user's body movements, posture, and emotional state.

[0598] "Means for acquiring information regarding the user's body movements and posture" refers to an apparatus or method for collecting data regarding the user's body movements and posture, and specifically includes a wearable device with a built-in sensor.

[0599] "Means for generating advice regarding a user's body movements and posture during an activity using a generative model" refers to a system for analyzing a user's body movements and posture data and generating advice based thereon, and includes a neural network trained with a machine learning algorithm.

[0600] The "means for providing the advice to the user" refers to a device or method for transmitting the generated advice to the user in real time, and uses a terminal or a display.

[0601] "Means including an emotion engine for analyzing acquired information and recognizing the user's emotions" refers to technology for analyzing the user's body movements and physiological data and recognizing the user's emotional state, including algorithms and software.

[0602] The "means for generating appropriate advice in accordance with the user's emotions" refers to a system for generating advice or messages suited to the user based on the recognized emotional state of the user, and includes an emotion analysis engine.

[0603] This system is designed to monitor the user's body movements and posture and provide appropriate advice. Specifically, the wearable device, terminal, and server work together to collect data, analyze it, and generate advice.

[0604] A wearable device is an electronic device that can be worn by a user and is equipped with various sensors such as an acceleration sensor and a gyro sensor. This makes it possible to obtain information about the user's body movements and posture in real time. For example, when a user is jogging, the wearable device records the movement of the user's legs and the angle of the back muscles. This information is transmitted to a terminal via wireless technologies such as Bluetooth and Wi-Fi.

[0605] The terminal is responsible for sending the received information to the server. The terminal is a device that the user can carry around, such as a smartphone or tablet. For example, the user's smartphone receives data from the wearable device and transfers the data to the server via the Internet.

[0606] The server inputs the received data into a generative AI model and analyzes the user's body movements and posture. This generative model is a neural network that uses machine learning algorithms to analyze the user's movement patterns and generate optimal advice. For example, the generated advice might be, "You can train more effectively if you run with your back straight."

[0607] In addition, the server also uses an emotion engine to analyze the user's emotional state. The emotion engine analyzes physiological data such as the user's heart rate and breathing patterns to recognize the user's emotional state, such as tiredness or concentration. For example, if the user is recognized as tired, an encouraging message such as "You're almost there, keep going!" is generated.

[0608] The generated advice is then provided to the user via the device. The user can check the advice in real time on their smartphone or tablet, correct their posture as necessary, and stay motivated. This allows the user to train more effectively, and is expected to increase their motivation.

[0609] Examples of prompts include: "Generate advice for when the user's posture is slouching." "Generate an encouraging message if you detect that the user is tired."

[0610] By using this system, users can receive specific and appropriate advice in real time based on their body movements, posture, and emotional state, enabling them to train more effectively.

[0611] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0612] Program processing flow

[0613] Step 1: The user puts on the wearable device.

[0614] Description: Before starting a jog or daily activity, the user puts on a wearable device on their arm or wrist. The device has built-in acceleration and gyro sensors, and collects data on the user's body movements and posture in real time.

[0615] Input: Sensor information from wearable devices

[0616] Output: User movement and posture data

[0617] Step 2: The wearable device monitors the user's body movements and posture.

[0618] Description: The wearable device continuously monitors the user's body movements and posture using built-in acceleration and gyro sensors. For example, while the user is jogging, it collects data such as foot movements, back angle, and hand movements.

[0619] Input: User movement and posture data

[0620] Output: Raw data as monitoring results

[0621] Step 3: The wearable device transmits the acquired data to the terminal or directly to the server.

[0622] Description: Wearable devices use wireless technologies such as Bluetooth or Wi-Fi to transmit the acquired data to a device such as a smartphone, or directly to a server.

[0623] Input: Raw data as monitoring results

[0624] Output: The data sent.

[0625] Step 4: The server inputs the received data into the generative AI model.

[0626] Description: The server receives the user's movement and posture data sent from the wearable device and inputs it into a generative AI model, which is a neural network using machine learning algorithms to analyze the data and generate appropriate advice.

[0627] Input: Data sent

[0628] Output: Input data for a generative AI model

[0629] Step 5: The generative model analyzes the data and generates appropriate advice.

[0630] Description: The generative AI model analyzes the data it receives. For example, if it detects that the user is slouching, it generates advice like "Stand up straight."

[0631] Input: Input data for the generative AI model

[0632] Output: Advice statement

[0633] Step 6: The server analyzes the user's emotions using the emotion engine.

[0634] Description: The server uses the emotion engine to analyze the user's heart rate, breathing patterns, etc., and recognizes the user's fatigue state and emotions. For example, if the user is recognized as tired, data is output to generate an encouraging message.

[0635] Input: User's physiological data

[0636] Output: Emotional state data

[0637] Step 7: The server generates advice according to the emotion.

[0638] Description: The server generates appropriate advice and encouraging messages for the user based on the emotional state data obtained from the emotion engine. For example, it generates an encouraging message such as "You're almost there, keep trying!"

[0639] Input: Emotional state data

[0640] Output: Advice sentences according to emotions

[0641] Step 8: The server sends the generated advice to the terminal.

[0642] Description: The server sends the generated advice to the user's device in real time, allowing the user to receive immediate feedback.

[0643] Input: Generated advice statement

[0644] Output: Advice text sent to terminal.

[0645] Step 9: The terminal provides advice to the user.

[0646] Description: The device provides the user with advice received from the server. For example, the device display shows "Please stand up straight" or "You're almost there, let's do our best!". The user can see this and correct their posture, which motivates them to continue training.

[0647] Input: Advice text sent to the terminal

[0648] Output: Provide advice to the user

[0649] By having each step work together in this way, users can receive appropriate advice in real time, enabling them to conduct more effective training and activities.

[0650] (Application example 2) Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server", and the headset type terminal 314 will be referred to as a "terminal".

[0651] Conventional fitness instruction systems using wearable devices have the problem that they can only obtain information about the user's body movements and posture, and are unable to provide advice that takes into account the user's emotional state. This can make it difficult for users to maintain their training effectiveness and motivation.

[0652] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means. In this invention, the server includes a means for acquiring information on the user's body movements and posture, a means for generating advice on the user's body movements and posture during activity using a generative model, a means for providing the advice to the user, and a means for recognizing the user's emotions and generating appropriate advice according to the emotions. This makes it possible not only to provide the user with real-time posture and form correction guidance during training, but also to provide encouragement and advice for improving motivation that takes into account the user's emotional state. This improves the training effect of the user and makes it possible to maintain motivation.

[0653] "Information regarding the user's body movements and posture" refers to motion data such as the position, direction, speed, and acceleration of the user's body, as well as posture information during a specific activity, obtained via a wearable device.

[0654] A "generative model" is a neural network trained using a machine learning algorithm, and is a model that analyzes a user's body movements and postures to generate advice.

[0655] "Means for providing advice to a user" refers to a mechanism for notifying a user of the generated advice, and specifically refers to a terminal such as a smartphone, smart glasses, or a head-mounted display.

[0656] "Means for recognizing the user's emotions and generating appropriate advice based on those emotions" refers to a mechanism for analyzing the user's motion data and vital signs to estimate their emotional state and generate advice or encouraging messages that are adapted to that state.

[0657] The present invention is a system that tracks the user's body movements and postures, analyzes the data, and provides appropriate advice. It also includes a function to recognize the user's emotional state and generate encouraging messages according to the emotion. The system is composed of a wearable device, a server, and a terminal.

[0658] A wearable device is an electronic device worn by a user and is equipped with multiple sensors, such as an acceleration sensor and a gyro sensor, to obtain information about the user's body movements and posture. The wearable device transmits the obtained data to a terminal or directly to a server.

[0659] The server inputs the received data into a generative model and analyzes the user's body movements and posture. The generative model is a neural network trained using machine learning algorithms. The server generates advice to provide to the user based on the analysis results. It also has an emotion engine that recognizes the user's emotional state. Depending on the user's emotional state, it generates appropriate advice or encouraging messages.

[0660] The terminal is a device that receives the advice sent from the server and provides it to the user. Examples of such devices include smartphones, smart glasses, and head-mounted displays. The user can check the advice provided on the terminal and correct their own movements and posture. This advice is provided in real time and is expected to improve the effectiveness of the user's training.

[0661] As a concrete example, consider the case where a user is doing squats at a fitness club. The wearable device detects the user's movements and sends them to the server. The server analyzes the received data and generates specific advice such as "Bend your knees a bit more." If the emotion engine recognizes that the user is tired, it also sends an encouraging message such as "You're doing great! Not much longer!" The user can train while checking these instructions and encouragement on the device.

[0662] Examples of prompts to be input to a generative AI model include the following:

[0663] "This shows posture data when the user is wearing the wearable device. Based on the following data, determine whether the user's current squat posture is correct and generate advice if necessary. Also, consider the user's emotions and include words of encouragement if the user seems tired. Sensor data: {Accelerometer value}, {Gyro sensor value}."

[0664] This system allows users to receive real-time guidance on how to correct their form while training, and also provides encouraging messages that take into account their emotional state, improving the effectiveness of their training and helping them maintain their motivation.

[0665] The flow of the specific process in the application example 2 will be described with reference to FIG.

[0666] Step 1: The wearable device acquires information about the user's body movements and posture. This information is data obtained from acceleration and gyro sensors, and specifically includes the user's position, direction, speed, acceleration, etc. The input is the user's motion data, and the output is compiled as sensor data.

[0667] Step 2: The wearable device transmits the acquired sensor data to the terminal or directly to the server. This data is transmitted in real time and communicated using an appropriate protocol. The input is the sensor data and the output is the data to be sent to the server.

[0668] Step 3: The server inputs the received sensor data into a generative model to analyze the user's body movements and posture. The generative model is a neural network trained with machine learning algorithms and generates appropriate advice based on the analyzed data. The input is the sensor data and the output is advice as a result of the analysis.

[0669] Step 4: The server generates advice based on the analysis results and sends it to the terminal. The advice includes specific posture correction instructions. The input is the advice as the analysis result, and the output is the data to be sent to the terminal.

[0670] Step 5: At the same time, the server recognizes the user's emotional state through the emotion engine. This uses the motion data and vital sign data. It estimates the emotional state and generates advice or encouraging messages accordingly. The input is the motion data and vital sign data, and the output is advice based on the emotion.

[0671] Step 6: The server sends appropriate advice or encouraging messages to the terminal based on the emotion recognition result. The input is the advice as the emotion recognition result, and the output is the data to be sent to the terminal.

[0672] Step 7: The terminal provides the advice received from the server to the user. This can be done using a smartphone, smart glasses, a head-mounted display, etc., and the user can view the advice and correct their movements and posture. The input is the advice data from the server, and the output is the display on the terminal.

[0673] Step 8: The user checks the display on the device and corrects their movements and posture in real time to improve the training effect. In addition, motivation is maintained by encouraging messages that take into account the user's emotional state. The input is the display data on the device, and the output is the user's actions.

[0674] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input for the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0675] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by making a neural network perform deep learning. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image is input. The data generation model 58 performs inference on the input inference data according to the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0676] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[0677] [Fourth embodiment] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[0678] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0679] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[0680] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. In addition, the microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[0681] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0682] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[0683] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54. The transmission and reception of various types of information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0684] The control target 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, legs, etc. The posture and behavior of the robot 414 are controlled by controlling the motors of the arms, hands, legs, etc. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[0685] Fig. 8 shows an example of main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0686] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0687] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0688] In the robot 414, the reception and output process is performed by the processor 46. A reception and output program 60 is stored in the storage 50. The processor 46 reads the reception and output program 60 from the storage 50, and executes the read reception and output program 60 on the RAM 48. The reception and output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception and output program 60 executed on the RAM 48.

[0689] Next, a description will be given of the specific processing by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal".

[0690] This embodiment is a system that is composed of the following elements.

[0691] 1. Wearable devices: A wearable device is an electronic device that can be worn by a user, such as a wristwatch, and is equipped with various sensors. The wearable device also acquires information about the user's body movements and posture and transmits the information to a terminal or a server.

[0692] 2. Server: The server inputs the received information into a generative model for analysis. The generative model is a neural network that uses a machine learning algorithm to generate advice on the user's body movements and posture during activities.

[0693] 3. Terminal: The terminal receives the generated advice from the server and provides it to the user. The advice is provided to the wearable device in real time. Note that the terminal may be the same as the wearable device.

[0694] An example of the advice providing process by the above-mentioned system will be described below. For example, consider a case where a user is jogging while wearing a wearable device. The wearable device constantly monitors the user's movements and posture using built-in sensors such as an acceleration sensor and a gyro sensor, and transmits the information obtained from the sensors to a server via a terminal or directly. The server inputs the received information into a generative model and analyzes the user's running style using a machine learning algorithm. If it is detected that the user's posture is incorrect, the generated advice is transmitted to the terminal as a specific instruction such as "run with your back straight for more effective training." The user can jog more effectively by checking the advice displayed on the terminal or on the wearable device received from the terminal and correcting their posture.

[0695] The process flow will be explained below.

[0696] Step 1: Acquire body movement and posture information using a wearable device. To acquire the information, the user wears a wearable device. The wearable device uses built-in sensors to collect information about the user's body movement and posture. The collected information is then sent to a server via the terminal or directly.

[0697] Step 2: Analyze using the generative model on the server. The server inputs the information received from the wearable device or terminal into the generative model. The generative model is a neural network that uses a machine learning algorithm and generates advice regarding the user's body movements and posture during activities. The server sends the advice generated by the generative model to the terminal.

[0698] Step 3: Display the advice on the terminal. The terminal displays the received advice on the terminal or on the wearable device by transmitting the advice to the wearable device. This allows the user to check the advice.

[0699] Example 1 Next, a description will be given of Example 1. In the following description, the data processing device 12 is referred to as a "server" and the robot 414 is referred to as a "terminal."

[0700] Conventional exercise support systems have had difficulty obtaining information about the user's body movements and posture in real time and providing specific and effective advice based on that information. In addition, the wearable device worn by the user has limited analytical capabilities, which limits the accuracy and quality of the information obtained. Furthermore, the advice generated was general and abstract, making it difficult for the user to immediately understand and implement specific improvement methods.

[0701] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0702] In this invention, the server includes means for acquiring information about the user's body movements and posture via the wearable device, means for transmitting the acquired information to the server via the terminal or directly, means for generating advice about the user's body movements and posture during activity from the acquired information using a generative model, means for transmitting the generated advice to the terminal and providing it to the user, and means for providing the advice to the user in real time. This makes it possible to analyze information about the user's body movements and posture with high accuracy and in real time, and provide specific and effective advice.

[0703] A "wearable device" is an electronic device that can be worn by a user and is equipped with various sensors for obtaining information about body movements and posture.

[0704] A "terminal" is a device that transmits and receives information between a wearable device and a server, and has a function of providing generated advice to a user.

[0705] A "server" is a computer system that receives data transmitted from a terminal via a network, analyzes the data using a generative model, and generates advice.

[0706] A "generative model" is a neural network that uses a machine learning algorithm and is a model that analyzes data related to the user's body movements and posture to generate advice.

[0707] "Advice" is specific and effective instructions for correcting or improving the user's behavior based on the user's body movements and posture analyzed by the generative model.

[0708] "Information acquisition means" is a general term for methods and apparatuses that use various sensors mounted on wearable devices to collect data regarding the user's body movements and posture.

[0709] "Data transmission means" refers to a communication function or means for transmitting collected data to a terminal or a server.

[0710] The "analysis means" is a means for analyzing collected data using a generative model and generating appropriate advice for the user.

[0711] "Advice providing means" is a general term for a method or device for conveying generated advice to a user via a terminal.

[0712] "Real-time" refers to processing and information being provided immediately, without delay.

[0713] The embodiments of the present invention will be specifically described below.

[0714] First, the system consists of three main components: a wearable device, a terminal, and a server.

[0715] Wearable Devices A user puts on a wearable device. This device may be a wristwatch or have another shape, and is equipped with multiple sensors. Specifically, these sensors include an acceleration sensor and a gyro sensor. These sensors acquire data on the user's body movements and posture in real time. The acquired data is temporarily stored in the device and then transmitted to a terminal or a server.

[0716] Terminal The terminal is an intermediary device that transmits acquired data to the server. Terminals include smartphones and tablets. They have the function of receiving data from the wearable device via Bluetooth or Wi-Fi and transmitting that data to the server in real time. They also have the role of providing advice received from the server to the user.

[0717] server The server is the central analysis device of the system. The server receives data sent from the wearable device or terminal and analyzes it using a generative model. The generative model is a neural network using a machine learning algorithm, and generates specific and effective advice from information about the user's body movements and posture. The generated advice is sent back to the terminal and provided to the user.

[0718] Examples For example, consider the case where a user is jogging while wearing a wearable device. In this case, the following actions are performed:

[0719] 1. Wearable devices constantly monitor the user's movements and posture using acceleration sensors and gyro sensors.

[0720] 2. The acquired data is sent to the device via Bluetooth.

[0721] 3. The device transmits the received data to the server in real time via Wi-Fi.

[0722] 4. The server inputs the received data into the generative model and analyzes the user's movements and posture.

[0723] 5. The generative model generates specific advice, such as "Run with your back straight for a more effective workout."

[0724] 6. The generated advice is sent to the terminal, which displays it to the user.

[0725] Examples of prompt statements "If users are hunching their back while jogging, provide advice on effective posture correction methods."

[0726] Additional example prompts: "Generate appropriate advice when a user's running style is detected as poor." "Provide posture advice to help joggers get a more effective workout."

[0727] In this way, the user can receive specific and effective advice in real time, enabling the user to train more effectively.

[0728] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0729] Step 1: The user puts on the wearable device. This device is equipped with multiple sensors, such as an acceleration sensor and a gyro sensor, and collects data on the user's body movements and posture in real time.

[0730] Input: User's body movements and posture

[0731] Data processing: Data acquisition by sensors (accelerometer, gyro sensor)

[0732] Output: Acquired movement and posture data (e.g. XYZ axis acceleration data, angular velocity data)

[0733] Specific operation: The acceleration sensor captures the up and down movement while jogging, and the gyro sensor detects the rotation of the body.

[0734] Step 2: The wearable device transmits the acquired data to the terminal using a wireless communication method such as Bluetooth.

[0735] Input: Acquired movement and posture data

[0736] Data processing: Wireless data transmission (Bluetooth)

[0737] Output: Data sent to the terminal

[0738] Specific operation: The wearable device uses Bluetooth to transmit the acquired acceleration and angular velocity data to the terminal.

[0739] Step 3: The terminal transmits the data received from the wearable device to the server. The terminal uploads the data in real time via Wi-Fi.

[0740] Input: Data received from a wearable device

[0741] Data processing: Wireless data transmission (Wi-Fi)

[0742] Output: Data sent to the server

[0743] Specific operation: The sensor data received by the device is sent to the server via Wi-Fi.

[0744] Step 4: The server inputs the received data into the generative model, which is a neural network that uses machine learning algorithms to analyze the data.

[0745] Input: Data sent from the terminal

[0746] Data processing: Data analysis using generative models (neural networks)

[0747] Output: Analysis results (e.g. advice on how to improve the user's posture)

[0748] Specific operation: The server inputs sensor data into a generative AI model for analysis, analyzing the user's posture and movements.

[0749] Step 5: The server sends the advice generated by the generative model to the terminal, which includes specific instructions for the user to modify their behavior.

[0750] Input: Analysis results

[0751] Data processing: generating and sending advice (over the network)

[0752] Output: Advice sent to terminal

[0753] Specific operation: The server generates advice such as "If you run with your back straight, you will get a more effective workout" and sends it to the terminal.

[0754] Step 6: The terminal displays the received advice to the user. The display method may be a pop-up display on the smartphone screen, a voice notification, or a message displayed on the screen of the wearable device.

[0755] Input: Advice sent by the server

[0756] Data processing: Display of advice

[0757] Output:Notification to the user

[0758] Specific action: The device will display a notification on the smartphone screen or wearable device display, such as "Running with your back straight will give you a more effective workout."

[0759] Through the above steps, the user can receive specific advice in real time.

[0760] (Application example 1) Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[0761] In conventional factory work, it is difficult to manage the movements and postures of workers, which has led to problems of reduced work efficiency and safety. In particular, poor posture and incorrect movements during work are cited as factors that can lead to industrial accidents and reduced production efficiency. To solve this, a system that can monitor the movements and postures of workers in real time and provide appropriate feedback is needed.

[0762] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0763] In this invention, the server includes a means for acquiring information on the user's body movements and posture, a means for generating advice on the user's body movements and posture during activity using a generative model, a means for providing the advice to the user, and a means for analyzing data on the movements and posture of factory workers and providing advice aimed at improving safety and efficiency. This makes it possible to improve safety and production efficiency by monitoring the movements and posture of workers in real time and providing specific advice.

[0764] A "user" is a person who uses the system or performs tasks.

[0765] "Information regarding body movement and posture" refers to data and measurement results regarding the user's body movement and posture.

[0766] A "generative model" is a neural network trained with machine learning algorithms to generate advice regarding a user's body movements and posture.

[0767] "Advice" refers to instructions or suggestions provided to a user that are intended to improve their body movement or posture.

[0768] A "wearable device" is an electronic device that can be worn by a user and is equipped with sensors related to body movements and posture.

[0769] A "factory worker" is a worker who performs work in a factory.

[0770] "Data analysis" refers to using a generative model to analyze the acquired information about body movements and postures.

[0771] "Safety" refers to reducing the risk of accidents and injuries when workers perform their work.

[0772] "Improving efficiency" refers to increasing work productivity and promoting optimal use of time and resources.

[0773] "Real-time" refers to information being obtained and advice being given almost instantly.

[0774] The present invention provides a system for monitoring the movements and postures of factory workers in real time to improve safety and work efficiency. The system is configured as follows.

[0775] Hardware configuration:

[0776] Wearable device: An electronic device with built-in sensors (accelerometer, gyro sensor, etc.) that is worn by the user to obtain information about the user's body movements and posture.

[0777] Server: Processes the received data and performs analysis using generative AI models. This server is a computer system that implements Python and machine learning libraries (e.g. TensorFlow, PyTorch).

[0778] Terminal: A device such as a smartphone or tablet that is a display device for providing advice from the server to the user. This terminal works in conjunction with the wearable device to send and receive data.

[0779] Software configuration:

[0780] 1. Data acquisition and transmission process: The wearable device collects information about the user's body movements and posture in real time through sensors. This data is then transmitted to a server via wireless communication.

[0781] 2. Data analysis process: The server inputs the received data into the generative AI model and uses machine learning algorithms to analyze the user's behavior. TensorFlow or PyTorch is used for the generative AI model.

[0782] 3. Advice generation process: The generative AI model generates specific advice for the user's movements and posture based on the analyzed data. For example, it may generate instructions such as "It is safer to work with your waist bent."

[0783] 4. Advice Providing Process: The server sends the generated advice to the terminal and displays it. The user can check the advice in real time on the terminal and perform safe and efficient work.

[0784] Examples: Consider a case where a worker is lifting heavy parts in a factory. The wearable device monitors the worker's body movements in real time and sends the data to a server. The server analyzes the received data with a generative AI model, and if it detects that the worker's posture is inappropriate, it generates advice such as "You should bend your waist more." This advice is sent to the terminal and can be checked by the worker in real time.

[0785] Example prompt: "If the user's acceleration is 0.01, 0.02, -0.98, and the gyroscope values ​​are 0.001, 0.002, 0.003, what body movement and posture corrections are required?"

[0786] This makes it possible to constantly optimize the movements of factory workers and support safe and efficient work.

[0787] The flow of the specific process in the application example 1 will be described with reference to FIG.

[0788] Step 1: The wearable device acquires information about the user's body movements and posture. Specifically, it uses an acceleration sensor and a gyro sensor to detect the user's body movements (e.g., 0.01, 0.02, -0.98) and posture (e.g., 0.001, 0.002, 0.003) and compiles them into a data packet.

[0789] Input: User's body movements and posture

[0790] Output: Sensor data packets (e.g. acceleration data, gyroscope data)

[0791] Step 2: The sensor data acquired by the wearable device is sent to the server via wireless communication. Specifically, data packets are transferred to the server using Bluetooth or Wi-Fi.

[0792] Input: Sensor data packet

[0793] Output: Data packets forwarded to the server

[0794] Step 3: The server inputs the received sensor data into the generative AI model for analysis. Specifically, the received data is preprocessed (e.g., noise filtering, normalization) and input into the generative AI model (using TensorFlow or PyTorch) to generate analysis results on the user's movements and postures.

[0795] Input: Data packet forwarded to the server

[0796] Output: Analysis results regarding the user's movements and postures

[0797] Step 4: The generative AI model generates advice based on the results of its analysis. Specifically, it generates specific feedback such as "bend your waist more" or "straighten your back" based on the analysis results.

[0798] Input: Analysis results regarding the user's movements and posture

[0799] Output: Specific advice message

[0800] Step 5: The server sends the generated advice to the terminal (e.g., a smartphone or tablet). Specifically, a protocol (e.g., HTTP, WebSocket) is used to send the generated advice message to the terminal application.

[0801] Input: A specific advice message

[0802] Output: Advice messages printed to the terminal.

[0803] Step 6: The device provides the received advice to the user. Specifically, the advice is displayed on the device display in real time. For example, a message such as "It would be safer if you bend your waist more" is displayed.

[0804] Input: Advice message displayed on terminal

[0805] Output: User-visible advice display

[0806] In addition, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0807] This embodiment is a system that is composed of the following elements.

[0808] 1. Wearable devices:

[0809] A wearable device is an electronic device that can be worn by a user and is equipped with various sensors. The wearable device also acquires information about the user's body movements and posture and transmits the information to a terminal or a server.

[0810] 2. Server:

[0811] The server inputs the received information into a generative model for analysis. The generative model is a neural network that uses a machine learning algorithm and generates advice regarding the user's body movements and posture during activities. The server is also combined with an emotion engine that recognizes the user's emotions and generates appropriate advice according to those emotions.

[0812] 3. Terminal:

[0813] The terminal receives the generated advice from the server and provides it to the user. The advice is provided to the user in real time. Note that the terminal may be the same as the wearable device.

[0814] An example of the advice providing process by the above-mentioned system will be described below. For example, consider a case where a user is jogging while wearing a wearable device. The wearable device constantly monitors the user's movements and posture using sensors such as an acceleration sensor and a gyro sensor built into it, and transmits the information obtained from the sensors to the server via the terminal or directly. The server inputs the received information into a generative model and analyzes the user's running style using a machine learning algorithm. If it is detected that the user's posture is incorrect, the generated advice is transmitted to the terminal as a specific instruction such as "run with your back straight, you can train more effectively." In addition, the server recognizes the user's emotions using an emotion engine, and generates encouraging advice if the user is tired. For example, if the user feels tired, the generated advice is transmitted to the terminal as a specific encouraging word such as "You're doing a great job! Let's run to the end!" The user can check the advice displayed on the terminal or on the wearable device received from the terminal and correct their posture to jog more effectively. In addition, if the advice includes encouraging words, it is expected that the user's motivation to continue jogging will increase.

[0815] The process flow will be explained below.

[0816] Step 1: Acquire information on body movements and posture using a wearable device. To acquire the information, the user wears the wearable device. The wearable device uses built-in sensors to collect information on the user's body movements and posture. The collected information is then sent to a server via a terminal or directly. The wearable device or the terminal also transmits information for recognizing the user's emotions. The information for recognizing the user's emotions may include images of the user's facial expressions, pulse rate, etc.

[0817] Step 2: Analyze using the generative model on the server. The server inputs the information received from the wearable device or terminal into the generative model. The generative model is a neural network using a machine learning algorithm, and generates advice regarding the user's body movements and posture during activity. The server sends the advice generated by the generative model to the terminal. The emotion engine equipped in the server is also a neural network using a machine learning algorithm for recognizing the user's emotions. The server uses the emotion engine to recognize the user's emotions from the received information for recognizing the user's emotions, and determines whether the user is tired.

[0818] Step 3: The server generates appropriate advice based on the emotion. The server combines the analysis results with the emotion recognition results to generate appropriate advice for the user. If the user is tired, the generated advice includes specific instructions as words of encouragement, such as "You're doing great! Let's run to the end!"

[0819] Step 4: Display the advice on the terminal. The terminal displays the received advice on the terminal or on the wearable device by transmitting it to the wearable device. This allows the user to confirm the advice. After confirming the advice, the user can continue jogging while correcting their posture and being encouraged by the encouraging words.

[0820] Example 2 Next, a description will be given of Example 2. In the following description, the data processing device 12 is referred to as a "server" and the robot 414 is referred to as a "terminal."

[0821] In recent years, there has been a demand for a system that monitors a user's body movements and posture and provides appropriate advice in training and daily life. However, conventional technology cannot provide advice that takes into account the user's emotional state, and there are limitations in terms of maintaining the user's motivation and providing effective training support. Therefore, the present invention aims to provide a system that improves the effectiveness of a user's training and increases the user's motivation by providing advice that takes into account not only the user's body movements and posture information but also the user's emotional state.

[0822] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0823] In this invention, the server includes a means for acquiring information on the user's body movements and posture, a means for generating advice on the user's body movements and posture during activity using a generative model, a means for providing the advice to the user, a means for analyzing the acquired information and including an emotion engine for recognizing the user's emotion, and a means for generating appropriate advice according to the user's emotion. This makes it possible to provide specific and appropriate advice in real time according to the user's body movements, posture, and emotional state.

[0824] "Means for acquiring information regarding the user's body movements and posture" refers to an apparatus or method for collecting data regarding the user's body movements and posture, and specifically includes a wearable device with a built-in sensor.

[0825] "Means for generating advice regarding a user's body movements and posture during an activity using a generative model" refers to a system for analyzing a user's body movements and posture data and generating advice based thereon, and includes a neural network trained with a machine learning algorithm.

[0826] The "means for providing the advice to the user" refers to a device or method for transmitting the generated advice to the user in real time, and uses a terminal or a display.

[0827] "Means including an emotion engine for analyzing acquired information and recognizing the user's emotions" refers to technology for analyzing the user's body movements and physiological data and recognizing the user's emotional state, including algorithms and software.

[0828] The "means for generating appropriate advice in accordance with the user's emotions" refers to a system for generating advice or messages suited to the user based on the recognized emotional state of the user, and includes an emotion analysis engine.

[0829] This system is designed to monitor the user's body movements and posture and provide appropriate advice. Specifically, the wearable device, terminal, and server work together to collect data, analyze it, and generate advice.

[0830] A wearable device is an electronic device that can be worn by a user and is equipped with various sensors such as an acceleration sensor and a gyro sensor. This makes it possible to obtain information about the user's body movements and posture in real time. For example, when a user is jogging, the wearable device records the movement of the user's legs and the angle of the back muscles. This information is transmitted to a terminal via wireless technologies such as Bluetooth and Wi-Fi.

[0831] The terminal is responsible for sending the received information to the server. The terminal is a device that the user can carry around, such as a smartphone or tablet. For example, the user's smartphone receives data from the wearable device and transfers the data to the server via the Internet.

[0832] The server inputs the received data into a generative AI model and analyzes the user's body movements and posture. This generative model is a neural network that uses machine learning algorithms to analyze the user's movement patterns and generate optimal advice. For example, the generated advice might be, "You can train more effectively if you run with your back straight."

[0833] In addition, the server also uses an emotion engine to analyze the user's emotional state. The emotion engine analyzes physiological data such as the user's heart rate and breathing patterns to recognize the user's emotional state, such as tiredness or concentration. For example, if the user is recognized as tired, an encouraging message such as "You're almost there, keep going!" is generated.

[0834] The generated advice is then provided to the user via the device. The user can check the advice in real time on their smartphone or tablet, correct their posture as necessary, and stay motivated. This allows the user to train more effectively, and is expected to increase their motivation.

[0835] Examples of prompts include: "Generate advice for when the user's posture is slouching." "Generate an encouraging message if you detect that the user is tired."

[0836] By using this system, users can receive specific and appropriate advice in real time based on their body movements, posture, and emotional state, enabling them to train more effectively.

[0837] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0838] Program processing flow

[0839] Step 1: The user puts on the wearable device.

[0840] Description: Before starting a jog or daily activity, the user puts on a wearable device on their arm or wrist. The device has built-in acceleration and gyro sensors, and collects data on the user's body movements and posture in real time.

[0841] Input: Sensor information from wearable devices

[0842] Output: User movement and posture data

[0843] Step 2: The wearable device monitors the user's body movements and posture.

[0844] Description: The wearable device continuously monitors the user's body movements and posture using built-in acceleration and gyro sensors. For example, while the user is jogging, it collects data such as foot movements, back angle, and hand movements.

[0845] Input: User movement and posture data

[0846] Output: Raw data as monitoring results

[0847] Step 3: The wearable device transmits the acquired data to the terminal or directly to the server.

[0848] Description: Wearable devices use wireless technologies such as Bluetooth or Wi-Fi to transmit the acquired data to a device such as a smartphone, or directly to a server.

[0849] Input: Raw data as monitoring results

[0850] Output: The data sent.

[0851] Step 4: The server inputs the received data into the generative AI model.

[0852] Description: The server receives the user's movement and posture data sent from the wearable device and inputs it into a generative AI model, which is a neural network using machine learning algorithms to analyze the data and generate appropriate advice.

[0853] Input: Data sent

[0854] Output: Input data for a generative AI model

[0855] Step 5: The generative model analyzes the data and generates appropriate advice.

[0856] Description: The generative AI model analyzes the data it receives. For example, if it detects that the user is slouching, it generates advice like "Stand up straight."

[0857] Input: Input data for the generative AI model

[0858] Output: Advice statement

[0859] Step 6: The server analyzes the user's emotions using the emotion engine.

[0860] Description: The server uses the emotion engine to analyze the user's heart rate, breathing patterns, etc., and recognizes the user's fatigue state and emotions. For example, if the user is recognized as tired, data is output to generate an encouraging message.

[0861] Input: User's physiological data

[0862] Output: Emotional state data

[0863] Step 7: The server generates advice according to the emotion.

[0864] Description: The server generates appropriate advice and encouraging messages for the user based on the emotional state data obtained from the emotion engine. For example, it generates an encouraging message such as "You're almost there, keep trying!"

[0865] Input: Emotional state data

[0866] Output: Advice sentences according to emotions

[0867] Step 8: The server sends the generated advice to the terminal.

[0868] Description: The server sends the generated advice to the user's device in real time, allowing the user to receive immediate feedback.

[0869] Input: Generated advice statement

[0870] Output: Advice text sent to terminal.

[0871] Step 9: The terminal provides advice to the user.

[0872] Description: The device provides the user with advice received from the server. For example, the device display shows "Please stand up straight" or "You're almost there, let's do our best!". The user can see this and correct their posture, which motivates them to continue training.

[0873] Input: Advice text sent to the terminal

[0874] Output: Provide advice to the user

[0875] By having each step work together in this way, users can receive appropriate advice in real time, enabling them to conduct more effective training and activities.

[0876] (Application example 2) Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal".

[0877] Conventional fitness instruction systems using wearable devices have the problem that they can only obtain information about the user's body movements and posture, and are unable to provide advice that takes into account the user's emotional state. This can make it difficult for users to maintain their training effectiveness and motivation.

[0878] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means. In this invention, the server includes a means for acquiring information on the user's body movements and posture, a means for generating advice on the user's body movements and posture during activity using a generative model, a means for providing the advice to the user, and a means for recognizing the user's emotions and generating appropriate advice according to the emotions. This makes it possible not only to provide the user with real-time posture and form correction guidance during training, but also to provide encouragement and advice for improving motivation that takes into account the user's emotional state. This improves the training effect of the user and makes it possible to maintain motivation.

[0879] "Information regarding the user's body movements and posture" refers to motion data such as the position, direction, speed, and acceleration of the user's body, as well as posture information during a specific activity, obtained via a wearable device.

[0880] A "generative model" is a neural network trained using a machine learning algorithm, and is a model that analyzes a user's body movements and postures to generate advice.

[0881] "Means for providing advice to a user" refers to a mechanism for notifying a user of the generated advice, and specifically refers to a terminal such as a smartphone, smart glasses, or a head-mounted display.

[0882] "Means for recognizing the user's emotions and generating appropriate advice based on those emotions" refers to a mechanism for analyzing the user's motion data and vital signs to estimate their emotional state and generate advice or encouraging messages that are adapted to that state.

[0883] The present invention is a system that tracks the user's body movements and postures, analyzes the data, and provides appropriate advice. It also includes a function to recognize the user's emotional state and generate encouraging messages according to the emotion. The system is composed of a wearable device, a server, and a terminal.

[0884] A wearable device is an electronic device worn by a user and is equipped with multiple sensors, such as an acceleration sensor and a gyro sensor, to obtain information about the user's body movements and posture. The wearable device transmits the obtained data to a terminal or directly to a server.

[0885] The server inputs the received data into a generative model and analyzes the user's body movements and posture. The generative model is a neural network trained using machine learning algorithms. The server generates advice to provide to the user based on the analysis results. It also has an emotion engine that recognizes the user's emotional state. Depending on the user's emotional state, it generates appropriate advice or encouraging messages.

[0886] The terminal is a device that receives the advice sent from the server and provides it to the user. Examples of such devices include smartphones, smart glasses, and head-mounted displays. The user can check the advice provided on the terminal and correct their own movements and posture. This advice is provided in real time and is expected to improve the effectiveness of the user's training.

[0887] As a concrete example, consider the case where a user is doing squats at a fitness club. The wearable device detects the user's movements and sends them to the server. The server analyzes the received data and generates specific advice such as "Bend your knees a bit more." If the emotion engine recognizes that the user is tired, it also sends an encouraging message such as "You're doing great! Not much longer!" The user can train while checking these instructions and encouragement on the device.

[0888] Examples of prompts to be input to a generative AI model include the following:

[0889] "This shows posture data when the user is wearing the wearable device. Based on the following data, determine whether the user's current squat posture is correct and generate advice if necessary. Also, consider the user's emotions and include words of encouragement if the user seems tired. Sensor data: {Accelerometer value}, {Gyro sensor value}."

[0890] This system allows users to receive real-time guidance on how to correct their form while training, and also provides encouraging messages that take into account their emotional state, improving the effectiveness of their training and helping them maintain their motivation.

[0891] The flow of the specific process in the application example 2 will be described with reference to FIG.

[0892] Step 1: The wearable device acquires information about the user's body movements and posture. This information is data obtained from acceleration and gyro sensors, and specifically includes the user's position, direction, speed, acceleration, etc. The input is the user's motion data, and the output is compiled as sensor data.

[0893] Step 2: The wearable device transmits the acquired sensor data to the terminal or directly to the server. This data is transmitted in real time and communicated using an appropriate protocol. The input is the sensor data and the output is the data to be sent to the server.

[0894] Step 3: The server inputs the received sensor data into a generative model to analyze the user's body movements and posture. The generative model is a neural network trained with machine learning algorithms and generates appropriate advice based on the analyzed data. The input is the sensor data and the output is advice as a result of the analysis.

[0895] Step 4: The server generates advice based on the analysis results and sends it to the terminal. The advice includes specific posture correction instructions. The input is the advice as the analysis result, and the output is the data to be sent to the terminal.

[0896] Step 5: At the same time, the server recognizes the user's emotional state through the emotion engine. This uses the motion data and vital sign data. It estimates the emotional state and generates advice or encouraging messages accordingly. The input is the motion data and vital sign data, and the output is advice based on the emotion.

[0897] Step 6: The server sends appropriate advice or encouraging messages to the terminal based on the emotion recognition result. The input is the advice as the emotion recognition result, and the output is the data to be sent to the terminal.

[0898] Step 7: The terminal provides the advice received from the server to the user. This can be done using a smartphone, smart glasses, a head-mounted display, etc., and the user can view the advice and correct their movements and posture. The input is the advice data from the server, and the output is the display on the terminal.

[0899] Step 8: The user checks the display on the device and corrects their movements and posture in real time to improve the training effect. In addition, motivation is maintained by encouraging messages that take into account the user's emotional state. The input is the display data on the device, and the output is the user's actions.

[0900] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0901] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by making a neural network perform deep learning. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image is input. The data generation model 58 performs inference on the input inference data according to the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0902] In the above embodiment, an example was given in which the specific process was performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the robot 414.

[0903] The emotion identification model 59 as an emotion engine may determine the emotion of the user according to a specific mapping. Specifically, the emotion identification model 59 may determine the emotion of the user according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the emotion of the robot, and the identification processing unit 290 may perform identification processing using the emotion of the robot.

[0904] FIG. 9 is a diagram showing an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive emotions are arranged. The more outside the concentric circles, the more emotions that represent states and actions that arise from a state of mind are arranged. Emotions are a concept that includes emotions and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions that occur in the brain are arranged. On the right side of the concentric circles, emotions that are generally induced by situational judgment are arranged. On the upper and lower sides of the concentric circles, emotions that are generally generated from reactions that occur in the brain and are induced by situational judgment are arranged. In addition, on the upper side of the concentric circles, emotions of "pleasure" are arranged, and on the lower side, emotions of "discomfort" are arranged. In this way, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[0905] These emotions are distributed in the three o'clock direction of emotion map 400 and usually fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[0906] The inside of emotion map 400 represents what is going on inside one's mind, and the outside of emotion map 400 represents behavior, so the further out you go on emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[0907] Here, human emotions are based on various balances such as posture and blood sugar level, and when these balances are far from the ideal, it indicates an unpleasant state, and when they are close to the ideal, it indicates a pleasant state. Emotions can also be created for robots, cars, motorcycles, etc., based on various balances such as posture and battery level, so that when these balances are far from the ideal, it indicates an unpleasant state, and when they are close to the ideal, it indicates a pleasant state. The emotion map may be generated, for example, based on the emotion map of Dr. Mitsuyoshi (Research on speech emotion recognition and emotion brain physiological signal analysis system, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). On the left half of the emotion map, emotions belonging to an area called "reaction" where sensation is dominant are lined up. On the right half of the emotion map, emotions belonging to an area called "situation" where situation recognition is dominant are lined up.

[0908] The emotion map defines two emotions that promote learning. The first is the negative emotion around the middle of "repentance" or "remorse" on the situation side. In other words, this is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the positive emotion around "desire" on the response side. In other words, this is when the robot has positive feelings such as "I want more" or "I want to know more."

[0909] The emotion identification model 59 inputs the user input to a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the emotion of the user. This neural network is pre-trained based on multiple learning data that are combinations of the user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in Fig. 10. Fig. 10 shows an example in which multiple emotions, "relief," "calm," and "encouraging," have similar emotion values.

[0910] Although the system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, the system according to the present disclosure is not necessarily implemented in a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program that runs on a personal computer, or an application that runs on a smartphone or the like. The method according to the present disclosure may be provided to a user in the form of SaaS (Software as a Service).

[0911] In the above embodiment, an example is given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to input data.

[0912] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable computer-readable non-transitory storage medium such as a Universal Serial Bus (USB) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[0913] In addition, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 upon request from the data processing device 12.

[0914] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[0915] As the hardware resource for executing the specific process, various processors as shown below can be used. An example of the processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing the specific process by executing software, i.e., a program. Another example of the processor is a dedicated electric circuit, which is a processor having a circuit configuration designed exclusively for executing the specific process, such as a Field-Programmable Gate Array (FPGA), a Programmable Logic Device (PLD), or an Application Specific Integrated Circuit (ASIC). Each processor has a built-in or connected memory, and each processor executes the specific process by using the memory.

[0916] The hardware resource that executes the specific process may be one of these various processors, or may be a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific process may be a single processor.

[0917] As an example of a configuration using one processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a configuration using a processor that realizes the functions of the entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[0918] Furthermore, more specifically, the hardware structure of these various processors can be an electric circuit that combines circuit elements such as semiconductor elements. The specific processes described above are merely examples. It goes without saying that unnecessary steps may be deleted, new steps may be added, or the order of processes may be changed without departing from the spirit of the invention.

[0919] The above description and illustrations are detailed descriptions of the parts related to the technology of the present disclosure, and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, function, action, and effect is an example of the configuration, function, action, and effect of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above description and illustrations, within the scope of the gist of the technology of the present disclosure. In addition, in order to avoid confusion and to facilitate understanding of the parts related to the technology of the present disclosure, the above description and illustrations omit explanations of technical common sense that do not require explanation in order to enable the implementation of the technology of the present disclosure.

[0920] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, and technical standard was specifically and individually indicated to be incorporated by reference.

[0921] The following is further disclosed regarding the above embodiment.

[0922] (First aspect) The system includes: a means for acquiring information on a user's body movements and posture; a means for generating advice on the user's body movements and posture during an activity using a generative model; and a means for providing the advice to the user. system.

[0923] (Second aspect) The information regarding the user's body movements and posture is acquired via a wearable device worn by the user. The system according to the first aspect.

[0924] (Third aspect) The generative model is a neural network trained using a machine learning algorithm. 2. The system according to the first or second aspect.

[0925] (Fourth aspect) further comprising an emotion engine for recognizing an emotion of the user; A system according to any one of the first to third aspects.

[0926] (Fifth aspect) The emotion engine is a neural network using machine learning algorithms to recognize the user's emotions. The system according to the fourth aspect.

[0927] (Sixth aspect) The emotion engine works in combination with a rule-based system to generate appropriate advice according to the user's emotions. 10. The system according to the fourth or fifth aspect.

[0928] (Seventh aspect) The wearable device includes smart glasses. The system according to the second aspect.

[0929] "Example 1"

[0930] (Claim 1) A means for acquiring information regarding a user's body movements and posture via a wearable device; A means for transmitting the acquired information to a server via a terminal or directly; A means for generating advice regarding the user's body movements and postures during an activity from the acquired information using a generative model; A means for transmitting the generated advice to a terminal and providing the advice to a user; means for providing said advice to a user in real time; A system including:

[0931] (Claim 2) The system of claim 1 , wherein the generative model is a neural network using a machine learning algorithm.

[0932] (Claim 3) 2. The system according to claim 1, wherein the terminal has a function of transmitting acquired information to a server and providing generated advice to the user.

[0933] "Application example 1"

[0934] (Claim 1) means for acquiring information about a user's body movements and posture; A means for generating advice regarding body movements and postures during a user's activity using the generative model; means for providing said advice to a user; A means of analyzing data on the movements and postures of factory workers and providing advice aimed at improving safety and efficiency; A system including:

[0935] (Claim 2) The information regarding the user's body movements and posture is acquired via a wearable device worn by the user. 2. The system of claim 1.

[0936] (Claim 3) The generative model is a neural network trained using a machine learning algorithm. 2. The system of claim 1.

[0937] "Example 2 of combining emotion engines"

[0938] (Claim 1) means for acquiring information about a user's body movements and posture; A means for generating advice regarding body movements and postures during a user's activity using the generative model; means for providing said advice to a user; A means including an emotion engine for analyzing the acquired information and recognizing an emotion of a user; A means for generating appropriate advice according to the user's emotions; A system including:

[0939] (Claim 2) The information regarding the user's body movements and posture is acquired via a wearable device worn by the user. 2. The system of claim 1.

[0940] (Claim 3) The generative model is a neural network trained using a machine learning algorithm. 2. The system of claim 1.

[0941] "Application example 2 when combining emotion engines"

[0942] (Claim 1) means for acquiring information about a user's body movements and posture; A means for generating advice regarding body movements and postures during a user's activity using the generative model; means for providing said advice to a user; A means for recognizing a user's emotion and generating appropriate advice according to the emotion; A system including:

[0943] (Claim 2) The information regarding the user's body movements and posture is acquired via a wearable device worn by the user. 2. The system of claim 1.

[0944] (Claim 3) The generative model is a neural network trained using a machine learning algorithm. 2. The system of claim 1. [Explanation of symbols]

[0945] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for acquiring information regarding a user's body movements and posture via a wearable device; A means for transmitting information acquired by the wearable device to a server via a terminal or directly from the wearable device; a means for generating advice regarding the user's body movement and posture during the activity, using a prompt sentence instructing the server to generate advice regarding the user's body movement and posture during the activity from the acquired information and a generative model; and means for providing the generated advice to the user. system.

2. The generative model is a neural network using a machine learning algorithm. The system of claim 1 .

3. The terminal transmits the acquired information to the server, and provides the generated advice to the user. The system of claim 1 .

4. The method further includes a means including an emotion engine for analyzing the acquired information and recognizing an emotion of a user, the means for generating advice generates the advice by using a prompt sentence instructing to generate the advice from the acquired information and the recognized emotion of the user, and by using the generative model. The system of claim 1 .

Images