System

Abstract

A system is provided.SOLUTION: A system comprising: means for collecting physiological and positional data during a run; means for transmitting the collected data in real time to a remote server; means for analyzing the collected data at the remote server and evaluating the user's performance status; means for generating appropriate music data based on the evaluation results; means for transmitting the generated music data to a user terminal; and means for playing the transmitted music data.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Listening to music while running is said to be effective in improving runners' performance. However, music selection is currently left up to the user, making it difficult to provide optimal music in real time. As a result, runners are unable to maximize their motivation and achieve sufficient performance improvement. There is a need to solve this situation, effectively improve runners' performance, and provide a comfortable running experience. [Means for solving the problem]

[0005] This invention provides a system that collects physiological and positional data obtained while running and transmits it to a remote server in real time. The remote server analyzes the received data and evaluates the user's performance. Based on the evaluation results, optimal music data is generated and transmitted to the user's device. Furthermore, a mechanism is introduced that receives feedback from the user and reflects it in the generation of music data, thereby continuously improving the suitability of the music. Furthermore, comparison with past performance data enables more accurate performance evaluation and music generation. This system allows users to run more effectively while listening to optimal music in real time.

[0006] "Running" is a sporting activity that improves physical function by running continuously at a constant speed.

[0007] "Physiological data" is data indicating physiological indicators obtained from a living body, such as the user's heart rate, blood pressure, and respiratory rate.

[0008] "Location data" refers to data including geographical location information indicating the user's current location and travel route.

[0009] A "remote server" is a computer system that provides services over a network and has the ability to receive, store, and analyze data from users.

[0010] "Analysis" is the process of evaluating and analyzing acquired data based on relevant algorithms and models.

[0011] "Performance status" is a collection of indicators that indicate the user's athletic ability and physical condition while running.

[0012] "Music data" refers to data that includes audio files and stream information necessary for playing music.

[0013] "Feedback" is information indicating opinions and evaluations provided by users regarding the output of the system.

[0014] "Past performance data" refers to data that records the user's athletic ability and physical condition from previous running sessions. [Brief explanation of the drawings]

[0015] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0016] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0017] First, the terms used in the following description will be explained.

[0018] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0019] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0020] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0021] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0023] [First embodiment]

[0024] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0025] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0028] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0031] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0035] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0036] The system of the present invention collects user performance data in real time while running and generates optimal music based on that data to help runners improve their performance. This system is implemented as follows.

[0037] User launches and configures the application

[0038] Users launch a dedicated application installed on their smartphone or wearable device. The application provides an option to start running, which users select to begin collecting running data. Users can also input their music preferences and past running data into the app.

[0039] Data collection and transmission

[0040] As soon as you start running, your device collects real-time physiological and location data, such as GPS data, heart rate, speed, and distance. This data is then sent to a server at regular intervals via HTTP POST requests, with the data encoded in JSON format.

[0041] Server-side data reception and analysis

[0042] The server sets up an API endpoint to receive data sent from the device, which is then immediately analyzed to assess the user's current performance. For example, metrics such as heart rate, speed, and distance are used to determine whether the user is running at a sustainable pace or whether fatigue is building up.

[0043] Optimal music data generation

[0044] Based on the analysis results, the server generates music data that is optimal for the user's running situation. This generation uses a generative AI model and takes into account the user's preferences and past feedback data. For example, if the user is running at a fast pace, fast-paced, energetic music will be generated. Conversely, if the user is running an endurance race, music with a steady rhythm will be generated.

[0045] Sending and playing music data

[0046] The generated music data is sent from the server to the user's device, which analyzes the received music data and plays it in a format suitable for the user. Playback is performed using the application's audio player, allowing users to listen to the perfect music while running.

[0047] User feedback and regeneration

[0048] Users can provide real-time feedback on music while running through the application. For example, they can send feedback such as "I like this song," "The tempo is too fast," or "The tempo is too slow" by pressing a button. The device then sends this feedback to the server, which then adjusts the parameters for generating the music data based on the feedback. The adjusted music data is then sent back to the user's device, and playback is updated.

[0049] This invention provides users with optimal music in real time, which helps maintain motivation, reduce fatigue, and improve performance, significantly improving the running experience and enhancing the training effect.

[0050] The processing flow will be explained below.

[0051] Step 1: The user launches the dedicated application installed on their smartphone or wearable device. Once the user logs in and selects the option to start running, data collection mode begins.

[0052] Step 2: The device uses the GPS sensor and heart rate sensor to collect real-time physiological and location data during the run, including current location, speed, heart rate, distance, etc.

[0053] Step 3: The device converts the collected data into JSON format at regular intervals (e.g., every second) and sends it to the server using an HTTP POST request, including the user ID.

[0054] Step 4: The server sets up an API endpoint to receive data sent from the device, which is then immediately passed to the analysis module.

[0055] Step 5: The server's analysis module evaluates the user's current performance based on the received data, and the evaluation result is expressed in the form of tags, such as "sustained pace" or "fast pace."

[0056] Step 6: The server generates optimal music data using a generative AI model based on the evaluation results, taking into account the user's musical preferences and past feedback.

[0057] Step 7: The server sends the generated music data to the user's terminal, where the music data is encoded in an appropriate format (e.g., MP3).

[0058] Step 8: The device analyzes the received music data and plays the music using the audio player in the application. The user can continue running while listening to the optimal music in real time.

[0059] Step 9: Users can provide real-time feedback on the music within the application, in the form of "I like this song" or "The tempo is too fast."

[0060] Step 10: The device sends the user feedback to the server in JSON format.

[0061] Step 11: The server analyzes the received feedback and inputs it as new parameters into the generative AI model. The music data is then regenerated based on this feedback.

[0062] Step 12: The server again sends the newly generated music data to the user's device. The device again receives the music data and plays it on the audio player. This process is repeated, allowing the system to continue providing the optimal music experience the user desires.

[0063] Through these steps, the system improves the user's running experience in real time and helps improve performance.

[0064] Example 1

[0065] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0066] Conventional running support systems have difficulty not only collecting physiological and location data from users but also providing appropriate feedback and support to improve their performance based on that data. In particular, there is a need for systems that can maintain motivation and reduce fatigue while running by evaluating the user's performance in real time and dynamically generating and providing optimal music based on that data.

[0067] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0068] In this invention, the server includes means for collecting physiological data and position data during running, means for transmitting the collected data to a remote server in real time, means for analyzing the collected data at the remote server and evaluating the user's performance status, means for using a generative AI model to generate appropriate music data based on the evaluation results, means for inputting a prompt sentence to the generative AI model to generate music data based on the evaluation results, means for transmitting the generated music data to a user terminal, and means for playing the transmitted music data. This makes it possible to dynamically generate and provide music tailored to each user's individual running status, thereby maintaining the user's motivation and contributing to improving performance.

[0069] "Physiological data" refers to physiological information such as heart rate, blood pressure, and respiratory rate collected from within the user's body in real time.

[0070] "Location Data" means information based on GPS coordinates that indicates a user's current location.

[0071] "Remote Server" refers to a central processing unit that transmits, receives, and analyzes data over a network.

[0072] "Analysis Engine" means a software component that evaluates a user's performance status based on collected data.

[0073] "Generative AI model" refers to an artificial intelligence model that generates optimal music based on the user's performance status and past feedback data.

[0074] A "prompt" is an instructional text that specifies conditions or requests and is input into a generative AI model.

[0075] "User terminal" refers to a portable electronic device used by a user, such as a smartphone or wearable device.

[0076] "Audio player" refers to a software or hardware component for playing music data within a user terminal.

[0077] "Feedback" refers to input information that provides the system with responses and impressions that the user has actually experienced in real time.

[0078] The system of this invention provides users with real-time music that optimizes their running performance. The system utilizes various hardware and software, including smartphones, wearable devices, remote servers, and generative AI models, to provide a dynamic and personalized experience for users.

[0079] User launches and configures the application

[0080] Users launch a dedicated application installed on their smartphone or wearable device. The application has a "Start Running" option, which users can select to begin collecting running data. Users can input their music preferences and past running data into the app in advance.

[0081] Data collection and transmission

[0082] The device collects real-time physiological and location data, such as GPS data, heart rate, speed, and distance, as soon as the user starts running. This data is encoded into JSON format at regular intervals and sent to the server via HTTP POST requests. The server uses the Python Flask framework to set up an API endpoint to receive the data.

[0083] Server-side data reception and analysis

[0084] The server immediately analyzes the received data and assesses the user's current performance status, using metrics such as heart rate, speed, and distance to determine whether the user is running at a sustainable pace or whether fatigue is building up.

[0085] Optimal music data generation

[0086] Based on the analysis results, the server uses a generative AI model to generate music data that is optimal for the user's running situation. This generation also takes into account the user's past feedback data and preferences. For example, if the user is running at a fast pace, fast-paced, energetic music will be generated. The following prompt sentence is input to the generative AI model:

[0087] The user's current heart rate is 160 BPM and their running speed is 5 km / min. The user's past running history has shown that they prefer music with a tempo of 120 BPM. Based on these conditions, generate music that will allow the user to maintain their current pace.

[0088] Sending and playing music data

[0089] The generated music data is sent from the server to the user's device, which analyzes the received music data and plays it using the application's audio player, allowing users to listen to the perfect music while running.

[0090] User feedback and regeneration

[0091] While running, users can provide real-time feedback on the music through the application. For example, they can send feedback such as "I like this song," "The tempo is too fast," or "The tempo is too slow" by pressing a button. The device sends this feedback to the server, which then adjusts the parameters of the generative AI model based on this feedback. The adjusted music data is then sent back to the user's device, and playback is updated.

[0092] These processes allow users to continue running while enjoying optimal music in real time, which is expected to maintain motivation and improve performance.

[0093] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0094] Step 1:

[0095] Starting and Configuring the Application

[0096] The user launches a dedicated application on their smartphone or wearable device. They then tap the "Start Running" button, which prepares the application to begin collecting running data. Input includes the user's music preferences and past running data. Based on this, user profile data is generated.

[0097] Step 2:

[0098] Collecting and sending running data

[0099] The device enters collection mode when the user starts running. It collects physiological and location data such as GPS data, heart rate, speed, and distance in real time from the GPS sensor, heart rate sensor, and accelerometer. The data is converted into JSON format and sent to the server at regular intervals via HTTP POST requests. The input is physiological and location data, and the output is JSON-formatted data.

[0100] Step 3:

[0101] Data reception and storage

[0102] The server uses the Python Flask framework to set up an API endpoint. The server receives data received as a POST request from the terminal and saves the data in temporary storage. The input is the received JSON data, and the output is the temporarily saved data. As an example of actual operation, when a request arrives at the / data endpoint, the data is stored in the database.

[0103] Step 4:

[0104] Data analysis

[0105] The server's analysis engine analyzes the stored data. The server evaluates indicators such as heart rate, speed, and distance to determine the user's performance. The analysis results may include information such as "heart rate is high," "speed is stable," and "distance still half covered." The input is the temporarily stored data, and the output is the analyzed performance evaluation.

[0106] Step 5:

[0107] Music data generation

[0108] Based on the analysis results, the server inputs a prompt into the generative AI model to generate music data that is optimal for the user's running situation. The user's preferences and past feedback data are also taken into consideration. The input is performance evaluation and user profile, and the output is the generated music data. An example of a prompt is, "The user's current heart rate is 160 BPM and speed is 5 minutes / km. The user's past running history has included feedback that they prefer music with a tempo of 120 BPM. Based on these conditions, please generate music that will allow the user to maintain their current pace."

[0109] Step 6:

[0110] Sending and playing music data

[0111] The server sends the generated music data in JSON format as an HTTP response to the device. The device analyzes the received data and plays the music using the audio player in the application. The input is the received music data, and the output is the played music. As a specific example of operation, the device downloads an MP3 file and plays it on the built-in player.

[0112] Step 7:

[0113] Receiving feedback and reanalyzing

[0114] The user sends feedback about the music through the application. For example, by pressing a button, they can indicate whether the tempo is too fast or too slow. The device encodes this feedback data into JSON format and sends it to the server. The server then adjusts the parameters of the generative AI model based on the received feedback and generates new music data. The input is the user feedback, and the output is the regenerated music data.

[0115] This series of processes allows the user to continue running while always enjoying the most suitable music in real time.

[0116] (Application example 1)

[0117] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0118] Maintaining and improving productivity is a challenge in modern factories. In particular, there is a need to promote efficient operation of factory machines and optimize operator motivation and the operating pace of the machines. However, current systems lack the technology to analyze factory machine operation data and environmental data in real time and automatically provide appropriate work music based on that data. This can result in a decline in factory productivity.

[0119] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0120] In this invention, the server includes a means for collecting operational data and environmental data of factory machines, a means for evaluating the productivity of the factory machines based on the collected data, and a means for generating appropriate work music based on the evaluation results. This makes it possible to evaluate the operating efficiency of factory machines in real time and automatically generate and provide music suited to the work environment.

[0121] "Physiological data" refers to data that indicates the physiological state of the body, such as heart rate, speed, and distance.

[0122] "Location data" refers to geographical location information of users and devices obtained by GPS or other means.

[0123] "Factory machinery" refers to automated equipment and robotic systems used in manufacturing operations.

[0124] "Operation data" refers to data that indicates the operating status of factory machines, such as operating time, operating speed, and travel distance.

[0125] "Environmental data" refers to data that indicates the surrounding environmental conditions such as temperature, humidity, and noise level.

[0126] A "server" is a remote computer system that analyzes the collected data and takes appropriate action.

[0127] "Analysis" refers to the process of making an evaluation or diagnosis based on collected data that is suitable for a specific purpose.

[0128] "Productivity" is an indicator of the efficiency and performance of factory machines and systems over a certain period of time.

[0129] "Work music" is music provided to improve work efficiency and motivation.

[0130] "Feedback" refers to evaluations and opinions given by users and systems, which are used by the system to adapt and improve.

[0131] MODE FOR CARRYING OUT THE INVENTION

[0132] The system for realizing this invention first incorporates sensors that collect operational data and environmental data from factory machines. This data is sent to a cloud server using Wi-Fi or wired connections installed in the factory. The data is sent using an HTTP POST request, and the data is encoded in JSON format.

[0133] The server receives the collected data and analyzes it in real time. Data analysis tools such as Python and R are used for the analysis, and machine learning models and AI technologies (e.g., TensorFlow, Scikit-learn) are used to evaluate the operating status and productivity of factory machinery. For example, the current performance and load status of machinery can be determined based on data such as operating time, operating speed, travel distance, temperature, humidity, and noise level.

[0134] Based on the analysis results, the server inputs specific prompt sentences into the generative AI model to generate appropriate work music. The prompt sentences include content that corresponds to the operating status and productivity of the factory machine. The generated music data is then sent from the server to the factory machine's control system. The factory machine's control system analyzes the received music data and plays it through the built-in speaker or speakers of peripheral devices.

[0135] Feedback from factory machines and operators is sent to the server in real time. This feedback includes an evaluation of whether the tempo of the music is appropriate and the impact of the music on productivity. The server uses this feedback to adjust the parameters of the generative AI model and reflect this in future music generation. This makes it possible to automatically provide optimized work music.

[0136] Specific examples

[0137] Hardware used

[0138] Sensors: Sensors built into factory machines to collect operational and environmental data

[0139] Communication equipment: Wi-Fi module or wired connection

[0140] Cloud server: A server that performs data analysis and music generation (e.g., AWS, Google Cloud)

[0141] Software used

[0142] Data analysis tools: Python, R

[0143] Machine learning models: TensorFlow, Scikit-learn

[0144] Music Generation Library: A tentative music generation library

[0145] Prompt Sentence Examples

[0146] "Based on the music feedback provided during the run, generate new music taking into account the following criteria:

[0147] Productivity has decreased in the last 30 minutes

[0148] Factory machines operate at lower than average speeds

[0149] There's a demand for high-energy, fast-paced music."

[0150] In this way, this invention makes it possible to generate and provide optimal work music in real time based on the operation data and environmental data of factory machines, which is expected to improve factory productivity and efficient machine operation.

[0151] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0152] Step 1:

[0153] Factory machines use built-in sensors to collect operational and environmental data, such as operating time, operating speed, travel distance, temperature, humidity, and noise level.

[0154] Step 2:

[0155] The collected data is sent to a cloud server via Wi-Fi or wired connection installed in the factory. The data is sent using an HTTP POST request, and is encoded in JSON format. The JSON format data is output.

[0156] Step 3:

[0157] The server analyzes the received data. Specifically, it uses data analysis tools such as Python and R to apply machine learning models (e.g., TensorFlow, Scikit-learn) to evaluate the operating status and productivity of factory machinery. The evaluation uses data such as operating time, operating speed, temperature, humidity, and noise level. The analysis results are output.

[0158] Step 4:

[0159] The server generates a prompt sentence based on the analysis results. The prompt sentence contains content that corresponds to the operating status and productivity of the factory machines. For example, "Productivity has decreased in the last 30 minutes" or "Energetic, fast-paced music is required." The prompt sentence is the output.

[0160] Step 5:

[0161] The server inputs the generated prompt sentence into a generative AI model to generate appropriate task music. The generative AI model (e.g., OpenAI's GPT-3) generates music data based on the prompt sentence. The generated music data is the output.

[0162] Step 6:

[0163] The generated music data is then sent from the server to the factory machine control system, where it becomes the output.

[0164] Step 7:

[0165] The control system of the factory machine analyzes the received music data and plays it through the built-in speaker or a speaker of a peripheral device. The playback of the music data becomes the output.

[0166] Step 8:

[0167] Feedback from factory machines and operators is sent to the server in real time. The feedback includes an assessment of whether the music tempo is appropriate and the impact of the music on productivity. The feedback data is the output.

[0168] Step 9:

[0169] The server adjusts the parameters of the generative AI model based on the received feedback and reflects this in future music generation. This further optimizes the generated music. The adjusted generative AI model is the output.

[0170] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0171] This invention is a system that collects and analyzes a user's performance and emotional data in real time while they are running, and generates optimal music based on that data, thereby improving a runner's performance and providing a comfortable running experience.

[0172] User launches and configures the application

[0173] The user launches a dedicated application installed on a smartphone or wearable device. The application provides an option to start running, which the user selects to begin data collection. The user can also input their music preferences and past running data into the app. Furthermore, the present invention incorporates an emotion engine that also collects user emotion data.

[0174] Data collection and transmission

[0175] As soon as the device starts running, it collects real-time physiological and location data, such as GPS data, heart rate, speed, and distance. Additionally, the emotion engine analyzes the user's voice input, facial expressions, and other physiological data to recognize emotions. This data is then sent to the server at regular intervals using an HTTP POST request, with the data encoded in JSON format.

[0176] Server-side data reception and analysis

[0177] The server has an API endpoint set up to receive data sent from the device. The received data is immediately passed to the analysis module, which evaluates the user's current performance and emotional state based on the running data and emotional data. For example, along with indicators such as heart rate, speed, and distance, the emotional state of the user, such as whether they are stressed or relaxed, is evaluated.

[0178] Optimal music data generation

[0179] Based on the analysis results, the server uses a generative AI model to generate optimal music data. The generative AI model takes into account the user's performance and emotional state, as well as their music preferences and past feedback. For example, if a user is running at a fast pace but feeling stressed, music with a fast tempo but a relaxing feel will be generated. On the other hand, if a user is running a long distance and feeling relaxed, music with a steady rhythm will be generated.

[0180] Sending and playing music data

[0181] The generated music data is sent from the server to the user's device, which analyzes the received music data and plays it in a format suitable for the user. Playback is performed using the application's audio player, allowing users to listen to the perfect music while running.

[0182] User feedback and regeneration

[0183] Users can provide real-time feedback on music through the application while running. For example, they can send feedback such as "I like this song," "The tempo is too fast," or "The tempo is too slow" by pressing a button. The device sends this feedback to the server, which then adjusts the parameters for generating the music data based on the feedback. The adjusted music data is then sent back to the user's device, and playback is updated.

[0184] Specific examples

[0185] For example, a user starts running, launches the application, and begins collecting data. If the user's heart rate increases while running and the emotion engine recognizes stress from the user's voice, the server generates fast-paced but relaxing music. The music data is immediately sent to the user's device and played on an audio player. The user can continue running while listening to the music and provide feedback.

[0186] In this way, by integrating the user's performance data and emotional data to provide an optimal music experience, the system improves running performance and provides a comfortable running experience.

[0187] The processing flow will be explained below.

[0188] Step 1: The user launches the dedicated application installed on their smartphone or wearable device, logs in to the application, and selects the option to start running.

[0189] Step 2: The device uses the GPS sensor, heart rate sensor, and emotion engine to collect physiological data, location data, and emotion data in real time while running, including current location, speed, heart rate, distance, smile level, voice tone, etc.

[0190] Step 3: The terminal converts the collected data into JSON format and sends it to the server at regular intervals (e.g., every second) using an HTTP POST request.

[0191] Step 4: The server receives the data sent from the device at the API endpoint, and passes the received data to the analysis module.

[0192] Step 5: The server's analysis module evaluates the user's current performance status and emotional state based on the received physiological, location, and emotional data, including categories such as "sustained pace," "fast pace," "relaxed," and "stressed."

[0193] Step 6: The server generates optimal music data based on the evaluation results using a generative AI model that takes into account the user's performance and emotional state, as well as their musical preferences and past feedback.

[0194] Step 7: The server sends the generated music data to the user's device, encoded in an appropriate format (e.g., MP3).

[0195] Step 8: The device analyzes the received music data and plays the music using the audio player in the application. The user can continue running while listening to the optimal music in real time.

[0196] Step 9: The user provides real-time feedback on the music within the application, using options such as "I like this song," "The tempo is too fast," or "The tempo is too slow."

[0197] Step 10: The device sends the user feedback to the server in JSON format.

[0198] Step 11: The server's analysis module analyzes the received feedback and inputs it as new parameters into the generative AI model, so that the next time music is generated, the user's preferences will be reflected.

[0199] Step 12: The server retransmits the newly generated music data to the user's device. The device again analyzes the received music data and plays it on the audio player. This process is repeated, allowing the user to continue running while always listening to the best music.

[0200] In this way, the system integrates the user's physiological data, location data, and emotional data to generate optimal music in real time, thereby improving performance and providing a comfortable running experience.

[0201] Example 2

[0202] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0203] The purpose of this invention is to improve runners' performance and provide a comfortable running experience by providing a means to collect and analyze a user's performance and emotional data in real time while running and generate optimal music based on that data. While conventional systems can collect and analyze physiological data, it is difficult to generate optimal music that takes the user's emotional state into account. Furthermore, there is a need for a more personalized experience by dynamically adjusting the music based on real-time feedback.

[0204] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0205] In this invention, the server includes a means for analyzing data collected by a remote server and evaluating the user's performance status and emotional state, a means including a generative AI model for generating optimal music data based on the evaluation results, and a means for transmitting the generated music data to the user terminal. This makes it possible to provide optimal music in real time according to the user's performance status and emotional state. Furthermore, by reflecting user feedback in real time and dynamically adjusting the music data, a personalized running experience can be achieved.

[0206] "Physiological data" is information related to a user's physical activity and physiological state, such as heart rate, speed, distance, etc.

[0207] "Location data" is information about the user's current location and travel route obtained using GPS.

[0208] A "remote server" is a server that is accessed via the Internet and is a device that receives and analyzes data sent from a user terminal.

[0209] A "generative AI model" is an artificial intelligence technology that generates music data taking into account the user's performance situation, emotional state, musical preferences, etc.

[0210] "Emotional state" refers to the user's emotional state, such as whether the user is stressed or relaxed.

[0211] "Feedback" refers to opinions and evaluations provided by users, and includes information such as responses to music tempo, likes and dislikes, etc.

[0212] An "analysis module" is a program or device that analyzes collected data and evaluates the user's performance status and emotional state.

[0213] An "audio player" is software or hardware for playing digital music data.

[0214] The present invention is a system that uses a smartphone or wearable device to collect real-time performance and emotional data while a user is running, and generates optimal music based on that data. The system analyzes the collected data, generates optimal music, plays the music, and incorporates user feedback.

[0215] First, the user launches a dedicated application installed on a smartphone or wearable device. The user enters their music preferences and past running data within the application, and the emotion engine also collects the user's emotional data. The hardware used in this case is a smartphone (iOS, Android) or a wearable device (Apple Watch, Fitbit, etc.), and the software is the dedicated application.

[0216] Next, as soon as the user starts running, the device collects real-time physiological and location data, such as GPS data, heart rate, speed, and distance. The emotion engine recognizes the user's emotions by analyzing voice input and facial expressions. The collected data is sent to the server at regular intervals. The data is sent using an HTTP POST request and encoded in JSON format.

[0217] The server passes the data received at the API endpoint to the analysis module. The analysis module evaluates the user's current performance status and emotional state based on the running data and emotional data. For example, it evaluates whether the user is feeling stressed or relaxed, along with indicators such as heart rate, speed, and distance. The analysis module used for this purpose is a program that evaluates the user's performance status and emotional state based on the collected data.

[0218] Based on the analysis results, the server uses a generative AI model to generate optimal music data. The generative AI model takes into account the user's performance status, emotional state, music preferences, and past feedback. This allows appropriate music to be generated in real time and sent to the user's device. For example, if a user is running at a fast pace but feeling stressed, music with a fast tempo but a relaxing feel will be generated. On the other hand, if a user is running a long distance and feeling relaxed, music with a steady rhythm will be generated. The generative AI model used for this is a program that generates optimal music data according to the user's performance status and emotional state.

[0219] The generated music data is sent from the server to the user's device, which then analyzes the received music data and plays it on the application's audio player, allowing the user to continue running while listening to the music.

[0220] Furthermore, users can provide real-time feedback on the music through the application while running. For example, they can send feedback such as "I like this song," "The tempo is too fast," or "The tempo is too slow" by pressing a button. The device sends this feedback to the server, which then adjusts the parameters for generating the music data based on the feedback. The adjusted music data is then sent back to the user's device, and the music playback is updated.

[0221] For example, when a user starts running and launches the application to begin collecting data, if their heart rate rises and the emotion engine detects stress in the user's voice, the server generates fast-paced but relaxing music. The music data is immediately sent to the user's device and played on an audio player. The user can continue running while listening to the music and provide feedback.

[0222] Examples of prompts include:

[0223] If the user's heart rate exceeds 150 and the emotion engine detects stress, what musical prompt should be generated?

[0224] Input data:

[0225] Heart rate: 150

[0226] Stress level: High

[0227] Music preference: Relaxing and fast-paced

[0228] Expected output:

[0229] Prompt: Generate fast-paced but relaxing music for a runner running at a fast pace.

[0230] In this way, the present invention integrates a user's performance data and emotional data to provide an optimal music experience, thereby improving performance while running and achieving a comfortable running experience.

[0231] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0232] Program processing flow

[0233] Step 1: User launches and configures the application

[0234] explanation

[0235] The user launches a dedicated application installed on a smartphone or wearable device, selects options to start running, and inputs music preferences and past running data. The emotion engine then begins collecting the user's emotional data.

[0236] input

[0237] User's music preferences

[0238] Past running data

[0239] output

[0240] Initial state after setup is complete

[0241] Activating the Emotion Engine

[0242] Specific actions

[0243] When a user launches the application and presses the "Start Running" button, a settings screen appears. This screen displays options for inputting "favorite music genre" and "average pace of past runs." This starts the emotion engine in the background.

[0244] Step 2: Start collecting data

[0245] explanation

[0246] As soon as the user starts running, the device collects real-time physiological and location data, including GPS data, heart rate, speed, and distance. The emotion engine analyzes voice input and facial expressions to recognize the user's emotions.

[0247] input

[0248] User's real-time physiological data (heart rate, speed, distance)

[0249] User's real-time location data (GPS information)

[0250] User emotion data (voice input, facial expression analysis)

[0251] output

[0252] Physiological and emotional data collection results

[0253] Specific actions

[0254] When a user starts running, their smartphone or wearable device automatically starts collecting sensor data, including heart rate monitors, GPS sensors, and microphones, and the data is stored locally at regular intervals.

[0255] Step 3: Sending data

[0256] explanation

[0257] The device sends the collected data to the server at regular intervals using HTTP POST requests, with the data encoded in JSON format.

[0258] input

[0259] Physiological and emotional data collection results

[0260] output

[0261] Data sent to the server

[0262] Specific actions

[0263] The collected data is automatically sent to the server at regular intervals. For example, every minute, heart rate, GPS data, speed, distance, and physiological data are sent to the server using an HTTP POST request. The data is formatted in JSON format and sent to the server's API endpoint.

[0264] Step 4: Receiving and parsing data on the server side

[0265] explanation

[0266] The server receives the data at the API endpoint and passes it to the analysis module, which evaluates the user's current performance status and emotional state based on the running data and emotional data.

[0267] input

[0268] Data sent to the server

[0269] output

[0270] Evaluation results of user performance and emotional state

[0271] Specific actions

[0272] The server passes the data received at the API endpoint to the analysis module, which analyzes heart rate, speed, distance, and emotional data. The analysis module uses this data to evaluate the user's level of stress or relaxation, and passes the results to the next step.

[0273] Step 5: Generate optimal music data

[0274] explanation

[0275] Based on the analysis results, the server generates optimal music data using a generative AI model that takes into account the user's performance, emotional state, musical preferences, and past feedback.

[0276] input

[0277] Evaluation results of user performance and emotional state

[0278] User's music preferences

[0279] Past Feedback

[0280] output

[0281] Optimal music data

[0282] Specific actions

[0283] Based on the analysis results, the server's generative AI model generates a prompt, which then generates optimal music data based on that prompt. For example, the generative AI model might be prompted to "generate fast-paced, relaxing music."

[0284] Step 6: Send and play music data

[0285] explanation

[0286] The generated music data is sent from the server to the user's device, which analyzes the received music data and plays it using the audio player within the application.

[0287] input

[0288] Optimal music data

[0289] output

[0290] Music played on an audio player

[0291] Specific actions

[0292] The generated music data is encoded in JSON format from the server and sent to the user's device, which interprets the data and automatically starts playing the music in the application's audio player.

[0293] Step 7: User feedback and adjustments

[0294] explanation

[0295] The user can provide feedback on the music through the application while running, and the device sends this feedback to the server, which then adjusts the parameters for generating the music data.

[0296] input

[0297] User Feedback

[0298] output

[0299] Adjusted music data

[0300] Specific actions

[0301] When a user sends feedback such as "The tempo is too fast" or "I like this song" using the buttons in the app, that feedback is sent from the device to the server. The server receives the feedback, sends a new prompt to the generative AI model, and regenerates the adjusted music data. The newly generated music data is then sent back to the user's device, and the music playback is updated.

[0302] (Application example 2)

[0303] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0304] In modern society, users want a comfortable experience while running or shopping, while also improving their performance and stabilizing their emotions. However, current systems lack the ability to collect users' physiological and emotional data in real time and provide optimal music based on that data. As a result, users must choose music that best suits their condition, which hinders efficient performance improvement and a comfortable experience.

[0305] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0306] In this invention, the server includes means for collecting physiological data and location data during running, means for transmitting the collected data to a remote server in real time, means for analyzing the collected data at the remote server and evaluating the user's performance status, means for generating appropriate music data based on the evaluation results, means for transmitting the generated music data to a user terminal, means for playing the transmitted music data, means for collecting the user's location data and emotional data during a shopping experience at a physical store, means for analyzing the collected data and evaluating the user's emotional state, means for generating optimal music data using a generative AI model based on the evaluation results, means for playing music data in real time according to the user's emotional state, and means for adjusting the user's emotional state and shopping pace. This allows the user to enjoy an optimal music experience based on their performance data and emotional data while running or shopping, ensuring a comfortable and efficient experience.

[0307] "Physiological data" refers to data that indicates the user's physical condition, and specifically refers to heart rate, respiratory rate, body temperature, sweat rate, etc.

[0308] "Location data" refers to data that indicates the user's current geographical location, and specifically includes GPS information and the like.

[0309] A "remote server" is a server device that receives data sent from a user's terminal and analyzes and processes the data.

[0310] "Evaluation means" refers to the analysis modules and algorithms used to evaluate a user's performance status and emotional state based on the collected data.

[0311] "Music Data" refers to music files or streams generated based on a user's performance status or emotional state.

[0312] "User terminal" refers to electronic devices used by users, such as smartphones, tablets, and wearable devices.

[0313] "Generative AI model" refers to an artificial intelligence model used to generate optimal music based on user data.

[0314] "Feedback" refers to input from users, such as opinions, impressions, and preferences, that the system uses to adjust its music generation.

[0315] "Shopping pace" refers to data that indicates the speed at which users move within a physical store, the length of time they stay there, and the speed at which they progress with their shopping.

[0316] "Emotional state" is data that indicates the user's feelings or psychological state, and specifically includes stress, relaxation, joy, sadness, and the like.

[0317] This invention is a system that collects and analyzes a user's performance and emotional data in real time while running or shopping in a physical store, and generates optimal music based on that data, thereby improving the user's performance and providing a comfortable experience.

[0318] User launches and configures the application

[0319] The user launches a dedicated application installed on a smartphone or wearable device. The application provides options for starting a run or shopping, and by selecting one, the user begins data collection. The user can also input past running data and music preferences. Furthermore, an emotion engine is built in, which collects user emotional data from voice input and facial expressions.

[0320] Data collection and transmission

[0321] As soon as the device starts running or shopping, it collects real-time physiological and location data, such as GPS data, heart rate, speed, and distance. The emotion engine analyzes the user's voice input and facial expressions to recognize emotional data. This data is sent to the server at regular intervals using HTTP POST requests, and the data is encoded in JSON format.

[0322] Server-side data reception and analysis

[0323] The server has an API endpoint that receives data sent from the device. The server then passes the data to an analysis module to evaluate the user's performance and emotional state while running or shopping. For example, the server evaluates the user's emotional state, such as whether they are stressed or relaxed, along with indicators such as heart rate, walking pace, and distance.

[0324] Optimal music data generation

[0325] Based on the analysis results, the server uses a generative AI model to generate optimal music data. The generative AI model takes into account the user's performance and emotional state, as well as their musical preferences and past feedback. For example, if a user is walking fast and feeling stressed while shopping in a physical store, fast-paced but relaxing music will be generated.

[0326] Sending and playing music data

[0327] The generated music data is sent from the server to the user's device. The device analyzes the received music data and plays it in a format suitable for the user. This playback is performed using the audio player within the application, allowing the user to listen to the music that is best suited to them.

[0328] User feedback and regeneration

[0329] Users can provide real-time feedback on music through the application while running or shopping. For example, they can send feedback such as "I like this song," "The tempo is too fast," or "The tempo is too slow" by pressing a button. The device sends this feedback to the server, which then adjusts the parameters for generating the music data and resends the regenerated music data to the user's device.

[0330] Examples of concrete examples and prompts

[0331] For example, suppose a user starts shopping at a physical store and launches the application. If the analysis reveals that the user is walking fast and feeling stressed based on their voice, the server will generate relaxing music. The generated music data will be sent to the user's device and played on an audio player.

[0332] Examples of prompts:

[0333] "Please input audio data of a user running, walking at a fast pace, and feeling stressed. Generate relaxing music based on that."

[0334] As described above, the present invention integrates a user's performance data and emotional data to provide an optimal music experience, thereby improving performance and providing a comfortable experience while running or shopping.

[0335] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0336] Step 1:

[0337] The user launches the application and selects the running or shopping option, which loads the user's past data and music preferences.

[0338] Input: User selected options, historical data and music preferences

[0339] Output: Data collection start trigger, user setting data

[0340] Step 2:

[0341] As soon as you start running or shopping, the device collects real-time physiological and location data, such as GPS data, heart rate, walking pace, and distance, while an emotion engine analyzes voice input and facial expressions to recognize emotional data.

[0342] Input: User's physiological data, location data, voice input and facial expressions

[0343] Output: Collected performance and sentiment data

[0344] Step 3:

[0345] The device collects data at regular intervals, encodes it in JSON format, and sends it to a remote server using an HTTP POST request.

[0346] Input: Collected performance and emotion data

[0347] Output: Data sent to the server

[0348] Step 4:

[0349] The server receives the received data through the API endpoint and passes it to the analysis module for analysis, which evaluates the user's performance status and emotional state.

[0350] Input: Remotely transmitted data

[0351] Output: Evaluation results of the user's performance status and emotional state

[0352] Step 5:

[0353] Based on the evaluation results, the server uses a generative AI model to generate optimal music data, which is customized based on the user's physiological and emotional data.

[0354] Input: Evaluation results of the user's performance status and emotional state

[0355] Output: Generated music data

[0356] Step 6:

[0357] The server transmits the generated music data to the user's terminal.

[0358] Input: Generated music data

[0359] Output: Music data sent to the user's device

[0360] Step 7:

[0361] The device analyzes the received music data and plays it using the audio player within the application.

[0362] Input: Music data sent

[0363] Output: Played music

[0364] Step 8:

[0365] Users can provide real-time feedback through the application, such as "I like this song," "The tempo is too fast," or "The tempo is too slow."

[0366] Input: User feedback

[0367] Output: Feedback sent to the server

[0368] Step 9:

[0369] The server receives feedback from the user and adjusts the parameters for generating the music data based on that feedback. The generative AI model then recreates the music using the new parameters and sends it back to the user's device.

[0370] Input: Received user feedback

[0371] Output: Regenerated music data

[0372] This concludes the processing flow at each step. This allows users to enjoy an optimal music experience based on their own data in real time, providing a comfortable and efficient experience.

[0373] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0374] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0375] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0376] [Second embodiment]

[0377] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0378] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0379] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0380] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0381] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0382] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0383] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0384] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0385] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0386] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0387] In the smart glasses 214, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0388] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0389] The system of the present invention collects user performance data in real time while running and generates optimal music based on that data to help runners improve their performance. This system is implemented as follows.

[0390] User launches and configures the application

[0391] Users launch a dedicated application installed on their smartphone or wearable device. The application provides an option to start running, which users select to begin collecting running data. Users can also input their music preferences and past running data into the app.

[0392] Data collection and transmission

[0393] As soon as you start running, your device collects real-time physiological and location data, such as GPS data, heart rate, speed, and distance. This data is then sent to a server at regular intervals via HTTP POST requests, with the data encoded in JSON format.

[0394] Server-side data reception and analysis

[0395] The server sets up an API endpoint to receive data sent from the device, which is then immediately analyzed to assess the user's current performance. For example, metrics such as heart rate, speed, and distance are used to determine whether the user is running at a sustainable pace or whether fatigue is building up.

[0396] Optimal music data generation

[0397] Based on the analysis results, the server generates music data that is optimal for the user's running situation. This generation uses a generative AI model and takes into account the user's preferences and past feedback data. For example, if the user is running at a fast pace, fast-paced, energetic music will be generated. Conversely, if the user is running an endurance race, music with a steady rhythm will be generated.

[0398] Sending and playing music data

[0399] The generated music data is sent from the server to the user's device, which analyzes the received music data and plays it in a format suitable for the user. Playback is performed using the application's audio player, allowing users to listen to the perfect music while running.

[0400] User feedback and regeneration

[0401] Users can provide real-time feedback on music while running through the application. For example, they can send feedback such as "I like this song," "The tempo is too fast," or "The tempo is too slow" by pressing a button. The device then sends this feedback to the server, which then adjusts the parameters for generating the music data based on the feedback. The adjusted music data is then sent back to the user's device, and playback is updated.

[0402] This invention provides users with optimal music in real time, which helps maintain motivation, reduce fatigue, and improve performance, significantly improving the running experience and enhancing the training effect.

[0403] The processing flow will be explained below.

[0404] Step 1: The user launches the dedicated application installed on their smartphone or wearable device. Once the user logs in and selects the option to start running, data collection mode begins.

[0405] Step 2: The device uses the GPS sensor and heart rate sensor to collect real-time physiological and location data during the run, including current location, speed, heart rate, distance, etc.

[0406] Step 3: The device converts the collected data into JSON format at regular intervals (e.g., every second) and sends it to the server using an HTTP POST request, including the user ID.

[0407] Step 4: The server sets up an API endpoint to receive data sent from the device, which is then immediately passed to the analysis module.

[0408] Step 5: The server's analysis module evaluates the user's current performance based on the received data, and the evaluation result is expressed in the form of tags, such as "sustained pace" or "fast pace."

[0409] Step 6: The server generates optimal music data using a generative AI model based on the evaluation results, taking into account the user's musical preferences and past feedback.

[0410] Step 7: The server sends the generated music data to the user's terminal, where the music data is encoded in an appropriate format (e.g., MP3).

[0411] Step 8: The device analyzes the received music data and plays the music using the audio player in the application. The user can continue running while listening to the optimal music in real time.

[0412] Step 9: Users can provide real-time feedback on the music within the application, in the form of "I like this song" or "The tempo is too fast."

[0413] Step 10: The device sends the user feedback to the server in JSON format.

[0414] Step 11: The server analyzes the received feedback and inputs it as new parameters into the generative AI model. The music data is then regenerated based on this feedback.

[0415] Step 12: The server again sends the newly generated music data to the user's device. The device again receives the music data and plays it on the audio player. This process is repeated, allowing the system to continue providing the optimal music experience the user desires.

[0416] Through these steps, the system improves the user's running experience in real time and helps improve performance.

[0417] Example 1

[0418] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0419] Conventional running support systems have difficulty not only collecting physiological and location data from users but also providing appropriate feedback and support to improve their performance based on that data. In particular, there is a need for systems that can maintain motivation and reduce fatigue while running by evaluating the user's performance in real time and dynamically generating and providing optimal music based on that data.

[0420] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0421] In this invention, the server includes means for collecting physiological data and position data during running, means for transmitting the collected data to a remote server in real time, means for analyzing the collected data at the remote server and evaluating the user's performance status, means for using a generative AI model to generate appropriate music data based on the evaluation results, means for inputting a prompt sentence to the generative AI model to generate music data based on the evaluation results, means for transmitting the generated music data to a user terminal, and means for playing the transmitted music data. This makes it possible to dynamically generate and provide music tailored to each user's individual running status, thereby maintaining the user's motivation and contributing to improving performance.

[0422] "Physiological data" refers to physiological information such as heart rate, blood pressure, and respiratory rate collected from within the user's body in real time.

[0423] "Location Data" means information based on GPS coordinates that indicates a user's current location.

[0424] "Remote Server" refers to a central processing unit that transmits, receives, and analyzes data over a network.

[0425] "Analysis Engine" means a software component that evaluates a user's performance status based on collected data.

[0426] "Generative AI model" refers to an artificial intelligence model that generates optimal music based on the user's performance status and past feedback data.

[0427] A "prompt" is an instructional text that specifies conditions or requests and is input into a generative AI model.

[0428] "User terminal" refers to a portable electronic device used by a user, such as a smartphone or wearable device.

[0429] "Audio player" refers to a software or hardware component for playing music data within a user terminal.

[0430] "Feedback" refers to input information that provides the system with responses and impressions that the user has actually experienced in real time.

[0431] The system of this invention provides users with real-time music that optimizes their running performance. The system utilizes various hardware and software, including smartphones, wearable devices, remote servers, and generative AI models, to provide a dynamic and personalized experience for users.

[0432] User launches and configures the application

[0433] Users launch a dedicated application installed on their smartphone or wearable device. The application has a "Start Running" option, which users can select to begin collecting running data. Users can input their music preferences and past running data into the app in advance.

[0434] Data collection and transmission

[0435] The device collects real-time physiological and location data, such as GPS data, heart rate, speed, and distance, as soon as the user starts running. This data is encoded into JSON format at regular intervals and sent to the server via HTTP POST requests. The server uses the Python Flask framework to set up an API endpoint to receive the data.

[0436] Server-side data reception and analysis

[0437] The server immediately analyzes the received data and assesses the user's current performance status, using metrics such as heart rate, speed, and distance to determine whether the user is running at a sustainable pace or whether fatigue is building up.

[0438] Optimal music data generation

[0439] Based on the analysis results, the server uses a generative AI model to generate music data that is optimal for the user's running situation. This generation also takes into account the user's past feedback data and preferences. For example, if the user is running at a fast pace, fast-paced, energetic music will be generated. The following prompt sentence is input to the generative AI model:

[0440] The user's current heart rate is 160 BPM and their running speed is 5 km / min. The user's past running history has shown that they prefer music with a tempo of 120 BPM. Based on these conditions, generate music that will allow the user to maintain their current pace.

[0441] Sending and playing music data

[0442] The generated music data is sent from the server to the user's device, which analyzes the received music data and plays it using the application's audio player, allowing users to listen to the perfect music while running.

[0443] User feedback and regeneration

[0444] While running, users can provide real-time feedback on the music through the application. For example, they can send feedback such as "I like this song," "The tempo is too fast," or "The tempo is too slow" by pressing a button. The device sends this feedback to the server, which then adjusts the parameters of the generative AI model based on this feedback. The adjusted music data is then sent back to the user's device, and playback is updated.

[0445] These processes allow users to continue running while enjoying optimal music in real time, which is expected to maintain motivation and improve performance.

[0446] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0447] Step 1:

[0448] Starting and Configuring the Application

[0449] The user launches a dedicated application on their smartphone or wearable device. They then tap the "Start Running" button, which prepares the application to begin collecting running data. Input includes the user's music preferences and past running data. Based on this, user profile data is generated.

[0450] Step 2:

[0451] Collecting and sending running data

[0452] The device enters collection mode when the user starts running. It collects physiological and location data such as GPS data, heart rate, speed, and distance in real time from the GPS sensor, heart rate sensor, and accelerometer. The data is converted into JSON format and sent to the server at regular intervals via HTTP POST requests. The input is physiological and location data, and the output is JSON-formatted data.

[0453] Step 3:

[0454] Data reception and storage

[0455] The server uses the Python Flask framework to set up an API endpoint. The server receives data received as a POST request from the terminal and saves the data in temporary storage. The input is the received JSON data, and the output is the temporarily saved data. As an example of actual operation, when a request arrives at the / data endpoint, the data is stored in the database.

[0456] Step 4:

[0457] Data analysis

[0458] The server's analysis engine analyzes the stored data. The server evaluates indicators such as heart rate, speed, and distance to determine the user's performance. The analysis results may include information such as "heart rate is high," "speed is stable," and "distance still half covered." The input is the temporarily stored data, and the output is the analyzed performance evaluation.

[0459] Step 5:

[0460] Music data generation

[0461] Based on the analysis results, the server inputs a prompt into the generative AI model to generate music data that is optimal for the user's running situation. The user's preferences and past feedback data are also taken into consideration. The input is performance evaluation and user profile, and the output is the generated music data. An example of a prompt is, "The user's current heart rate is 160 BPM and speed is 5 minutes / km. The user's past running history has included feedback that they prefer music with a tempo of 120 BPM. Based on these conditions, please generate music that will allow the user to maintain their current pace."

[0462] Step 6:

[0463] Sending and playing music data

[0464] The server sends the generated music data in JSON format as an HTTP response to the device. The device analyzes the received data and plays the music using the audio player in the application. The input is the received music data, and the output is the played music. As a specific example of operation, the device downloads an MP3 file and plays it on the built-in player.

[0465] Step 7:

[0466] Receiving feedback and reanalyzing

[0467] The user sends feedback about the music through the application. For example, by pressing a button, they can indicate whether the tempo is too fast or too slow. The device encodes this feedback data into JSON format and sends it to the server. The server then adjusts the parameters of the generative AI model based on the received feedback and generates new music data. The input is the user feedback, and the output is the regenerated music data.

[0468] This series of processes allows the user to continue running while always enjoying the most suitable music in real time.

[0469] (Application example 1)

[0470] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0471] Maintaining and improving productivity is a challenge in modern factories. In particular, there is a need to promote efficient operation of factory machines and optimize operator motivation and the operating pace of the machines. However, current systems lack the technology to analyze factory machine operation data and environmental data in real time and automatically provide appropriate work music based on that data. This can result in a decline in factory productivity.

[0472] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0473] In this invention, the server includes a means for collecting operational data and environmental data of factory machines, a means for evaluating the productivity of the factory machines based on the collected data, and a means for generating appropriate work music based on the evaluation results. This makes it possible to evaluate the operating efficiency of factory machines in real time and automatically generate and provide music suited to the work environment.

[0474] "Physiological data" refers to data that indicates the physiological state of the body, such as heart rate, speed, and distance.

[0475] "Location data" refers to geographical location information of users and devices obtained by GPS or other means.

[0476] "Factory machinery" refers to automated equipment and robotic systems used in manufacturing operations.

[0477] "Operation data" refers to data that indicates the operating status of factory machines, such as operating time, operating speed, and travel distance.

[0478] "Environmental data" refers to data that indicates the surrounding environmental conditions such as temperature, humidity, and noise level.

[0479] A "server" is a remote computer system that analyzes the collected data and takes appropriate action.

[0480] "Analysis" refers to the process of making an evaluation or diagnosis based on collected data that is suitable for a specific purpose.

[0481] "Productivity" is an indicator of the efficiency and performance of factory machines and systems over a certain period of time.

[0482] "Work music" is music provided to improve work efficiency and motivation.

[0483] "Feedback" refers to evaluations and opinions given by users and systems, which are used by the system to adapt and improve.

[0484] MODE FOR CARRYING OUT THE INVENTION

[0485] The system for realizing this invention first incorporates sensors that collect operational data and environmental data from factory machines. This data is sent to a cloud server using Wi-Fi or wired connections installed in the factory. The data is sent using an HTTP POST request, and the data is encoded in JSON format.

[0486] The server receives the collected data and analyzes it in real time. Data analysis tools such as Python and R are used for the analysis, and machine learning models and AI technologies (e.g., TensorFlow, Scikit-learn) are used to evaluate the operating status and productivity of factory machinery. For example, the current performance and load status of machinery can be determined based on data such as operating time, operating speed, travel distance, temperature, humidity, and noise level.

[0487] Based on the analysis results, the server inputs specific prompt sentences into the generative AI model to generate appropriate work music. The prompt sentences include content that corresponds to the operating status and productivity of the factory machine. The generated music data is then sent from the server to the factory machine's control system. The factory machine's control system analyzes the received music data and plays it through the built-in speaker or speakers of peripheral devices.

[0488] Feedback from factory machines and operators is sent to the server in real time. This feedback includes an evaluation of whether the tempo of the music is appropriate and the impact of the music on productivity. The server uses this feedback to adjust the parameters of the generative AI model and reflect this in future music generation. This makes it possible to automatically provide optimized work music.

[0489] Specific examples

[0490] Hardware used

[0491] Sensors: Sensors built into factory machines to collect operational and environmental data

[0492] Communication equipment: Wi-Fi module or wired connection

[0493] Cloud server: A server that performs data analysis and music generation (e.g., AWS, Google Cloud)

[0494] Software used

[0495] Data analysis tools: Python, R

[0496] Machine learning models: TensorFlow, Scikit-learn

[0497] Music Generation Library: A tentative music generation library

[0498] Prompt Sentence Examples

[0499] "Based on the music feedback provided during the run, generate new music taking into account the following criteria:

[0500] Productivity has decreased in the last 30 minutes

[0501] Factory machines operate at lower than average speeds

[0502] There's a demand for high-energy, fast-paced music."

[0503] In this way, this invention makes it possible to generate and provide optimal work music in real time based on the operation data and environmental data of factory machines, which is expected to improve factory productivity and efficient machine operation.

[0504] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0505] Step 1:

[0506] Factory machines use built-in sensors to collect operational and environmental data, such as operating time, operating speed, travel distance, temperature, humidity, and noise level.

[0507] Step 2:

[0508] The collected data is sent to a cloud server via Wi-Fi or wired connection installed in the factory. The data is sent using an HTTP POST request, and is encoded in JSON format. The JSON format data is output.

[0509] Step 3:

[0510] The server analyzes the received data. Specifically, it uses data analysis tools such as Python and R to apply machine learning models (e.g., TensorFlow, Scikit-learn) to evaluate the operating status and productivity of factory machinery. The evaluation uses data such as operating time, operating speed, temperature, humidity, and noise level. The analysis results are output.

[0511] Step 4:

[0512] The server generates a prompt sentence based on the analysis results. The prompt sentence contains content that corresponds to the operating status and productivity of the factory machines. For example, "Productivity has decreased in the last 30 minutes" or "Energetic, fast-paced music is required." The prompt sentence is the output.

[0513] Step 5:

[0514] The server inputs the generated prompt sentence into a generative AI model to generate appropriate task music. The generative AI model (e.g., OpenAI's GPT-3) generates music data based on the prompt sentence. The generated music data is the output.

[0515] Step 6:

[0516] The generated music data is then sent from the server to the factory machine control system, where it becomes the output.

[0517] Step 7:

[0518] The control system of the factory machine analyzes the received music data and plays it through the built-in speaker or a speaker of a peripheral device. The playback of the music data becomes the output.

[0519] Step 8:

[0520] Feedback from factory machines and operators is sent to the server in real time. The feedback includes an assessment of whether the music tempo is appropriate and the impact of the music on productivity. The feedback data is the output.

[0521] Step 9:

[0522] The server adjusts the parameters of the generative AI model based on the received feedback and reflects this in future music generation. This further optimizes the generated music. The adjusted generative AI model is the output.

[0523] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0524] This invention is a system that collects and analyzes a user's performance and emotional data in real time while they are running, and generates optimal music based on that data, thereby improving a runner's performance and providing a comfortable running experience.

[0525] User launches and configures the application

[0526] The user launches a dedicated application installed on a smartphone or wearable device. The application provides an option to start running, which the user selects to begin data collection. The user can also input their music preferences and past running data into the app. Furthermore, the present invention incorporates an emotion engine that also collects user emotion data.

[0527] Data collection and transmission

[0528] As soon as the device starts running, it collects real-time physiological and location data, such as GPS data, heart rate, speed, and distance. Additionally, the emotion engine analyzes the user's voice input, facial expressions, and other physiological data to recognize emotions. This data is then sent to the server at regular intervals using an HTTP POST request, with the data encoded in JSON format.

[0529] Server-side data reception and analysis

[0530] The server has an API endpoint set up to receive data sent from the device. The received data is immediately passed to the analysis module, which evaluates the user's current performance and emotional state based on the running data and emotional data. For example, along with indicators such as heart rate, speed, and distance, the emotional state of the user, such as whether they are stressed or relaxed, is evaluated.

[0531] Optimal music data generation

[0532] Based on the analysis results, the server uses a generative AI model to generate optimal music data. The generative AI model takes into account the user's performance and emotional state, as well as their music preferences and past feedback. For example, if a user is running at a fast pace but feeling stressed, music with a fast tempo but a relaxing feel will be generated. On the other hand, if a user is running a long distance and feeling relaxed, music with a steady rhythm will be generated.

[0533] Sending and playing music data

[0534] The generated music data is sent from the server to the user's device, which analyzes the received music data and plays it in a format suitable for the user. Playback is performed using the application's audio player, allowing users to listen to the perfect music while running.

[0535] User feedback and regeneration

[0536] Users can provide real-time feedback on music through the application while running. For example, they can send feedback such as "I like this song," "The tempo is too fast," or "The tempo is too slow" by pressing a button. The device sends this feedback to the server, which then adjusts the parameters for generating the music data based on the feedback. The adjusted music data is then sent back to the user's device, and playback is updated.

[0537] Specific examples

[0538] For example, a user starts running, launches the application, and begins collecting data. If the user's heart rate increases while running and the emotion engine recognizes stress from the user's voice, the server generates fast-paced but relaxing music. The music data is immediately sent to the user's device and played on an audio player. The user can continue running while listening to the music and provide feedback.

[0539] In this way, by integrating the user's performance data and emotional data to provide an optimal music experience, the system improves running performance and provides a comfortable running experience.

[0540] The processing flow will be explained below.

[0541] Step 1: The user launches the dedicated application installed on their smartphone or wearable device, logs in to the application, and selects the option to start running.

[0542] Step 2: The device uses the GPS sensor, heart rate sensor, and emotion engine to collect physiological data, location data, and emotion data in real time while running, including current location, speed, heart rate, distance, smile level, voice tone, etc.

[0543] Step 3: The terminal converts the collected data into JSON format and sends it to the server at regular intervals (e.g., every second) using an HTTP POST request.

[0544] Step 4: The server receives the data sent from the device at the API endpoint, and passes the received data to the analysis module.

[0545] Step 5: The server's analysis module evaluates the user's current performance status and emotional state based on the received physiological, location, and emotional data, including categories such as "sustained pace," "fast pace," "relaxed," and "stressed."

[0546] Step 6: The server generates optimal music data based on the evaluation results using a generative AI model that takes into account the user's performance and emotional state, as well as their musical preferences and past feedback.

[0547] Step 7: The server sends the generated music data to the user's device, encoded in an appropriate format (e.g., MP3).

[0548] Step 8: The device analyzes the received music data and plays the music using the audio player in the application. The user can continue running while listening to the optimal music in real time.

[0549] Step 9: The user provides real-time feedback on the music within the application, using options such as "I like this song," "The tempo is too fast," or "The tempo is too slow."

[0550] Step 10: The device sends the user feedback to the server in JSON format.

[0551] Step 11: The server's analysis module analyzes the received feedback and inputs it as new parameters into the generative AI model, so that the next time music is generated, the user's preferences will be reflected.

[0552] Step 12: The server retransmits the newly generated music data to the user's device. The device again analyzes the received music data and plays it on the audio player. This process is repeated, allowing the user to continue running while always listening to the best music.

[0553] In this way, the system integrates the user's physiological data, location data, and emotional data to generate optimal music in real time, thereby improving performance and providing a comfortable running experience.

[0554] Example 2

[0555] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0556] The purpose of this invention is to improve runners' performance and provide a comfortable running experience by providing a means to collect and analyze a user's performance and emotional data in real time while running and generate optimal music based on that data. While conventional systems can collect and analyze physiological data, it is difficult to generate optimal music that takes the user's emotional state into account. Furthermore, there is a need for a more personalized experience by dynamically adjusting the music based on real-time feedback.

[0557] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0558] In this invention, the server includes a means for analyzing data collected by a remote server and evaluating the user's performance status and emotional state, a means including a generative AI model for generating optimal music data based on the evaluation results, and a means for transmitting the generated music data to the user terminal. This makes it possible to provide optimal music in real time according to the user's performance status and emotional state. Furthermore, by reflecting user feedback in real time and dynamically adjusting the music data, a personalized running experience can be achieved.

[0559] "Physiological data" is information related to a user's physical activity and physiological state, such as heart rate, speed, distance, etc.

[0560] "Location data" is information about the user's current location and travel route obtained using GPS.

[0561] A "remote server" is a server that is accessed via the Internet and is a device that receives and analyzes data sent from a user terminal.

[0562] A "generative AI model" is an artificial intelligence technology that generates music data taking into account the user's performance situation, emotional state, musical preferences, etc.

[0563] "Emotional state" refers to the user's emotional state, such as whether the user is stressed or relaxed.

[0564] "Feedback" refers to opinions and evaluations provided by users, and includes information such as responses to music tempo, likes and dislikes, etc.

[0565] An "analysis module" is a program or device that analyzes collected data and evaluates the user's performance status and emotional state.

[0566] An "audio player" is software or hardware for playing digital music data.

[0567] The present invention is a system that uses a smartphone or wearable device to collect real-time performance and emotional data while a user is running, and generates optimal music based on that data. The system analyzes the collected data, generates optimal music, plays the music, and incorporates user feedback.

[0568] First, the user launches a dedicated application installed on a smartphone or wearable device. The user enters their music preferences and past running data within the application, and the emotion engine also collects the user's emotional data. The hardware used in this case is a smartphone (iOS, Android) or a wearable device (Apple Watch, Fitbit, etc.), and the software is the dedicated application.

[0569] Next, as soon as the user starts running, the device collects real-time physiological and location data, such as GPS data, heart rate, speed, and distance. The emotion engine recognizes the user's emotions by analyzing voice input and facial expressions. The collected data is sent to the server at regular intervals. The data is sent using an HTTP POST request and encoded in JSON format.

[0570] The server passes the data received at the API endpoint to the analysis module. The analysis module evaluates the user's current performance status and emotional state based on the running data and emotional data. For example, it evaluates whether the user is feeling stressed or relaxed, along with indicators such as heart rate, speed, and distance. The analysis module used for this purpose is a program that evaluates the user's performance status and emotional state based on the collected data.

[0571] Based on the analysis results, the server uses a generative AI model to generate optimal music data. The generative AI model takes into account the user's performance status, emotional state, music preferences, and past feedback. This allows appropriate music to be generated in real time and sent to the user's device. For example, if a user is running at a fast pace but feeling stressed, music with a fast tempo but a relaxing feel will be generated. On the other hand, if a user is running a long distance and feeling relaxed, music with a steady rhythm will be generated. The generative AI model used for this is a program that generates optimal music data according to the user's performance status and emotional state.

[0572] The generated music data is sent from the server to the user's device, which then analyzes the received music data and plays it on the application's audio player, allowing the user to continue running while listening to the music.

[0573] Furthermore, users can provide real-time feedback on the music through the application while running. For example, they can send feedback such as "I like this song," "The tempo is too fast," or "The tempo is too slow" by pressing a button. The device sends this feedback to the server, which then adjusts the parameters for generating the music data based on the feedback. The adjusted music data is then sent back to the user's device, and the music playback is updated.

[0574] For example, when a user starts running and launches the application to begin collecting data, if their heart rate rises and the emotion engine detects stress in the user's voice, the server generates fast-paced but relaxing music. The music data is immediately sent to the user's device and played on an audio player. The user can continue running while listening to the music and provide feedback.

[0575] Examples of prompts include:

[0576] If the user's heart rate exceeds 150 and the emotion engine detects stress, what musical prompt should be generated?

[0577] Input data:

[0578] Heart rate: 150

[0579] Stress level: High

[0580] Music preference: Relaxing and fast-paced

[0581] Expected output:

[0582] Prompt: Generate fast-paced but relaxing music for a runner running at a fast pace.

[0583] In this way, the present invention integrates a user's performance data and emotional data to provide an optimal music experience, thereby improving performance while running and achieving a comfortable running experience.

[0584] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0585] Program processing flow

[0586] Step 1: User launches and configures the application

[0587] explanation

[0588] The user launches a dedicated application installed on a smartphone or wearable device, selects options to start running, and inputs music preferences and past running data. The emotion engine then begins collecting the user's emotional data.

[0589] input

[0590] User's music preferences

[0591] Past running data

[0592] output

[0593] Initial state after setup is complete

[0594] Activating the Emotion Engine

[0595] Specific actions

[0596] When a user launches the application and presses the "Start Running" button, a settings screen appears. This screen displays options for inputting "favorite music genre" and "average pace of past runs." This starts the emotion engine in the background.

[0597] Step 2: Start collecting data

[0598] explanation

[0599] As soon as the user starts running, the device collects real-time physiological and location data, including GPS data, heart rate, speed, and distance. The emotion engine analyzes voice input and facial expressions to recognize the user's emotions.

[0600] input

[0601] User's real-time physiological data (heart rate, speed, distance)

[0602] User's real-time location data (GPS information)

[0603] User emotion data (voice input, facial expression analysis)

[0604] output

[0605] Physiological and emotional data collection results

[0606] Specific actions

[0607] When a user starts running, their smartphone or wearable device automatically starts collecting sensor data, including heart rate monitors, GPS sensors, and microphones, and the data is stored locally at regular intervals.

[0608] Step 3: Sending data

[0609] explanation

[0610] The device sends the collected data to the server at regular intervals using HTTP POST requests, with the data encoded in JSON format.

[0611] input

[0612] Physiological and emotional data collection results

[0613] output

[0614] Data sent to the server

[0615] Specific actions

[0616] The collected data is automatically sent to the server at regular intervals. For example, every minute, heart rate, GPS data, speed, distance, and physiological data are sent to the server using an HTTP POST request. The data is formatted in JSON format and sent to the server's API endpoint.

[0617] Step 4: Receiving and parsing data on the server side

[0618] explanation

[0619] The server receives the data at the API endpoint and passes it to the analysis module, which evaluates the user's current performance status and emotional state based on the running data and emotional data.

[0620] input

[0621] Data sent to the server

[0622] output

[0623] Evaluation results of user performance and emotional state

[0624] Specific actions

[0625] The server passes the data received at the API endpoint to the analysis module, which analyzes heart rate, speed, distance, and emotional data. The analysis module uses this data to evaluate the user's level of stress or relaxation, and passes the results to the next step.

[0626] Step 5: Generate optimal music data

[0627] explanation

[0628] Based on the analysis results, the server generates optimal music data using a generative AI model that takes into account the user's performance, emotional state, musical preferences, and past feedback.

[0629] input

[0630] Evaluation results of user performance and emotional state

[0631] User's music preferences

[0632] Past Feedback

[0633] output

[0634] Optimal music data

[0635] Specific actions

[0636] Based on the analysis results, the server's generative AI model generates a prompt, which then generates optimal music data based on that prompt. For example, the generative AI model might be prompted to "generate fast-paced, relaxing music."

[0637] Step 6: Send and play music data

[0638] explanation

[0639] The generated music data is sent from the server to the user's device, which analyzes the received music data and plays it using the audio player within the application.

[0640] input

[0641] Optimal music data

[0642] output

[0643] Music played on an audio player

[0644] Specific actions

[0645] The generated music data is encoded in JSON format from the server and sent to the user's device, which interprets the data and automatically starts playing the music in the application's audio player.

[0646] Step 7: User feedback and adjustments

[0647] explanation

[0648] The user can provide feedback on the music through the application while running, and the device sends this feedback to the server, which then adjusts the parameters for generating the music data.

[0649] input

[0650] User Feedback

[0651] output

[0652] Adjusted music data

[0653] Specific actions

[0654] When a user sends feedback such as "The tempo is too fast" or "I like this song" using the buttons in the app, that feedback is sent from the device to the server. The server receives the feedback, sends a new prompt to the generative AI model, and regenerates the adjusted music data. The newly generated music data is then sent back to the user's device, and the music playback is updated.

[0655] (Application example 2)

[0656] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0657] In modern society, users want a comfortable experience while running or shopping, while also improving their performance and stabilizing their emotions. However, current systems lack the ability to collect users' physiological and emotional data in real time and provide optimal music based on that data. As a result, users must choose music that best suits their condition, which hinders efficient performance improvement and a comfortable experience.

[0658] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0659] In this invention, the server includes means for collecting physiological data and location data during running, means for transmitting the collected data to a remote server in real time, means for analyzing the collected data at the remote server and evaluating the user's performance status, means for generating appropriate music data based on the evaluation results, means for transmitting the generated music data to a user terminal, means for playing the transmitted music data, means for collecting the user's location data and emotional data during a shopping experience at a physical store, means for analyzing the collected data and evaluating the user's emotional state, means for generating optimal music data using a generative AI model based on the evaluation results, means for playing music data in real time according to the user's emotional state, and means for adjusting the user's emotional state and shopping pace. This allows the user to enjoy an optimal music experience based on their performance data and emotional data while running or shopping, ensuring a comfortable and efficient experience.

[0660] "Physiological data" refers to data that indicates the user's physical condition, and specifically refers to heart rate, respiratory rate, body temperature, sweat rate, etc.

[0661] "Location data" refers to data that indicates the user's current geographical location, and specifically includes GPS information and the like.

[0662] A "remote server" is a server device that receives data sent from a user's terminal and analyzes and processes the data.

[0663] "Evaluation means" refers to the analysis modules and algorithms used to evaluate a user's performance status and emotional state based on the collected data.

[0664] "Music Data" refers to music files or streams generated based on a user's performance status or emotional state.

[0665] "User terminal" refers to electronic devices used by users, such as smartphones, tablets, and wearable devices.

[0666] "Generative AI model" refers to an artificial intelligence model used to generate optimal music based on user data.

[0667] "Feedback" refers to input from users, such as opinions, impressions, and preferences, that the system uses to adjust its music generation.

[0668] "Shopping pace" refers to data that indicates the speed at which users move within a physical store, the length of time they stay there, and the speed at which they progress with their shopping.

[0669] "Emotional state" is data that indicates the user's feelings or psychological state, and specifically includes stress, relaxation, joy, sadness, and the like.

[0670] This invention is a system that collects and analyzes a user's performance and emotional data in real time while running or shopping in a physical store, and generates optimal music based on that data, thereby improving the user's performance and providing a comfortable experience.

[0671] User launches and configures the application

[0672] The user launches a dedicated application installed on a smartphone or wearable device. The application provides options for starting a run or shopping, and by selecting one, the user begins data collection. The user can also input past running data and music preferences. Furthermore, an emotion engine is built in, which collects user emotional data from voice input and facial expressions.

[0673] Data collection and transmission

[0674] As soon as the device starts running or shopping, it collects real-time physiological and location data, such as GPS data, heart rate, speed, and distance. The emotion engine analyzes the user's voice input and facial expressions to recognize emotional data. This data is sent to the server at regular intervals using HTTP POST requests, and the data is encoded in JSON format.

[0675] Server-side data reception and analysis

[0676] The server has an API endpoint that receives data sent from the device. The server then passes the data to an analysis module to evaluate the user's performance and emotional state while running or shopping. For example, the server evaluates the user's emotional state, such as whether they are stressed or relaxed, along with indicators such as heart rate, walking pace, and distance.

[0677] Optimal music data generation

[0678] Based on the analysis results, the server uses a generative AI model to generate optimal music data. The generative AI model takes into account the user's performance and emotional state, as well as their musical preferences and past feedback. For example, if a user is walking fast and feeling stressed while shopping in a physical store, fast-paced but relaxing music will be generated.

[0679] Sending and playing music data

[0680] The generated music data is sent from the server to the user's device. The device analyzes the received music data and plays it in a format suitable for the user. This playback is performed using the audio player within the application, allowing the user to listen to the music that is best suited to them.

[0681] User feedback and regeneration

[0682] Users can provide real-time feedback on music through the application while running or shopping. For example, they can send feedback such as "I like this song," "The tempo is too fast," or "The tempo is too slow" by pressing a button. The device sends this feedback to the server, which then adjusts the parameters for generating the music data and resends the regenerated music data to the user's device.

[0683] Examples of concrete examples and prompts

[0684] For example, suppose a user starts shopping at a physical store and launches the application. If the analysis reveals that the user is walking fast and feeling stressed based on their voice, the server will generate relaxing music. The generated music data will be sent to the user's device and played on an audio player.

[0685] Examples of prompts:

[0686] "Please input audio data of a user running, walking at a fast pace, and feeling stressed. Generate relaxing music based on that."

[0687] As described above, the present invention integrates a user's performance data and emotional data to provide an optimal music experience, thereby improving performance and providing a comfortable experience while running or shopping.

[0688] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0689] Step 1:

[0690] The user launches the application and selects the running or shopping option, which loads the user's past data and music preferences.

[0691] Input: User selected options, historical data and music preferences

[0692] Output: Data collection start trigger, user setting data

[0693] Step 2:

[0694] As soon as you start running or shopping, the device collects real-time physiological and location data, such as GPS data, heart rate, walking pace, and distance, while an emotion engine analyzes voice input and facial expressions to recognize emotional data.

[0695] Input: User's physiological data, location data, voice input and facial expressions

[0696] Output: Collected performance and sentiment data

[0697] Step 3:

[0698] The device collects data at regular intervals, encodes it in JSON format, and sends it to a remote server using an HTTP POST request.

[0699] Input: Collected performance and emotion data

[0700] Output: Data sent to the server

[0701] Step 4:

[0702] The server receives the received data through the API endpoint and passes it to the analysis module for analysis, which evaluates the user's performance status and emotional state.

[0703] Input: Remotely transmitted data

[0704] Output: Evaluation results of the user's performance status and emotional state

[0705] Step 5:

[0706] Based on the evaluation results, the server uses a generative AI model to generate optimal music data, which is customized based on the user's physiological and emotional data.

[0707] Input: Evaluation results of the user's performance status and emotional state

[0708] Output: Generated music data

[0709] Step 6:

[0710] The server transmits the generated music data to the user's terminal.

[0711] Input: Generated music data

[0712] Output: Music data sent to the user's device

[0713] Step 7:

[0714] The device analyzes the received music data and plays it using the audio player within the application.

[0715] Input: Music data sent

[0716] Output: Played music

[0717] Step 8:

[0718] Users can provide real-time feedback through the application, such as "I like this song," "The tempo is too fast," or "The tempo is too slow."

[0719] Input: User feedback

[0720] Output: Feedback sent to the server

[0721] Step 9:

[0722] The server receives feedback from the user and adjusts the parameters for generating the music data based on that feedback. The generative AI model then recreates the music using the new parameters and sends it back to the user's device.

[0723] Input: Received user feedback

[0724] Output: Regenerated music data

[0725] This concludes the processing flow at each step. This allows users to enjoy an optimal music experience based on their own data in real time, providing a comfortable and efficient experience.

[0726] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0727] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0728] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0729] [Third embodiment]

[0730] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0731] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0732] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0733] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0734] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0735] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0736] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0737] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0738] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0739] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0740] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0741] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0742] The system of the present invention collects user performance data in real time while running and generates optimal music based on that data to help runners improve their performance. This system is implemented as follows.

[0743] User launches and configures the application

[0744] Users launch a dedicated application installed on their smartphone or wearable device. The application provides an option to start running, which users select to begin collecting running data. Users can also input their music preferences and past running data into the app.

[0745] Data collection and transmission

[0746] As soon as you start running, your device collects real-time physiological and location data, such as GPS data, heart rate, speed, and distance. This data is then sent to a server at regular intervals via HTTP POST requests, with the data encoded in JSON format.

[0747] Server-side data reception and analysis

[0748] The server sets up an API endpoint to receive data sent from the device, which is then immediately analyzed to assess the user's current performance. For example, metrics such as heart rate, speed, and distance are used to determine whether the user is running at a sustainable pace or whether fatigue is building up.

[0749] Optimal music data generation

[0750] Based on the analysis results, the server generates music data that is optimal for the user's running situation. This generation uses a generative AI model and takes into account the user's preferences and past feedback data. For example, if the user is running at a fast pace, fast-paced, energetic music will be generated. Conversely, if the user is running an endurance race, music with a steady rhythm will be generated.

[0751] Sending and playing music data

[0752] The generated music data is sent from the server to the user's device, which analyzes the received music data and plays it in a format suitable for the user. Playback is performed using the application's audio player, allowing users to listen to the perfect music while running.

[0753] User feedback and regeneration

[0754] Users can provide real-time feedback on music while running through the application. For example, they can send feedback such as "I like this song," "The tempo is too fast," or "The tempo is too slow" by pressing a button. The device then sends this feedback to the server, which then adjusts the parameters for generating the music data based on the feedback. The adjusted music data is then sent back to the user's device, and playback is updated.

[0755] This invention provides users with optimal music in real time, which helps maintain motivation, reduce fatigue, and improve performance, significantly improving the running experience and enhancing the training effect.

[0756] The processing flow will be explained below.

[0757] Step 1: The user launches the dedicated application installed on their smartphone or wearable device. Once the user logs in and selects the option to start running, data collection mode begins.

[0758] Step 2: The device uses the GPS sensor and heart rate sensor to collect real-time physiological and location data during the run, including current location, speed, heart rate, distance, etc.

[0759] Step 3: The device converts the collected data into JSON format at regular intervals (e.g., every second) and sends it to the server using an HTTP POST request, including the user ID.

[0760] Step 4: The server sets up an API endpoint to receive data sent from the device, which is then immediately passed to the analysis module.

[0761] Step 5: The server's analysis module evaluates the user's current performance based on the received data, and the evaluation result is expressed in the form of tags, such as "sustained pace" or "fast pace."

[0762] Step 6: The server generates optimal music data using a generative AI model based on the evaluation results, taking into account the user's musical preferences and past feedback.

[0763] Step 7: The server sends the generated music data to the user's terminal, where the music data is encoded in an appropriate format (e.g., MP3).

[0764] Step 8: The device analyzes the received music data and plays the music using the audio player in the application. The user can continue running while listening to the optimal music in real time.

[0765] Step 9: Users can provide real-time feedback on the music within the application, in the form of "I like this song" or "The tempo is too fast."

[0766] Step 10: The device sends the user feedback to the server in JSON format.

[0767] Step 11: The server analyzes the received feedback and inputs it as new parameters into the generative AI model. The music data is then regenerated based on this feedback.

[0768] Step 12: The server again sends the newly generated music data to the user's device. The device again receives the music data and plays it on the audio player. This process is repeated, allowing the system to continue providing the optimal music experience the user desires.

[0769] Through these steps, the system improves the user's running experience in real time and helps improve performance.

[0770] Example 1

[0771] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0772] Conventional running support systems have difficulty not only collecting physiological and location data from users but also providing appropriate feedback and support to improve their performance based on that data. In particular, there is a need for systems that can maintain motivation and reduce fatigue while running by evaluating the user's performance in real time and dynamically generating and providing optimal music based on that data.

[0773] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0774] In this invention, the server includes means for collecting physiological data and position data during running, means for transmitting the collected data to a remote server in real time, means for analyzing the collected data at the remote server and evaluating the user's performance status, means for using a generative AI model to generate appropriate music data based on the evaluation results, means for inputting a prompt sentence to the generative AI model to generate music data based on the evaluation results, means for transmitting the generated music data to a user terminal, and means for playing the transmitted music data. This makes it possible to dynamically generate and provide music tailored to each user's individual running status, thereby maintaining the user's motivation and contributing to improving performance.

[0775] "Physiological data" refers to physiological information such as heart rate, blood pressure, and respiratory rate collected from within the user's body in real time.

[0776] "Location Data" means information based on GPS coordinates that indicates a user's current location.

[0777] "Remote Server" refers to a central processing unit that transmits, receives, and analyzes data over a network.

[0778] "Analysis Engine" means a software component that evaluates a user's performance status based on collected data.

[0779] "Generative AI model" refers to an artificial intelligence model that generates optimal music based on the user's performance status and past feedback data.

[0780] A "prompt" is an instructional text that specifies conditions or requests and is input into a generative AI model.

[0781] "User terminal" refers to a portable electronic device used by a user, such as a smartphone or wearable device.

[0782] "Audio player" refers to a software or hardware component for playing music data within a user terminal.

[0783] "Feedback" refers to input information that provides the system with responses and impressions that the user has actually experienced in real time.

[0784] The system of this invention provides users with real-time music that optimizes their running performance. The system utilizes various hardware and software, including smartphones, wearable devices, remote servers, and generative AI models, to provide a dynamic and personalized experience for users.

[0785] User launches and configures the application

[0786] Users launch a dedicated application installed on their smartphone or wearable device. The application has a "Start Running" option, which users can select to begin collecting running data. Users can input their music preferences and past running data into the app in advance.

[0787] Data collection and transmission

[0788] The device collects real-time physiological and location data, such as GPS data, heart rate, speed, and distance, as soon as the user starts running. This data is encoded into JSON format at regular intervals and sent to the server via HTTP POST requests. The server uses the Python Flask framework to set up an API endpoint to receive the data.

[0789] Server-side data reception and analysis

[0790] The server immediately analyzes the received data and assesses the user's current performance status, using metrics such as heart rate, speed, and distance to determine whether the user is running at a sustainable pace or whether fatigue is building up.

[0791] Optimal music data generation

[0792] Based on the analysis results, the server uses a generative AI model to generate music data that is optimal for the user's running situation. This generation also takes into account the user's past feedback data and preferences. For example, if the user is running at a fast pace, fast-paced, energetic music will be generated. The following prompt sentence is input to the generative AI model:

[0793] The user's current heart rate is 160 BPM and their running speed is 5 km / min. The user's past running history has shown that they prefer music with a tempo of 120 BPM. Based on these conditions, generate music that will allow the user to maintain their current pace.

[0794] Sending and playing music data

[0795] The generated music data is sent from the server to the user's device, which analyzes the received music data and plays it using the application's audio player, allowing users to listen to the perfect music while running.

[0796] User feedback and regeneration

[0797] While running, users can provide real-time feedback on the music through the application. For example, they can send feedback such as "I like this song," "The tempo is too fast," or "The tempo is too slow" by pressing a button. The device sends this feedback to the server, which then adjusts the parameters of the generative AI model based on this feedback. The adjusted music data is then sent back to the user's device, and playback is updated.

[0798] These processes allow users to continue running while enjoying optimal music in real time, which is expected to maintain motivation and improve performance.

[0799] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0800] Step 1:

[0801] Starting and Configuring the Application

[0802] The user launches a dedicated application on their smartphone or wearable device. They then tap the "Start Running" button, which prepares the application to begin collecting running data. Input includes the user's music preferences and past running data. Based on this, user profile data is generated.

[0803] Step 2:

[0804] Collecting and sending running data

[0805] The device enters collection mode when the user starts running. It collects physiological and location data such as GPS data, heart rate, speed, and distance in real time from the GPS sensor, heart rate sensor, and accelerometer. The data is converted into JSON format and sent to the server at regular intervals via HTTP POST requests. The input is physiological and location data, and the output is JSON-formatted data.

[0806] Step 3:

[0807] Data reception and storage

[0808] The server uses the Python Flask framework to set up an API endpoint. The server receives data received as a POST request from the terminal and saves the data in temporary storage. The input is the received JSON data, and the output is the temporarily saved data. As an example of actual operation, when a request arrives at the / data endpoint, the data is stored in the database.

[0809] Step 4:

[0810] Data analysis

[0811] The server's analysis engine analyzes the stored data. The server evaluates indicators such as heart rate, speed, and distance to determine the user's performance. The analysis results may include information such as "heart rate is high," "speed is stable," and "distance still half covered." The input is the temporarily stored data, and the output is the analyzed performance evaluation.

[0812] Step 5:

[0813] Music data generation

[0814] Based on the analysis results, the server inputs a prompt into the generative AI model to generate music data that is optimal for the user's running situation. The user's preferences and past feedback data are also taken into consideration. The input is performance evaluation and user profile, and the output is the generated music data. An example of a prompt is, "The user's current heart rate is 160 BPM and speed is 5 minutes / km. The user's past running history has included feedback that they prefer music with a tempo of 120 BPM. Based on these conditions, please generate music that will allow the user to maintain their current pace."

[0815] Step 6:

[0816] Sending and playing music data

[0817] The server sends the generated music data in JSON format as an HTTP response to the device. The device analyzes the received data and plays the music using the audio player in the application. The input is the received music data, and the output is the played music. As a specific example of operation, the device downloads an MP3 file and plays it on the built-in player.

[0818] Step 7:

[0819] Receiving feedback and reanalyzing

[0820] The user sends feedback about the music through the application. For example, by pressing a button, they can indicate whether the tempo is too fast or too slow. The device encodes this feedback data into JSON format and sends it to the server. The server then adjusts the parameters of the generative AI model based on the received feedback and generates new music data. The input is the user feedback, and the output is the regenerated music data.

[0821] This series of processes allows the user to continue running while always enjoying the most suitable music in real time.

[0822] (Application example 1)

[0823] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0824] Maintaining and improving productivity is a challenge in modern factories. In particular, there is a need to promote efficient operation of factory machines and optimize operator motivation and the operating pace of the machines. However, current systems lack the technology to analyze factory machine operation data and environmental data in real time and automatically provide appropriate work music based on that data. This can result in a decline in factory productivity.

[0825] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0826] In this invention, the server includes a means for collecting operational data and environmental data of factory machines, a means for evaluating the productivity of the factory machines based on the collected data, and a means for generating appropriate work music based on the evaluation results. This makes it possible to evaluate the operating efficiency of factory machines in real time and automatically generate and provide music suited to the work environment.

[0827] "Physiological data" refers to data that indicates the physiological state of the body, such as heart rate, speed, and distance.

[0828] "Location data" refers to geographical location information of users and devices obtained by GPS or other means.

[0829] "Factory machinery" refers to automated equipment and robotic systems used in manufacturing operations.

[0830] "Operation data" refers to data that indicates the operating status of factory machines, such as operating time, operating speed, and travel distance.

[0831] "Environmental data" refers to data that indicates the surrounding environmental conditions such as temperature, humidity, and noise level.

[0832] A "server" is a remote computer system that analyzes the collected data and takes appropriate action.

[0833] "Analysis" refers to the process of making an evaluation or diagnosis based on collected data that is suitable for a specific purpose.

[0834] "Productivity" is an indicator of the efficiency and performance of factory machines and systems over a certain period of time.

[0835] "Work music" is music provided to improve work efficiency and motivation.

[0836] "Feedback" refers to evaluations and opinions given by users and systems, which are used by the system to adapt and improve.

[0837] MODE FOR CARRYING OUT THE INVENTION

[0838] The system for realizing this invention first incorporates sensors that collect operational data and environmental data from factory machines. This data is sent to a cloud server using Wi-Fi or wired connections installed in the factory. The data is sent using an HTTP POST request, and the data is encoded in JSON format.

[0839] The server receives the collected data and analyzes it in real time. Data analysis tools such as Python and R are used for the analysis, and machine learning models and AI technologies (e.g., TensorFlow, Scikit-learn) are used to evaluate the operating status and productivity of factory machinery. For example, the current performance and load status of machinery can be determined based on data such as operating time, operating speed, travel distance, temperature, humidity, and noise level.

[0840] Based on the analysis results, the server inputs specific prompt sentences into the generative AI model to generate appropriate work music. The prompt sentences include content that corresponds to the operating status and productivity of the factory machine. The generated music data is then sent from the server to the factory machine's control system. The factory machine's control system analyzes the received music data and plays it through the built-in speaker or speakers of peripheral devices.

[0841] Feedback from factory machines and operators is sent to the server in real time. This feedback includes an evaluation of whether the tempo of the music is appropriate and the impact of the music on productivity. The server uses this feedback to adjust the parameters of the generative AI model and reflect this in future music generation. This makes it possible to automatically provide optimized work music.

[0842] Specific examples

[0843] Hardware used

[0844] Sensors: Sensors built into factory machines to collect operational and environmental data

[0845] Communication equipment: Wi-Fi module or wired connection

[0846] Cloud server: A server that performs data analysis and music generation (e.g., AWS, Google Cloud)

[0847] Software used

[0848] Data analysis tools: Python, R

[0849] Machine learning models: TensorFlow, Scikit-learn

[0850] Music Generation Library: A tentative music generation library

[0851] Prompt Sentence Examples

[0852] "Based on the music feedback provided during the run, generate new music taking into account the following criteria:

[0853] Productivity has decreased in the last 30 minutes

[0854] Factory machines operate at lower than average speeds

[0855] There's a demand for high-energy, fast-paced music."

[0856] In this way, this invention makes it possible to generate and provide optimal work music in real time based on the operation data and environmental data of factory machines, which is expected to improve factory productivity and efficient machine operation.

[0857] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0858] Step 1:

[0859] Factory machines use built-in sensors to collect operational and environmental data, such as operating time, operating speed, travel distance, temperature, humidity, and noise level.

[0860] Step 2:

[0861] The collected data is sent to a cloud server via Wi-Fi or wired connection installed in the factory. The data is sent using an HTTP POST request, and is encoded in JSON format. The JSON format data is output.

[0862] Step 3:

[0863] The server analyzes the received data. Specifically, it uses data analysis tools such as Python and R to apply machine learning models (e.g., TensorFlow, Scikit-learn) to evaluate the operating status and productivity of factory machinery. The evaluation uses data such as operating time, operating speed, temperature, humidity, and noise level. The analysis results are output.

[0864] Step 4:

[0865] The server generates a prompt sentence based on the analysis results. The prompt sentence contains content that corresponds to the operating status and productivity of the factory machines. For example, "Productivity has decreased in the last 30 minutes" or "Energetic, fast-paced music is required." The prompt sentence is the output.

[0866] Step 5:

[0867] The server inputs the generated prompt sentence into a generative AI model to generate appropriate task music. The generative AI model (e.g., OpenAI's GPT-3) generates music data based on the prompt sentence. The generated music data is the output.

[0868] Step 6:

[0869] The generated music data is then sent from the server to the factory machine control system, where it becomes the output.

[0870] Step 7:

[0871] The control system of the factory machine analyzes the received music data and plays it through the built-in speaker or a speaker of a peripheral device. The playback of the music data becomes the output.

[0872] Step 8:

[0873] Feedback from factory machines and operators is sent to the server in real time. The feedback includes an assessment of whether the music tempo is appropriate and the impact of the music on productivity. The feedback data is the output.

[0874] Step 9:

[0875] The server adjusts the parameters of the generative AI model based on the received feedback and reflects this in future music generation. This further optimizes the generated music. The adjusted generative AI model is the output.

[0876] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0877] This invention is a system that collects and analyzes a user's performance and emotional data in real time while they are running, and generates optimal music based on that data, thereby improving a runner's performance and providing a comfortable running experience.

[0878] User launches and configures the application

[0879] The user launches a dedicated application installed on a smartphone or wearable device. The application provides an option to start running, which the user selects to begin data collection. The user can also input their music preferences and past running data into the app. Furthermore, the present invention incorporates an emotion engine that also collects user emotion data.

[0880] Data collection and transmission

[0881] As soon as the device starts running, it collects real-time physiological and location data, such as GPS data, heart rate, speed, and distance. Additionally, the emotion engine analyzes the user's voice input, facial expressions, and other physiological data to recognize emotions. This data is then sent to the server at regular intervals using an HTTP POST request, with the data encoded in JSON format.

[0882] Server-side data reception and analysis

[0883] The server has an API endpoint set up to receive data sent from the device. The received data is immediately passed to the analysis module, which evaluates the user's current performance and emotional state based on the running data and emotional data. For example, along with indicators such as heart rate, speed, and distance, the emotional state of the user, such as whether they are stressed or relaxed, is evaluated.

[0884] Optimal music data generation

[0885] Based on the analysis results, the server uses a generative AI model to generate optimal music data. The generative AI model takes into account the user's performance and emotional state, as well as their music preferences and past feedback. For example, if a user is running at a fast pace but feeling stressed, music with a fast tempo but a relaxing feel will be generated. On the other hand, if a user is running a long distance and feeling relaxed, music with a steady rhythm will be generated.

[0886] Sending and playing music data

[0887] The generated music data is sent from the server to the user's device, which analyzes the received music data and plays it in a format suitable for the user. Playback is performed using the application's audio player, allowing users to listen to the perfect music while running.

[0888] User feedback and regeneration

[0889] Users can provide real-time feedback on music through the application while running. For example, they can send feedback such as "I like this song," "The tempo is too fast," or "The tempo is too slow" by pressing a button. The device sends this feedback to the server, which then adjusts the parameters for generating the music data based on the feedback. The adjusted music data is then sent back to the user's device, and playback is updated.

[0890] Specific examples

[0891] For example, a user starts running, launches the application, and begins collecting data. If the user's heart rate increases while running and the emotion engine recognizes stress from the user's voice, the server generates fast-paced but relaxing music. The music data is immediately sent to the user's device and played on an audio player. The user can continue running while listening to the music and provide feedback.

[0892] In this way, by integrating the user's performance data and emotional data to provide an optimal music experience, the system improves running performance and provides a comfortable running experience.

[0893] The processing flow will be explained below.

[0894] Step 1: The user launches the dedicated application installed on their smartphone or wearable device, logs in to the application, and selects the option to start running.

[0895] Step 2: The device uses the GPS sensor, heart rate sensor, and emotion engine to collect physiological data, location data, and emotion data in real time while running, including current location, speed, heart rate, distance, smile level, voice tone, etc.

[0896] Step 3: The terminal converts the collected data into JSON format and sends it to the server at regular intervals (e.g., every second) using an HTTP POST request.

[0897] Step 4: The server receives the data sent from the device at the API endpoint, and passes the received data to the analysis module.

[0898] Step 5: The server's analysis module evaluates the user's current performance status and emotional state based on the received physiological, location, and emotional data, including categories such as "sustained pace," "fast pace," "relaxed," and "stressed."

[0899] Step 6: The server generates optimal music data based on the evaluation results using a generative AI model that takes into account the user's performance and emotional state, as well as their musical preferences and past feedback.

[0900] Step 7: The server sends the generated music data to the user's device, encoded in an appropriate format (e.g., MP3).

[0901] Step 8: The device analyzes the received music data and plays the music using the audio player in the application. The user can continue running while listening to the optimal music in real time.

[0902] Step 9: The user provides real-time feedback on the music within the application, using options such as "I like this song," "The tempo is too fast," or "The tempo is too slow."

[0903] Step 10: The device sends the user feedback to the server in JSON format.

[0904] Step 11: The server's analysis module analyzes the received feedback and inputs it as new parameters into the generative AI model, so that the next time music is generated, the user's preferences will be reflected.

[0905] Step 12: The server retransmits the newly generated music data to the user's device. The device again analyzes the received music data and plays it on the audio player. This process is repeated, allowing the user to continue running while always listening to the best music.

[0906] In this way, the system integrates the user's physiological data, location data, and emotional data to generate optimal music in real time, thereby improving performance and providing a comfortable running experience.

[0907] Example 2

[0908] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0909] The purpose of this invention is to improve runners' performance and provide a comfortable running experience by providing a means to collect and analyze a user's performance and emotional data in real time while running and generate optimal music based on that data. While conventional systems can collect and analyze physiological data, it is difficult to generate optimal music that takes the user's emotional state into account. Furthermore, there is a need for a more personalized experience by dynamically adjusting the music based on real-time feedback.

[0910] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0911] In this invention, the server includes a means for analyzing data collected by a remote server and evaluating the user's performance status and emotional state, a means including a generative AI model for generating optimal music data based on the evaluation results, and a means for transmitting the generated music data to the user terminal. This makes it possible to provide optimal music in real time according to the user's performance status and emotional state. Furthermore, by reflecting user feedback in real time and dynamically adjusting the music data, a personalized running experience can be achieved.

[0912] "Physiological data" is information related to a user's physical activity and physiological state, such as heart rate, speed, distance, etc.

[0913] "Location data" is information about the user's current location and travel route obtained using GPS.

[0914] A "remote server" is a server that is accessed via the Internet and is a device that receives and analyzes data sent from a user terminal.

[0915] A "generative AI model" is an artificial intelligence technology that generates music data taking into account the user's performance situation, emotional state, musical preferences, etc.

[0916] "Emotional state" refers to the user's emotional state, such as whether the user is stressed or relaxed.

[0917] "Feedback" refers to opinions and evaluations provided by users, and includes information such as responses to music tempo, likes and dislikes, etc.

[0918] An "analysis module" is a program or device that analyzes collected data and evaluates the user's performance status and emotional state.

[0919] An "audio player" is software or hardware for playing digital music data.

[0920] The present invention is a system that uses a smartphone or wearable device to collect real-time performance and emotional data while a user is running, and generates optimal music based on that data. The system analyzes the collected data, generates optimal music, plays the music, and incorporates user feedback.

[0921] First, the user launches a dedicated application installed on a smartphone or wearable device. The user enters their music preferences and past running data within the application, and the emotion engine also collects the user's emotional data. The hardware used in this case is a smartphone (iOS, Android) or a wearable device (Apple Watch, Fitbit, etc.), and the software is the dedicated application.

[0922] Next, as soon as the user starts running, the device collects real-time physiological and location data, such as GPS data, heart rate, speed, and distance. The emotion engine recognizes the user's emotions by analyzing voice input and facial expressions. The collected data is sent to the server at regular intervals. The data is sent using an HTTP POST request and encoded in JSON format.

[0923] The server passes the data received at the API endpoint to the analysis module. The analysis module evaluates the user's current performance status and emotional state based on the running data and emotional data. For example, it evaluates whether the user is feeling stressed or relaxed, along with indicators such as heart rate, speed, and distance. The analysis module used for this purpose is a program that evaluates the user's performance status and emotional state based on the collected data.

[0924] Based on the analysis results, the server uses a generative AI model to generate optimal music data. The generative AI model takes into account the user's performance status, emotional state, music preferences, and past feedback. This allows appropriate music to be generated in real time and sent to the user's device. For example, if a user is running at a fast pace but feeling stressed, music with a fast tempo but a relaxing feel will be generated. On the other hand, if a user is running a long distance and feeling relaxed, music with a steady rhythm will be generated. The generative AI model used for this is a program that generates optimal music data according to the user's performance status and emotional state.

[0925] The generated music data is sent from the server to the user's device, which then analyzes the received music data and plays it on the application's audio player, allowing the user to continue running while listening to the music.

[0926] Furthermore, users can provide real-time feedback on the music through the application while running. For example, they can send feedback such as "I like this song," "The tempo is too fast," or "The tempo is too slow" by pressing a button. The device sends this feedback to the server, which then adjusts the parameters for generating the music data based on the feedback. The adjusted music data is then sent back to the user's device, and the music playback is updated.

[0927] For example, when a user starts running and launches the application to begin collecting data, if their heart rate rises and the emotion engine detects stress in the user's voice, the server generates fast-paced but relaxing music. The music data is immediately sent to the user's device and played on an audio player. The user can continue running while listening to the music and provide feedback.

[0928] Examples of prompts include:

[0929] If the user's heart rate exceeds 150 and the emotion engine detects stress, what musical prompt should be generated?

[0930] Input data:

[0931] Heart rate: 150

[0932] Stress level: High

[0933] Music preference: Relaxing and fast-paced

[0934] Expected output:

[0935] Prompt: Generate fast-paced but relaxing music for a runner running at a fast pace.

[0936] In this way, the present invention integrates a user's performance data and emotional data to provide an optimal music experience, thereby improving performance while running and achieving a comfortable running experience.

[0937] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0938] Program processing flow

[0939] Step 1: User launches and configures the application

[0940] explanation

[0941] The user launches a dedicated application installed on a smartphone or wearable device, selects options to start running, and inputs music preferences and past running data. The emotion engine then begins collecting the user's emotional data.

[0942] input

[0943] User's music preferences

[0944] Past running data

[0945] output

[0946] Initial state after setup is complete

[0947] Activating the Emotion Engine

[0948] Specific actions

[0949] When a user launches the application and presses the "Start Running" button, a settings screen appears. This screen displays options for inputting "favorite music genre" and "average pace of past runs." This starts the emotion engine in the background.

[0950] Step 2: Start collecting data

[0951] explanation

[0952] As soon as the user starts running, the device collects real-time physiological and location data, including GPS data, heart rate, speed, and distance. The emotion engine analyzes voice input and facial expressions to recognize the user's emotions.

[0953] input

[0954] User's real-time physiological data (heart rate, speed, distance)

[0955] User's real-time location data (GPS information)

[0956] User emotion data (voice input, facial expression analysis)

[0957] output

[0958] Physiological and emotional data collection results

[0959] Specific actions

[0960] When a user starts running, their smartphone or wearable device automatically starts collecting sensor data, including heart rate monitors, GPS sensors, and microphones, and the data is stored locally at regular intervals.

[0961] Step 3: Sending data

[0962] explanation

[0963] The device sends the collected data to the server at regular intervals using HTTP POST requests, with the data encoded in JSON format.

[0964] input

[0965] Physiological and emotional data collection results

[0966] output

[0967] Data sent to the server

[0968] Specific actions

[0969] The collected data is automatically sent to the server at regular intervals. For example, every minute, heart rate, GPS data, speed, distance, and physiological data are sent to the server using an HTTP POST request. The data is formatted in JSON format and sent to the server's API endpoint.

[0970] Step 4: Receiving and parsing data on the server side

[0971] explanation

[0972] The server receives the data at the API endpoint and passes it to the analysis module, which evaluates the user's current performance status and emotional state based on the running data and emotional data.

[0973] input

[0974] Data sent to the server

[0975] output

[0976] Evaluation results of user performance and emotional state

[0977] Specific actions

[0978] The server passes the data received at the API endpoint to the analysis module, which analyzes heart rate, speed, distance, and emotional data. The analysis module uses this data to evaluate the user's level of stress or relaxation, and passes the results to the next step.

[0979] Step 5: Generate optimal music data

[0980] explanation

[0981] Based on the analysis results, the server generates optimal music data using a generative AI model that takes into account the user's performance, emotional state, musical preferences, and past feedback.

[0982] input

[0983] Evaluation results of user performance and emotional state

[0984] User's music preferences

[0985] Past Feedback

[0986] output

[0987] Optimal music data

[0988] Specific actions

[0989] Based on the analysis results, the server's generative AI model generates a prompt, which then generates optimal music data based on that prompt. For example, the generative AI model might be prompted to "generate fast-paced, relaxing music."

[0990] Step 6: Send and play music data

[0991] explanation

[0992] The generated music data is sent from the server to the user's device, which analyzes the received music data and plays it using the audio player within the application.

[0993] input

[0994] Optimal music data

[0995] output

[0996] Music played on an audio player

[0997] Specific actions

[0998] The generated music data is encoded in JSON format from the server and sent to the user's device, which interprets the data and automatically starts playing the music in the application's audio player.

[0999] Step 7: User feedback and adjustments

[1000] explanation

[1001] The user can provide feedback on the music through the application while running, and the device sends this feedback to the server, which then adjusts the parameters for generating the music data.

[1002] input

[1003] User Feedback

[1004] output

[1005] Adjusted music data

[1006] Specific actions

[1007] When a user sends feedback such as "The tempo is too fast" or "I like this song" using the buttons in the app, that feedback is sent from the device to the server. The server receives the feedback, sends a new prompt to the generative AI model, and regenerates the adjusted music data. The newly generated music data is then sent back to the user's device, and the music playback is updated.

[1008] (Application example 2)

[1009] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1010] In modern society, users want a comfortable experience while running or shopping, while also improving their performance and stabilizing their emotions. However, current systems lack the ability to collect users' physiological and emotional data in real time and provide optimal music based on that data. As a result, users must choose music that best suits their condition, which hinders efficient performance improvement and a comfortable experience.

[1011] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1012] In this invention, the server includes means for collecting physiological data and location data during running, means for transmitting the collected data to a remote server in real time, means for analyzing the collected data at the remote server and evaluating the user's performance status, means for generating appropriate music data based on the evaluation results, means for transmitting the generated music data to a user terminal, means for playing the transmitted music data, means for collecting the user's location data and emotional data during a shopping experience at a physical store, means for analyzing the collected data and evaluating the user's emotional state, means for generating optimal music data using a generative AI model based on the evaluation results, means for playing music data in real time according to the user's emotional state, and means for adjusting the user's emotional state and shopping pace. This allows the user to enjoy an optimal music experience based on their performance data and emotional data while running or shopping, ensuring a comfortable and efficient experience.

[1013] "Physiological data" refers to data that indicates the user's physical condition, and specifically refers to heart rate, respiratory rate, body temperature, sweat rate, etc.

[1014] "Location data" refers to data that indicates the user's current geographical location, and specifically includes GPS information and the like.

[1015] A "remote server" is a server device that receives data sent from a user's terminal and analyzes and processes the data.

[1016] "Evaluation means" refers to the analysis modules and algorithms used to evaluate a user's performance status and emotional state based on the collected data.

[1017] "Music Data" refers to music files or streams generated based on a user's performance status or emotional state.

[1018] "User terminal" refers to electronic devices used by users, such as smartphones, tablets, and wearable devices.

[1019] "Generative AI model" refers to an artificial intelligence model used to generate optimal music based on user data.

[1020] "Feedback" refers to input from users, such as opinions, impressions, and preferences, that the system uses to adjust its music generation.

[1021] "Shopping pace" refers to data that indicates the speed at which users move within a physical store, the length of time they stay there, and the speed at which they progress with their shopping.

[1022] "Emotional state" is data that indicates the user's feelings or psychological state, and specifically includes stress, relaxation, joy, sadness, and the like.

[1023] This invention is a system that collects and analyzes a user's performance and emotional data in real time while running or shopping in a physical store, and generates optimal music based on that data, thereby improving the user's performance and providing a comfortable experience.

[1024] User launches and configures the application

[1025] The user launches a dedicated application installed on a smartphone or wearable device. The application provides options for starting a run or shopping, and by selecting one, the user begins data collection. The user can also input past running data and music preferences. Furthermore, an emotion engine is built in, which collects user emotional data from voice input and facial expressions.

[1026] Data collection and transmission

[1027] As soon as the device starts running or shopping, it collects real-time physiological and location data, such as GPS data, heart rate, speed, and distance. The emotion engine analyzes the user's voice input and facial expressions to recognize emotional data. This data is sent to the server at regular intervals using HTTP POST requests, and the data is encoded in JSON format.

[1028] Server-side data reception and analysis

[1029] The server has an API endpoint that receives data sent from the device. The server then passes the data to an analysis module to evaluate the user's performance and emotional state while running or shopping. For example, the server evaluates the user's emotional state, such as whether they are stressed or relaxed, along with indicators such as heart rate, walking pace, and distance.

[1030] Optimal music data generation

[1031] Based on the analysis results, the server uses a generative AI model to generate optimal music data. The generative AI model takes into account the user's performance and emotional state, as well as their musical preferences and past feedback. For example, if a user is walking fast and feeling stressed while shopping in a physical store, fast-paced but relaxing music will be generated.

[1032] Sending and playing music data

[1033] The generated music data is sent from the server to the user's device. The device analyzes the received music data and plays it in a format suitable for the user. This playback is performed using the audio player within the application, allowing the user to listen to the music that is best suited to them.

[1034] User feedback and regeneration

[1035] Users can provide real-time feedback on music through the application while running or shopping. For example, they can send feedback such as "I like this song," "The tempo is too fast," or "The tempo is too slow" by pressing a button. The device sends this feedback to the server, which then adjusts the parameters for generating the music data and resends the regenerated music data to the user's device.

[1036] Examples of concrete examples and prompts

[1037] For example, suppose a user starts shopping at a physical store and launches the application. If the analysis reveals that the user is walking fast and feeling stressed based on their voice, the server will generate relaxing music. The generated music data will be sent to the user's device and played on an audio player.

[1038] Examples of prompts:

[1039] "Please input audio data of a user running, walking at a fast pace, and feeling stressed. Generate relaxing music based on that."

[1040] As described above, the present invention integrates a user's performance data and emotional data to provide an optimal music experience, thereby improving performance and providing a comfortable experience while running or shopping.

[1041] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1042] Step 1:

[1043] The user launches the application and selects the running or shopping option, which loads the user's past data and music preferences.

[1044] Input: User selected options, historical data and music preferences

[1045] Output: Data collection start trigger, user setting data

[1046] Step 2:

[1047] As soon as you start running or shopping, the device collects real-time physiological and location data, such as GPS data, heart rate, walking pace, and distance, while an emotion engine analyzes voice input and facial expressions to recognize emotional data.

[1048] Input: User's physiological data, location data, voice input and facial expressions

[1049] Output: Collected performance and sentiment data

[1050] Step 3:

[1051] The device collects data at regular intervals, encodes it in JSON format, and sends it to a remote server using an HTTP POST request.

[1052] Input: Collected performance and emotion data

[1053] Output: Data sent to the server

[1054] Step 4:

[1055] The server receives the received data through the API endpoint and passes it to the analysis module for analysis, which evaluates the user's performance status and emotional state.

[1056] Input: Remotely transmitted data

[1057] Output: Evaluation results of the user's performance status and emotional state

[1058] Step 5:

[1059] Based on the evaluation results, the server uses a generative AI model to generate optimal music data, which is customized based on the user's physiological and emotional data.

[1060] Input: Evaluation results of the user's performance status and emotional state

[1061] Output: Generated music data

[1062] Step 6:

[1063] The server transmits the generated music data to the user's terminal.

[1064] Input: Generated music data

[1065] Output: Music data sent to the user's device

[1066] Step 7:

[1067] The device analyzes the received music data and plays it using the audio player within the application.

[1068] Input: Music data sent

[1069] Output: Played music

[1070] Step 8:

[1071] Users can provide real-time feedback through the application, such as "I like this song," "The tempo is too fast," or "The tempo is too slow."

[1072] Input: User feedback

[1073] Output: Feedback sent to the server

[1074] Step 9:

[1075] The server receives feedback from the user and adjusts the parameters for generating the music data based on that feedback. The generative AI model then recreates the music using the new parameters and sends it back to the user's device.

[1076] Input: Received user feedback

[1077] Output: Regenerated music data

[1078] This concludes the processing flow at each step. This allows users to enjoy an optimal music experience based on their own data in real time, providing a comfortable and efficient experience.

[1079] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1080] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1081] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1082] [Fourth embodiment]

[1083] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1084] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1085] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1086] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1087] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1088] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1089] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1090] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1091] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1092] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1093] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1094] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1095] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1096] The system of the present invention collects user performance data in real time while running and generates optimal music based on that data to help runners improve their performance. This system is implemented as follows.

[1097] User launches and configures the application

[1098] Users launch a dedicated application installed on their smartphone or wearable device. The application provides an option to start running, which users select to begin collecting running data. Users can also input their music preferences and past running data into the app.

[1099] Data collection and transmission

[1100] As soon as you start running, your device collects real-time physiological and location data, such as GPS data, heart rate, speed, and distance. This data is then sent to a server at regular intervals via HTTP POST requests, with the data encoded in JSON format.

[1101] Server-side data reception and analysis

[1102] The server sets up an API endpoint to receive data sent from the device, which is then immediately analyzed to assess the user's current performance. For example, metrics such as heart rate, speed, and distance are used to determine whether the user is running at a sustainable pace or whether fatigue is building up.

[1103] Optimal music data generation

[1104] Based on the analysis results, the server generates music data that is optimal for the user's running situation. This generation uses a generative AI model and takes into account the user's preferences and past feedback data. For example, if the user is running at a fast pace, fast-paced, energetic music will be generated. Conversely, if the user is running an endurance race, music with a steady rhythm will be generated.

[1105] Sending and playing music data

[1106] The generated music data is sent from the server to the user's device, which analyzes the received music data and plays it in a format suitable for the user. Playback is performed using the application's audio player, allowing users to listen to the perfect music while running.

[1107] User feedback and regeneration

[1108] Users can provide real-time feedback on music while running through the application. For example, they can send feedback such as "I like this song," "The tempo is too fast," or "The tempo is too slow" by pressing a button. The device then sends this feedback to the server, which then adjusts the parameters for generating the music data based on the feedback. The adjusted music data is then sent back to the user's device, and playback is updated.

[1109] This invention provides users with optimal music in real time, which helps maintain motivation, reduce fatigue, and improve performance, significantly improving the running experience and enhancing the training effect.

[1110] The processing flow will be explained below.

[1111] Step 1: The user launches the dedicated application installed on their smartphone or wearable device. Once the user logs in and selects the option to start running, data collection mode begins.

[1112] Step 2: The device uses the GPS sensor and heart rate sensor to collect real-time physiological and location data during the run, including current location, speed, heart rate, distance, etc.

[1113] Step 3: The device converts the collected data into JSON format at regular intervals (e.g., every second) and sends it to the server using an HTTP POST request, including the user ID.

[1114] Step 4: The server sets up an API endpoint to receive data sent from the device, which is then immediately passed to the analysis module.

[1115] Step 5: The server's analysis module evaluates the user's current performance based on the received data, and the evaluation result is expressed in the form of tags, such as "sustained pace" or "fast pace."

[1116] Step 6: The server generates optimal music data using a generative AI model based on the evaluation results, taking into account the user's musical preferences and past feedback.

[1117] Step 7: The server sends the generated music data to the user's terminal, where the music data is encoded in an appropriate format (e.g., MP3).

[1118] Step 8: The device analyzes the received music data and plays the music using the audio player in the application. The user can continue running while listening to the optimal music in real time.

[1119] Step 9: Users can provide real-time feedback on the music within the application, in the form of "I like this song" or "The tempo is too fast."

[1120] Step 10: The device sends the user feedback to the server in JSON format.

[1121] Step 11: The server analyzes the received feedback and inputs it as new parameters into the generative AI model. The music data is then regenerated based on this feedback.

[1122] Step 12: The server again sends the newly generated music data to the user's device. The device again receives the music data and plays it on the audio player. This process is repeated, allowing the system to continue providing the optimal music experience the user desires.

[1123] Through these steps, the system improves the user's running experience in real time and helps improve performance.

[1124] Example 1

[1125] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1126] Conventional running support systems have difficulty not only collecting physiological and location data from users but also providing appropriate feedback and support to improve their performance based on that data. In particular, there is a need for systems that can maintain motivation and reduce fatigue while running by evaluating the user's performance in real time and dynamically generating and providing optimal music based on that data.

[1127] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1128] In this invention, the server includes means for collecting physiological data and position data during running, means for transmitting the collected data to a remote server in real time, means for analyzing the collected data at the remote server and evaluating the user's performance status, means for using a generative AI model to generate appropriate music data based on the evaluation results, means for inputting a prompt sentence to the generative AI model to generate music data based on the evaluation results, means for transmitting the generated music data to a user terminal, and means for playing the transmitted music data. This makes it possible to dynamically generate and provide music tailored to each user's individual running status, thereby maintaining the user's motivation and contributing to improving performance.

[1129] "Physiological data" refers to physiological information such as heart rate, blood pressure, and respiratory rate collected from within the user's body in real time.

[1130] "Location Data" means information based on GPS coordinates that indicates a user's current location.

[1131] "Remote Server" refers to a central processing unit that transmits, receives, and analyzes data over a network.

[1132] "Analysis Engine" means a software component that evaluates a user's performance status based on collected data.

[1133] "Generative AI model" refers to an artificial intelligence model that generates optimal music based on the user's performance status and past feedback data.

[1134] A "prompt" is an instructional text that specifies conditions or requests and is input into a generative AI model.

[1135] "User terminal" refers to a portable electronic device used by a user, such as a smartphone or wearable device.

[1136] "Audio player" refers to a software or hardware component for playing music data within a user terminal.

[1137] "Feedback" refers to input information that provides the system with responses and impressions that the user has actually experienced in real time.

[1138] The system of this invention provides users with real-time music that optimizes their running performance. The system utilizes various hardware and software, including smartphones, wearable devices, remote servers, and generative AI models, to provide a dynamic and personalized experience for users.

[1139] User launches and configures the application

[1140] Users launch a dedicated application installed on their smartphone or wearable device. The application has a "Start Running" option, which users can select to begin collecting running data. Users can input their music preferences and past running data into the app in advance.

[1141] Data collection and transmission

[1142] The device collects real-time physiological and location data, such as GPS data, heart rate, speed, and distance, as soon as the user starts running. This data is encoded into JSON format at regular intervals and sent to the server via HTTP POST requests. The server uses the Python Flask framework to set up an API endpoint to receive the data.

[1143] Server-side data reception and analysis

[1144] The server immediately analyzes the received data and assesses the user's current performance status, using metrics such as heart rate, speed, and distance to determine whether the user is running at a sustainable pace or whether fatigue is building up.

[1145] Optimal music data generation

[1146] Based on the analysis results, the server uses a generative AI model to generate music data that is optimal for the user's running situation. This generation also takes into account the user's past feedback data and preferences. For example, if the user is running at a fast pace, fast-paced, energetic music will be generated. The following prompt sentence is input to the generative AI model:

[1147] The user's current heart rate is 160 BPM and their running speed is 5 km / min. The user's past running history has shown that they prefer music with a tempo of 120 BPM. Based on these conditions, generate music that will allow the user to maintain their current pace.

[1148] Sending and playing music data

[1149] The generated music data is sent from the server to the user's device, which analyzes the received music data and plays it using the application's audio player, allowing users to listen to the perfect music while running.

[1150] User feedback and regeneration

[1151] While running, users can provide real-time feedback on the music through the application. For example, they can send feedback such as "I like this song," "The tempo is too fast," or "The tempo is too slow" by pressing a button. The device sends this feedback to the server, which then adjusts the parameters of the generative AI model based on this feedback. The adjusted music data is then sent back to the user's device, and playback is updated.

[1152] These processes allow users to continue running while enjoying optimal music in real time, which is expected to maintain motivation and improve performance.

[1153] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1154] Step 1:

[1155] Starting and Configuring the Application

[1156] The user launches a dedicated application on their smartphone or wearable device. They then tap the "Start Running" button, which prepares the application to begin collecting running data. Input includes the user's music preferences and past running data. Based on this, user profile data is generated.

[1157] Step 2:

[1158] Collecting and sending running data

[1159] The device enters collection mode when the user starts running. It collects physiological and location data such as GPS data, heart rate, speed, and distance in real time from the GPS sensor, heart rate sensor, and accelerometer. The data is converted into JSON format and sent to the server at regular intervals via HTTP POST requests. The input is physiological and location data, and the output is JSON-formatted data.

[1160] Step 3:

[1161] Data reception and storage

[1162] The server uses the Python Flask framework to set up an API endpoint. The server receives data received as a POST request from the terminal and saves the data in temporary storage. The input is the received JSON data, and the output is the temporarily saved data. As an example of actual operation, when a request arrives at the / data endpoint, the data is stored in the database.

[1163] Step 4:

[1164] Data analysis

[1165] The server's analysis engine analyzes the stored data. The server evaluates indicators such as heart rate, speed, and distance to determine the user's performance. The analysis results may include information such as "heart rate is high," "speed is stable," and "distance still half covered." The input is the temporarily stored data, and the output is the analyzed performance evaluation.

[1166] Step 5:

[1167] Music data generation

[1168] Based on the analysis results, the server inputs a prompt into the generative AI model to generate music data that is optimal for the user's running situation. The user's preferences and past feedback data are also taken into consideration. The input is performance evaluation and user profile, and the output is the generated music data. An example of a prompt is, "The user's current heart rate is 160 BPM and speed is 5 minutes / km. The user's past running history has included feedback that they prefer music with a tempo of 120 BPM. Based on these conditions, please generate music that will allow the user to maintain their current pace."

[1169] Step 6:

[1170] Sending and playing music data

[1171] The server sends the generated music data in JSON format as an HTTP response to the device. The device analyzes the received data and plays the music using the audio player in the application. The input is the received music data, and the output is the played music. As a specific example of operation, the device downloads an MP3 file and plays it on the built-in player.

[1172] Step 7:

[1173] Receiving feedback and reanalyzing

[1174] The user sends feedback about the music through the application. For example, by pressing a button, they can indicate whether the tempo is too fast or too slow. The device encodes this feedback data into JSON format and sends it to the server. The server then adjusts the parameters of the generative AI model based on the received feedback and generates new music data. The input is the user feedback, and the output is the regenerated music data.

[1175] This series of processes allows the user to continue running while always enjoying the most suitable music in real time.

[1176] (Application example 1)

[1177] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1178] Maintaining and improving productivity is a challenge in modern factories. In particular, there is a need to promote efficient operation of factory machines and optimize operator motivation and the operating pace of the machines. However, current systems lack the technology to analyze factory machine operation data and environmental data in real time and automatically provide appropriate work music based on that data. This can result in a decline in factory productivity.

[1179] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1180] In this invention, the server includes a means for collecting operational data and environmental data of factory machines, a means for evaluating the productivity of the factory machines based on the collected data, and a means for generating appropriate work music based on the evaluation results. This makes it possible to evaluate the operating efficiency of factory machines in real time and automatically generate and provide music suited to the work environment.

[1181] "Physiological data" refers to data that indicates the physiological state of the body, such as heart rate, speed, and distance.

[1182] "Location data" refers to geographical location information of users and devices obtained by GPS or other means.

[1183] "Factory machinery" refers to automated equipment and robotic systems used in manufacturing operations.

[1184] "Operation data" refers to data that indicates the operating status of factory machines, such as operating time, operating speed, and travel distance.

[1185] "Environmental data" refers to data that indicates the surrounding environmental conditions such as temperature, humidity, and noise level.

[1186] A "server" is a remote computer system that analyzes the collected data and takes appropriate action.

[1187] "Analysis" refers to the process of making an evaluation or diagnosis based on collected data that is suitable for a specific purpose.

[1188] "Productivity" is an indicator of the efficiency and performance of factory machines and systems over a certain period of time.

[1189] "Work music" is music provided to improve work efficiency and motivation.

[1190] "Feedback" refers to evaluations and opinions given by users and systems, which are used by the system to adapt and improve.

[1191] MODE FOR CARRYING OUT THE INVENTION

[1192] The system for realizing this invention first incorporates sensors that collect operational data and environmental data from factory machines. This data is sent to a cloud server using Wi-Fi or wired connections installed in the factory. The data is sent using an HTTP POST request, and the data is encoded in JSON format.

[1193] The server receives the collected data and analyzes it in real time. Data analysis tools such as Python and R are used for the analysis, and machine learning models and AI technologies (e.g., TensorFlow, Scikit-learn) are used to evaluate the operating status and productivity of factory machinery. For example, the current performance and load status of machinery can be determined based on data such as operating time, operating speed, travel distance, temperature, humidity, and noise level.

[1194] Based on the analysis results, the server inputs specific prompt sentences into the generative AI model to generate appropriate work music. The prompt sentences include content that corresponds to the operating status and productivity of the factory machine. The generated music data is then sent from the server to the factory machine's control system. The factory machine's control system analyzes the received music data and plays it through the built-in speaker or speakers of peripheral devices.

[1195] Feedback from factory machines and operators is sent to the server in real time. This feedback includes an evaluation of whether the tempo of the music is appropriate and the impact of the music on productivity. The server uses this feedback to adjust the parameters of the generative AI model and reflect this in future music generation. This makes it possible to automatically provide optimized work music.

[1196] Specific examples

[1197] Hardware used

[1198] Sensors: Sensors built into factory machines to collect operational and environmental data

[1199] Communication equipment: Wi-Fi module or wired connection

[1200] Cloud server: A server that performs data analysis and music generation (e.g., AWS, Google Cloud)

[1201] Software used

[1202] Data analysis tools: Python, R

[1203] Machine learning models: TensorFlow, Scikit-learn

[1204] Music Generation Library: A tentative music generation library

[1205] Prompt Sentence Examples

[1206] "Based on the music feedback provided during the run, generate new music taking into account the following criteria:

[1207] Productivity has decreased in the last 30 minutes

[1208] Factory machines operate at lower than average speeds

[1209] There's a demand for high-energy, fast-paced music."

[1210] In this way, this invention makes it possible to generate and provide optimal work music in real time based on the operation data and environmental data of factory machines, which is expected to improve factory productivity and efficient machine operation.

[1211] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1212] Step 1:

[1213] Factory machines use built-in sensors to collect operational and environmental data, such as operating time, operating speed, travel distance, temperature, humidity, and noise level.

[1214] Step 2:

[1215] The collected data is sent to a cloud server via Wi-Fi or wired connection installed in the factory. The data is sent using an HTTP POST request, and is encoded in JSON format. The JSON format data is output.

[1216] Step 3:

[1217] The server analyzes the received data. Specifically, it uses data analysis tools such as Python and R to apply machine learning models (e.g., TensorFlow, Scikit-learn) to evaluate the operating status and productivity of factory machinery. The evaluation uses data such as operating time, operating speed, temperature, humidity, and noise level. The analysis results are output.

[1218] Step 4:

[1219] The server generates a prompt sentence based on the analysis results. The prompt sentence contains content that corresponds to the operating status and productivity of the factory machines. For example, "Productivity has decreased in the last 30 minutes" or "Energetic, fast-paced music is required." The prompt sentence is the output.

[1220] Step 5:

[1221] The server inputs the generated prompt sentence into a generative AI model to generate appropriate task music. The generative AI model (e.g., OpenAI's GPT-3) generates music data based on the prompt sentence. The generated music data is the output.

[1222] Step 6:

[1223] The generated music data is then sent from the server to the factory machine control system, where it becomes the output.

[1224] Step 7:

[1225] The control system of the factory machine analyzes the received music data and plays it through the built-in speaker or a speaker of a peripheral device. The playback of the music data becomes the output.

[1226] Step 8:

[1227] Feedback from factory machines and operators is sent to the server in real time. The feedback includes an assessment of whether the music tempo is appropriate and the impact of the music on productivity. The feedback data is the output.

[1228] Step 9:

[1229] The server adjusts the parameters of the generative AI model based on the received feedback and reflects this in future music generation. This further optimizes the generated music. The adjusted generative AI model is the output.

[1230] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1231] This invention is a system that collects and analyzes a user's performance and emotional data in real time while they are running, and generates optimal music based on that data, thereby improving a runner's performance and providing a comfortable running experience.

[1232] User launches and configures the application

[1233] The user launches a dedicated application installed on a smartphone or wearable device. The application provides an option to start running, which the user selects to begin data collection. The user can also input their music preferences and past running data into the app. Furthermore, the present invention incorporates an emotion engine that also collects user emotion data.

[1234] Data collection and transmission

[1235] As soon as the device starts running, it collects real-time physiological and location data, such as GPS data, heart rate, speed, and distance. Additionally, the emotion engine analyzes the user's voice input, facial expressions, and other physiological data to recognize emotions. This data is then sent to the server at regular intervals using an HTTP POST request, with the data encoded in JSON format.

[1236] Server-side data reception and analysis

[1237] The server has an API endpoint set up to receive data sent from the device. The received data is immediately passed to the analysis module, which evaluates the user's current performance and emotional state based on the running data and emotional data. For example, along with indicators such as heart rate, speed, and distance, the emotional state of the user, such as whether they are stressed or relaxed, is evaluated.

[1238] Optimal music data generation

[1239] Based on the analysis results, the server uses a generative AI model to generate optimal music data. The generative AI model takes into account the user's performance and emotional state, as well as their music preferences and past feedback. For example, if a user is running at a fast pace but feeling stressed, music with a fast tempo but a relaxing feel will be generated. On the other hand, if a user is running a long distance and feeling relaxed, music with a steady rhythm will be generated.

[1240] Sending and playing music data

[1241] The generated music data is sent from the server to the user's device, which analyzes the received music data and plays it in a format suitable for the user. Playback is performed using the application's audio player, allowing users to listen to the perfect music while running.

[1242] User feedback and regeneration

[1243] Users can provide real-time feedback on music through the application while running. For example, they can send feedback such as "I like this song," "The tempo is too fast," or "The tempo is too slow" by pressing a button. The device sends this feedback to the server, which then adjusts the parameters for generating the music data based on the feedback. The adjusted music data is then sent back to the user's device, and playback is updated.

[1244] Specific examples

[1245] For example, a user starts running, launches the application, and begins collecting data. If the user's heart rate increases while running and the emotion engine recognizes stress from the user's voice, the server generates fast-paced but relaxing music. The music data is immediately sent to the user's device and played on an audio player. The user can continue running while listening to the music and provide feedback.

[1246] In this way, by integrating the user's performance data and emotional data to provide an optimal music experience, the system improves running performance and provides a comfortable running experience.

[1247] The processing flow will be explained below.

[1248] Step 1: The user launches the dedicated application installed on their smartphone or wearable device, logs in to the application, and selects the option to start running.

[1249] Step 2: The device uses the GPS sensor, heart rate sensor, and emotion engine to collect physiological data, location data, and emotion data in real time while running, including current location, speed, heart rate, distance, smile level, voice tone, etc.

[1250] Step 3: The terminal converts the collected data into JSON format and sends it to the server at regular intervals (e.g., every second) using an HTTP POST request.

[1251] Step 4: The server receives the data sent from the device at the API endpoint, and passes the received data to the analysis module.

[1252] Step 5: The server's analysis module evaluates the user's current performance status and emotional state based on the received physiological, location, and emotional data, including categories such as "sustained pace," "fast pace," "relaxed," and "stressed."

[1253] Step 6: The server generates optimal music data based on the evaluation results using a generative AI model that takes into account the user's performance and emotional state, as well as their musical preferences and past feedback.

[1254] Step 7: The server sends the generated music data to the user's device, encoded in an appropriate format (e.g., MP3).

[1255] Step 8: The device analyzes the received music data and plays the music using the audio player in the application. The user can continue running while listening to the optimal music in real time.

[1256] Step 9: The user provides real-time feedback on the music within the application, using options such as "I like this song," "The tempo is too fast," or "The tempo is too slow."

[1257] Step 10: The device sends the user feedback to the server in JSON format.

[1258] Step 11: The server's analysis module analyzes the received feedback and inputs it as new parameters into the generative AI model, so that the next time music is generated, the user's preferences will be reflected.

[1259] Step 12: The server retransmits the newly generated music data to the user's device. The device again analyzes the received music data and plays it on the audio player. This process is repeated, allowing the user to continue running while always listening to the best music.

[1260] In this way, the system integrates the user's physiological data, location data, and emotional data to generate optimal music in real time, thereby improving performance and providing a comfortable running experience.

[1261] Example 2

[1262] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1263] The purpose of this invention is to improve runners' performance and provide a comfortable running experience by providing a means to collect and analyze a user's performance and emotional data in real time while running and generate optimal music based on that data. While conventional systems can collect and analyze physiological data, it is difficult to generate optimal music that takes the user's emotional state into account. Furthermore, there is a need for a more personalized experience by dynamically adjusting the music based on real-time feedback.

[1264] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1265] In this invention, the server includes a means for analyzing data collected by a remote server and evaluating the user's performance status and emotional state, a means including a generative AI model for generating optimal music data based on the evaluation results, and a means for transmitting the generated music data to the user terminal. This makes it possible to provide optimal music in real time according to the user's performance status and emotional state. Furthermore, by reflecting user feedback in real time and dynamically adjusting the music data, a personalized running experience can be achieved.

[1266] "Physiological data" is information related to a user's physical activity and physiological state, such as heart rate, speed, distance, etc.

[1267] "Location data" is information about the user's current location and travel route obtained using GPS.

[1268] A "remote server" is a server that is accessed via the Internet and is a device that receives and analyzes data sent from a user terminal.

[1269] A "generative AI model" is an artificial intelligence technology that generates music data taking into account the user's performance situation, emotional state, musical preferences, etc.

[1270] "Emotional state" refers to the user's emotional state, such as whether the user is stressed or relaxed.

[1271] "Feedback" refers to opinions and evaluations provided by users, and includes information such as responses to music tempo, likes and dislikes, etc.

[1272] An "analysis module" is a program or device that analyzes collected data and evaluates the user's performance status and emotional state.

[1273] An "audio player" is software or hardware for playing digital music data.

[1274] The present invention is a system that uses a smartphone or wearable device to collect real-time performance and emotional data while a user is running, and generates optimal music based on that data. The system analyzes the collected data, generates optimal music, plays the music, and incorporates user feedback.

[1275] First, the user launches a dedicated application installed on a smartphone or wearable device. The user enters their music preferences and past running data within the application, and the emotion engine also collects the user's emotional data. The hardware used in this case is a smartphone (iOS, Android) or a wearable device (Apple Watch, Fitbit, etc.), and the software is the dedicated application.

[1276] Next, as soon as the user starts running, the device collects real-time physiological and location data, such as GPS data, heart rate, speed, and distance. The emotion engine recognizes the user's emotions by analyzing voice input and facial expressions. The collected data is sent to the server at regular intervals. The data is sent using an HTTP POST request and encoded in JSON format.

[1277] The server passes the data received at the API endpoint to the analysis module. The analysis module evaluates the user's current performance status and emotional state based on the running data and emotional data. For example, it evaluates whether the user is feeling stressed or relaxed, along with indicators such as heart rate, speed, and distance. The analysis module used for this purpose is a program that evaluates the user's performance status and emotional state based on the collected data.

[1278] Based on the analysis results, the server uses a generative AI model to generate optimal music data. The generative AI model takes into account the user's performance status, emotional state, music preferences, and past feedback. This allows appropriate music to be generated in real time and sent to the user's device. For example, if a user is running at a fast pace but feeling stressed, music with a fast tempo but a relaxing feel will be generated. On the other hand, if a user is running a long distance and feeling relaxed, music with a steady rhythm will be generated. The generative AI model used for this is a program that generates optimal music data according to the user's performance status and emotional state.

[1279] The generated music data is sent from the server to the user's device, which then analyzes the received music data and plays it on the application's audio player, allowing the user to continue running while listening to the music.

[1280] Furthermore, users can provide real-time feedback on the music through the application while running. For example, they can send feedback such as "I like this song," "The tempo is too fast," or "The tempo is too slow" by pressing a button. The device sends this feedback to the server, which then adjusts the parameters for generating the music data based on the feedback. The adjusted music data is then sent back to the user's device, and the music playback is updated.

[1281] For example, when a user starts running and launches the application to begin collecting data, if their heart rate rises and the emotion engine detects stress in the user's voice, the server generates fast-paced but relaxing music. The music data is immediately sent to the user's device and played on an audio player. The user can continue running while listening to the music and provide feedback.

[1282] Examples of prompts include:

[1283] If the user's heart rate exceeds 150 and the emotion engine detects stress, what musical prompt should be generated?

[1284] Input data:

[1285] Heart rate: 150

[1286] Stress level: High

[1287] Music preference: Relaxing and fast-paced

[1288] Expected output:

[1289] Prompt: Generate fast-paced but relaxing music for a runner running at a fast pace.

[1290] In this way, the present invention integrates a user's performance data and emotional data to provide an optimal music experience, thereby improving performance while running and achieving a comfortable running experience.

[1291] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1292] Program processing flow

[1293] Step 1: User launches and configures the application

[1294] explanation

[1295] The user launches a dedicated application installed on a smartphone or wearable device, selects options to start running, and inputs music preferences and past running data. The emotion engine then begins collecting the user's emotional data.

[1296] input

[1297] User's music preferences

[1298] Past running data

[1299] output

[1300] Initial state after setup is complete

[1301] Activating the Emotion Engine

[1302] Specific actions

[1303] When a user launches the application and presses the "Start Running" button, a settings screen appears. This screen displays options for inputting "favorite music genre" and "average pace of past runs." This starts the emotion engine in the background.

[1304] Step 2: Start collecting data

[1305] explanation

[1306] As soon as the user starts running, the device collects real-time physiological and location data, including GPS data, heart rate, speed, and distance. The emotion engine analyzes voice input and facial expressions to recognize the user's emotions.

[1307] input

[1308] User's real-time physiological data (heart rate, speed, distance)

[1309] User's real-time location data (GPS information)

[1310] User emotion data (voice input, facial expression analysis)

[1311] output

[1312] Physiological and emotional data collection results

[1313] Specific actions

[1314] When a user starts running, their smartphone or wearable device automatically starts collecting sensor data, including heart rate monitors, GPS sensors, and microphones, and the data is stored locally at regular intervals.

[1315] Step 3: Sending data

[1316] explanation

[1317] The device sends the collected data to the server at regular intervals using HTTP POST requests, with the data encoded in JSON format.

[1318] input

[1319] Physiological and emotional data collection results

[1320] output

[1321] Data sent to the server

[1322] Specific actions

[1323] The collected data is automatically sent to the server at regular intervals. For example, every minute, heart rate, GPS data, speed, distance, and physiological data are sent to the server using an HTTP POST request. The data is formatted in JSON format and sent to the server's API endpoint.

[1324] Step 4: Receiving and parsing data on the server side

[1325] explanation

[1326] The server receives the data at the API endpoint and passes it to the analysis module, which evaluates the user's current performance status and emotional state based on the running data and emotional data.

[1327] input

[1328] Data sent to the server

[1329] output

[1330] Evaluation results of user performance and emotional state

[1331] Specific actions

[1332] The server passes the data received at the API endpoint to the analysis module, which analyzes heart rate, speed, distance, and emotional data. The analysis module uses this data to evaluate the user's level of stress or relaxation, and passes the results to the next step.

[1333] Step 5: Generate optimal music data

[1334] explanation

[1335] Based on the analysis results, the server generates optimal music data using a generative AI model that takes into account the user's performance, emotional state, musical preferences, and past feedback.

[1336] input

[1337] Evaluation results of user performance and emotional state

[1338] User's music preferences

[1339] Past Feedback

[1340] output

[1341] Optimal music data

[1342] Specific actions

[1343] Based on the analysis results, the server's generative AI model generates a prompt, which then generates optimal music data based on that prompt. For example, the generative AI model might be prompted to "generate fast-paced, relaxing music."

[1344] Step 6: Send and play music data

[1345] explanation

[1346] The generated music data is sent from the server to the user's device, which analyzes the received music data and plays it using the audio player within the application.

[1347] input

[1348] Optimal music data

[1349] output

[1350] Music played on an audio player

[1351] Specific actions

[1352] The generated music data is encoded in JSON format from the server and sent to the user's device, which interprets the data and automatically starts playing the music in the application's audio player.

[1353] Step 7: User feedback and adjustments

[1354] explanation

[1355] The user can provide feedback on the music through the application while running, and the device sends this feedback to the server, which then adjusts the parameters for generating the music data.

[1356] input

[1357] User Feedback

[1358] output

[1359] Adjusted music data

[1360] Specific actions

[1361] When a user sends feedback such as "The tempo is too fast" or "I like this song" using the buttons in the app, that feedback is sent from the device to the server. The server receives the feedback, sends a new prompt to the generative AI model, and regenerates the adjusted music data. The newly generated music data is then sent back to the user's device, and the music playback is updated.

[1362] (Application example 2)

[1363] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1364] In modern society, users want a comfortable experience while running or shopping, while also improving their performance and stabilizing their emotions. However, current systems lack the ability to collect users' physiological and emotional data in real time and provide optimal music based on that data. As a result, users must choose music that best suits their condition, which hinders efficient performance improvement and a comfortable experience.

[1365] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1366] In this invention, the server includes means for collecting physiological data and location data during running, means for transmitting the collected data to a remote server in real time, means for analyzing the collected data at the remote server and evaluating the user's performance status, means for generating appropriate music data based on the evaluation results, means for transmitting the generated music data to a user terminal, means for playing the transmitted music data, means for collecting the user's location data and emotional data during a shopping experience at a physical store, means for analyzing the collected data and evaluating the user's emotional state, means for generating optimal music data using a generative AI model based on the evaluation results, means for playing music data in real time according to the user's emotional state, and means for adjusting the user's emotional state and shopping pace. This allows the user to enjoy an optimal music experience based on their performance data and emotional data while running or shopping, ensuring a comfortable and efficient experience.

[1367] "Physiological data" refers to data that indicates the user's physical condition, and specifically refers to heart rate, respiratory rate, body temperature, sweat rate, etc.

[1368] "Location data" refers to data that indicates the user's current geographical location, and specifically includes GPS information and the like.

[1369] A "remote server" is a server device that receives data sent from a user's terminal and analyzes and processes the data.

[1370] "Evaluation means" refers to the analysis modules and algorithms used to evaluate a user's performance status and emotional state based on the collected data.

[1371] "Music Data" refers to music files or streams generated based on a user's performance status or emotional state.

[1372] "User terminal" refers to electronic devices used by users, such as smartphones, tablets, and wearable devices.

[1373] "Generative AI model" refers to an artificial intelligence model used to generate optimal music based on user data.

[1374] "Feedback" refers to input from users, such as opinions, impressions, and preferences, that the system uses to adjust its music generation.

[1375] "Shopping pace" refers to data that indicates the speed at which users move within a physical store, the length of time they stay there, and the speed at which they progress with their shopping.

[1376] "Emotional state" is data that indicates the user's feelings or psychological state, and specifically includes stress, relaxation, joy, sadness, and the like.

[1377] This invention is a system that collects and analyzes a user's performance and emotional data in real time while running or shopping in a physical store, and generates optimal music based on that data, thereby improving the user's performance and providing a comfortable experience.

[1378] User launches and configures the application

[1379] The user launches a dedicated application installed on a smartphone or wearable device. The application provides options for starting a run or shopping, and by selecting one, the user begins data collection. The user can also input past running data and music preferences. Furthermore, an emotion engine is built in, which collects user emotional data from voice input and facial expressions.

[1380] Data collection and transmission

[1381] As soon as the device starts running or shopping, it collects real-time physiological and location data, such as GPS data, heart rate, speed, and distance. The emotion engine analyzes the user's voice input and facial expressions to recognize emotional data. This data is sent to the server at regular intervals using HTTP POST requests, and the data is encoded in JSON format.

[1382] Server-side data reception and analysis

[1383] The server has an API endpoint that receives data sent from the device. The server then passes the data to an analysis module to evaluate the user's performance and emotional state while running or shopping. For example, the server evaluates the user's emotional state, such as whether they are stressed or relaxed, along with indicators such as heart rate, walking pace, and distance.

[1384] Optimal music data generation

[1385] Based on the analysis results, the server uses a generative AI model to generate optimal music data. The generative AI model takes into account the user's performance and emotional state, as well as their musical preferences and past feedback. For example, if a user is walking fast and feeling stressed while shopping in a physical store, fast-paced but relaxing music will be generated.

[1386] Sending and playing music data

[1387] The generated music data is sent from the server to the user's device. The device analyzes the received music data and plays it in a format suitable for the user. This playback is performed using the audio player within the application, allowing the user to listen to the music that is best suited to them.

[1388] User feedback and regeneration

[1389] Users can provide real-time feedback on music through the application while running or shopping. For example, they can send feedback such as "I like this song," "The tempo is too fast," or "The tempo is too slow" by pressing a button. The device sends this feedback to the server, which then adjusts the parameters for generating the music data and resends the regenerated music data to the user's device.

[1390] Examples of concrete examples and prompts

[1391] For example, suppose a user starts shopping at a physical store and launches the application. If the analysis reveals that the user is walking fast and feeling stressed based on their voice, the server will generate relaxing music. The generated music data will be sent to the user's device and played on an audio player.

[1392] Examples of prompts:

[1393] "Please input audio data of a user running, walking at a fast pace, and feeling stressed. Generate relaxing music based on that."

[1394] As described above, the present invention integrates a user's performance data and emotional data to provide an optimal music experience, thereby improving performance and providing a comfortable experience while running or shopping.

[1395] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1396] Step 1:

[1397] The user launches the application and selects the running or shopping option, which loads the user's past data and music preferences.

[1398] Input: User selected options, historical data and music preferences

[1399] Output: Data collection start trigger, user setting data

[1400] Step 2:

[1401] As soon as you start running or shopping, the device collects real-time physiological and location data, such as GPS data, heart rate, walking pace, and distance, while an emotion engine analyzes voice input and facial expressions to recognize emotional data.

[1402] Input: User's physiological data, location data, voice input and facial expressions

[1403] Output: Collected performance and sentiment data

[1404] Step 3:

[1405] The device collects data at regular intervals, encodes it in JSON format, and sends it to a remote server using an HTTP POST request.

[1406] Input: Collected performance and emotion data

[1407] Output: Data sent to the server

[1408] Step 4:

[1409] The server receives the received data through the API endpoint and passes it to the analysis module for analysis, which evaluates the user's performance status and emotional state.

[1410] Input: Remotely transmitted data

[1411] Output: Evaluation results of the user's performance status and emotional state

[1412] Step 5:

[1413] Based on the evaluation results, the server uses a generative AI model to generate optimal music data, which is customized based on the user's physiological and emotional data.

[1414] Input: Evaluation results of the user's performance status and emotional state

[1415] Output: Generated music data

[1416] Step 6:

[1417] The server transmits the generated music data to the user's terminal.

[1418] Input: Generated music data

[1419] Output: Music data sent to the user's device

[1420] Step 7:

[1421] The device analyzes the received music data and plays it using the audio player within the application.

[1422] Input: Music data sent

[1423] Output: Played music

[1424] Step 8:

[1425] Users can provide real-time feedback through the application, such as "I like this song," "The tempo is too fast," or "The tempo is too slow."

[1426] Input: User feedback

[1427] Output: Feedback sent to the server

[1428] Step 9:

[1429] The server receives feedback from the user and adjusts the parameters for generating the music data based on that feedback. The generative AI model then recreates the music using the new parameters and sends it back to the user's device.

[1430] Input: Received user feedback

[1431] Output: Regenerated music data

[1432] This concludes the processing flow at each step. This allows users to enjoy an optimal music experience based on their own data in real time, providing a comfortable and efficient experience.

[1433] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1434] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1435] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1436] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1437] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1438] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1439] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1440] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1441] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1442] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1443] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1444] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1445] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1446] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1447] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1448] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1449] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1450] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1451] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1452] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1453] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1454] The following is further disclosed regarding the above embodiment.

[1455] (Claim 1)

[1456] means for collecting physiological and location data during a run;

[1457] means for transmitting the collected data to a remote server in real time;

[1458] means for analyzing the collected data at the remote server to evaluate the user's performance status;

[1459] means for generating appropriate music data based on the evaluation results;

[1460] means for transmitting the generated music data to a user terminal;

[1461] The system includes means for playing the transmitted music data.

[1462] (Claim 2)

[1463] means for receiving real-time user feedback during a run;

[1464] 10. The system of claim 1, further comprising means for modifying the generation of the music data based on the received feedback.

[1465] (Claim 3)

[1466] 10. The system of claim 1, further comprising means for comparing the collected data with the user's past performance data in the analysis of the collected data.

[1467] "Example 1"

[1468] (Claim 1)

[1469] means for collecting physiological and location data during a run;

[1470] means for transmitting the collected data to a remote server in real time;

[1471] means for analyzing the collected data at the remote server to evaluate the user's performance status;

[1472] a means for using a generative AI model to generate appropriate music data based on the evaluation results;

[1473] A means for inputting a prompt sentence into the generative AI model and generating music data based on the evaluation results;

[1474] means for transmitting the generated music data to a user terminal;

[1475] The system includes means for playing the transmitted music data.

[1476] (Claim 2)

[1477] means for receiving real-time user feedback during a run;

[1478] 10. The system of claim 1, further comprising means for modifying the generation of the music data based on the received feedback.

[1479] (Claim 3)

[1480] 10. The system of claim 1, further comprising means for comparing the collected data with the user's past performance data in the analysis of the collected data.

[1481] "Application Example 1"

[1482] (Claim 1)

[1483] means for collecting physiological and location data during a run;

[1484] means for transmitting the collected data to a remote server in real time;

[1485] means for analyzing the collected data at the remote server to evaluate the user's performance status;

[1486] means for generating appropriate music data based on the evaluation results;

[1487] means for transmitting the generated music data to a user terminal;

[1488] means for playing the transmitted music data;

[1489] means for collecting operational and environmental data of the factory machine;

[1490] A means for evaluating the productivity of factory machines based on collected data;

[1491] means for generating appropriate work music based on the evaluation results;

[1492] The system includes means for transmitting and playing the generated work music to factory machines.

[1493] (Claim 2)

[1494] means for receiving real-time user feedback during a run;

[1495] A means for modifying the generation of music data based on received feedback.

[1496] The system of claim 1 further comprising:

[1497] (Claim 3)

[1498] In analyzing the collected data, a means of comparing it with users' past performance data is provided.

[1499] The system of claim 1 further comprising:

[1500] "Example 2: Combining Emotion Engines"

[1501] (Claim 1)

[1502] means for collecting physiological and location data during a run;

[1503] means for transmitting the collected data to a remote server in real time;

[1504] means for analyzing the collected data at the remote server to assess the user's performance status and emotional state;

[1505] means including a generative AI model for generating optimal music data based on the evaluation results;

[1506] means for transmitting the generated music data to a user terminal;

[1507] A system including means for playing the transmitted music data in a format suitable for the user terminal.

[1508] (Claim 2)

[1509] means for receiving real-time user feedback during a run;

[1510] 10. The system of claim 1, further comprising means for modifying the generation of the music data based on the received feedback.

[1511] (Claim 3)

[1512] 10. The system of claim 1, further comprising means for comparing the collected data with the user's past performance data and musical preferences in analyzing the collected data.

[1513] "Application example 2 when combining emotion engines"

[1514] (Claim 1)

[1515] means for collecting physiological and location data during a run;

[1516] means for transmitting the collected data to a remote server in real time;

[1517] means for analyzing the collected data at the remote server to evaluate the user's performance status;

[1518] means for generating appropriate music data based on the evaluation results;

[1519] means for transmitting the generated music data to a user terminal;

[1520] means for playing the transmitted music data;

[1521] means for collecting user location data and emotional data during a physical store shopping experience;

[1522] means for analyzing the collected data and assessing the user's emotional state;

[1523] A means for generating optimal music data using a generative AI model based on the evaluation results;

[1524] means for playing music data in real time according to the emotional state of a user;

[1525] A system including a means for adjusting a user's emotional state and shopping pace.

[1526] (Claim 2)

[1527] means for receiving real-time user feedback during a run;

[1528] means for modifying the generation of the music data based on the received feedback;

[1529] means for analyzing the collected data and comparing it with the user's past performance data;

[1530] Receive real-time feedback while users are shopping in-store,

[1531] 10. The system of claim 1, further comprising means for readjusting the generation of music data based on emotional state and shopping pace.

[1532] (Claim 3)

[1533] means for analyzing the collected data and comparing it with the user's past performance data;

[1534] 10. The system of claim 1, further comprising means for comparing with past shopping feedback and music preference data. [Explanation of symbols]

[1535] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. means for collecting physiological and location data during a run; means for transmitting the collected data to a remote server in real time; means for analyzing the collected data at the remote server to evaluate the user's performance status; means for generating appropriate music data based on the evaluation results; means for transmitting the generated music data to a user terminal; The system includes means for playing the transmitted music data.

2. means for receiving real-time user feedback during a run; 10. The system of claim 1, further comprising means for modifying the generation of the music data based on the received feedback.

3. 10. The system of claim 1, further comprising means for comparing the collected data with past performance data of the user in the analysis of the collected data.

Images