system

Abstract

We provide the system. [Solution] A means of receiving information from users regarding their dance style, age, physical fitness, and preferences, A means for analyzing the received information and generating original dance videos and music based on an existing dance database, A means for transmitting the generated original dance video and music to the user terminal, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] [[ID=X]]Many people want to perform dances at school festivals, wedding receptions, flash mobs for SNS, etc., but there are restrictions on the cost, time, and place for receiving professional guidance. Solving such problems of users and providing an environment where they can easily enjoy dancing is an object of this invention. Also, it has high marketability in that it is useful as an exercise for school education where dancing is made compulsory, maintaining the health of the elderly, and dieting.

Means for Solving the Problems

[0005] It should be noted that there seems to be an error in the original text where "X" is added in the translation of line 35. If this is not an intended addition, please correct the original text for a more accurate translation.This invention solves a predetermined problem by combining the following means. First, it provides a system that includes means for receiving information from a user regarding dance style, age, physical fitness, and preferences; means for analyzing the received information and generating original dance videos and music based on an existing dance database; and means for transmitting the generated original dance videos and music to a user terminal. It also provides a system that includes means for acquiring motion data in real time through a wearable device worn by the user, analyzing the acquired motion data and comparing it with the generated original dance video to identify problems, providing guidance and feedback to the user based on the identified problems, and guiding the user to correct their movements through the wearable device. Furthermore, it provides a system that includes means for receiving individual dance style and preference information from multiple users, generating an integrated dance performance as a group, assigning individual choreography parts to each member, acquiring motion data from each member in real time, analyzing the performance of the entire group, providing feedback to the entire group and individual members, and supporting performance improvement.

[0006] "User information" refers to data entered by the user regarding their dance style, age, physical fitness, and preferences.

[0007] "Receiving means" refers to the function that acquires user information and performs data communication between devices.

[0008] "Analysis means" refers to a function that selects dance moves and music data based on received user information and generates original dances.

[0009] "Generation means" refers to the function that creates original dance videos and music using data selected by the analysis means.

[0010] "Transmission means" refers to the function that transfers the original dance video and music created by the generation means to the user's device.

[0011] A "wearable device" refers to a device that is attached to a user's forehead, abdomen, wrists, or ankles to acquire motion data in real time.

[0012] "Motion data" refers to data about the user's body movements acquired by a wearable device.

[0013] "Comparison method" refers to a function that evaluates acquired motion data by comparing it with the generated original dance.

[0014] "Identification means" refers to a function that uses comparison means to identify problems in the user's operation.

[0015] A "feedback mechanism" refers to a function that provides users with instructions for correction or advice based on identified problems.

[0016] "Guidance means" refers to a function that guides the user through a wearable device to perform the correct actions based on instructions from feedback means.

[0017] "Group information" refers to data about individual dance styles and preferences entered by multiple users.

[0018] "Integration means" refers to a function that consolidates group information and generates a unified dance performance.

[0019] "Role-sharing means" refers to the function of distributing the performance generated by the integration means to each member.

[0020] "Group analysis means" refers to a function that integrates operational data from the entire group and evaluates its performance.

[0021] "Group feedback means" refers to a function that provides improvement instructions to the entire group and individual members based on the results of group analysis means. [Brief explanation of the drawing]

[0022] [Figure 1] It is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] It is a conceptual diagram showing an example of the main functions of a data processing device and a smart device according to the first embodiment. [Figure 3] It is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] It is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] It is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] It is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] It is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] It is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] It shows an emotion map to which a plurality of emotions are mapped. [Figure 10] It shows an emotion map to which a plurality of emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Example 2 when an emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when an emotion engine is combined.

Embodiments for Carrying Out the Invention

[0023] Hereinafter, an example of an embodiment of the system relating to the technology of this disclosure will be described with reference to the attached drawings.

[0024] First, let's explain the terminology used in the following explanation.

[0025] In the following embodiments, the signed processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Furthermore, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include CPU (Central Processing Unit), GPU (Graphics Processing Unit), GPGPU (General-Purpose computing on Graphics Processing Units), and APU (Accelerated Processing Unit).

[0026] In the following embodiments, signed RAM (Random Access Memory) is a memory that temporarily stores information and is used as work memory by the processor.

[0027] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0028] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0029] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0030] [First Embodiment]

[0031] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0032] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0033] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0034] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0035] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0036] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0037] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0038] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0039] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0040] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0041] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0042] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0043] This invention is a system that generates original dances tailored to the user by allowing the user to communicate their dance style, image, age, physical ability, and preferences to an AI, and then supports their practice and instruction. The system is configured as follows:

[0044] 1. Creating an original dance

[0045] User:

[0046] Users launch the app on their smartphones, tablets, PCs, or other devices and enter information such as their dance style and image, age, physical condition, and preferences through an interface such as a chat window. Once they have finished entering the information, they press the submit button.

[0047] Terminal:

[0048] The terminal receives information entered by the user, formats the data, and sends it to the server.

[0049] server:

[0050] The server analyzes the received data and searches its database for dance moves and music that match the specified dance style and conditions. Using an AI model, it integrates these elements to generate an original dance video and music. The generated video and music are then sent to the terminal.

[0051] Specific example

[0052] For example, if a user requests "a jazz dance style, intermediate level, aimed at young people in their 20s, and with an upbeat tempo," the device sends this information to the server. The server uses an AI model to extract elements of jazz dance and generate original choreography and music. The generated jazz dance video and music are sent to the user's device and presented to them.

[0053] 2. Support for dance practice

[0054] User:

[0055] When practicing choreography, users wear AI-enabled wearable devices on their forehead, abdomen, both wrists, and both ankles. Next, they select a dance practice mode in the app and start playing the dance.

[0056] Terminal:

[0057] The device works in conjunction with a wearable device to collect user movement data in real time. It also uses the device's camera to record the user's dance. The collected data and video are sent to a server.

[0058] server:

[0059] The server analyzes the received data and video, compares them to the generated original dance, and identifies problems. Based on the identified problems, it generates feedback to the user, providing specific instructions on how to correct them, and sends it to the terminal.

[0060] Terminal:

[0061] The device displays feedback to the user and provides support to correct actions according to the instructions. It also uses wearable devices such as vibration to guide the user in correcting their actions.

[0062] Specific example

[0063] The system uses a wearable device and camera worn by the user to acquire movement data from the jazz dance they are practicing. The data sent to the server is analyzed by AI to identify problems such as insufficient hip movement. As a result, the device provides the user with specific feedback such as, "Let's move our hips a little more."

[0064] 3. Creating and teaching dances with multiple people

[0065] User:

[0066] If multiple people wish to dance together, each member must launch the app and set themselves to belong to the same group. All members then enter their dance request information.

[0067] Terminal:

[0068] The information of each user is converted into a data packet and sent to the server along with the group ID.

[0069] server:

[0070] The server generates an integrated, original dance based on information from the entire group. It then determines individual choreography parts for each member and generates corresponding choreography and music for each. The generated data is then sent to the terminal.

[0071] Terminal:

[0072] Each member's device displays their individual choreography part and music. Wearable devices are used to collect real-time movement data from each member and send it to a server.

[0073] server:

[0074] The server analyzes the operational data of multiple users and evaluates the performance of the group as a whole and each individual. Based on the evaluation results, it generates individual and group-wide feedback and sends it to the terminal.

[0075] Terminal:

[0076] The terminal displays feedback to each member and supports performance correction. Additionally, wearable devices are used to guide the correction process as needed.

[0077] Specific example

[0078] When a group of five requests a "spectacular and energetic dance for a wedding," information about each member is sent to the server. The server generates a cohesive dance and assigns individual choreography and music. Each member's movement data is analyzed, and improvement suggestions are provided for the group as a whole and for individual members. This entire process allows users to efficiently master the dance.

[0079] The following describes the processing flow.

[0080] 1. Creating an original dance

[0081] Step 1:

[0082] User:

[0083] Launch the app and enter information such as dance style, image, age, physical condition, and preferences into the chat window.

[0084] Step 2:

[0085] Terminal:

[0086] The system receives the input information, formats it into a data packet, and sends it to the server.

[0087] Step 3:

[0088] server:

[0089] The system analyzes received data packets and searches a database for dance styles and music genres based on the input information.

[0090] Step 4:

[0091] server:

[0092] Using an AI model, original dance videos and music are generated based on search results.

[0093] Step 5:

[0094] server:

[0095] The generated original dance video and music are sent to the device.

[0096] Step 6:

[0097] Terminal:

[0098] The received original dance video and music are displayed and played on the user interface.

[0099] 2. Support for dance practice

[0100] Step 1:

[0101] User:

[0102] AI-enabled wearable devices are attached to the head, abdomen, both wrists, and both ankles.

[0103] Select dance practice mode in the app.

[0104] Step 2:

[0105] Terminal:

[0106] Wearable devices are connected via Bluetooth or other communication methods to acquire operational data in real time.

[0107] The device's camera records the user's dance.

[0108] Step 3:

[0109] Terminal:

[0110] The collected motion data and recorded video are bundled into data packets and sent to the server.

[0111] Step 4:

[0112] server:

[0113] The received motion data and recorded video are analyzed using an AI model, and then compared to the generated original dance.

[0114] Step 5:

[0115] server:

[0116] Identify operational problems and generate specific correction instructions and advice.

[0117] Step 6:

[0118] server:

[0119] The generated feedback information is sent to the device.

[0120] Step 7:

[0121] Terminal:

[0122] Display feedback information in the user interface.

[0123] Wearable devices are used to provide guidance through vibrations and other means.

[0124] 3. Creating and teaching dances with multiple people

[0125] Step 1:

[0126] User:

[0127] Each member launches the app and sets themselves to belong to the same group.

[0128] Enter your dance style, image, and performance preferences.

[0129] Step 2:

[0130] Terminal:

[0131] The input information of each member is converted into a data packet and sent to the server along with the group ID.

[0132] Step 3:

[0133] server:

[0134] Based on information from the entire group, it generates an original dance that conveys a sense of unity.

[0135] Assign choreography parts to each member.

[0136] Step 4:

[0137] server:

[0138] The generated dance video, music, and choreography parts are sent to the device.

[0139] Step 5:

[0140] Terminal:

[0141] The system displays the choreography part and music assigned to each member on their device.

[0142] The system connects to wearable devices to acquire real-time motion data and transmits it to a server.

[0143] Step 6:

[0144] server:

[0145] Analyze the motion data of multiple individuals to evaluate the performance of the group as a whole and individual members.

[0146] Step 7:

[0147] server:

[0148] Generate and send group-wide and individual feedback to the device.

[0149] Step 8:

[0150] Terminal:

[0151] Provide feedback to each member and offer suggestions for revisions.

[0152] Provide guidance via vibration from wearable devices as needed.

[0153] Through these steps, users can efficiently learn dance and improve their performance.

[0154] (Example 1)

[0155] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0156] In recent years, the demand for dance has been increasing as a way to maintain health and as a form of entertainment, but finding a dance style and choreography that suits individual users is difficult. Furthermore, obtaining individualized feedback requires a professional instructor, which is time-consuming and costly. Additionally, efficient practice of dance performances by multiple people requires coordination and appropriate instruction for each member. There is a need for a system that can solve these problems and enable users to learn dance efficiently.

[0157] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0158] In this invention, the server includes means for receiving information from the user regarding dance style, age, physical fitness, and preferences; means for generating original dance videos and music based on an existing dance database using a generative AI model; and means for transmitting the generated original dance videos and music to the user terminal. This makes it possible to automatically provide the user with the most suitable dance style and choreography.

[0159] Furthermore, this invention includes a server that includes means for acquiring motion data in real time through a wearable device worn by the user, means for analyzing the acquired motion data and identifying problems by comparing it with the generated original dance video, means for providing guidance and feedback to the user based on the identified problems, and means for guiding the user to correct their movements through the wearable device. As a result, the user can receive individualized feedback in real time, enabling efficient dance practice.

[0160] Furthermore, this invention includes a server that receives individual dance style and preference information from multiple users and generates an integrated dance performance as a group; a server that assigns individual choreography parts to each member; a server that acquires movement data of each member in real time and analyzes the performance of the entire group; and a server that provides feedback to the entire group and individual members to support performance improvement. This enables multiple users to efficiently learn group dance performances.

[0161] "Dance style" refers to a type or form of dance based on a specific type of music or culture. Examples include jazz dance, hip hop, and ballet.

[0162] "Age" refers to the number of years elapsed since the user's birth. It is a factor that can be used as a reference for health status and physical fitness.

[0163] "Physical strength" refers to the user's physical abilities, such as physical endurance and muscle strength.

[0164] "Preferences" refer to personal tastes such as the dance style or type of music that a user particularly enjoys.

[0165] A "generative AI model" refers to a model that uses artificial intelligence technology to generate and analyze data. Examples include deep learning models and natural language processing models.

[0166] A "dance database" refers to a database containing data on various dance styles, choreography, and music.

[0167] An "original dance video" refers to a unique dance video generated by a generative AI model based on the user's requests.

[0168] "Music" refers to the music tracks used to accompany the dance. These are also selected based on the user's preferences and requests.

[0169] A "user terminal" refers to a device used by a user to operate the dance system. Examples include smartphones, tablets, and PCs.

[0170] A "wearable device" refers to a device that a user wears on their body. It is used to acquire motion data in real time.

[0171] "Motion data" refers to data about the user's body movements acquired through wearable devices.

[0172] "Feedback" refers to advice and suggestions for improvement provided by the server to the user.

[0173] "Choreography part" refers to the individual dance movements and roles assigned to each member in a dance performance involving multiple people.

[0174] A "group" refers to a team or collective formed by multiple users to perform dances together.

[0175] "Performance" refers to a dance demonstration by a user or group.

[0176] This invention is a system that utilizes specific hardware and software to allow users to communicate information such as their dance style and image, age, physical fitness, and preferences to an AI. Based on this information, the AI ​​generates original dances and further supports practice and instruction. Each element of this system is described in detail below.

[0177] Creating original dances

[0178] User:

[0179] Users launch a dedicated application on their smartphone, tablet, or PC. Using the application's chat window or form, they enter their desired dance style (e.g., jazz dance), level (e.g., intermediate), target age group (e.g., 20s), and musical preference (e.g., uptempo). Once they have finished entering the information, they press the submit button.

[0180] Terminal:

[0181] The terminal receives information entered by the user, converts it to a specified data format, and sends it to the server. During this process, the terminal uses an internet connection to transmit data in real time.

[0182] server:

[0183] The server analyzes the information sent from the user's device. Based on the analyzed information, it searches the database for choreography and music suitable for the specified dance style and conditions. Using a generative AI model (e.g., a deep learning model), it integrates these elements to generate an original dance video and music. The generated dance video and music are then sent to the user's device.

[0184] Specific example

[0185] For example, if a user requests "a jazz dance style, intermediate level, suitable for young people in their 20s, and they like upbeat music," the device sends this information to the server. The server uses an AI model to extract elements of jazz dance and generates original choreography and music. This generated jazz dance video and music are sent to the user's device and presented to the user.

[0186] Support for dance practice

[0187] User:

[0188] When practicing dance choreography, the user wears an AI-enabled wearable device (e.g., a smart band) on their forehead, abdomen, wrists, and ankles. Afterward, they select dance practice mode in the app and play a dance video.

[0189] Terminal:

[0190] The device works in conjunction with a wearable device to collect user movement data in real time. It also uses the device's camera to record the user's dance. This collected data and video are then sent to a server.

[0191] server:

[0192] The server analyzes the received motion data and video, comparing them to the generated original dance to identify problems. Based on the problems identified using the AI ​​model, it generates specific instructions and feedback for the user. The feedback is sent to the user's device.

[0193] Terminal:

[0194] The device displays feedback received from the server to the user. It also uses wearable devices to guide the user in correcting their actions through vibrations and other means.

[0195] Specific example

[0196] For example, when a user is practicing jazz dance, a wearable device and camera are used to collect motion data. The data sent to the server is analyzed by AI, which identifies specific problems such as "insufficient hip movement." As a result, the device displays feedback to the user, such as "move your hips a little more."

[0197] Dance creation and instruction by multiple people

[0198] User:

[0199] When multiple people dance together, each member launches the app and sets themselves to belong to the same group. Each member then enters their own dance request information.

[0200] Terminal:

[0201] The information of each user is converted into a data packet and sent to the server along with the group ID.

[0202] server:

[0203] The server generates an integrated, original dance based on information from the entire group. It then determines individual choreography parts for each member and generates corresponding choreography and music. The generated dance data is then sent to each terminal.

[0204] Terminal:

[0205] Each member's device displays their individual choreography part and music. Furthermore, it works in conjunction with wearable devices to collect real-time motion data from each member and send it to the server.

[0206] server:

[0207] The server analyzes the operational data of multiple users and evaluates the performance of the group as a whole and each individual. Based on the evaluation results, it generates feedback and sends it to each terminal.

[0208] Terminal:

[0209] The device displays feedback to each member and supports performance correction. It also uses wearable devices to guide the correction process as needed.

[0210] Specific example

[0211] For example, if a group of five requests a "spectacular and energetic dance for a wedding," information about each member is sent to the server. The server generates a cohesive dance and assigns individual choreography and music. Each member's movement data is analyzed, and improvement suggestions are provided for the group as a whole and for individual members. This entire process allows users to efficiently master the dance.

[0212] Example of a prompt

[0213] For example, the prompt message if the user wants to do jazz dance would be as follows:

[0214] "Please create an intermediate-level jazz dance routine for young people in their 20s. It should be to an upbeat song."

[0215] Furthermore, the prompt message for when multiple people are involved is as follows:

[0216] "Please create a vibrant and energetic dance for a group of five to perform at a wedding."

[0217] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0218] Step 1:

[0219] Users launch a dedicated app on their smartphone, tablet, or PC.

[0220] Input: Users enter their desired dance style, level, target age group, and musical preferences in the chat window or form, and then press the submit button.

[0221] Output: The input data is received by the terminal.

[0222] Step 2:

[0223] After receiving information entered by the user, the terminal converts the data into a specified format.

[0224] Input: User input data.

[0225] Data processing: Conversion to data format.

[0226] Output: The converted data is sent from the terminal to the server.

[0227] Step 3:

[0228] The server receives data sent from the user's terminal and begins analysis.

[0229] Input: Converted data.

[0230] Data processing: The system analyzes incoming data and searches the database for choreography and music suitable for the specified dance style and conditions. A generative AI model is used to integrate these elements and generate original dance videos and music.

[0231] Output: The generated dance video and music are sent from the server to the user's device.

[0232] Step 4:

[0233] The user's device plays the dance video and music received from the server and presents them to the user.

[0234] Input: Generated dance video and music.

[0235] Data processing: Convert to a data format for playback (if necessary).

[0236] Output: The user watches the original dance video and listens to the music.

[0237] Step 5:

[0238] Users wear an AI-powered wearable device while practicing dance choreography.

[0239] Input: Wearing a wearable device and selecting a dance practice mode in the app.

[0240] Output: The wearable device is ready to collect motion data.

[0241] Step 6:

[0242] The device works in conjunction with a wearable device to collect user movement data in real time. It also records the user's dance using the device's camera.

[0243] Input: User action data and video.

[0244] Data processing: Acquisition of motion data and video, and format conversion.

[0245] Output: The collected data and video are sent to the server.

[0246] Step 7:

[0247] The server analyzes the received motion data and video, compares it to the generated original dance, and identifies any problems.

[0248] Input: Collected motion data and video.

[0249] Data processing: Comparison of motion data with generated dance videos. Identification and analysis of problems.

[0250] Output: Specific guidance and feedback based on the problems are generated and sent to the user's device.

[0251] Step 8:

[0252] The device displays feedback received from the server to the user. It also uses wearable devices to guide the user in correcting their actions through vibrations and other means.

[0253] Input: Feedback data and instruction content.

[0254] Data processing: Display of feedback and vibration control.

[0255] Output: The feedback is displayed to the user, and guidance is provided for correcting the behavior.

[0256] Step 9:

[0257] When dancing with multiple people, each member launches the app and sets themselves to belong to the same group.

[0258] Input: Dance request information for each member.

[0259] Output: Information about each member is sent from the terminal to the server.

[0260] Step 10:

[0261] The server generates an integrated, original dance based on information from the entire group. It then determines individual choreography parts for each member and generates the corresponding choreography and music.

[0262] Input: Information for each member.

[0263] Data processing: Information analysis of the entire group, generation of integrated dance, and determination of individual choreography parts.

[0264] Output: The generated dance data is sent from the server to each terminal.

[0265] Step 11:

[0266] Each member's device displays their individual choreography part and music. Furthermore, it works in conjunction with wearable devices to collect real-time motion data from each member and send it to the server.

[0267] Input: Choreography data and motion data.

[0268] Data processing: Data display and collection and transmission of operational data.

[0269] Output: Operation data is sent to the server.

[0270] Step 12:

[0271] The server analyzes the operational data of multiple users and evaluates the performance of the group as a whole and each individual. Based on the evaluation results, it generates feedback and sends it to each terminal.

[0272] Input: Action data for each member.

[0273] Data processing: Analysis and evaluation of operational data, and generation of feedback.

[0274] Output: Feedback is sent to each device.

[0275] Step 13:

[0276] The device displays feedback to each member and supports performance correction. It also uses wearable devices to guide the correction process as needed.

[0277] Input: Feedback data.

[0278] Data processing: Controlling the display of feedback and correction guidelines.

[0279] Output: Feedback is displayed, and support is provided for correcting the behavior.

[0280] (Application Example 1)

[0281] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0282] While many factories currently utilize robots and automation technologies, efficiency can decrease if their operation patterns are not optimized. Furthermore, optimizing overall work efficiency is challenging when multiple robots and workers collaborate. These challenges need to be addressed because they lead to decreased productivity and increased costs.

[0283] The specific processing by the specific processing unit 290 of the data processing apparatus 12 in Application Example 1 is realized by the following means.

[0284] In this invention, the server includes means for receiving from a user information on the factory environment and the characteristics of a robot, means for using a generation AI model for generating a factory environment and a robot operation pattern based on the received information, and means for transmitting the generated operation pattern to the robot. Thereby, it becomes possible to optimize the operation patterns of robots and workers in a factory and improve the overall work efficiency.

[0285] A "user" is a person or organization that uses the system and is the entity that inputs information and gives operation instructions.

[0286] A "dance style" refers to the form of body movements and choreography in accordance with specific music and rhythms, and is selected based on the user's preferences.

[0287] "Age" refers to the number of years calculated from the user's date of birth and is a criterion for adjusting the difficulty level of dancing and movements.

[0288] "Physical strength" refers to the exercise ability and physical ability of the user and affects the intensity and duration of dancing and movements.

[0289] [[ID=三]] "Preferences" are information indicating the user's preferences and interests and are used for customizing the generated dance and operation patterns.

[0290] "Means for receiving information" refers to an interface or device for receiving input data from a user and analyzing it.

[0291] A "generation AI model" refers to an artificial intelligence algorithm or framework that automatically generates an optimal dance or operation pattern based on the input data.

[0292] A "database" refers to a system that systematically stores existing data and information, and allows for efficient retrieval and utilization of that data.

[0293] An "original dance video" refers to video content that includes unique dance choreography generated based on user input.

[0294] "Music" refers to a combination of sounds used in conjunction with dance or movement, and is an integrated form of sound data containing various elements.

[0295] A "user terminal" refers to devices such as smartphones, tablets, and PCs that users directly operate to input information or receive feedback.

[0296] A "wearable device" refers to a device that a user wears on their body to collect motion data in real time and transmit it to a system.

[0297] "Real-time" refers to a state where data is acquired and processed almost instantly, and information is updated without delay.

[0298] A "motion pattern" refers to a sequence of movements optimized to achieve a specific purpose.

[0299] A "robot" is an automated mechanical device designed to perform a specified task or action.

[0300] "Feedback" refers to improvement suggestions and correction instructions provided to users or robots based on analyzed data.

[0301] "Factory environment" refers to the physical conditions, layout, temperature, humidity, and other factors of the site where the robot operates.

[0302] This invention is a system for optimizing the motion patterns of robots in a factory and improving overall work efficiency. The embodiments for carrying out this invention are described below.

[0303] Users input information about the factory environment and robot characteristics using devices such as smartphones, tablets, and PCs. For example, they input information such as the factory's temperature, humidity, layout, robot type, speed, and accuracy. This information is then transmitted from the device to the server.

[0304] The server analyzes the received information and uses a generative AI model to generate factory environment and robot operation patterns. The generative AI model automatically generates the optimal operation pattern based on the input data. This operation pattern is a sequence designed to allow the robot to operate with maximum efficiency.

[0305] The generated motion patterns are sent from the server to the robot. The robot performs the task according to the received motion pattern. Motion data from each robot is collected in real time and sent back to the server. The server analyzes this motion data and generates feedback to improve efficiency. This feedback is provided to the user terminal and the robot, allowing for modifications and improvements to the robot's movements.

[0306] For example, if a user specifies that the factory temperature is 22°C, humidity is 40%, layout is Type A, the robot is an arm type, and it operates at medium speed and high precision, the AI ​​model will generate an optimal motion pattern based on these conditions. The generated motion pattern is sent to the robot as a command sequence such as "move_arm_to_XYZ, pick_and_place, verify_and_adjust". The robot operates according to this command sequence, and the server analyzes the collected motion data and provides feedback with correction instructions as needed.

[0307] This series of processes optimizes the operation of robots in the factory, making it possible to improve overall work efficiency.

[0308] The hardware used includes user terminals (smartphones, tablets, PCs), servers, and factory robots. The software used includes a generative AI model (implemented with Keras / TENSORFLOW (registered trademark)), Python scripts for data analysis, and a front-end application for the user interface.

[0309] Examples of prompt sentences:

[0310] "The temperature in the factory is 22°C, the humidity is 40%, the layout is Type A, the robot is an arm type, medium speed, high precision"

[0311] As a result, the system can generate an optimal robot operation pattern suitable for the factory environment and improve work efficiency.

[0312] The flow of specific processing in Application Example 1 will be described using FIG. 12.

[0313] Step 1:

[0314] The user uses a terminal such as a smartphone, tablet, or PC to input information about the factory environment and the characteristics of the robot. Specifically, details such as the temperature, humidity, layout, type of robot, speed, and accuracy of the factory are input, and the send button is pressed. The input is in the following format: "The temperature in the factory is 22°C, the humidity is 40%, the layout is Type A, the robot is an arm type, medium speed, high precision"

[0315] Step 2:

[0316] The terminal formats the information about the factory environment and the characteristics of the robot received from the user and sends it to the server. Here, the terminal converts the input data into JSON format and sends it to the server through the network. This is a data transmission operation, where the input is the user's input information and the output is the data transmission to the server.

[0317] Step 3:

[0318] The server analyzes the received information. Specifically, the server loads a generative AI model for analyzing factory environment and robot characteristic information, and supplies the received data to the generative AI model as input data. The AI ​​model generates the optimal operation pattern based on this data. The input is JSON data received from the terminal, and the output is the optimal operation pattern.

[0319] Step 4:

[0320] The server sends the generated motion patterns to the robots. These generated motion patterns are formatted as motion sequences for each robot and applied to specific robots within the factory. Specifically, the server sends the optimal motion pattern to the robot control unit. The input is the motion pattern generated from the AI ​​model, and the output is the transmission of the motion pattern to the robot.

[0321] Step 5:

[0322] The robots perform tasks according to motion patterns received from the server. Each robot executes the specified movements based on the received sequence. This allows the robots to efficiently perform specific work processes within the factory. The input is the motion pattern received from the server, and the output is the actual robot movement.

[0323] Step 6:

[0324] Robot motion data is collected in real time and transmitted to a server. The robot uses its built-in sensors and monitoring system to record motion data and transmit it to the server. Input is data obtained from the robot's motion sensors, and output is data transmission to the server.

[0325] Step 7:

[0326] The server analyzes the received robot motion data. Specifically, a data analysis module within the server analyzes the motion data and generates feedback necessary to improve the efficiency of factory work. The input is the motion data received from the robot, and the output is the feedback for efficiency improvement.

[0327] Step 8:

[0328] The server provides the generated feedback to the user terminal and the robot. The server sends the feedback to the user terminal, presenting the user with specific suggestions for improvement. Similarly, feedback is sent to the robot, allowing for modifications and adjustments to its operation. The input is the server's analysis results, and the output is the transmission of feedback to the user terminal and the robot.

[0329] By executing the above steps sequentially, the operation of factory robots is optimized, resulting in improved overall work efficiency.

[0330] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0331] This invention is a system that generates original dances tailored to the user by allowing the user to communicate their dance style, image, emotions, age, physical condition, and preferences to an AI, and then supports their practice and instruction. In addition to this system, an emotion engine that recognizes the user's emotions is incorporated to achieve more advanced feedback and customization. It is configured as follows:

[0332] 1. Creating an original dance

[0333] User:

[0334] Users launch the app on their smartphones, tablets, PCs, or other devices and input information such as their dance style, image, emotions, age, physical condition, and preferences through an interface such as a chat window. Once they have finished inputting the information, they press the submit button.

[0335] Terminal:

[0336] The terminal receives information entered by the user, formats the data, and sends it to the server.

[0337] server:

[0338] The server analyzes the received data and searches its database for dance moves and music that are suitable for the specified dance style, conditions, and the user's emotions. Using an AI model, it integrates these elements to generate an original dance video and music. The generated video and music are then sent to the device.

[0339] Specific example

[0340] For example, if a user requests "a jazz dance style, a fun mood, intermediate level, suitable for young people in their 20s, and they like upbeat music," the device sends this information to the server. The server uses an AI model and emotion engine to generate choreography and music that reflect the elements of jazz dance and a "fun" atmosphere. The generated jazz dance video and music are sent to the user's device and presented to the user.

[0341] 2. Support for dance practice

[0342] User:

[0343] When practicing choreography, users wear AI-enabled wearable devices on their forehead, abdomen, both wrists, and both ankles. Next, they select a dance practice mode in the app and start playing the dance.

[0344] Terminal:

[0345] The device works in conjunction with a wearable device to collect user movement data in real time. It also uses the device's camera to record the user's dance. The collected data and video are sent to a server.

[0346] server:

[0347] The server analyzes the received data and video, comparing them to the generated original dance to identify problems. Furthermore, it uses an emotion engine to analyze the user's emotions during practice in real time and adaptively adjust the feedback. Based on the identified problems and the emotion analysis results, it generates feedback to specifically instruct the user on how to correct them and sends it to the terminal.

[0348] Terminal:

[0349] The device displays feedback to the user and provides support to correct actions according to the instructions. It also uses wearable devices such as vibration to guide the user in correcting their actions.

[0350] Specific example

[0351] The system uses a wearable device and camera worn by the user to acquire movement data from the jazz dance they are practicing. The data sent to the server is analyzed by AI and an emotion engine to identify problems such as "insufficient hip movement" and emotions such as "feeling frustrated." As a result, the device provides the user with specific feedback such as "move your hips a little more" and emotionally responsive advice such as "relax and enjoy yourself."

[0352] 3. Creating and teaching dances with multiple people

[0353] User:

[0354] If multiple people wish to dance together, each member must launch the app and set themselves to belong to the same group. All members then enter their dance request information and emotional information.

[0355] Terminal:

[0356] The information of each user is converted into a data packet and sent to the server along with the group ID.

[0357] server:

[0358] The server generates an integrated, original dance based on information from the entire group. It then determines individual choreography parts for each member and generates corresponding choreography and music for each. The generated data is then sent to the terminal.

[0359] Terminal:

[0360] Each member's device displays their individual choreography part and music. Wearable devices are used to collect real-time movement data from each member and send it to a server.

[0361] server:

[0362] The server analyzes the behavioral data of multiple individuals and evaluates the performance of the entire group and each individual. Furthermore, it uses an emotion engine to analyze the emotional state of the entire group and generates emotion-based feedback. Based on the evaluation results and emotion analysis results, it generates individual and group-wide feedback and sends it to the terminal.

[0363] Terminal:

[0364] The device displays feedback to each member, supporting performance improvements and emotional enhancement. It also uses wearable devices to guide behavioral corrections as needed.

[0365] Specific example

[0366] When a group of five requests a "spectacular and energetic dance for a wedding," information about each member is sent to the server. The server uses AI and an emotion engine to generate a cohesive dance, assigning individual choreography and music. The emotion engine analyzes each member's emotional state and provides feedback to ensure the entire group enjoys the performance together. Each member's movement data is analyzed, and improvement instructions are provided for the group as a whole and for individual members. Through this process, users can efficiently master the dance while also receiving emotional support.

[0367] The following describes the processing flow.

[0368] 1. Creating an original dance

[0369] Step 1:

[0370] User:

[0371] Launch the app and enter information such as dance style, image, emotions, age, physical condition, and preferences into the chat window.

[0372] Step 2:

[0373] Terminal:

[0374] The system receives the input information, formats it into a data packet, and sends it to the server.

[0375] Step 3:

[0376] server:

[0377] The system analyzes received data packets and searches a database for dance style, music genre, and emotional data based on the input information.

[0378] Step 4:

[0379] server:

[0380] Using AI models and an emotion engine, the system generates original dance videos and music based on search results.

[0381] Step 5:

[0382] server:

[0383] The generated original dance video and music are sent to the device.

[0384] Step 6:

[0385] Terminal:

[0386] The received original dance video and music are displayed and played on the user interface.

[0387] 2. Support for dance practice

[0388] Step 1:

[0389] User:

[0390] AI-enabled wearable devices are attached to the head, abdomen, both wrists, and both ankles.

[0391] Select dance practice mode in the app.

[0392] Step 2:

[0393] Terminal:

[0394] Wearable devices are connected via Bluetooth or other communication methods to acquire motion data and emotion data in real time.

[0395] The device's camera records the user's dance.

[0396] Step 3:

[0397] Terminal:

[0398] The collected motion data, emotion data, and recorded video are bundled into data packets and sent to the server.

[0399] Step 4:

[0400] server:

[0401] The received motion data, emotion data, and recorded video are analyzed using an AI model and emotion engine, and compared to the generated original dance.

[0402] Step 5:

[0403] server:

[0404] Identify operational problems, refine feedback based on real-time sentiment data, and generate specific corrective instructions and advice.

[0405] Step 6:

[0406] server:

[0407] The generated feedback information is sent to the device.

[0408] Step 7:

[0409] Terminal:

[0410] Display feedback information in the user interface.

[0411] Wearable devices are used to provide guidance through vibrations and other means.

[0412] 3. Creating and teaching dances with multiple people

[0413] Step 1:

[0414] User:

[0415] Each member launches the app and sets themselves to belong to the same group.

[0416] Enter your dance style, image, emotions, and performance preferences.

[0417] Step 2:

[0418] Terminal:

[0419] The input information of each member is converted into a data packet and sent to the server along with the group ID.

[0420] Step 3:

[0421] server:

[0422] Based on information from the entire group, it generates an original dance that conveys a sense of unity.

[0423] Assign choreography parts to each member.

[0424] Step 4:

[0425] server:

[0426] The generated dance video, music, and choreography parts are sent to the device.

[0427] Step 5:

[0428] Terminal:

[0429] The system displays the choreography part and music assigned to each member on their device.

[0430] The system connects to wearable devices to acquire motion and emotion data in real time and transmits it to a server.

[0431] Step 6:

[0432] server:

[0433] We analyze the behavioral and emotional data of multiple individuals to evaluate the performance of the group as a whole and individual members.

[0434] Step 7:

[0435] server:

[0436] It generates group-wide and individual feedback, further refines the feedback based on sentiment data, and sends it to the device.

[0437] Step 8:

[0438] Terminal:

[0439] Provide feedback to each member and offer suggestions for revisions.

[0440] Provide guidance via vibration from wearable devices as needed.

[0441] Through these steps, users can efficiently learn dance and receive support that takes their emotional state into consideration.

[0442] (Example 2)

[0443] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0444] Conventional dance generation and practice support systems failed to adequately reflect users' individual preferences and emotions, and their motion corrections and feedback during practice were not adapted to the user's emotional state. Furthermore, when generating dance performances for multiple people, there were insufficient methods for coordinating the group's overall choreography with individual choreography. In addition, real-time motion data collection and analysis lacked integration with emotion analysis.

[0445] The identification processing performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for receiving information from the user regarding dance style, age, physical strength, preferences, and emotions; means for analyzing the received information and generating original dance videos and music using a generation AI model based on an existing dance database; and means for transmitting the generated original dance videos and music to the user terminal. This makes it possible to generate original dance videos and music that reflect individual preferences and emotions.

[0446] The server also includes means for acquiring motion data in real time through a wearable device worn by the user; means for analyzing the acquired motion data and identifying problems by comparing it with the generated original dance video; means for analyzing the user's emotions using an emotion engine and generating feedback based on that; means for providing guidance and feedback to the user based on the identified problems and emotions; and means for guiding the user's motion correction through the wearable device. This makes it possible for motion correction and feedback during practice to be adapted to the user's emotional state.

[0447] Furthermore, the server includes means for receiving individual dance styles, preferences, and emotional information from multiple users to generate an integrated dance performance as a group; means for assigning individual choreography parts to each member; means for acquiring movement data of each member in real time and analyzing the overall performance of the group; and means for generating feedback to the group as a whole and to individual members using a generative AI model and an emotion engine to support performance improvement. This makes it possible to generate and improve dance performances involving multiple people.

[0448] "Dance style" refers to the type or form of dance desired by the user, and examples include jazz dance, ballet, and hip hop.

[0449] "Age" refers to the user's age group and is information used to determine appropriate dance choreography and exercise intensity.

[0450] "Physical fitness" refers to the user's physical ability and endurance, and serves as a criterion for adjusting the difficulty and intensity of the dance.

[0451] "Preferences" refer to a user's likes and interests, and include preferences such as favorite music genres and dance styles.

[0452] "Emotions" refers to the emotional state a user experiences while dancing, and includes feelings such as happiness, sadness, and relaxation.

[0453] A "wearable device" is a device worn and used by a user to acquire movement data in real time. Examples include smartwatches and fitness trackers.

[0454] A "generative AI model" is a model that uses artificial intelligence to analyze and generate data, and is used to generate original dance videos and music based on specific inputs.

[0455] An "emotion engine" is a system that analyzes a user's emotional state and provides optimal feedback and guidance based on that analysis.

[0456] "Feedback" refers to information that provides suggestions and advice for improving a user's actions and practice methods.

[0457] A "database" is a system that stores information on dance moves and music, and is used to search for and retrieve appropriate data based on user input.

[0458] An "original dance video" is a video of a custom dance, generated using an AI model and tailored to the user's requests.

[0459] "Original music" refers to custom music generated using AI models, tailored to the user's requests.

[0460] This invention is a system that generates personalized, original dances based on the user's dance style, image, emotions, age, physical condition, and preferences communicated to the AI, and supports practice and instruction. The system incorporates an emotion engine that recognizes the user's emotions in real time and reflects them in the feedback.

[0461] Creating original dances

[0462] User:

[0463] Users launch the app using their smartphone, tablet, or PC and enter information such as their dance style, image, emotions, age, fitness level, and preferences through the app's chat window. For example, they might enter information such as, "I like jazz dance, I'm in a fun mood, I'm intermediate level, I'm in my 20s, and I like upbeat music."

[0464] Terminal:

[0465] The terminal receives user input information, formats it into the required format, and sends it to the server. This formatting may involve using data formats such as JSON.

[0466] server:

[0467] The server analyzes the received data and searches its database for dance moves and music that are suitable for the specified dance style, conditions, and the user's emotions. A generative AI model is used for this, and an emotion engine is also utilized to generate original dance videos and music that are adapted to the user's emotions. The generated videos and music are then sent back to the terminal.

[0468] Support for dance practice

[0469] User:

[0470] When practicing dance choreography, users wear AI-enabled wearable devices on their forehead, abdomen, both wrists, and both ankles. Next, they launch the app, select dance practice mode, and start playing the dance.

[0471] Terminal:

[0472] The device works in conjunction with a wearable device to collect user movement data in real time. It also uses the device's camera to record the user's dance movements. The collected data and video are sent to a server.

[0473] server:

[0474] The server analyzes the received data and video, comparing them to the generated original dance to identify problems. Furthermore, it uses an emotion engine to analyze the user's emotions during practice in real time and generates feedback based on that analysis. Based on the identified problems and the emotion analysis results, it generates feedback to provide the user with specific instructions on how to correct their behavior and sends it to the terminal.

[0475] Dance creation and instruction by multiple people

[0476] User:

[0477] When multiple people dance together, each member launches the app and sets themselves to belong to the same group. Each member then enters dance request information and emotion information.

[0478] Terminal:

[0479] Each member's data is converted into a data packet and sent to the server along with the group ID.

[0480] server:

[0481] The server analyzes the received information from the entire group and generates an integrated original dance. It determines individual choreography parts for each member and generates corresponding choreography and music. The generated data is sent to each member's terminal.

[0482] Terminal:

[0483] Each member's device displays their individual choreography part and music, and in conjunction with wearable devices, movement data from each member is collected in real time and sent to a server.

[0484] server:

[0485] The server analyzes the behavioral data of multiple individuals and evaluates the performance of the group as a whole and each individual. Furthermore, it uses an emotion engine to analyze the emotional state of the entire group and generates feedback based on that analysis. This feedback is sent to each member's terminal.

[0486] Specific example

[0487] For example, if a user enters a prompt such as "Jazz dance style, fun mood, intermediate level, for people in their 20s, likes upbeat music," the device sends this to the server. The server uses a generative AI model and emotion engine to generate choreography and music that reflects the elements of jazz dance and a fun atmosphere based on this information. This is then sent to the user's device, and the user can view the content in the app.

[0488] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0489] Creating original dances

[0490] Step 1:

[0491] User: Users launch the app using a smartphone, tablet, or PC and enter information about their dance style, image, emotions, age, fitness level, and preferences into the app's chat window. Specifically, they might enter information such as, "I like jazz dance, I'm in a fun mood, I'm intermediate level, I'm in my 20s, and I like upbeat music." The input is detailed information about the user's preferences and emotions, and the output is the transmission of this information.

[0492] Step 2:

[0493] Terminal: The terminal receives user input, converts the data into a format such as JSON, and sends it to the server. The input is information entered by the user, and the output is formatted data. Specifically, it performs the operation of converting the text data entered by the user into an appropriate format.

[0494] Step 3:

[0495] Server: The server receives formatted data and uses a generating AI model to search and analyze a database for dance moves and music that are suitable for the specified dance style, conditions, and the user's emotions. The input is formatted data received from the terminal, and the output is the generated dance moves and music. The server uses the AI ​​model to generate dance choreography patterns and music based on text information.

[0496] Step 4:

[0497] Server: The server searches and integrates generated dances and music from a database to produce original dance videos and music. The input is the search and analysis results, and the output is the integrated original dance video and music. The server uses a generation AI model to integrate the elements and perform the operation of generating new content.

[0498] Step 5:

[0499] Server: Sends the generated original dance video and music to the user's device. The input is the generated content, and the output is the data sent to the device.

[0500] Step 6:

[0501] Terminal: The terminal displays received dance videos and music to the user. Input is data sent from the server, and output is what is displayed to the user.

[0502] Support for dance practice

[0503] Step 1:

[0504] User: The user attaches AI-enabled wearable devices to their forehead, abdomen, both wrists, and both ankles. Next, they launch the app, select dance practice mode, and start playback. The input is the original dance video and the attached wearable devices, and the output is the start of practice mode.

[0505] Step 2:

[0506] Terminal: The terminal works in conjunction with a wearable device to collect user movement data in real time. It also uses the terminal's camera to record video of the user dancing. The input is user movement data and video, and the output is the collected real-time data.

[0507] Step 3:

[0508] Terminal: Sends collected data and video to the server. Inputs are real-time data and recorded video, and outputs are data sent to the server.

[0509] Step 4:

[0510] Server: The server analyzes the received data and video, compares them to the generated original dance, and identifies problems. Furthermore, it uses an emotion engine to analyze the user's emotions during practice in real time. The input is the received data and video, and the output is the problems and the emotion analysis results. The server performs the analysis using an AI model and an emotion engine.

[0511] Step 5:

[0512] Server: Based on the identified problems and sentiment analysis results, the server generates feedback to provide the user with specific instructions on how to correct them and sends it to the terminal. The input is the problems and analysis results, and the output is the generated feedback.

[0513] Step 6:

[0514] Terminal: The terminal displays feedback to the user and provides support to correct actions according to instructions. It also uses wearable devices to guide the user's actions through vibration, etc. Input is feedback received from the server, and output is the display and guidance to the user.

[0515] Dance creation and instruction by multiple people

[0516] Step 1:

[0517] User: When multiple people want to dance together, each member launches the app and sets themselves to belong to the same group. All members enter dance request information and emotion information. Input is group settings and request information, and output is setting complete and input complete.

[0518] Step 2:

[0519] Terminal: Converts each member's information into a data packet and sends it to the server along with the group ID. Input is information from group members, and output is a data packet.

[0520] Step 3:

[0521] Server: The server generates an integrated, original dance based on information from the entire group. It then determines individual choreography parts for each member and generates corresponding choreography and music for each. The input is multiple request information, and the output is the integrated dance and individual choreography data. The server uses an AI model and an emotion engine for generation.

[0522] Step 4:

[0523] Server: Sends the generated dance and music to each member's terminal. The input is the generated data, and the output is the data sent to the terminal.

[0524] Step 5:

[0525] Terminals: Each member's terminal displays their individual choreography part and music. Wearable devices are used to collect real-time motion data from each member and send it to the server. Inputs are received data and motion data, while outputs are data sent to the server.

[0526] Step 6:

[0527] Server: The server analyzes the received behavioral data and evaluates the performance of the entire group and individual members. Furthermore, it uses an emotion engine to analyze the emotional state of the entire group and generate feedback. Inputs are behavioral and emotion data, and outputs are evaluation results and feedback.

[0528] Step 7:

[0529] Terminal: The terminal displays feedback to each member, supporting performance correction and emotional improvement. It also guides behavioral correction using wearable devices as needed. Input is feedback received from the server, and output is the display and guidance to the user.

[0530] (Application Example 2)

[0531] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0532] Traditional dance practice systems and teaching methods have struggled to adequately customize and accommodate individual user preferences and emotions. Furthermore, the lack of systems capable of analyzing user movement data and emotional states in real time and providing adaptive feedback based on those results often resulted in inefficient practice instruction and inadequate emotional support. Additionally, in group dance practice, there was a lack of means to analyze the overall group performance and emotional state and provide integrated feedback.

[0533] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving information from the user regarding dance style, age, physical strength, preferences, and emotions; means for analyzing the received information and generating original dance videos and music based on an existing dance database and a generation AI model; means for transmitting the generated original dance videos and music to the user terminal; and means for analyzing the user's emotions and providing feedback based on the analysis results. This enables the generation of customized dances that correspond to the user's individual preferences and emotions, efficient practice guidance, and adaptive emotional support.

[0534] Furthermore, the system includes means for acquiring motion data in real time through a wearable device worn by the user, means for analyzing the acquired motion data and identifying problems by comparing it with the generated original dance video, means for providing guidance and emotion-responsive feedback to the user based on the identified problems and emotion analysis results, and means for guiding the user to correct their movements through the wearable device, thereby enabling the user to correct their movements and improve their skills.

[0535] Furthermore, by including means for receiving individual dance style, preference, and emotional information from multiple users to generate an integrated dance performance as a group, means for assigning individual choreography parts to each member, means for acquiring movement data of each member in real time and analyzing the overall performance and emotions of the group, and means for providing feedback to the group as a whole and to individual members to support performance improvement and emotional enhancement, efficient dance practice for multiple people and improvement of the overall group performance can be achieved.

[0536] A "user" is someone who provides the system with information about their dance style, age, physical fitness, preferences, and emotions, and receives generated original dance videos and music, as well as practice and instruction.

[0537] "Dance style" refers to information that describes the specific type or form of dance that the user desires.

[0538] "Age" refers to information indicating the user's actual age.

[0539] "Physical fitness" refers to information that indicates the user's physical ability and endurance.

[0540] "Preferences" refer to information that indicates an individual user's tastes and tendencies, such as the type of music or dance style they prefer.

[0541] "Emotions" refer to information that indicates the feelings and mental state that a user is currently experiencing.

[0542] A "generative AI model" is an artificial intelligence model used to generate original dance videos and music based on user input.

[0543] A "prompt message" is a statement that indicates the format and content of the information that the user should input into the system.

[0544] An "original dance video" is a video of a dance that has been individually customized and generated based on information provided by the user.

[0545] "Music" refers to the audio data corresponding to the generated original dance video.

[0546] A "user terminal" is a hardware device used by a user to receive and play original dance videos and music.

[0547] "Emotional analysis" is the process of analyzing a user's emotional state and incorporating the results into the feedback.

[0548] "Feedback" refers to information that provides guidance and advice based on the user's dance performance and emotional state.

[0549] A "wearable device" is a device that a user wears to acquire real-time motion data.

[0550] "Motion data" refers to data that shows the user's body movements and location information.

[0551] "Performance" refers to the technical skill and expressiveness of a user's dance moves.

[0552] A "group" refers to a collective of multiple users performing a dance together.

[0553] A "choreographed part" refers to the individual dance movements assigned to each member of the group.

[0554] One embodiment of this invention is a system that generates original dance videos and music customized based on the dance style, age, physical fitness, preferences, and emotions entered by the user, and supports their practice and instruction. Specifically, it involves the following processing steps.

[0555] 1. Collecting user input

[0556] Users launch the application using devices such as smartphones, tablets, or PCs. Through the application's interface, users input their dance style, age, physical fitness level, preferences, and current emotions. This information is sent to the server.

[0557] 2. Generation of original dance videos and music

[0558] The server analyzes the received user information. Using an existing dance database and a generative AI model, it generates original dance videos and music tailored to the user's preferences and emotions. This generated content is then sent to the user's device.

[0559] 3. Emotional analysis and provision of feedback

[0560] The server uses an emotion engine to analyze the emotional information entered by the user. Based on the analysis results, it provides appropriate feedback to the user. For example, if the emotion during practice is "frustration," it will provide feedback such as "relax and enjoy yourself."

[0561] 4. Acquisition of motion data using wearable devices

[0562] During practice, the user wears AI-enabled wearable devices on their forehead, abdomen, both wrists, and both ankles. The terminal interacts with these wearable devices to collect the user's movement data in real time. It also uses the terminal's camera to record the user's dance. This data is sent to a server.

[0563] 5. Comparison of motion data and the generated original dance video

[0564] The server analyzes the received motion data and video, compares it to the generated original dance, and identifies problems. Based on the identified problems, it generates feedback to instruct the user on how to correct them and sends it to the user's terminal.

[0565] 6. Creating and teaching dances with multiple people

[0566] When multiple users register as a group, each member enters their individual dance style, preferences, and emotional information into the application. This information is sent to the server, which generates an integrated original dance performance for the entire group. The server then assigns individual choreography parts to each member. In addition, it acquires movement data from each member in real time, analyzes the group's overall and individual performances, and provides feedback.

[0567] Examples

[0568] Possible prompt statements include the following:

[0569] Favorite dance style: Hip Hop

[0570] Current emotion: Happy

[0571] Dance level: Intermediate

[0572] Age: 25

[0573] Preferences: Upbeat music

[0574] Based on this prompt, the server generates a fun, intermediate-level hip-hop style dance. Users can practice the dance while receiving real-time feedback. If frustration is detected, advice such as "Relax and have fun" is also provided.

[0575] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0576] Step 1:

[0577] Users launch the application on their smartphones, tablets, or PCs and input information about their dance style, age, physical fitness, preferences, and emotions. The input data is as follows: "Dance Style: Hip Hop," "Emotion: Happy," "Dance Level: Intermediate," "Age: 25," "Preference: Upbeat Music." This input data is then sent from the device to the server.

[0578] Step 2:

[0579] The server analyzes the user's input data. Based on the input information, it uses an existing dance database and a generative AI model to generate an original dance video and music tailored to the user. The generated dance video and music files are sent from the server to the user's terminal. The output is a customized original dance video and music file.

[0580] Step 3:

[0581] The user wears AI-enabled wearable devices on their forehead, abdomen, both wrists, and both ankles, and selects a dance practice mode within the application. The terminal interacts with the wearable devices to collect the user's movement data in real time. This movement data includes information such as the position of each joint and the speed of movement. This movement data is transmitted from the terminal to the server.

[0582] Step 4:

[0583] The server analyzes the received motion data and compares it to the generated original dance video. Using an AI model, it compares the user's movements to accurate dance moves and identifies problems. For example, it might identify a problem such as "insufficient hip movement." The analysis results are stored on the server as feedback information.

[0584] Step 5:

[0585] The server uses an emotion engine to analyze the emotional information entered by the user. Based on the analysis results, it evaluates the user's emotional state during practice and generates adaptive feedback. For example, if the user is feeling "frustrated," it will generate feedback such as "relax and enjoy yourself." The output consists of the emotional state analysis result and the feedback statement.

[0586] Step 6:

[0587] Based on the identified problems and emotional state, the server generates specific instructions and emotionally responsive feedback for the user and sends them to the user's terminal. The terminal displays this to the user, providing instructions for correcting actions such as "Move your hips a little more" and emotional advice such as "Relax and enjoy yourself."

[0588] Step 7:

[0589] Based on the feedback received from the user, the dance is modified, and motion data is collected again. Then, by repeating the process from steps 3 to 6, the user's performance improves and emotional support is continuously provided.

[0590] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0591] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0592] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0593] [Second Embodiment]

[0594] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0595] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0596] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0597] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0598] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0599] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0600] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0601] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0602] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0603] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0604] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0605] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0606] This invention is a system that generates original dances tailored to the user by allowing the user to communicate their dance style, image, age, physical ability, and preferences to an AI, and then supports their practice and instruction. The system is configured as follows:

[0607] 1. Creating an original dance

[0608] User:

[0609] Users launch the app on their smartphones, tablets, PCs, or other devices and enter information such as their dance style and image, age, physical condition, and preferences through an interface such as a chat window. Once they have finished entering the information, they press the submit button.

[0610] Terminal:

[0611] The terminal receives information entered by the user, formats the data, and sends it to the server.

[0612] server:

[0613] The server analyzes the received data and searches its database for dance moves and music that match the specified dance style and conditions. Using an AI model, it integrates these elements to generate an original dance video and music. The generated video and music are then sent to the terminal.

[0614] Specific example

[0615] For example, if a user requests "a jazz dance style, intermediate level, aimed at young people in their 20s, and with an upbeat tempo," the device sends this information to the server. The server uses an AI model to extract elements of jazz dance and generate original choreography and music. The generated jazz dance video and music are sent to the user's device and presented to them.

[0616] 2. Support for dance practice

[0617] User:

[0618] When practicing choreography, users wear AI-enabled wearable devices on their forehead, abdomen, both wrists, and both ankles. Next, they select a dance practice mode in the app and start playing the dance.

[0619] Terminal:

[0620] The device works in conjunction with a wearable device to collect user movement data in real time. It also uses the device's camera to record the user's dance. The collected data and video are sent to a server.

[0621] server:

[0622] The server analyzes the received data and video, compares them to the generated original dance, and identifies problems. Based on the identified problems, it generates feedback to the user, providing specific instructions on how to correct them, and sends it to the terminal.

[0623] Terminal:

[0624] The device displays feedback to the user and provides support to correct actions according to the instructions. It also uses wearable devices such as vibration to guide the user in correcting their actions.

[0625] Specific example

[0626] The system uses a wearable device and camera worn by the user to acquire movement data from the jazz dance they are practicing. The data sent to the server is analyzed by AI to identify problems such as insufficient hip movement. As a result, the device provides the user with specific feedback such as, "Let's move our hips a little more."

[0627] 3. Creating and teaching dances with multiple people

[0628] User:

[0629] If multiple people wish to dance together, each member must launch the app and set themselves to belong to the same group. All members then enter their dance request information.

[0630] Terminal:

[0631] The information of each user is converted into a data packet and sent to the server along with the group ID.

[0632] server:

[0633] The server generates an integrated, original dance based on information from the entire group. It then determines individual choreography parts for each member and generates corresponding choreography and music for each. The generated data is then sent to the terminal.

[0634] Terminal:

[0635] Each member's device displays their individual choreography part and music. Wearable devices are used to collect real-time movement data from each member and send it to a server.

[0636] server:

[0637] The server analyzes the operational data of multiple users and evaluates the performance of the group as a whole and each individual. Based on the evaluation results, it generates individual and group-wide feedback and sends it to the terminal.

[0638] Terminal:

[0639] The terminal displays feedback to each member and supports performance correction. Additionally, wearable devices are used to guide the correction process as needed.

[0640] Specific example

[0641] When a group of five requests a "spectacular and energetic dance for a wedding," information about each member is sent to the server. The server generates a cohesive dance and assigns individual choreography and music. Each member's movement data is analyzed, and improvement suggestions are provided for the group as a whole and for individual members. This entire process allows users to efficiently master the dance.

[0642] The following describes the processing flow.

[0643] 1. Creating an original dance

[0644] Step 1:

[0645] User:

[0646] Launch the app and enter information such as dance style, image, age, physical condition, and preferences into the chat window.

[0647] Step 2:

[0648] Terminal:

[0649] The system receives the input information, formats it into a data packet, and sends it to the server.

[0650] Step 3:

[0651] server:

[0652] The system analyzes received data packets and searches the database for dance styles and music genres based on the input information.

[0653] Step 4:

[0654] server:

[0655] Using an AI model, original dance videos and music are generated based on search results.

[0656] Step 5:

[0657] server:

[0658] The generated original dance video and music are sent to the device.

[0659] Step 6:

[0660] Terminal:

[0661] The received original dance video and music are displayed and played on the user interface.

[0662] 2. Support for dance practice

[0663] Step 1:

[0664] User:

[0665] AI-enabled wearable devices are attached to the head, abdomen, both wrists, and both ankles.

[0666] Select dance practice mode in the app.

[0667] Step 2:

[0668] Terminal:

[0669] Wearable devices are connected via Bluetooth or other communication methods to acquire operational data in real time.

[0670] The device's camera records the user's dance.

[0671] Step 3:

[0672] Terminal:

[0673] The collected motion data and recorded video are bundled into data packets and sent to the server.

[0674] Step 4:

[0675] server:

[0676] The received motion data and recorded video are analyzed using an AI model, and then compared to the generated original dance.

[0677] Step 5:

[0678] server:

[0679] Identify operational problems and generate specific correction instructions and advice.

[0680] Step 6:

[0681] server:

[0682] The generated feedback information is sent to the device.

[0683] Step 7:

[0684] Terminal:

[0685] Display feedback information in the user interface.

[0686] Wearable devices are used to provide guidance through vibrations and other means.

[0687] 3. Creating and teaching dances with multiple people

[0688] Step 1:

[0689] User:

[0690] Each member launches the app and sets themselves to belong to the same group.

[0691] Enter your dance style, image, and performance preferences.

[0692] Step 2:

[0693] Terminal:

[0694] The input information of each member is converted into a data packet and sent to the server along with the group ID.

[0695] Step 3:

[0696] server:

[0697] Based on information from the entire group, it generates an original dance that conveys a sense of unity.

[0698] Assign choreography parts to each member.

[0699] Step 4:

[0700] server:

[0701] The generated dance video, music, and choreography parts are sent to the device.

[0702] Step 5:

[0703] Terminal:

[0704] The system displays the choreography part and music assigned to each member on their device.

[0705] The system connects to wearable devices to acquire real-time motion data and transmits it to a server.

[0706] Step 6:

[0707] server:

[0708] Analyze the motion data of multiple individuals to evaluate the performance of the group as a whole and individual members.

[0709] Step 7:

[0710] server:

[0711] Generate and send group-wide and individual feedback to the device.

[0712] Step 8:

[0713] Terminal:

[0714] Provide feedback to each member and offer suggestions for revisions.

[0715] Provide guidance via vibration from wearable devices as needed.

[0716] Through these steps, users can efficiently learn dance and improve their performance.

[0717] (Example 1)

[0718] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0719] In recent years, the demand for dance has been increasing as a way to maintain health and as a form of entertainment, but finding a dance style and choreography that suits individual users is difficult. Furthermore, obtaining individualized feedback requires a professional instructor, which is time-consuming and costly. Additionally, efficient practice of dance performances by multiple people requires coordination and appropriate instruction for each member. There is a need for a system that can solve these problems and enable users to learn dance efficiently.

[0720] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0721] In this invention, the server includes means for receiving information from the user regarding dance style, age, physical fitness, and preferences; means for generating original dance videos and music based on an existing dance database using a generative AI model; and means for transmitting the generated original dance videos and music to the user terminal. This makes it possible to automatically provide the user with the most suitable dance style and choreography.

[0722] Furthermore, this invention includes a server that includes means for acquiring motion data in real time through a wearable device worn by the user, means for analyzing the acquired motion data and identifying problems by comparing it with the generated original dance video, means for providing guidance and feedback to the user based on the identified problems, and means for guiding the user to correct their movements through the wearable device. As a result, the user can receive individualized feedback in real time, enabling efficient dance practice.

[0723] Furthermore, this invention includes a server that receives individual dance style and preference information from multiple users and generates an integrated dance performance as a group; a server that assigns individual choreography parts to each member; a server that acquires movement data of each member in real time and analyzes the performance of the entire group; and a server that provides feedback to the entire group and individual members to support performance improvement. This enables multiple users to efficiently learn group dance performances.

[0724] A "dance style" refers to a type or form of dance based on a specific type of music or culture. Examples include jazz dance, hip hop, and ballet.

[0725] "Age" refers to the number of years elapsed since the user's birth. It is a factor that can be used as a reference for health status and physical fitness.

[0726] "Physical strength" refers to the user's physical abilities, such as physical endurance and muscle strength.

[0727] "Preferences" refer to personal tastes such as the dance style or type of music that a user particularly enjoys.

[0728] A "generative AI model" refers to a model that uses artificial intelligence technology to generate and analyze data. Examples include deep learning models and natural language processing models.

[0729] A "dance database" refers to a database containing data on various dance styles, choreography, and music.

[0730] An "original dance video" refers to a unique dance video generated by a generative AI model based on the user's requests.

[0731] "Music" refers to the music tracks used to accompany the dance. These are also selected based on the user's preferences and requests.

[0732] A "user terminal" refers to a device used by a user to operate the dance system. Examples include smartphones, tablets, and PCs.

[0733] A "wearable device" refers to a device that a user wears on their body. It is used to acquire motion data in real time.

[0734] "Motion data" refers to data about the user's body movements acquired through wearable devices.

[0735] "Feedback" refers to advice and suggestions for improvement provided by the server to the user.

[0736] "Choreography part" refers to the individual dance movements and roles assigned to each member in a dance performance involving multiple people.

[0737] A "group" refers to a team or collective formed by multiple users to perform dances together.

[0738] "Performance" refers to a dance demonstration by a user or group.

[0739] This invention is a system that utilizes specific hardware and software to allow users to communicate information such as their dance style and image, age, physical fitness, and preferences to an AI. Based on this information, the AI ​​generates original dances and further supports practice and instruction. Each element of this system is described in detail below.

[0740] Creating original dances

[0741] User:

[0742] Users launch a dedicated application on their smartphone, tablet, or PC. Using the application's chat window or form, they enter their desired dance style (e.g., jazz dance), level (e.g., intermediate), target age group (e.g., 20s), and musical preference (e.g., uptempo). Once they have finished entering the information, they press the submit button.

[0743] Terminal:

[0744] The terminal receives information entered by the user, converts it to a specified data format, and sends it to the server. During this process, the terminal uses an internet connection to transmit data in real time.

[0745] server:

[0746] The server analyzes the information sent from the user's device. Based on the analyzed information, it searches the database for choreography and music suitable for the specified dance style and conditions. Using a generative AI model (e.g., a deep learning model), it integrates these elements to generate an original dance video and music. The generated dance video and music are then sent to the user's device.

[0747] Specific example

[0748] For example, if a user requests "a jazz dance style, intermediate level, suitable for young people in their 20s, and they like upbeat music," the device sends this information to the server. The server uses an AI model to extract elements of jazz dance and generates original choreography and music. This generated jazz dance video and music are sent to the user's device and presented to the user.

[0749] Support for dance practice

[0750] User:

[0751] When practicing dance choreography, the user wears an AI-enabled wearable device (e.g., a smart band) on their forehead, abdomen, wrists, and ankles. Afterward, they select dance practice mode in the app and play a dance video.

[0752] Terminal:

[0753] The device works in conjunction with a wearable device to collect user movement data in real time. It also uses the device's camera to record the user's dance. This collected data and video are then sent to a server.

[0754] server:

[0755] The server analyzes the received motion data and video, comparing them to the generated original dance to identify problems. Based on the problems identified using the AI ​​model, it generates specific instructions and feedback for the user. The feedback is sent to the user's device.

[0756] Terminal:

[0757] The device displays feedback received from the server to the user. It also uses wearable devices to guide the user in correcting their actions through vibrations and other means.

[0758] Specific example

[0759] For example, when a user is practicing jazz dance, a wearable device and camera are used to collect motion data. The data sent to the server is analyzed by AI, which identifies specific problems such as "insufficient hip movement." As a result, the device displays feedback to the user, such as "move your hips a little more."

[0760] Dance creation and instruction by multiple people

[0761] User:

[0762] When multiple people dance together, each member launches the app and sets themselves to belong to the same group. Each member then enters their own dance request information.

[0763] Terminal:

[0764] The information of each user is converted into a data packet and sent to the server along with the group ID.

[0765] server:

[0766] The server generates an integrated, original dance based on information from the entire group. It then determines individual choreography parts for each member and generates corresponding choreography and music. The generated dance data is then sent to each terminal.

[0767] Terminal:

[0768] Each member's device displays their individual choreography part and music. Furthermore, it works in conjunction with wearable devices to collect real-time motion data from each member and send it to the server.

[0769] server:

[0770] The server analyzes the operational data of multiple users and evaluates the performance of the group as a whole and each individual. Based on the evaluation results, it generates feedback and sends it to each terminal.

[0771] Terminal:

[0772] The terminal displays feedback to each member and supports performance correction. It also uses wearable devices to guide the correction process as needed.

[0773] Specific example

[0774] For example, if a group of five requests a "spectacular and energetic dance for a wedding," information about each member is sent to the server. The server generates a cohesive dance and assigns individual choreography and music. Each member's movement data is analyzed, and improvement suggestions are provided for the group as a whole and for individual members. This entire process allows users to efficiently master the dance.

[0775] Example of a prompt

[0776] For example, the prompt message if the user wants to do jazz dance would be as follows:

[0777] "Please create an intermediate-level jazz dance routine for young people in their 20s. It should be to an upbeat song."

[0778] Furthermore, the prompt message for when multiple people are involved is as follows:

[0779] "Please create a vibrant and energetic dance for a group of five to perform at a wedding."

[0780] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0781] Step 1:

[0782] Users launch a dedicated app on their smartphone, tablet, or PC.

[0783] Input: Users enter their desired dance style, level, target age group, and musical preferences in the chat window or form, and then press the submit button.

[0784] Output: The input data is received by the terminal.

[0785] Step 2:

[0786] After receiving information entered by the user, the terminal converts the data into a specified format.

[0787] Input: User input data.

[0788] Data processing: Conversion to data format.

[0789] Output: The converted data is sent from the terminal to the server.

[0790] Step 3:

[0791] The server receives data sent from the user's terminal and begins analysis.

[0792] Input: Converted data.

[0793] Data processing: The system analyzes incoming data and searches the database for choreography and music suitable for the specified dance style and conditions. A generative AI model is used to integrate these elements and generate original dance videos and music.

[0794] Output: The generated dance video and music are sent from the server to the user's device.

[0795] Step 4:

[0796] The user's device plays the dance video and music received from the server and presents them to the user.

[0797] Input: Generated dance video and music.

[0798] Data processing: Conversion to a data format for playback (if necessary).

[0799] Output: The user watches the original dance video and listens to the music.

[0800] Step 5:

[0801] Users wear an AI-powered wearable device while practicing dance choreography.

[0802] Input: Wearing a wearable device and selecting a dance practice mode in the app.

[0803] Output: The wearable device is ready to collect motion data.

[0804] Step 6:

[0805] The device works in conjunction with a wearable device to collect user movement data in real time. It also records the user's dance using the device's camera.

[0806] Input: User action data and video.

[0807] Data processing: Acquisition of motion data and video, and format conversion.

[0808] Output: The collected data and video are sent to the server.

[0809] Step 7:

[0810] The server analyzes the received motion data and video, compares it to the generated original dance, and identifies any problems.

[0811] Input: Collected motion data and video.

[0812] Data processing: Comparison of motion data with generated dance videos. Identification and analysis of problems.

[0813] Output: Specific guidance and feedback based on the problems are generated and sent to the user's device.

[0814] Step 8:

[0815] The device displays feedback received from the server to the user. It also uses wearable devices to guide the user in correcting their actions through vibrations and other means.

[0816] Input: Feedback data and instruction content.

[0817] Data processing: Display of feedback and vibration control.

[0818] Output: The feedback is displayed to the user, and guidance is provided for correcting the behavior.

[0819] Step 9:

[0820] When dancing with multiple people, each member launches the app and sets themselves to belong to the same group.

[0821] Input: Dance request information for each member.

[0822] Output: Information about each member is sent from the terminal to the server.

[0823] Step 10:

[0824] The server generates an integrated, original dance based on information from the entire group. It then determines individual choreography parts for each member and generates the corresponding choreography and music.

[0825] Input: Information for each member.

[0826] Data processing: Information analysis of the entire group, generation of integrated dance, and determination of individual choreography parts.

[0827] Output: The generated dance data is sent from the server to each terminal.

[0828] Step 11:

[0829] Each member's device displays their individual choreography part and music. Furthermore, it works in conjunction with wearable devices to collect real-time motion data from each member and send it to the server.

[0830] Input: Choreography data and motion data.

[0831] Data processing: Data display and collection and transmission of operational data.

[0832] Output: Operation data is sent to the server.

[0833] Step 12:

[0834] The server analyzes the operational data of multiple users and evaluates the performance of the group as a whole and each individual. Based on the evaluation results, it generates feedback and sends it to each terminal.

[0835] Input: Action data for each member.

[0836] Data processing: Analysis and evaluation of operational data, and generation of feedback.

[0837] Output: Feedback is sent to each device.

[0838] Step 13:

[0839] The terminal displays feedback to each member and supports performance correction. It also uses wearable devices to guide the correction process as needed.

[0840] Input: Feedback data.

[0841] Data processing: Controlling the display of feedback and correction guidelines.

[0842] Output: Feedback is displayed, and support is provided for correcting the behavior.

[0843] (Application Example 1)

[0844] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0845] While many factories currently utilize robots and automation technologies, efficiency can decrease if their operation patterns are not optimized. Furthermore, optimizing overall work efficiency is challenging when multiple robots and workers collaborate. These challenges need to be addressed because they lead to decreased productivity and increased costs.

[0846] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0847] In this invention, the server includes means for receiving information on the factory environment and robot characteristics from a user, means for using a generative AI model to generate operation patterns for the factory environment and robots based on the received information, and means for transmitting the generated operation patterns to the robots. This makes it possible to optimize the operation patterns of robots and workers in the factory and improve overall work efficiency.

[0848] A "user" is a person or organization that uses a system, and is the entity that inputs information or gives instructions for its actions.

[0849] "Dance style" refers to the form of body movements and choreography that are synchronized with a specific piece of music and rhythm, and is selected based on the user's preferences.

[0850] "Age" refers to the number of years calculated from the user's date of birth, and serves as a basis for adjusting the difficulty level of dances and movements.

[0851] "Physical fitness" refers to the user's athletic ability and physical capabilities, which affect the intensity and duration of dance and movement.

[0852] "Preferences" refer to information that indicates a user's likes and interests, and are used to customize the generated dance and movement patterns.

[0853] "Means of receiving information" refers to interfaces and devices that receive input data from users and analyze it.

[0854] A "generative AI model" refers to an artificial intelligence algorithm or framework that automatically generates optimal dance or movement patterns based on input data.

[0855] A "database" refers to a system that systematically stores existing data and information, and allows for efficient retrieval and utilization of that data.

[0856] An "original dance video" refers to video content that includes unique dance choreography generated based on user input.

[0857] "Music" refers to a combination of sounds used in conjunction with dance or movement, and is an integrated form of sound data containing various elements.

[0858] A "user terminal" refers to devices such as smartphones, tablets, and PCs that users directly operate to input information or receive feedback.

[0859] A "wearable device" refers to a device that a user wears on their body to collect motion data in real time and transmit it to a system.

[0860] "Real-time" refers to a state where data is acquired and processed almost instantly, and information is updated without delay.

[0861] A "motion pattern" refers to a sequence of movements optimized to achieve a specific purpose.

[0862] A "robot" is an automated mechanical device designed to perform a specified task or action.

[0863] "Feedback" refers to improvement suggestions and correction instructions provided to users or robots based on analyzed data.

[0864] "Factory environment" refers to the physical conditions, layout, temperature, humidity, and other factors of the site where the robot operates.

[0865] This invention is a system for optimizing the motion patterns of robots in a factory and improving overall work efficiency. The embodiments for carrying out this invention are described below.

[0866] Users input information about the factory environment and robot characteristics using devices such as smartphones, tablets, and PCs. For example, they input information such as the factory's temperature, humidity, layout, robot type, speed, and accuracy. This information is then transmitted from the device to the server.

[0867] The server analyzes the received information and uses a generative AI model to generate factory environment and robot operation patterns. The generative AI model automatically generates the optimal operation pattern based on the input data. This operation pattern is a sequence designed to allow the robot to operate with maximum efficiency.

[0868] The generated motion patterns are sent from the server to the robot. The robot performs the task according to the received motion pattern. Motion data from each robot is collected in real time and sent back to the server. The server analyzes this motion data and generates feedback to improve efficiency. This feedback is provided to the user terminal and the robot, allowing for modifications and improvements to the robot's movements.

[0869] For example, if a user specifies that the factory temperature is 22°C, humidity is 40%, layout is Type A, the robot is an arm type, and it operates at medium speed and high precision, the AI ​​model will generate an optimal motion pattern based on these conditions. The generated motion pattern is sent to the robot as a command sequence such as "move_arm_to_XYZ, pick_and_place, verify_and_adjust". The robot operates according to this command sequence, and the server analyzes the collected motion data and provides feedback with correction instructions as needed.

[0870] This series of processes optimizes the operation of robots in the factory, making it possible to improve overall work efficiency.

[0871] The hardware used includes user terminals (smartphones, tablets, PCs), servers, and factory robots. The software used includes generative AI models (implemented in Keras / TensorFlow), Python scripts for data analysis, and front-end applications for the user interface.

[0872] Example of a prompt:

[0873] "Factory temperature is 22°C, humidity is 40%, layout is Type A, robot is arm-type, medium speed, high precision."

[0874] This allows the system to generate optimal robot motion patterns suited to the factory environment, thereby improving work efficiency.

[0875] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0876] Step 1:

[0877] Users input information about the factory environment and robot characteristics using devices such as smartphones, tablets, and PCs. Specifically, they input details such as the factory temperature, humidity, layout, robot type, speed, and accuracy, and then press the submit button. The input format is as follows: "Factory temperature: 22°C, humidity: 40%, layout: Type A, robot: arm type, medium speed, high accuracy."

[0878] Step 2:

[0879] The terminal formats the information about the factory environment and robot characteristics received from the user and sends it to the server. Here, the terminal converts the input data into JSON format and sends it to the server over the network. This is a data transmission operation, where the input is the user's input information and the output is the data transmission to the server.

[0880] Step 3:

[0881] The server analyzes the received information. Specifically, the server loads a generative AI model for analyzing factory environment and robot characteristic information, and supplies the received data to the generative AI model as input data. The AI ​​model generates the optimal operation pattern based on this data. The input is JSON data received from the terminal, and the output is the optimal operation pattern.

[0882] Step 4:

[0883] The server sends the generated motion patterns to the robots. These generated motion patterns are formatted as motion sequences for each robot and applied to specific robots within the factory. Specifically, the server sends the optimal motion pattern to the robot control unit. The input is the motion pattern generated from the AI ​​model, and the output is the transmission of the motion pattern to the robot.

[0884] Step 5:

[0885] The robots perform tasks according to motion patterns received from the server. Each robot executes the specified movements based on the received sequence. This allows the robots to efficiently perform specific work processes within the factory. The input is the motion pattern received from the server, and the output is the actual robot movement.

[0886] Step 6:

[0887] Robot motion data is collected in real time and transmitted to a server. The robot uses its built-in sensors and monitoring system to record motion data and transmit it to the server. Input is data obtained from the robot's motion sensors, and output is data transmission to the server.

[0888] Step 7:

[0889] The server analyzes the received robot motion data. Specifically, a data analysis module within the server analyzes the motion data and generates feedback necessary to improve the efficiency of factory work. The input is the motion data received from the robot, and the output is the feedback for efficiency improvement.

[0890] Step 8:

[0891] The server provides the generated feedback to the user terminal and the robot. The server sends the feedback to the user terminal, presenting the user with specific suggestions for improvement. Similarly, feedback is sent to the robot, allowing for modifications and adjustments to its operation. The input is the server's analysis results, and the output is the transmission of feedback to the user terminal and the robot.

[0892] By executing the above steps sequentially, the operation of factory robots is optimized, resulting in improved overall work efficiency.

[0893] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0894] This invention is a system that generates original dances tailored to the user by allowing the user to communicate their dance style, image, emotions, age, physical condition, and preferences to an AI, and then supports their practice and instruction. In addition to this system, an emotion engine that recognizes the user's emotions is incorporated to achieve more advanced feedback and customization. It is configured as follows:

[0895] 1. Creating an original dance

[0896] User:

[0897] Users launch the app on their smartphones, tablets, PCs, or other devices and input information such as their dance style, image, emotions, age, physical condition, and preferences through an interface such as a chat window. Once they have finished inputting the information, they press the submit button.

[0898] Terminal:

[0899] The terminal receives information entered by the user, formats the data, and sends it to the server.

[0900] server:

[0901] The server analyzes the received data and searches its database for dance moves and music that are suitable for the specified dance style, conditions, and the user's emotions. Using an AI model, it integrates these elements to generate an original dance video and music. The generated video and music are then sent to the device.

[0902] Specific example

[0903] For example, if a user requests "a jazz dance style, a fun mood, intermediate level, suitable for young people in their 20s, and they like upbeat music," the device sends this information to the server. The server uses an AI model and emotion engine to generate choreography and music that reflect the elements of jazz dance and a "fun" atmosphere. The generated jazz dance video and music are sent to the user's device and presented to the user.

[0904] 2. Support for dance practice

[0905] User:

[0906] When practicing choreography, users wear AI-enabled wearable devices on their forehead, abdomen, both wrists, and both ankles. Next, they select a dance practice mode in the app and start playing the dance.

[0907] Terminal:

[0908] The device works in conjunction with a wearable device to collect user movement data in real time. It also uses the device's camera to record the user's dance. The collected data and video are sent to a server.

[0909] server:

[0910] The server analyzes the received data and video, comparing them to the generated original dance to identify problems. Furthermore, it uses an emotion engine to analyze the user's emotions during practice in real time and adaptively adjust the feedback. Based on the identified problems and the emotion analysis results, it generates feedback to specifically instruct the user on how to correct them and sends it to the terminal.

[0911] Terminal:

[0912] The device displays feedback to the user and provides support to correct actions according to the instructions. It also uses wearable devices such as vibration to guide the user in correcting their actions.

[0913] Specific example

[0914] The system uses a wearable device and camera worn by the user to acquire movement data from the jazz dance they are practicing. The data sent to the server is analyzed by AI and an emotion engine to identify problems such as "insufficient hip movement" and emotions such as "feeling frustrated." As a result, the device provides the user with specific feedback such as "move your hips a little more" and emotionally responsive advice such as "relax and enjoy yourself."

[0915] 3. Creating and teaching dances with multiple people

[0916] User:

[0917] If multiple people wish to dance together, each member must launch the app and set themselves to belong to the same group. All members then enter their dance request information and emotional information.

[0918] Terminal:

[0919] The information of each user is converted into a data packet and sent to the server along with the group ID.

[0920] server:

[0921] The server generates an integrated, original dance based on information from the entire group. It then determines individual choreography parts for each member and generates corresponding choreography and music for each. The generated data is then sent to the terminal.

[0922] Terminal:

[0923] Each member's device displays their individual choreography part and music. Wearable devices are used to collect real-time movement data from each member and send it to a server.

[0924] server:

[0925] The server analyzes the behavioral data of multiple individuals and evaluates the performance of the entire group and each individual. Furthermore, it uses an emotion engine to analyze the emotional state of the entire group and generates emotion-based feedback. Based on the evaluation results and emotion analysis results, it generates individual and group-wide feedback and sends it to the terminal.

[0926] Terminal:

[0927] The device displays feedback to each member, supporting performance improvements and emotional enhancement. It also uses wearable devices to guide behavioral corrections as needed.

[0928] Specific example

[0929] When a group of five requests a "spectacular and energetic dance for a wedding," information about each member is sent to the server. The server uses AI and an emotion engine to generate a cohesive dance, assigning individual choreography and music. The emotion engine analyzes each member's emotional state and provides feedback to ensure the entire group enjoys the performance together. Each member's movement data is analyzed, and improvement instructions are provided for the group as a whole and for individual members. Through this process, users can efficiently master the dance while also receiving emotional support.

[0930] The following describes the processing flow.

[0931] 1. Creating an original dance

[0932] Step 1:

[0933] User:

[0934] Launch the app and enter information such as dance style, image, emotions, age, physical condition, and preferences into the chat window.

[0935] Step 2:

[0936] Terminal:

[0937] The system receives the input information, formats it into a data packet, and sends it to the server.

[0938] Step 3:

[0939] server:

[0940] The system analyzes received data packets and searches a database for dance style, music genre, and emotional data based on the input information.

[0941] Step 4:

[0942] server:

[0943] Using AI models and an emotion engine, the system generates original dance videos and music based on search results.

[0944] Step 5:

[0945] server:

[0946] The generated original dance video and music are sent to the device.

[0947] Step 6:

[0948] Terminal:

[0949] The received original dance video and music are displayed and played on the user interface.

[0950] 2. Support for dance practice

[0951] Step 1:

[0952] User:

[0953] AI-enabled wearable devices are attached to the head, abdomen, both wrists, and both ankles.

[0954] Select dance practice mode in the app.

[0955] Step 2:

[0956] Terminal:

[0957] Wearable devices are connected via Bluetooth or other communication methods to acquire motion data and emotion data in real time.

[0958] The device's camera records the user's dance.

[0959] Step 3:

[0960] Terminal:

[0961] The collected motion data, emotion data, and recorded video are bundled into data packets and sent to the server.

[0962] Step 4:

[0963] server:

[0964] The received motion data, emotion data, and recorded video are analyzed using an AI model and emotion engine, and compared to the generated original dance.

[0965] Step 5:

[0966] server:

[0967] Identify operational problems, refine feedback based on real-time sentiment data, and generate specific corrective instructions and advice.

[0968] Step 6:

[0969] server:

[0970] The generated feedback information is sent to the device.

[0971] Step 7:

[0972] Terminal:

[0973] Display feedback information in the user interface.

[0974] Wearable devices are used to provide guidance through vibrations and other means.

[0975] 3. Creating and teaching dances with multiple people

[0976] Step 1:

[0977] User:

[0978] Each member launches the app and sets themselves to belong to the same group.

[0979] Enter your dance style, image, emotions, and performance preferences.

[0980] Step 2:

[0981] Terminal:

[0982] The input information of each member is converted into a data packet and sent to the server along with the group ID.

[0983] Step 3:

[0984] server:

[0985] Based on information from the entire group, it generates an original dance that conveys a sense of unity.

[0986] Assign choreography parts to each member.

[0987] Step 4:

[0988] server:

[0989] The generated dance video, music, and choreography parts are sent to the device.

[0990] Step 5:

[0991] Terminal:

[0992] The system displays the choreography part and music assigned to each member on their device.

[0993] The system connects to wearable devices to acquire motion and emotion data in real time and transmits it to a server.

[0994] Step 6:

[0995] server:

[0996] We analyze the behavioral and emotional data of multiple individuals to evaluate the performance of the group as a whole and individual members.

[0997] Step 7:

[0998] server:

[0999] It generates group-wide and individual feedback, further refines the feedback based on sentiment data, and sends it to the device.

[1000] Step 8:

[1001] Terminal:

[1002] Provide feedback to each member and offer suggestions for revisions.

[1003] Provide guidance via vibration from wearable devices as needed.

[1004] Through these steps, users can efficiently learn dance and receive support that takes their emotional state into consideration.

[1005] (Example 2)

[1006] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[1007] Conventional dance generation and practice support systems failed to adequately reflect users' individual preferences and emotions, and their motion corrections and feedback during practice were not adapted to the user's emotional state. Furthermore, when generating dance performances for multiple people, there were insufficient methods for coordinating the group's overall choreography with individual choreography. In addition, real-time motion data collection and analysis lacked integration with emotion analysis.

[1008] The identification processing performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for receiving information from the user regarding dance style, age, physical strength, preferences, and emotions; means for analyzing the received information and generating original dance videos and music using a generation AI model based on an existing dance database; and means for transmitting the generated original dance videos and music to the user terminal. This makes it possible to generate original dance videos and music that reflect individual preferences and emotions.

[1009] The server also includes means for acquiring motion data in real time through a wearable device worn by the user; means for analyzing the acquired motion data and identifying problems by comparing it with the generated original dance video; means for analyzing the user's emotions using an emotion engine and generating feedback based on that; means for providing guidance and feedback to the user based on the identified problems and emotions; and means for guiding the user's motion correction through the wearable device. This makes it possible for motion correction and feedback during practice to be adapted to the user's emotional state.

[1010] Furthermore, the server includes means for receiving individual dance styles, preferences, and emotional information from multiple users to generate an integrated dance performance as a group; means for assigning individual choreography parts to each member; means for acquiring movement data of each member in real time and analyzing the overall performance of the group; and means for generating feedback to the group as a whole and to individual members using a generative AI model and an emotion engine to support performance improvement. This makes it possible to generate and improve dance performances involving multiple people.

[1011] "Dance style" refers to the type or form of dance desired by the user, and examples include jazz dance, ballet, and hip hop.

[1012] "Age" refers to the user's age group and is information used to determine appropriate dance choreography and exercise intensity.

[1013] "Physical fitness" refers to the user's physical ability and endurance, and serves as a criterion for adjusting the difficulty and intensity of the dance.

[1014] "Preferences" refer to a user's likes and interests, and include preferences such as favorite music genres and dance styles.

[1015] "Emotions" refers to the emotional state a user experiences while dancing, and includes feelings such as happiness, sadness, and relaxation.

[1016] A "wearable device" is a device worn and used by a user to acquire movement data in real time. Examples include smartwatches and fitness trackers.

[1017] A "generative AI model" is a model that uses artificial intelligence to analyze and generate data, and is used to generate original dance videos and music based on specific inputs.

[1018] An "emotion engine" is a system that analyzes a user's emotional state and provides optimal feedback and guidance based on that analysis.

[1019] "Feedback" refers to information that provides suggestions and advice for improving a user's actions and practice methods.

[1020] A "database" is a system that stores information on dance moves and music, and is used to search for and retrieve appropriate data based on user input.

[1021] An "original dance video" is a video of a custom dance, generated using an AI model and tailored to the user's requests.

[1022] "Original music" refers to custom music generated using AI models, tailored to the user's requests.

[1023] This invention is a system that generates personalized, original dances based on the user's dance style, image, emotions, age, physical condition, and preferences communicated to the AI, and supports practice and instruction. The system incorporates an emotion engine that recognizes the user's emotions in real time and reflects them in the feedback.

[1024] Creating original dances

[1025] User:

[1026] Users launch the app using their smartphone, tablet, or PC and enter information such as their dance style, image, emotions, age, fitness level, and preferences through the app's chat window. For example, they might enter information such as, "I like jazz dance, I'm in a fun mood, I'm intermediate level, I'm in my 20s, and I like upbeat music."

[1027] Terminal:

[1028] The terminal receives user input information, formats it into the required format, and sends it to the server. This formatting may involve using data formats such as JSON.

[1029] server:

[1030] The server analyzes the received data and searches its database for dance moves and music that are suitable for the specified dance style, conditions, and the user's emotions. A generative AI model is used for this, and an emotion engine is also utilized to generate original dance videos and music that are adapted to the user's emotions. The generated videos and music are then sent back to the terminal.

[1031] Support for dance practice

[1032] User:

[1033] When practicing dance choreography, users wear AI-enabled wearable devices on their forehead, abdomen, both wrists, and both ankles. Next, they launch the app, select dance practice mode, and start playing the dance.

[1034] Terminal:

[1035] The device works in conjunction with a wearable device to collect user movement data in real time. It also uses the device's camera to record the user's dance movements. The collected data and video are sent to a server.

[1036] server:

[1037] The server analyzes the received data and video, comparing them to the generated original dance to identify problems. Furthermore, it uses an emotion engine to analyze the user's emotions during practice in real time and generates feedback based on that analysis. Based on the identified problems and the emotion analysis results, it generates feedback to provide the user with specific instructions on how to correct their behavior and sends it to the terminal.

[1038] Dance creation and instruction by multiple people

[1039] User:

[1040] When multiple people dance together, each member launches the app and sets themselves to belong to the same group. Each member then enters dance request information and emotion information.

[1041] Terminal:

[1042] Each member's data is converted into a data packet and sent to the server along with the group ID.

[1043] server:

[1044] The server analyzes the received information from the entire group and generates an integrated original dance. It determines individual choreography parts for each member and generates corresponding choreography and music. The generated data is sent to each member's terminal.

[1045] Terminal:

[1046] Each member's device displays their individual choreography part and music, and in conjunction with wearable devices, movement data from each member is collected in real time and sent to a server.

[1047] server:

[1048] The server analyzes the behavioral data of multiple individuals and evaluates the performance of the group as a whole and each individual. Furthermore, it uses an emotion engine to analyze the emotional state of the entire group and generates feedback based on that analysis. This feedback is sent to each member's terminal.

[1049] Specific example

[1050] For example, if a user enters a prompt such as "Jazz dance style, fun mood, intermediate level, for people in their 20s, likes upbeat music," the device sends this to the server. The server uses a generative AI model and emotion engine to generate choreography and music that reflects the elements of jazz dance and a fun atmosphere based on this information. This is then sent to the user's device, and the user can view the content in the app.

[1051] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1052] Creating original dances

[1053] Step 1:

[1054] User: Users launch the app using a smartphone, tablet, or PC and enter information about their dance style, image, emotions, age, fitness level, and preferences into the app's chat window. Specifically, they might enter information such as, "I like jazz dance, I'm in a fun mood, I'm intermediate level, I'm in my 20s, and I like upbeat music." The input is detailed information about the user's preferences and emotions, and the output is the transmission of this information.

[1055] Step 2:

[1056] Terminal: The terminal receives user input, converts the data into a format such as JSON, and sends it to the server. The input is information entered by the user, and the output is formatted data. Specifically, it performs the operation of converting the text data entered by the user into an appropriate format.

[1057] Step 3:

[1058] Server: The server receives formatted data and uses a generating AI model to search and analyze a database for dance moves and music that are suitable for the specified dance style, conditions, and the user's emotions. The input is formatted data received from the terminal, and the output is the generated dance moves and music. The server uses the AI ​​model to generate dance choreography patterns and music based on text information.

[1059] Step 4:

[1060] Server: The server searches and integrates generated dances and music from a database to produce original dance videos and music. The input is the search and analysis results, and the output is the integrated original dance video and music. The server uses a generation AI model to integrate the elements and perform the operation of generating new content.

[1061] Step 5:

[1062] Server: Sends the generated original dance video and music to the user's device. The input is the generated content, and the output is the data sent to the device.

[1063] Step 6:

[1064] Terminal: The terminal displays received dance videos and music to the user. Input is data sent from the server, and output is what is displayed to the user.

[1065] Support for dance practice

[1066] Step 1:

[1067] User: The user attaches AI-enabled wearable devices to their forehead, abdomen, both wrists, and both ankles. Next, they launch the app, select dance practice mode, and start playback. The input is the original dance video and the attached wearable devices, and the output is the start of practice mode.

[1068] Step 2:

[1069] Terminal: The terminal works in conjunction with a wearable device to collect user movement data in real time. It also uses the terminal's camera to record video of the user dancing. The input is user movement data and video, and the output is the collected real-time data.

[1070] Step 3:

[1071] Terminal: Sends collected data and video to the server. Inputs are real-time data and recorded video, and outputs are data sent to the server.

[1072] Step 4:

[1073] Server: The server analyzes the received data and video, compares them to the generated original dance, and identifies problems. Furthermore, it uses an emotion engine to analyze the user's emotions during practice in real time. The input is the received data and video, and the output is the problems and the emotion analysis results. The server performs the analysis using an AI model and an emotion engine.

[1074] Step 5:

[1075] Server: Based on the identified problems and sentiment analysis results, the server generates feedback to provide the user with specific instructions on how to correct them and sends it to the terminal. The input is the problems and analysis results, and the output is the generated feedback.

[1076] Step 6:

[1077] Terminal: The terminal displays feedback to the user and provides support to correct actions according to instructions. It also uses wearable devices to guide the user's actions through vibration, etc. Input is feedback received from the server, and output is the display and guidance to the user.

[1078] Dance creation and instruction by multiple people

[1079] Step 1:

[1080] User: When multiple people want to dance together, each member launches the app and sets themselves to belong to the same group. All members enter dance request information and emotion information. Input is group settings and request information, and output is setting complete and input complete.

[1081] Step 2:

[1082] Terminal: Converts each member's information into a data packet and sends it to the server along with the group ID. Input is information from group members, and output is a data packet.

[1083] Step 3:

[1084] Server: The server generates an integrated, original dance based on information from the entire group. It then determines individual choreography parts for each member and generates corresponding choreography and music for each. The input is multiple request information, and the output is the integrated dance and individual choreography data. The server uses an AI model and an emotion engine for generation.

[1085] Step 4:

[1086] Server: Sends the generated dance and music to each member's terminal. The input is the generated data, and the output is the data sent to the terminal.

[1087] Step 5:

[1088] Terminal: Each member's terminal displays their individual choreography part and music. It works in conjunction with wearable devices to collect real-time motion data from each member and send it to the server. Inputs are received data and motion data, and outputs are data sent to the server.

[1089] Step 6:

[1090] Server: The server analyzes the received behavioral data and evaluates the performance of the entire group and individual members. Furthermore, it uses an emotion engine to analyze the emotional state of the entire group and generate feedback. Inputs are behavioral and emotion data, and outputs are evaluation results and feedback.

[1091] Step 7:

[1092] Terminal: The terminal displays feedback to each member, supporting performance correction and emotional improvement. It also guides behavioral correction using wearable devices as needed. Input is feedback received from the server, and output is the display and guidance to the user.

[1093] (Application Example 2)

[1094] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[1095] Traditional dance practice systems and teaching methods have struggled to adequately customize and accommodate individual user preferences and emotions. Furthermore, the lack of systems capable of analyzing user movement data and emotional states in real time and providing adaptive feedback based on those results often resulted in inefficient practice instruction and inadequate emotional support. Additionally, in group dance practice, there was a lack of means to analyze the overall group performance and emotional state and provide integrated feedback.

[1096] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving information from the user regarding dance style, age, physical strength, preferences, and emotions; means for analyzing the received information and generating original dance videos and music based on an existing dance database and a generation AI model; means for transmitting the generated original dance videos and music to the user terminal; and means for analyzing the user's emotions and providing feedback based on the analysis results. This enables the generation of customized dances that correspond to the user's individual preferences and emotions, efficient practice guidance, and adaptive emotional support.

[1097] Furthermore, the system includes means for acquiring motion data in real time through a wearable device worn by the user, means for analyzing the acquired motion data and identifying problems by comparing it with the generated original dance video, means for providing guidance and emotion-responsive feedback to the user based on the identified problems and emotion analysis results, and means for guiding the user to correct their movements through the wearable device, thereby enabling the user to correct their movements and improve their skills.

[1098] Furthermore, by including means for receiving individual dance style, preference, and emotional information from multiple users to generate an integrated dance performance as a group, means for assigning individual choreography parts to each member, means for acquiring movement data of each member in real time and analyzing the overall performance and emotions of the group, and means for providing feedback to the group as a whole and to individual members to support performance improvement and emotional enhancement, efficient dance practice for multiple people and improvement of the overall group performance can be achieved.

[1099] A "user" is someone who provides the system with information about their dance style, age, physical fitness, preferences, and emotions, and receives generated original dance videos and music, as well as practice and instruction.

[1100] "Dance style" refers to information that describes the specific type or form of dance that the user desires.

[1101] "Age" refers to information indicating the user's actual age.

[1102] "Physical fitness" refers to information that indicates the user's physical ability and endurance.

[1103] "Preferences" refer to information that indicates an individual user's tastes and tendencies, such as the type of music or dance style they prefer.

[1104] "Emotions" refer to information that indicates the feelings and mental state that the user is currently experiencing.

[1105] A "generative AI model" is an artificial intelligence model used to generate original dance videos and music based on user input.

[1106] A "prompt message" is a statement that indicates the format and content of the information that the user should input into the system.

[1107] An "original dance video" is a video of a dance that has been individually customized and generated based on information provided by the user.

[1108] "Music" refers to the audio data corresponding to the generated original dance video.

[1109] A "user terminal" is a hardware device used by a user to receive and play original dance videos and music.

[1110] "Emotional analysis" is the process of analyzing a user's emotional state and incorporating the results into the feedback.

[1111] "Feedback" refers to information that provides guidance and advice based on the user's dance performance and emotional state.

[1112] A "wearable device" is a device that a user wears to acquire real-time motion data.

[1113] "Motion data" refers to data that shows the user's body movements and location information.

[1114] "Performance" refers to the technical skill and expressiveness of a user's dance moves.

[1115] A "group" refers to a collective of multiple users performing a dance together.

[1116] A "choreographed part" refers to the individual dance movements assigned to each member of the group.

[1117] One embodiment of this invention is a system that generates original dance videos and music customized based on the dance style, age, physical fitness, preferences, and emotions entered by the user, and supports their practice and instruction. Specifically, it involves the following processing steps.

[1118] 1. Collecting user input

[1119] Users launch the application using devices such as smartphones, tablets, or PCs. Through the application's interface, users input their dance style, age, physical fitness level, preferences, and current emotions. This information is sent to the server.

[1120] 2. Generation of original dance videos and music

[1121] The server analyzes the received user information. Using an existing dance database and a generative AI model, it generates original dance videos and music tailored to the user's preferences and emotions. This generated content is then sent to the user's device.

[1122] 3. Emotional analysis and provision of feedback

[1123] The server uses an emotion engine to analyze the emotional information entered by the user. Based on the analysis results, it provides appropriate feedback to the user. For example, if the user's emotion during practice is "frustration," it will provide feedback such as "relax and enjoy yourself."

[1124] 4. Acquisition of motion data using wearable devices

[1125] During practice, the user wears AI-enabled wearable devices on their forehead, abdomen, both wrists, and both ankles. The terminal interacts with these wearable devices to collect the user's movement data in real time. It also uses the terminal's camera to record the user's dance. This data is sent to a server.

[1126] 5. Comparison of motion data and the generated original dance video

[1127] The server analyzes the received motion data and video, compares it to the generated original dance, and identifies problems. Based on the identified problems, it generates feedback to instruct the user on how to correct them and sends it to the user's terminal.

[1128] 6. Creating and teaching dances with multiple people

[1129] When multiple users register as a group, each member enters their individual dance style, preferences, and emotional information into the application. This information is sent to the server, which generates an integrated original dance performance for the entire group. The server then assigns individual choreography parts to each member. In addition, it acquires movement data from each member in real time, analyzes the group's overall and individual performances, and provides feedback.

[1130] Examples

[1131] Possible prompt statements include the following:

[1132] Favorite dance style: Hip Hop

[1133] Current emotion: Happy

[1134] Dance level: Intermediate

[1135] Age: 25

[1136] Preferences: Upbeat music

[1137] Based on this prompt, the server generates a fun, intermediate-level hip-hop style dance. Users can practice the dance while receiving real-time feedback. If frustration is detected, advice such as "Relax and have fun" is also provided.

[1138] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1139] Step 1:

[1140] Users launch the application on their smartphones, tablets, or PCs and input information about their dance style, age, physical fitness, preferences, and emotions. The input data is as follows: "Dance Style: Hip Hop," "Emotion: Happy," "Dance Level: Intermediate," "Age: 25," "Preference: Upbeat Music." This input data is then sent from the device to the server.

[1141] Step 2:

[1142] The server analyzes the user's input data. Based on the input information, it uses an existing dance database and a generative AI model to generate an original dance video and music tailored to the user. The generated dance video and music files are sent from the server to the user's terminal. The output is a customized original dance video and music file.

[1143] Step 3:

[1144] The user wears AI-enabled wearable devices on their forehead, abdomen, both wrists, and both ankles, and selects a dance practice mode within the application. The terminal interacts with the wearable devices to collect the user's movement data in real time. This movement data includes information such as the position of each joint and the speed of movement. This movement data is transmitted from the terminal to the server.

[1145] Step 4:

[1146] The server analyzes the received motion data and compares it to the generated original dance video. Using an AI model, it compares the user's movements to accurate dance moves and identifies problems. For example, it might identify a problem such as "insufficient hip movement." The analysis results are stored on the server as feedback information.

[1147] Step 5:

[1148] The server uses an emotion engine to analyze the emotional information entered by the user. Based on the analysis results, it evaluates the user's emotional state during practice and generates adaptive feedback. For example, if the user is feeling "frustrated," it will generate feedback such as "relax and enjoy yourself." The output consists of the emotional state analysis result and the feedback statement.

[1149] Step 6:

[1150] Based on the identified problems and emotional state, the server generates specific instructions and emotionally responsive feedback for the user and sends them to the user's terminal. The terminal displays this to the user, providing instructions for correcting actions such as "Move your hips a little more" and emotional advice such as "Relax and enjoy yourself."

[1151] Step 7:

[1152] Based on the feedback received from the user, the dance is modified, and motion data is collected again. Then, by repeating the process from steps 3 to 6, the user's performance improves and emotional support is continuously provided.

[1153] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1154] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1155] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[1156] [Third Embodiment]

[1157] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[1158] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1159] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1160] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[1161] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1162] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1163] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1164] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1165] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1166] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1167] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1168] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[1169] This invention is a system that generates original dances tailored to the user by allowing the user to communicate their dance style, image, age, physical ability, and preferences to an AI, and then supports their practice and instruction. The system is configured as follows:

[1170] 1. Creating an original dance

[1171] User:

[1172] Users launch the app on their smartphones, tablets, PCs, or other devices and enter information such as their dance style and image, age, physical condition, and preferences through an interface such as a chat window. Once they have finished entering the information, they press the submit button.

[1173] Terminal:

[1174] The terminal receives information entered by the user, formats the data, and sends it to the server.

[1175] server:

[1176] The server analyzes the received data and searches its database for dance moves and music that match the specified dance style and conditions. Using an AI model, it integrates these elements to generate an original dance video and music. The generated video and music are then sent to the terminal.

[1177] Specific example

[1178] For example, if a user requests "a jazz dance style, intermediate level, aimed at young people in their 20s, and with an upbeat tempo," the device sends this information to the server. The server uses an AI model to extract elements of jazz dance and generate original choreography and music. The generated jazz dance video and music are sent to the user's device and presented to them.

[1179] 2. Support for dance practice

[1180] User:

[1181] When practicing choreography, users wear AI-enabled wearable devices on their forehead, abdomen, both wrists, and both ankles. Next, they select a dance practice mode in the app and start playing the dance.

[1182] Terminal:

[1183] The device works in conjunction with a wearable device to collect user movement data in real time. It also uses the device's camera to record the user's dance. The collected data and video are sent to a server.

[1184] server:

[1185] The server analyzes the received data and video, compares them to the generated original dance, and identifies problems. Based on the identified problems, it generates feedback to the user, providing specific instructions on how to correct them, and sends it to the terminal.

[1186] Terminal:

[1187] The device displays feedback to the user and provides support to correct actions according to the instructions. It also uses wearable devices such as vibration to guide the user in correcting their actions.

[1188] Specific example

[1189] The system uses a wearable device and camera worn by the user to acquire movement data from the jazz dance they are practicing. The data sent to the server is analyzed by AI to identify problems such as insufficient hip movement. As a result, the device provides the user with specific feedback such as, "Let's move our hips a little more."

[1190] 3. Creating and teaching dances with multiple people

[1191] User:

[1192] If multiple people wish to dance together, each member must launch the app and set themselves to belong to the same group. All members then enter their dance request information.

[1193] Terminal:

[1194] The information of each user is converted into a data packet and sent to the server along with the group ID.

[1195] server:

[1196] The server generates an integrated, original dance based on information from the entire group. It then determines individual choreography parts for each member and generates corresponding choreography and music for each. The generated data is then sent to the terminal.

[1197] Terminal:

[1198] Each member's device displays their individual choreography part and music. Wearable devices are used to collect real-time movement data from each member and send it to a server.

[1199] server:

[1200] The server analyzes the operational data of multiple users and evaluates the performance of the group as a whole and each individual. Based on the evaluation results, it generates individual and group-wide feedback and sends it to the terminal.

[1201] Terminal:

[1202] The terminal displays feedback to each member and supports performance correction. Additionally, wearable devices are used to guide the correction process as needed.

[1203] Specific example

[1204] When a group of five requests a "spectacular and energetic dance for a wedding," information about each member is sent to the server. The server generates a cohesive dance and assigns individual choreography and music. Each member's movement data is analyzed, and improvement suggestions are provided for the group as a whole and for individual members. This entire process allows users to efficiently master the dance.

[1205] The following describes the processing flow.

[1206] 1. Creating an original dance

[1207] Step 1:

[1208] User:

[1209] Launch the app and enter information such as dance style, image, age, physical condition, and preferences into the chat window.

[1210] Step 2:

[1211] Terminal:

[1212] The system receives the input information, formats it into a data packet, and sends it to the server.

[1213] Step 3:

[1214] server:

[1215] The system analyzes received data packets and searches the database for dance styles and music genres based on the input information.

[1216] Step 4:

[1217] server:

[1218] Using an AI model, original dance videos and music are generated based on search results.

[1219] Step 5:

[1220] server:

[1221] The generated original dance video and music are sent to the device.

[1222] Step 6:

[1223] Terminal:

[1224] The received original dance video and music are displayed and played on the user interface.

[1225] 2. Support for dance practice

[1226] Step 1:

[1227] User:

[1228] AI-enabled wearable devices are attached to the head, abdomen, both wrists, and both ankles.

[1229] Select dance practice mode in the app.

[1230] Step 2:

[1231] Terminal:

[1232] Wearable devices are connected via Bluetooth or other communication methods to acquire operational data in real time.

[1233] The device's camera records the user's dance.

[1234] Step 3:

[1235] Terminal:

[1236] The collected motion data and recorded video are bundled into data packets and sent to the server.

[1237] Step 4:

[1238] server:

[1239] The received motion data and recorded video are analyzed using an AI model, and then compared to the generated original dance.

[1240] Step 5:

[1241] server:

[1242] Identify operational problems and generate specific correction instructions and advice.

[1243] Step 6:

[1244] server:

[1245] The generated feedback information is sent to the device.

[1246] Step 7:

[1247] Terminal:

[1248] Display feedback information in the user interface.

[1249] Wearable devices are used to provide guidance through vibrations and other means.

[1250] 3. Creating and teaching dances with multiple people

[1251] Step 1:

[1252] User:

[1253] Each member launches the app and sets themselves to belong to the same group.

[1254] Enter your dance style, image, and performance preferences.

[1255] Step 2:

[1256] Terminal:

[1257] The input information of each member is converted into a data packet and sent to the server along with the group ID.

[1258] Step 3:

[1259] server:

[1260] Based on information from the entire group, it generates an original dance that conveys a sense of unity.

[1261] Assign choreography parts to each member.

[1262] Step 4:

[1263] server:

[1264] The generated dance video, music, and choreography parts are sent to the device.

[1265] Step 5:

[1266] Terminal:

[1267] The system displays the choreography part and music assigned to each member on their device.

[1268] The system connects to wearable devices to acquire real-time motion data and transmits it to a server.

[1269] Step 6:

[1270] server:

[1271] Analyze the motion data of multiple individuals to evaluate the performance of the group as a whole and individual members.

[1272] Step 7:

[1273] server:

[1274] Generate and send group-wide and individual feedback to the device.

[1275] Step 8:

[1276] Terminal:

[1277] Provide feedback to each member and offer suggestions for revisions.

[1278] Provide guidance via vibration from wearable devices as needed.

[1279] Through these steps, users can efficiently learn dance and improve their performance.

[1280] (Example 1)

[1281] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1282] In recent years, the demand for dance has been increasing as a way to maintain health and as a form of entertainment, but finding a dance style and choreography that suits individual users is difficult. Furthermore, obtaining individualized feedback requires a professional instructor, which is time-consuming and costly. Additionally, efficient practice of dance performances by multiple people requires coordination and appropriate instruction for each member. There is a need for a system that can solve these problems and enable users to learn dance efficiently.

[1283] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1284] In this invention, the server includes means for receiving information from the user regarding dance style, age, physical fitness, and preferences; means for generating original dance videos and music based on an existing dance database using a generative AI model; and means for transmitting the generated original dance videos and music to the user terminal. This makes it possible to automatically provide the user with the most suitable dance style and choreography.

[1285] Furthermore, this invention includes a server that includes means for acquiring motion data in real time through a wearable device worn by the user, means for analyzing the acquired motion data and identifying problems by comparing it with the generated original dance video, means for providing guidance and feedback to the user based on the identified problems, and means for guiding the user to correct their movements through the wearable device. As a result, the user can receive individualized feedback in real time, enabling efficient dance practice.

[1286] Furthermore, this invention includes a server that receives individual dance style and preference information from multiple users and generates an integrated dance performance as a group; a server that assigns individual choreography parts to each member; a server that acquires movement data of each member in real time and analyzes the performance of the entire group; and a server that provides feedback to the entire group and individual members to support performance improvement. This enables multiple users to efficiently learn group dance performances.

[1287] A "dance style" refers to a type or form of dance based on a specific type of music or culture. Examples include jazz dance, hip hop, and ballet.

[1288] "Age" refers to the number of years elapsed since the user's birth. It is a factor that can be used as a reference for health status and physical fitness.

[1289] "Physical strength" refers to the user's physical abilities, such as physical endurance and muscle strength.

[1290] "Preferences" refer to personal tastes such as the dance style or type of music that a user particularly enjoys.

[1291] A "generative AI model" refers to a model that uses artificial intelligence technology to generate and analyze data. Examples include deep learning models and natural language processing models.

[1292] A "dance database" refers to a database containing data on various dance styles, choreography, and music.

[1293] An "original dance video" refers to a unique dance video generated by a generative AI model based on the user's requests.

[1294] "Music" refers to the music tracks used to accompany the dance. These are also selected based on the user's preferences and requests.

[1295] A "user terminal" refers to a device used by a user to operate the dance system. Examples include smartphones, tablets, and PCs.

[1296] A "wearable device" refers to a device that a user wears on their body. It is used to acquire motion data in real time.

[1297] "Motion data" refers to data about the user's body movements acquired through wearable devices.

[1298] "Feedback" refers to advice and suggestions for improvement provided by the server to the user.

[1299] "Choreography part" refers to the individual dance movements and roles assigned to each member in a dance performance involving multiple people.

[1300] A "group" refers to a team or collective formed by multiple users to perform dances together.

[1301] "Performance" refers to a dance demonstration by a user or group.

[1302] This invention is a system that utilizes specific hardware and software to allow users to communicate information such as their dance style and image, age, physical fitness, and preferences to an AI. Based on this information, the AI ​​generates original dances and further supports practice and instruction. Each element of this system is described in detail below.

[1303] Creating original dances

[1304] User:

[1305] Users launch a dedicated application on their smartphone, tablet, or PC. Using the application's chat window or form, they enter their desired dance style (e.g., jazz dance), level (e.g., intermediate), target age group (e.g., 20s), and musical preference (e.g., uptempo). Once they have finished entering the information, they press the submit button.

[1306] Terminal:

[1307] The terminal receives information entered by the user, converts it to a specified data format, and sends it to the server. During this process, the terminal uses an internet connection to transmit data in real time.

[1308] server:

[1309] The server analyzes the information sent from the user's device. Based on the analyzed information, it searches the database for choreography and music suitable for the specified dance style and conditions. Using a generative AI model (e.g., a deep learning model), it integrates these elements to generate an original dance video and music. The generated dance video and music are then sent to the user's device.

[1310] Specific example

[1311] For example, if a user requests "a jazz dance style, intermediate level, suitable for young people in their 20s, and they like upbeat music," the device sends this information to the server. The server uses an AI model to extract elements of jazz dance and generates original choreography and music. This generated jazz dance video and music are sent to the user's device and presented to the user.

[1312] Support for dance practice

[1313] User:

[1314] When practicing dance choreography, the user wears an AI-enabled wearable device (e.g., a smart band) on their forehead, abdomen, wrists, and ankles. Afterward, they select dance practice mode in the app and play a dance video.

[1315] Terminal:

[1316] The device works in conjunction with a wearable device to collect user movement data in real time. It also uses the device's camera to record the user's dance. This collected data and video are then sent to a server.

[1317] server:

[1318] The server analyzes the received motion data and video, comparing them to the generated original dance to identify problems. Based on the problems identified using the AI ​​model, it generates specific instructions and feedback for the user. The feedback is sent to the user's device.

[1319] Terminal:

[1320] The device displays feedback received from the server to the user. It also uses wearable devices to guide the user in correcting their actions through vibrations and other means.

[1321] Specific example

[1322] For example, when a user is practicing jazz dance, a wearable device and camera are used to collect motion data. The data sent to the server is analyzed by AI, which identifies specific problems such as "insufficient hip movement." As a result, the device displays feedback to the user, such as "move your hips a little more."

[1323] Dance creation and instruction by multiple people

[1324] User:

[1325] When multiple people dance together, each member launches the app and sets themselves to belong to the same group. Each member then enters their own dance request information.

[1326] Terminal:

[1327] The information of each user is converted into a data packet and sent to the server along with the group ID.

[1328] server:

[1329] The server generates an integrated, original dance based on information from the entire group. It then determines individual choreography parts for each member and generates corresponding choreography and music. The generated dance data is then sent to each terminal.

[1330] Terminal:

[1331] Each member's device displays their individual choreography part and music. Furthermore, it works in conjunction with wearable devices to collect real-time motion data from each member and send it to the server.

[1332] server:

[1333] The server analyzes the operational data of multiple users and evaluates the performance of the group as a whole and each individual. Based on the evaluation results, it generates feedback and sends it to each terminal.

[1334] Terminal:

[1335] The terminal displays feedback to each member and supports performance correction. It also uses wearable devices to guide the correction process as needed.

[1336] Specific example

[1337] For example, if a group of five requests a "spectacular and energetic dance for a wedding," information about each member is sent to the server. The server generates a cohesive dance and assigns individual choreography and music. Each member's movement data is analyzed, and improvement suggestions are provided for the group as a whole and for individual members. This entire process allows users to efficiently master the dance.

[1338] Example of a prompt

[1339] For example, the prompt message if the user wants to do jazz dance would be as follows:

[1340] "Please create an intermediate-level jazz dance routine for young people in their 20s. It should be to an upbeat song."

[1341] Furthermore, the prompt message for when multiple people are involved is as follows:

[1342] "Please create a vibrant and energetic dance for a group of five to perform at a wedding."

[1343] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1344] Step 1:

[1345] Users launch a dedicated app on their smartphone, tablet, or PC.

[1346] Input: Users enter their desired dance style, level, target age group, and musical preferences in the chat window or form, and then press the submit button.

[1347] Output: The input data is received by the terminal.

[1348] Step 2:

[1349] After receiving information entered by the user, the terminal converts the data into a specified format.

[1350] Input: User input data.

[1351] Data processing: Conversion to data format.

[1352] Output: The converted data is sent from the terminal to the server.

[1353] Step 3:

[1354] The server receives data sent from the user's terminal and begins analysis.

[1355] Input: Converted data.

[1356] Data processing: The system analyzes incoming data and searches the database for choreography and music suitable for the specified dance style and conditions. A generative AI model is used to integrate these elements and generate original dance videos and music.

[1357] Output: The generated dance video and music are sent from the server to the user's device.

[1358] Step 4:

[1359] The user's device plays the dance video and music received from the server and presents them to the user.

[1360] Input: Generated dance video and music.

[1361] Data processing: Conversion to a data format for playback (if necessary).

[1362] Output: The user watches the original dance video and listens to the music.

[1363] Step 5:

[1364] Users wear an AI-powered wearable device while practicing dance choreography.

[1365] Input: Wearing a wearable device and selecting a dance practice mode in the app.

[1366] Output: The wearable device is ready to collect motion data.

[1367] Step 6:

[1368] The device works in conjunction with a wearable device to collect user movement data in real time. It also records the user's dance using the device's camera.

[1369] Input: User action data and video.

[1370] Data processing: Acquisition of motion data and video, and format conversion.

[1371] Output: The collected data and video are sent to the server.

[1372] Step 7:

[1373] The server analyzes the received motion data and video, compares it to the generated original dance, and identifies any problems.

[1374] Input: Collected motion data and video.

[1375] Data processing: Comparison of motion data with generated dance videos. Identification and analysis of problems.

[1376] Output: Specific guidance and feedback based on the problems are generated and sent to the user's device.

[1377] Step 8:

[1378] The device displays feedback received from the server to the user. It also uses wearable devices to guide the user in correcting their actions through vibrations and other means.

[1379] Input: Feedback data and instruction content.

[1380] Data processing: Display of feedback and vibration control.

[1381] Output: The feedback is displayed to the user, and guidance is provided for correcting the behavior.

[1382] Step 9:

[1383] When dancing with multiple people, each member launches the app and sets themselves to belong to the same group.

[1384] Input: Dance request information for each member.

[1385] Output: Information about each member is sent from the terminal to the server.

[1386] Step 10:

[1387] The server generates an integrated, original dance based on information from the entire group. It then determines individual choreography parts for each member and generates the corresponding choreography and music.

[1388] Input: Information for each member.

[1389] Data processing: Information analysis of the entire group, generation of integrated dance, and determination of individual choreography parts.

[1390] Output: The generated dance data is sent from the server to each terminal.

[1391] Step 11:

[1392] Each member's device displays their individual choreography part and music. Furthermore, it works in conjunction with wearable devices to collect real-time motion data from each member and send it to the server.

[1393] Input: Choreography data and motion data.

[1394] Data processing: Data display and collection and transmission of operational data.

[1395] Output: Operation data is sent to the server.

[1396] Step 12:

[1397] The server analyzes the operational data of multiple users and evaluates the performance of the group as a whole and each individual. Based on the evaluation results, it generates feedback and sends it to each terminal.

[1398] Input: Action data for each member.

[1399] Data processing: Analysis and evaluation of operational data, and generation of feedback.

[1400] Output: Feedback is sent to each device.

[1401] Step 13:

[1402] The terminal displays feedback to each member and supports performance correction. It also uses wearable devices to guide the correction process as needed.

[1403] Input: Feedback data.

[1404] Data processing: Controlling the display of feedback and correction guidelines.

[1405] Output: Feedback is displayed, and support is provided for correcting the behavior.

[1406] (Application Example 1)

[1407] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1408] While many factories currently utilize robots and automation technologies, efficiency can decrease if their operation patterns are not optimized. Furthermore, optimizing overall work efficiency is challenging when multiple robots and workers collaborate. These challenges need to be addressed because they lead to decreased productivity and increased costs.

[1409] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1410] In this invention, the server includes means for receiving information on the factory environment and robot characteristics from a user, means for using a generative AI model to generate operation patterns for the factory environment and robots based on the received information, and means for transmitting the generated operation patterns to the robots. This makes it possible to optimize the operation patterns of robots and workers in the factory and improve overall work efficiency.

[1411] A "user" is a person or organization that uses a system, and is the entity that inputs information or gives instructions for its actions.

[1412] "Dance style" refers to the form of body movements and choreography that are synchronized with a specific piece of music and rhythm, and is selected based on the user's preferences.

[1413] "Age" refers to the number of years calculated from the user's date of birth, and serves as a basis for adjusting the difficulty level of dances and movements.

[1414] "Physical fitness" refers to the user's athletic ability and physical capabilities, which affect the intensity and duration of dance and movement.

[1415] "Preferences" refer to information that indicates a user's likes and interests, and are used to customize the generated dance and movement patterns.

[1416] "Means of receiving information" refers to interfaces and devices that receive input data from users and analyze it.

[1417] A "generative AI model" refers to an artificial intelligence algorithm or framework that automatically generates optimal dance or movement patterns based on input data.

[1418] A "database" refers to a system that systematically stores existing data and information, and allows for efficient retrieval and utilization of that data.

[1419] An "original dance video" refers to video content that includes unique dance choreography generated based on user input.

[1420] "Music" refers to a combination of sounds used in conjunction with dance or movement, and is an integrated form of sound data containing various elements.

[1421] A "user terminal" refers to devices such as smartphones, tablets, and PCs that users directly operate to input information or receive feedback.

[1422] A "wearable device" refers to a device that a user wears on their body to collect motion data in real time and transmit it to a system.

[1423] "Real-time" refers to a state where data is acquired and processed almost instantly, and information is updated without delay.

[1424] A "motion pattern" refers to a sequence of movements optimized to achieve a specific purpose.

[1425] A "robot" is an automated mechanical device designed to perform a specified task or action.

[1426] "Feedback" refers to improvement suggestions and correction instructions provided to users or robots based on analyzed data.

[1427] "Factory environment" refers to the physical conditions, layout, temperature, humidity, and other factors of the site where the robot operates.

[1428] This invention is a system for optimizing the motion patterns of robots in a factory and improving overall work efficiency. The embodiments for carrying out this invention are described below.

[1429] Users input information about the factory environment and robot characteristics using devices such as smartphones, tablets, and PCs. For example, they input information such as the factory's temperature, humidity, layout, robot type, speed, and accuracy. This information is then transmitted from the device to the server.

[1430] The server analyzes the received information and uses a generative AI model to generate factory environment and robot operation patterns. The generative AI model automatically generates the optimal operation pattern based on the input data. This operation pattern is a sequence designed to allow the robot to operate with maximum efficiency.

[1431] The generated motion patterns are sent from the server to the robot. The robot performs the task according to the received motion pattern. Motion data from each robot is collected in real time and sent back to the server. The server analyzes this motion data and generates feedback to improve efficiency. This feedback is provided to the user terminal and the robot, allowing for modifications and improvements to the robot's movements.

[1432] For example, if a user specifies that the factory temperature is 22°C, humidity is 40%, layout is Type A, the robot is an arm type, and it operates at medium speed and high precision, the AI ​​model will generate an optimal motion pattern based on these conditions. The generated motion pattern is sent to the robot as a command sequence such as "move_arm_to_XYZ, pick_and_place, verify_and_adjust". The robot operates according to this command sequence, and the server analyzes the collected motion data and provides feedback with correction instructions as needed.

[1433] This series of processes optimizes the operation of robots in the factory, making it possible to improve overall work efficiency.

[1434] The hardware used includes user terminals (smartphones, tablets, PCs), servers, and factory robots. The software used includes generative AI models (implemented in Keras / TensorFlow), Python scripts for data analysis, and front-end applications for the user interface.

[1435] Example of a prompt:

[1436] "Factory temperature is 22°C, humidity is 40%, layout is Type A, robot is arm-type, medium speed, high precision."

[1437] This allows the system to generate optimal robot motion patterns suited to the factory environment, thereby improving work efficiency.

[1438] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1439] Step 1:

[1440] Users input information about the factory environment and robot characteristics using devices such as smartphones, tablets, and PCs. Specifically, they input details such as the factory temperature, humidity, layout, robot type, speed, and accuracy, and then press the submit button. The input format is as follows: "Factory temperature: 22°C, humidity: 40%, layout: Type A, robot: arm type, medium speed, high accuracy."

[1441] Step 2:

[1442] The terminal formats the information about the factory environment and robot characteristics received from the user and sends it to the server. Here, the terminal converts the input data into JSON format and sends it to the server over the network. This is a data transmission operation, where the input is the user's input information and the output is the data transmission to the server.

[1443] Step 3:

[1444] The server analyzes the received information. Specifically, the server loads a generative AI model for analyzing factory environment and robot characteristic information, and supplies the received data to the generative AI model as input data. The AI ​​model generates the optimal operation pattern based on this data. The input is JSON data received from the terminal, and the output is the optimal operation pattern.

[1445] Step 4:

[1446] The server sends the generated motion patterns to the robots. These generated motion patterns are formatted as motion sequences for each robot and applied to specific robots within the factory. Specifically, the server sends the optimal motion pattern to the robot control unit. The input is the motion pattern generated from the AI ​​model, and the output is the transmission of the motion pattern to the robot.

[1447] Step 5:

[1448] The robots perform tasks according to motion patterns received from the server. Each robot executes the specified movements based on the received sequence. This allows the robots to efficiently perform specific work processes within the factory. The input is the motion pattern received from the server, and the output is the actual robot movement.

[1449] Step 6:

[1450] Robot motion data is collected in real time and transmitted to a server. The robot uses its built-in sensors and monitoring system to record motion data and transmit it to the server. Input is data obtained from the robot's motion sensors, and output is data transmission to the server.

[1451] Step 7:

[1452] The server analyzes the received robot motion data. Specifically, a data analysis module within the server analyzes the motion data and generates feedback necessary to improve the efficiency of factory work. The input is the motion data received from the robot, and the output is the feedback for efficiency improvement.

[1453] Step 8:

[1454] The server provides the generated feedback to the user terminal and the robot. The server sends the feedback to the user terminal, presenting the user with specific suggestions for improvement. Similarly, feedback is sent to the robot, allowing for modifications and adjustments to its operation. The input is the server's analysis results, and the output is the transmission of feedback to the user terminal and the robot.

[1455] By executing the above steps sequentially, the operation of factory robots is optimized, resulting in improved overall work efficiency.

[1456] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1457] This invention is a system that generates original dances tailored to the user by allowing the user to communicate their dance style, image, emotions, age, physical condition, and preferences to an AI, and then supports their practice and instruction. In addition to this system, an emotion engine that recognizes the user's emotions is incorporated to achieve more advanced feedback and customization. It is configured as follows:

[1458] 1. Creating an original dance

[1459] User:

[1460] Users launch the app on their smartphones, tablets, PCs, or other devices and input information such as their dance style, image, emotions, age, physical condition, and preferences through an interface such as a chat window. Once they have finished inputting the information, they press the submit button.

[1461] Terminal:

[1462] The terminal receives information entered by the user, formats the data, and sends it to the server.

[1463] server:

[1464] The server analyzes the received data and searches its database for dance moves and music that are suitable for the specified dance style, conditions, and the user's emotions. Using an AI model, it integrates these elements to generate an original dance video and music. The generated video and music are then sent to the device.

[1465] Specific example

[1466] For example, if a user requests "a jazz dance style, a fun mood, intermediate level, suitable for young people in their 20s, and they like upbeat music," the device sends this information to the server. The server uses an AI model and emotion engine to generate choreography and music that reflect the elements of jazz dance and a "fun" atmosphere. The generated jazz dance video and music are sent to the user's device and presented to the user.

[1467] 2. Support for dance practice

[1468] User:

[1469] When practicing choreography, users wear AI-enabled wearable devices on their forehead, abdomen, both wrists, and both ankles. Next, they select a dance practice mode in the app and start playing the dance.

[1470] Terminal:

[1471] The device works in conjunction with a wearable device to collect user movement data in real time. It also uses the device's camera to record the user's dance. The collected data and video are sent to a server.

[1472] server:

[1473] The server analyzes the received data and video, comparing them to the generated original dance to identify problems. Furthermore, it uses an emotion engine to analyze the user's emotions during practice in real time and adaptively adjust the feedback. Based on the identified problems and the emotion analysis results, it generates feedback to specifically instruct the user on how to correct them and sends it to the terminal.

[1474] Terminal:

[1475] The device displays feedback to the user and provides support to correct actions according to the instructions. It also uses wearable devices such as vibration to guide the user in correcting their actions.

[1476] Specific example

[1477] The system uses a wearable device and camera worn by the user to acquire movement data from the jazz dance they are practicing. The data sent to the server is analyzed by AI and an emotion engine to identify problems such as "insufficient hip movement" and emotions such as "feeling frustrated." As a result, the device provides the user with specific feedback such as "move your hips a little more" and emotionally responsive advice such as "relax and enjoy yourself."

[1478] 3. Creating and teaching dances with multiple people

[1479] User:

[1480] If multiple people wish to dance together, each member must launch the app and set themselves to belong to the same group. All members then enter their dance request information and emotional information.

[1481] Terminal:

[1482] The information of each user is converted into a data packet and sent to the server along with the group ID.

[1483] server:

[1484] The server generates an integrated, original dance based on information from the entire group. It then determines individual choreography parts for each member and generates corresponding choreography and music for each. The generated data is then sent to the terminal.

[1485] Terminal:

[1486] Each member's device displays their individual choreography part and music. Wearable devices are used to collect real-time movement data from each member and send it to a server.

[1487] server:

[1488] The server analyzes the behavioral data of multiple individuals and evaluates the performance of the entire group and each individual. Furthermore, it uses an emotion engine to analyze the emotional state of the entire group and generates emotion-based feedback. Based on the evaluation results and emotion analysis results, it generates individual and group-wide feedback and sends it to the terminal.

[1489] Terminal:

[1490] The device displays feedback to each member, supporting performance improvements and emotional enhancement. It also uses wearable devices to guide behavioral corrections as needed.

[1491] Specific example

[1492] When a group of five requests a "spectacular and energetic dance for a wedding," information about each member is sent to the server. The server uses AI and an emotion engine to generate a cohesive dance, assigning individual choreography and music. The emotion engine analyzes each member's emotional state and provides feedback to ensure the entire group enjoys the performance together. Each member's movement data is analyzed, and improvement instructions are provided for the group as a whole and for individual members. Through this process, users can efficiently master the dance while also receiving emotional support.

[1493] The following describes the processing flow.

[1494] 1. Creating an original dance

[1495] Step 1:

[1496] User:

[1497] Launch the app and enter information such as dance style, image, emotions, age, physical condition, and preferences into the chat window.

[1498] Step 2:

[1499] Terminal:

[1500] The system receives the input information, formats it into a data packet, and sends it to the server.

[1501] Step 3:

[1502] server:

[1503] The system analyzes received data packets and searches a database for dance style, music genre, and emotional data based on the input information.

[1504] Step 4:

[1505] server:

[1506] Using AI models and an emotion engine, the system generates original dance videos and music based on search results.

[1507] Step 5:

[1508] server:

[1509] The generated original dance video and music are sent to the device.

[1510] Step 6:

[1511] Terminal:

[1512] The received original dance video and music are displayed and played on the user interface.

[1513] 2. Support for dance practice

[1514] Step 1:

[1515] User:

[1516] AI-enabled wearable devices are attached to the head, abdomen, both wrists, and both ankles.

[1517] Select dance practice mode in the app.

[1518] Step 2:

[1519] Terminal:

[1520] Wearable devices are connected via Bluetooth or other communication methods to acquire motion data and emotion data in real time.

[1521] The device's camera records the user's dance.

[1522] Step 3:

[1523] Terminal:

[1524] The collected motion data, emotion data, and recorded video are bundled into data packets and sent to the server.

[1525] Step 4:

[1526] server:

[1527] The received motion data, emotion data, and recorded video are analyzed using an AI model and emotion engine, and compared to the generated original dance.

[1528] Step 5:

[1529] server:

[1530] Identify operational problems, refine feedback based on real-time sentiment data, and generate specific corrective instructions and advice.

[1531] Step 6:

[1532] server:

[1533] The generated feedback information is sent to the device.

[1534] Step 7:

[1535] Terminal:

[1536] Display feedback information in the user interface.

[1537] Wearable devices are used to provide guidance through vibrations and other means.

[1538] 3. Creating and teaching dances with multiple people

[1539] Step 1:

[1540] User:

[1541] Each member launches the app and sets themselves to belong to the same group.

[1542] Enter your dance style, image, emotions, and performance preferences.

[1543] Step 2:

[1544] Terminal:

[1545] The input information of each member is converted into a data packet and sent to the server along with the group ID.

[1546] Step 3:

[1547] server:

[1548] Based on information from the entire group, it generates an original dance that conveys a sense of unity.

[1549] Assign choreography parts to each member.

[1550] Step 4:

[1551] server:

[1552] The generated dance video, music, and choreography parts are sent to the device.

[1553] Step 5:

[1554] Terminal:

[1555] The system displays the choreography part and music assigned to each member on their device.

[1556] The system connects to wearable devices to acquire motion and emotion data in real time and transmits it to a server.

[1557] Step 6:

[1558] server:

[1559] We analyze the behavioral and emotional data of multiple individuals to evaluate the performance of the group as a whole and individual members.

[1560] Step 7:

[1561] server:

[1562] It generates group-wide and individual feedback, further refines the feedback based on sentiment data, and sends it to the device.

[1563] Step 8:

[1564] Terminal:

[1565] Provide feedback to each member and offer suggestions for revisions.

[1566] Provide guidance via vibration from wearable devices as needed.

[1567] Through these steps, users can efficiently learn dance and receive support that takes their emotional state into consideration.

[1568] (Example 2)

[1569] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1570] Conventional dance generation and practice support systems failed to adequately reflect users' individual preferences and emotions, and their motion corrections and feedback during practice were not adapted to the user's emotional state. Furthermore, when generating dance performances for multiple people, there were insufficient methods for coordinating the group's overall choreography with individual choreography. In addition, real-time motion data collection and analysis lacked integration with emotion analysis.

[1571] The identification processing performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for receiving information from the user regarding dance style, age, physical strength, preferences, and emotions; means for analyzing the received information and generating original dance videos and music using a generation AI model based on an existing dance database; and means for transmitting the generated original dance videos and music to the user terminal. This makes it possible to generate original dance videos and music that reflect individual preferences and emotions.

[1572] The server also includes means for acquiring motion data in real time through a wearable device worn by the user; means for analyzing the acquired motion data and identifying problems by comparing it with the generated original dance video; means for analyzing the user's emotions using an emotion engine and generating feedback based on that; means for providing guidance and feedback to the user based on the identified problems and emotions; and means for guiding the user's motion correction through the wearable device. This makes it possible for motion correction and feedback during practice to be adapted to the user's emotional state.

[1573] Furthermore, the server includes means for receiving individual dance styles, preferences, and emotional information from multiple users to generate an integrated dance performance as a group; means for assigning individual choreography parts to each member; means for acquiring movement data of each member in real time and analyzing the overall performance of the group; and means for generating feedback to the group as a whole and to individual members using a generative AI model and an emotion engine to support performance improvement. This makes it possible to generate and improve dance performances involving multiple people.

[1574] "Dance style" refers to the type or form of dance desired by the user, and examples include jazz dance, ballet, and hip hop.

[1575] "Age" refers to the user's age group and is information used to determine appropriate dance choreography and exercise intensity.

[1576] "Physical fitness" refers to the user's physical ability and endurance, and serves as a criterion for adjusting the difficulty and intensity of the dance.

[1577] "Preferences" refer to a user's likes and interests, and include preferences such as favorite music genres and dance styles.

[1578] "Emotions" refers to the emotional state a user experiences while dancing, and includes feelings such as happiness, sadness, and relaxation.

[1579] A "wearable device" is a device worn and used by a user to acquire movement data in real time. Examples include smartwatches and fitness trackers.

[1580] A "generative AI model" is a model that uses artificial intelligence to analyze and generate data, and is used to generate original dance videos and music based on specific inputs.

[1581] An "emotion engine" is a system that analyzes a user's emotional state and provides optimal feedback and guidance based on that analysis.

[1582] "Feedback" refers to information that provides suggestions and advice for improving a user's actions and practice methods.

[1583] A "database" is a system that stores information on dance moves and music, and is used to search for and retrieve appropriate data based on user input.

[1584] An "original dance video" is a video of a custom dance, generated using an AI model and tailored to the user's requests.

[1585] "Original music" refers to custom music generated using AI models, tailored to the user's requests.

[1586] This invention is a system that generates personalized, original dances based on the user's dance style, image, emotions, age, physical condition, and preferences communicated to the AI, and supports practice and instruction. The system incorporates an emotion engine that recognizes the user's emotions in real time and reflects them in the feedback.

[1587] Creating original dances

[1588] User:

[1589] Users launch the app using their smartphone, tablet, or PC and enter information such as their dance style, image, emotions, age, fitness level, and preferences through the app's chat window. For example, they might enter information such as, "I like jazz dance, I'm in a fun mood, I'm intermediate level, I'm in my 20s, and I like upbeat music."

[1590] Terminal:

[1591] The terminal receives user input information, formats it into the required format, and sends it to the server. This formatting may involve using data formats such as JSON.

[1592] server:

[1593] The server analyzes the received data and searches its database for dance moves and music that are suitable for the specified dance style, conditions, and the user's emotions. A generative AI model is used for this, and an emotion engine is also utilized to generate original dance videos and music that are adapted to the user's emotions. The generated videos and music are then sent back to the terminal.

[1594] Support for dance practice

[1595] User:

[1596] When practicing dance choreography, users wear AI-enabled wearable devices on their forehead, abdomen, both wrists, and both ankles. Next, they launch the app, select dance practice mode, and start playing the dance.

[1597] Terminal:

[1598] The device works in conjunction with a wearable device to collect user movement data in real time. It also uses the device's camera to record the user's dance movements. The collected data and video are sent to a server.

[1599] server:

[1600] The server analyzes the received data and video, comparing them to the generated original dance to identify problems. Furthermore, it uses an emotion engine to analyze the user's emotions during practice in real time and generates feedback based on that analysis. Based on the identified problems and the emotion analysis results, it generates feedback to provide the user with specific instructions on how to correct their behavior and sends it to the terminal.

[1601] Dance creation and instruction by multiple people

[1602] User:

[1603] When multiple people dance together, each member launches the app and sets themselves to belong to the same group. Each member then enters dance request information and emotion information.

[1604] Terminal:

[1605] Each member's data is converted into a data packet and sent to the server along with the group ID.

[1606] server:

[1607] The server analyzes the received information from the entire group and generates an integrated original dance. It determines individual choreography parts for each member and generates corresponding choreography and music. The generated data is sent to each member's terminal.

[1608] Terminal:

[1609] Each member's device displays their individual choreography part and music, and in conjunction with wearable devices, movement data from each member is collected in real time and sent to a server.

[1610] server:

[1611] The server analyzes the behavioral data of multiple individuals and evaluates the performance of the group as a whole and each individual. Furthermore, it uses an emotion engine to analyze the emotional state of the entire group and generates feedback based on that analysis. This feedback is sent to each member's terminal.

[1612] Specific example

[1613] For example, if a user enters a prompt such as "Jazz dance style, fun mood, intermediate level, for people in their 20s, likes upbeat music," the device sends this to the server. The server uses a generative AI model and emotion engine to generate choreography and music that reflects the elements of jazz dance and a fun atmosphere based on this information. This is then sent to the user's device, and the user can view the content in the app.

[1614] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1615] Creating original dances

[1616] Step 1:

[1617] User: Users launch the app using a smartphone, tablet, or PC and enter information about their dance style, image, emotions, age, fitness level, and preferences into the app's chat window. Specifically, they might enter information such as, "I like jazz dance, I'm in a fun mood, I'm intermediate level, I'm in my 20s, and I like upbeat music." The input is detailed information about the user's preferences and emotions, and the output is the transmission of this information.

[1618] Step 2:

[1619] Terminal: The terminal receives user input, converts the data into a format such as JSON, and sends it to the server. The input is information entered by the user, and the output is formatted data. Specifically, it performs the operation of converting the text data entered by the user into an appropriate format.

[1620] Step 3:

[1621] Server: The server receives formatted data and uses a generating AI model to search and analyze a database for dance moves and music that are suitable for the specified dance style, conditions, and the user's emotions. The input is formatted data received from the terminal, and the output is the generated dance moves and music. The server uses the AI ​​model to generate dance choreography patterns and music based on text information.

[1622] Step 4:

[1623] Server: The server searches and integrates generated dances and music from a database to produce original dance videos and music. The input is the search and analysis results, and the output is the integrated original dance video and music. The server uses a generation AI model to integrate the elements and perform the operation of generating new content.

[1624] Step 5:

[1625] Server: Sends the generated original dance video and music to the user's device. The input is the generated content, and the output is the data sent to the device.

[1626] Step 6:

[1627] Terminal: The terminal displays received dance videos and music to the user. Input is data sent from the server, and output is what is displayed to the user.

[1628] Support for dance practice

[1629] Step 1:

[1630] User: The user attaches AI-enabled wearable devices to their forehead, abdomen, both wrists, and both ankles. Next, they launch the app, select dance practice mode, and start playback. The input is the original dance video and the attached wearable devices, and the output is the start of practice mode.

[1631] Step 2:

[1632] Terminal: The terminal works in conjunction with a wearable device to collect user movement data in real time. It also uses the terminal's camera to record video of the user dancing. The input is user movement data and video, and the output is the collected real-time data.

[1633] Step 3:

[1634] Terminal: Sends collected data and video to the server. Inputs are real-time data and recorded video, and outputs are data sent to the server.

[1635] Step 4:

[1636] Server: The server analyzes the received data and video, compares them to the generated original dance, and identifies problems. Furthermore, it uses an emotion engine to analyze the user's emotions during practice in real time. The input is the received data and video, and the output is the problems and the emotion analysis results. The server performs the analysis using an AI model and an emotion engine.

[1637] Step 5:

[1638] Server: Based on the identified problems and sentiment analysis results, the server generates feedback to provide the user with specific instructions on how to correct them and sends it to the terminal. The input is the problems and analysis results, and the output is the generated feedback.

[1639] Step 6:

[1640] Terminal: The terminal displays feedback to the user and provides support to correct actions according to instructions. It also uses wearable devices to guide the user's actions through vibration, etc. Input is feedback received from the server, and output is the display and guidance to the user.

[1641] Dance creation and instruction by multiple people

[1642] Step 1:

[1643] User: When multiple people want to dance together, each member launches the app and sets themselves to belong to the same group. All members enter dance request information and emotion information. Input is group settings and request information, and output is setting complete and input complete.

[1644] Step 2:

[1645] Terminal: Converts each member's information into a data packet and sends it to the server along with the group ID. Input is information from group members, and output is a data packet.

[1646] Step 3:

[1647] Server: The server generates an integrated, original dance based on information from the entire group. It then determines individual choreography parts for each member and generates corresponding choreography and music for each. The input is multiple request information, and the output is the integrated dance and individual choreography data. The server uses an AI model and an emotion engine for generation.

[1648] Step 4:

[1649] Server: Sends the generated dance and music to each member's terminal. The input is the generated data, and the output is the data sent to the terminal.

[1650] Step 5:

[1651] Terminal: Each member's terminal displays their individual choreography part and music. It works in conjunction with wearable devices to collect real-time motion data from each member and send it to the server. Inputs are received data and motion data, and outputs are data sent to the server.

[1652] Step 6:

[1653] Server: The server analyzes the received behavioral data and evaluates the performance of the entire group and individual members. Furthermore, it uses an emotion engine to analyze the emotional state of the entire group and generate feedback. Inputs are behavioral and emotion data, and outputs are evaluation results and feedback.

[1654] Step 7:

[1655] Terminal: The terminal displays feedback to each member, supporting performance correction and emotional improvement. It also guides behavioral correction using wearable devices as needed. Input is feedback received from the server, and output is the display and guidance to the user.

[1656] (Application Example 2)

[1657] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1658] Traditional dance practice systems and teaching methods have struggled to adequately customize and accommodate individual user preferences and emotions. Furthermore, the lack of systems capable of analyzing user movement data and emotional states in real time and providing adaptive feedback based on those results often resulted in inefficient practice instruction and inadequate emotional support. Additionally, in group dance practice, there was a lack of means to analyze the overall group performance and emotional state and provide integrated feedback.

[1659] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving information from the user regarding dance style, age, physical strength, preferences, and emotions; means for analyzing the received information and generating original dance videos and music based on an existing dance database and a generation AI model; means for transmitting the generated original dance videos and music to the user terminal; and means for analyzing the user's emotions and providing feedback based on the analysis results. This enables the generation of customized dances that correspond to the user's individual preferences and emotions, efficient practice guidance, and adaptive emotional support.

[1660] Furthermore, the system includes means for acquiring motion data in real time through a wearable device worn by the user, means for analyzing the acquired motion data and identifying problems by comparing it with the generated original dance video, means for providing guidance and emotion-responsive feedback to the user based on the identified problems and emotion analysis results, and means for guiding the user to correct their movements through the wearable device, thereby enabling the user to correct their movements and improve their skills.

[1661] Furthermore, by including means for receiving individual dance style, preference, and emotional information from multiple users to generate an integrated dance performance as a group, means for assigning individual choreography parts to each member, means for acquiring movement data of each member in real time and analyzing the overall performance and emotions of the group, and means for providing feedback to the group as a whole and to individual members to support performance improvement and emotional enhancement, efficient dance practice for multiple people and improvement of the overall group performance can be achieved.

[1662] A "user" is someone who provides the system with information about their dance style, age, physical fitness, preferences, and emotions, and receives generated original dance videos and music, as well as practice and instruction.

[1663] "Dance style" refers to information that describes the specific type or form of dance that the user desires.

[1664] "Age" refers to information indicating the user's actual age.

[1665] "Physical fitness" refers to information that indicates the user's physical ability and endurance.

[1666] "Preferences" refer to information that indicates an individual user's tastes and tendencies, such as the type of music or dance style they prefer.

[1667] "Emotions" refer to information that indicates the feelings and mental state that the user is currently experiencing.

[1668] A "generative AI model" is an artificial intelligence model used to generate original dance videos and music based on user input.

[1669] A "prompt message" is a statement that indicates the format and content of the information that the user should input into the system.

[1670] An "original dance video" is a video of a dance that has been individually customized and generated based on information provided by the user.

[1671] "Music" refers to the audio data corresponding to the generated original dance video.

[1672] A "user terminal" is a hardware device used by a user to receive and play original dance videos and music.

[1673] "Emotional analysis" is the process of analyzing a user's emotional state and incorporating the results into the feedback.

[1674] "Feedback" refers to information that provides guidance and advice based on the user's dance performance and emotional state.

[1675] A "wearable device" is a device that a user wears to acquire real-time motion data.

[1676] "Motion data" refers to data that shows the user's body movements and location information.

[1677] "Performance" refers to the technical skill and expressiveness of a user's dance moves.

[1678] A "group" refers to a collective of multiple users performing a dance together.

[1679] A "choreographed part" refers to the individual dance movements assigned to each member of the group.

[1680] One embodiment of this invention is a system that generates original dance videos and music customized based on the dance style, age, physical fitness, preferences, and emotions entered by the user, and supports their practice and instruction. Specifically, it involves the following processing steps.

[1681] 1. Collecting user input

[1682] Users launch the application using devices such as smartphones, tablets, or PCs. Through the application's interface, users input their dance style, age, physical fitness level, preferences, and current emotions. This information is sent to the server.

[1683] 2. Generation of original dance videos and music

[1684] The server analyzes the received user information. Using an existing dance database and a generative AI model, it generates original dance videos and music tailored to the user's preferences and emotions. This generated content is then sent to the user's device.

[1685] 3. Emotional analysis and provision of feedback

[1686] The server uses an emotion engine to analyze the emotional information entered by the user. Based on the analysis results, it provides appropriate feedback to the user. For example, if the user's emotion during practice is "frustration," it will provide feedback such as "relax and enjoy yourself."

[1687] 4. Acquisition of motion data using wearable devices

[1688] During practice, the user wears AI-enabled wearable devices on their forehead, abdomen, both wrists, and both ankles. The terminal interacts with these wearable devices to collect the user's movement data in real time. It also uses the terminal's camera to record the user's dance. This data is sent to a server.

[1689] 5. Comparison of motion data and the generated original dance video

[1690] The server analyzes the received motion data and video, compares it to the generated original dance, and identifies problems. Based on the identified problems, it generates feedback to instruct the user on how to correct them and sends it to the user's terminal.

[1691] 6. Creating and teaching dances with multiple people

[1692] When multiple users register as a group, each member enters their individual dance style, preferences, and emotional information into the application. This information is sent to the server, which generates an integrated original dance performance for the entire group. The server then assigns individual choreography parts to each member. In addition, it acquires movement data from each member in real time, analyzes the group's overall and individual performances, and provides feedback.

[1693] Examples

[1694] Possible prompt statements include the following:

[1695] Favorite dance style: Hip Hop

[1696] Current emotion: Happy

[1697] Dance level: Intermediate

[1698] Age: 25

[1699] Preferences: Upbeat music

[1700] Based on this prompt, the server generates a fun, intermediate-level hip-hop style dance. Users can practice the dance while receiving real-time feedback. If frustration is detected, advice such as "Relax and have fun" is also provided.

[1701] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1702] Step 1:

[1703] Users launch the application on their smartphones, tablets, or PCs and input information about their dance style, age, physical fitness, preferences, and emotions. The input data is as follows: "Dance Style: Hip Hop," "Emotion: Happy," "Dance Level: Intermediate," "Age: 25," "Preference: Upbeat Music." This input data is then sent from the device to the server.

[1704] Step 2:

[1705] The server analyzes the user's input data. Based on the input information, it uses an existing dance database and a generative AI model to generate an original dance video and music tailored to the user. The generated dance video and music files are sent from the server to the user's terminal. The output is a customized original dance video and music file.

[1706] Step 3:

[1707] The user wears AI-enabled wearable devices on their forehead, abdomen, both wrists, and both ankles, and selects a dance practice mode within the application. The terminal interacts with the wearable devices to collect the user's movement data in real time. This movement data includes information such as the position of each joint and the speed of movement. This movement data is transmitted from the terminal to the server.

[1708] Step 4:

[1709] The server analyzes the received motion data and compares it to the generated original dance video. Using an AI model, it compares the user's movements to accurate dance moves and identifies problems. For example, it might identify a problem such as "insufficient hip movement." The analysis results are stored on the server as feedback information.

[1710] Step 5:

[1711] The server uses an emotion engine to analyze the emotional information entered by the user. Based on the analysis results, it evaluates the user's emotional state during practice and generates adaptive feedback. For example, if the user is feeling "frustrated," it will generate feedback such as "relax and enjoy yourself." The output consists of the emotional state analysis result and the feedback statement.

[1712] Step 6:

[1713] Based on the identified problems and emotional state, the server generates specific instructions and emotionally responsive feedback for the user and sends them to the user's terminal. The terminal displays this to the user, providing instructions for correcting actions such as "Move your hips a little more" and emotional advice such as "Relax and enjoy yourself."

[1714] Step 7:

[1715] Based on the feedback received from the user, the dance is modified, and motion data is collected again. Then, by repeating the process from steps 3 to 6, the user's performance improves and emotional support is continuously provided.

[1716] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1717] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1718] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[1719] [Fourth Embodiment]

[1720] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[1721] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1722] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1723] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[1724] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1725] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1726] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1727] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[1728] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1729] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1730] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1731] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1732] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1733] This invention is a system that generates original dances tailored to the user by allowing the user to communicate their dance style, image, age, physical ability, and preferences to an AI, and then supports their practice and instruction. The system is configured as follows:

[1734] 1. Creating an original dance

[1735] User:

[1736] Users launch the app on their smartphones, tablets, PCs, or other devices and enter information such as their dance style and image, age, physical condition, and preferences through an interface such as a chat window. Once they have finished entering the information, they press the submit button.

[1737] Terminal:

[1738] The terminal receives information entered by the user, formats the data, and sends it to the server.

[1739] server:

[1740] The server analyzes the received data and searches its database for dance moves and music that match the specified dance style and conditions. Using an AI model, it integrates these elements to generate an original dance video and music. The generated video and music are then sent to the terminal.

[1741] Specific example

[1742] For example, if a user requests "a jazz dance style, intermediate level, aimed at young people in their 20s, and with an upbeat tempo," the device sends this information to the server. The server uses an AI model to extract elements of jazz dance and generate original choreography and music. The generated jazz dance video and music are sent to the user's device and presented to them.

[1743] 2. Support for dance practice

[1744] User:

[1745] When practicing choreography, users wear AI-enabled wearable devices on their forehead, abdomen, both wrists, and both ankles. Next, they select a dance practice mode in the app and start playing the dance.

[1746] Terminal:

[1747] The device works in conjunction with a wearable device to collect user movement data in real time. It also uses the device's camera to record the user's dance. The collected data and video are sent to a server.

[1748] server:

[1749] The server analyzes the received data and video, compares them to the generated original dance, and identifies problems. Based on the identified problems, it generates feedback to the user, providing specific instructions on how to correct them, and sends it to the terminal.

[1750] Terminal:

[1751] The device displays feedback to the user and provides support to correct actions according to the instructions. It also uses wearable devices such as vibration to guide the user in correcting their actions.

[1752] Specific example

[1753] The system uses a wearable device and camera worn by the user to acquire movement data from the jazz dance they are practicing. The data sent to the server is analyzed by AI to identify problems such as insufficient hip movement. As a result, the device provides the user with specific feedback such as, "Let's move our hips a little more."

[1754] 3. Creating and teaching dances with multiple people

[1755] User:

[1756] If multiple people wish to dance together, each member must launch the app and set themselves to belong to the same group. All members then enter their dance request information.

[1757] Terminal:

[1758] The information of each user is converted into a data packet and sent to the server along with the group ID.

[1759] server:

[1760] The server generates an integrated, original dance based on information from the entire group. It then determines individual choreography parts for each member and generates corresponding choreography and music for each. The generated data is then sent to the terminal.

[1761] Terminal:

[1762] Each member's device displays their individual choreography part and music. Wearable devices are used to collect real-time movement data from each member and send it to a server.

[1763] server:

[1764] The server analyzes the operational data of multiple users and evaluates the performance of the group as a whole and each individual. Based on the evaluation results, it generates individual and group-wide feedback and sends it to the terminal.

[1765] Terminal:

[1766] The terminal displays feedback to each member and supports performance correction. Additionally, wearable devices are used to guide the correction process as needed.

[1767] Specific example

[1768] When a group of five requests a "spectacular and energetic dance for a wedding," information about each member is sent to the server. The server generates a cohesive dance and assigns individual choreography and music. Each member's movement data is analyzed, and improvement suggestions are provided for the group as a whole and for individual members. This entire process allows users to efficiently master the dance.

[1769] The following describes the processing flow.

[1770] 1. Creating an original dance

[1771] Step 1:

[1772] User:

[1773] Launch the app and enter information such as dance style, image, age, physical condition, and preferences into the chat window.

[1774] Step 2:

[1775] Terminal:

[1776] The system receives the input information, formats it into a data packet, and sends it to the server.

[1777] Step 3:

[1778] server:

[1779] The system analyzes received data packets and searches the database for dance styles and music genres based on the input information.

[1780] Step 4:

[1781] server:

[1782] Using an AI model, original dance videos and music are generated based on search results.

[1783] Step 5:

[1784] server:

[1785] The generated original dance video and music are sent to the device.

[1786] Step 6:

[1787] Terminal:

[1788] The received original dance video and music are displayed and played on the user interface.

[1789] 2. Support for dance practice

[1790] Step 1:

[1791] User:

[1792] AI-enabled wearable devices are attached to the head, abdomen, both wrists, and both ankles.

[1793] Select dance practice mode in the app.

[1794] Step 2:

[1795] Terminal:

[1796] Wearable devices are connected via Bluetooth or other communication methods to acquire operational data in real time.

[1797] The device's camera records the user's dance.

[1798] Step 3:

[1799] Terminal:

[1800] The collected motion data and recorded video are bundled into data packets and sent to the server.

[1801] Step 4:

[1802] server:

[1803] The received motion data and recorded video are analyzed using an AI model, and then compared to the generated original dance.

[1804] Step 5:

[1805] server:

[1806] Identify operational problems and generate specific correction instructions and advice.

[1807] Step 6:

[1808] server:

[1809] The generated feedback information is sent to the device.

[1810] Step 7:

[1811] Terminal:

[1812] Display feedback information in the user interface.

[1813] Wearable devices are used to provide guidance through vibrations and other means.

[1814] 3. Creating and teaching dances with multiple people

[1815] Step 1:

[1816] User:

[1817] Each member launches the app and sets themselves to belong to the same group.

[1818] Enter your dance style, image, and performance preferences.

[1819] Step 2:

[1820] Terminal:

[1821] The input information of each member is converted into a data packet and sent to the server along with the group ID.

[1822] Step 3:

[1823] server:

[1824] Based on information from the entire group, it generates an original dance that conveys a sense of unity.

[1825] Assign choreography parts to each member.

[1826] Step 4:

[1827] server:

[1828] The generated dance video, music, and choreography parts are sent to the device.

[1829] Step 5:

[1830] Terminal:

[1831] The system displays the choreography part and music assigned to each member on their device.

[1832] The system connects to wearable devices to acquire real-time motion data and transmits it to a server.

[1833] Step 6:

[1834] server:

[1835] Analyze the motion data of multiple individuals to evaluate the performance of the group as a whole and individual members.

[1836] Step 7:

[1837] server:

[1838] Generate and send group-wide and individual feedback to the device.

[1839] Step 8:

[1840] Terminal:

[1841] Provide feedback to each member and offer suggestions for revisions.

[1842] Provide guidance via vibration from wearable devices as needed.

[1843] Through these steps, users can efficiently learn dance and improve their performance.

[1844] (Example 1)

[1845] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1846] In recent years, the demand for dance has been increasing as a way to maintain health and as a form of entertainment, but finding a dance style and choreography that suits individual users is difficult. Furthermore, obtaining individualized feedback requires a professional instructor, which is time-consuming and costly. Additionally, efficient practice of dance performances by multiple people requires coordination and appropriate instruction for each member. There is a need for a system that can solve these problems and enable users to learn dance efficiently.

[1847] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1848] In this invention, the server includes means for receiving information from the user regarding dance style, age, physical fitness, and preferences; means for generating original dance videos and music based on an existing dance database using a generative AI model; and means for transmitting the generated original dance videos and music to the user terminal. This makes it possible to automatically provide the user with the most suitable dance style and choreography.

[1849] Furthermore, this invention includes a server that includes means for acquiring motion data in real time through a wearable device worn by the user, means for analyzing the acquired motion data and identifying problems by comparing it with the generated original dance video, means for providing guidance and feedback to the user based on the identified problems, and means for guiding the user to correct their movements through the wearable device. As a result, the user can receive individualized feedback in real time, enabling efficient dance practice.

[1850] Furthermore, this invention includes a server that receives individual dance style and preference information from multiple users and generates an integrated dance performance as a group; a server that assigns individual choreography parts to each member; a server that acquires movement data of each member in real time and analyzes the performance of the entire group; and a server that provides feedback to the entire group and individual members to support performance improvement. This enables multiple users to efficiently learn group dance performances.

[1851] A "dance style" refers to a type or form of dance based on a specific type of music or culture. Examples include jazz dance, hip hop, and ballet.

[1852] "Age" refers to the number of years elapsed since the user's birth. It is a factor that can be used as a reference for health status and physical fitness.

[1853] "Physical strength" refers to the user's physical abilities, such as physical endurance and muscle strength.

[1854] "Preferences" refer to personal tastes such as the dance style or type of music that a user particularly enjoys.

[1855] A "generative AI model" refers to a model that uses artificial intelligence technology to generate and analyze data. Examples include deep learning models and natural language processing models.

[1856] A "dance database" refers to a database containing data on various dance styles, choreography, and music.

[1857] An "original dance video" refers to a unique dance video generated by a generative AI model based on the user's requests.

[1858] "Music" refers to the music tracks used to accompany the dance. These are also selected based on the user's preferences and requests.

[1859] A "user terminal" refers to a device used by a user to operate the dance system. Examples include smartphones, tablets, and PCs.

[1860] A "wearable device" refers to a device that a user wears on their body. It is used to acquire motion data in real time.

[1861] "Motion data" refers to data about the user's body movements acquired through wearable devices.

[1862] "Feedback" refers to advice and suggestions for improvement provided by the server to the user.

[1863] "Choreography part" refers to the individual dance movements and roles assigned to each member in a dance performance involving multiple people.

[1864] A "group" refers to a team or collective formed by multiple users to perform dances together.

[1865] "Performance" refers to a dance demonstration by a user or group.

[1866] This invention is a system that utilizes specific hardware and software to allow users to communicate information such as their dance style and image, age, physical fitness, and preferences to an AI. Based on this information, the AI ​​generates original dances and further supports practice and instruction. Each element of this system is described in detail below.

[1867] Creating original dances

[1868] User:

[1869] Users launch a dedicated application on their smartphone, tablet, or PC. Using the application's chat window or form, they enter their desired dance style (e.g., jazz dance), level (e.g., intermediate), target age group (e.g., 20s), and musical preference (e.g., uptempo). Once they have finished entering the information, they press the submit button.

[1870] Terminal:

[1871] The terminal receives information entered by the user, converts it to a specified data format, and sends it to the server. During this process, the terminal uses an internet connection to transmit data in real time.

[1872] server:

[1873] The server analyzes the information sent from the user's device. Based on the analyzed information, it searches the database for choreography and music suitable for the specified dance style and conditions. Using a generative AI model (e.g., a deep learning model), it integrates these elements to generate an original dance video and music. The generated dance video and music are then sent to the user's device.

[1874] Specific example

[1875] For example, if a user requests "a jazz dance style, intermediate level, suitable for young people in their 20s, and they like upbeat music," the device sends this information to the server. The server uses an AI model to extract elements of jazz dance and generates original choreography and music. This generated jazz dance video and music are sent to the user's device and presented to the user.

[1876] Support for dance practice

[1877] User:

[1878] When practicing dance choreography, the user wears an AI-enabled wearable device (e.g., a smart band) on their forehead, abdomen, wrists, and ankles. Afterward, they select dance practice mode in the app and play a dance video.

[1879] Terminal:

[1880] The device works in conjunction with a wearable device to collect user movement data in real time. It also uses the device's camera to record the user's dance. This collected data and video are then sent to a server.

[1881] server:

[1882] The server analyzes the received motion data and video, comparing them to the generated original dance to identify problems. Based on the problems identified using the AI ​​model, it generates specific instructions and feedback for the user. The feedback is sent to the user's device.

[1883] Terminal:

[1884] The device displays feedback received from the server to the user. It also uses wearable devices to guide the user in correcting their actions through vibrations and other means.

[1885] Specific example

[1886] For example, when a user is practicing jazz dance, a wearable device and camera are used to collect motion data. The data sent to the server is analyzed by AI, which identifies specific problems such as "insufficient hip movement." As a result, the device displays feedback to the user, such as "move your hips a little more."

[1887] Dance creation and instruction by multiple people

[1888] User:

[1889] When multiple people dance together, each member launches the app and sets themselves to belong to the same group. Each member then enters their own dance request information.

[1890] Terminal:

[1891] The information of each user is converted into a data packet and sent to the server along with the group ID.

[1892] server:

[1893] The server generates an integrated, original dance based on information from the entire group. It then determines individual choreography parts for each member and generates corresponding choreography and music. The generated dance data is then sent to each terminal.

[1894] Terminal:

[1895] Each member's device displays their individual choreography part and music. Furthermore, it works in conjunction with wearable devices to collect real-time motion data from each member and send it to the server.

[1896] server:

[1897] The server analyzes the operational data of multiple users and evaluates the performance of the group as a whole and each individual. Based on the evaluation results, it generates feedback and sends it to each terminal.

[1898] Terminal:

[1899] The terminal displays feedback to each member and supports performance correction. It also uses wearable devices to guide the correction process as needed.

[1900] Specific example

[1901] For example, if a group of five requests a "spectacular and energetic dance for a wedding," information about each member is sent to the server. The server generates a cohesive dance and assigns individual choreography and music. Each member's movement data is analyzed, and improvement suggestions are provided for the group as a whole and for individual members. This entire process allows users to efficiently master the dance.

[1902] Example of a prompt

[1903] For example, the prompt message if the user wants to do jazz dance would be as follows:

[1904] "Please create an intermediate-level jazz dance routine for young people in their 20s. It should be to an upbeat song."

[1905] Furthermore, the prompt message for when multiple people are involved is as follows:

[1906] "Please create a vibrant and energetic dance for a group of five to perform at a wedding."

[1907] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1908] Step 1:

[1909] Users launch a dedicated app on their smartphone, tablet, or PC.

[1910] Input: Users enter their desired dance style, level, target age group, and musical preferences in the chat window or form, and then press the submit button.

[1911] Output: The input data is received by the terminal.

[1912] Step 2:

[1913] After receiving information entered by the user, the terminal converts the data into a specified format.

[1914] Input: User input data.

[1915] Data processing: Conversion to data format.

[1916] Output: The converted data is sent from the terminal to the server.

[1917] Step 3:

[1918] The server receives data sent from the user's terminal and begins analysis.

[1919] Input: Converted data.

[1920] Data processing: The system analyzes incoming data and searches the database for choreography and music suitable for the specified dance style and conditions. A generative AI model is used to integrate these elements and generate original dance videos and music.

[1921] Output: The generated dance video and music are sent from the server to the user's device.

[1922] Step 4:

[1923] The user's device plays the dance video and music received from the server and presents them to the user.

[1924] Input: Generated dance video and music.

[1925] Data processing: Conversion to a data format for playback (if necessary).

[1926] Output: The user watches the original dance video and listens to the music.

[1927] Step 5:

[1928] Users wear an AI-powered wearable device while practicing dance choreography.

[1929] Input: Wearing a wearable device and selecting a dance practice mode in the app.

[1930] Output: The wearable device is ready to collect motion data.

[1931] Step 6:

[1932] The device works in conjunction with a wearable device to collect user movement data in real time. It also records the user's dance using the device's camera.

[1933] Input: User action data and video.

[1934] Data processing: Acquisition of motion data and video, and format conversion.

[1935] Output: The collected data and video are sent to the server.

[1936] Step 7:

[1937] The server analyzes the received motion data and video, compares it to the generated original dance, and identifies any problems.

[1938] Input: Collected motion data and video.

[1939] Data processing: Comparison of motion data with generated dance videos. Identification and analysis of problems.

[1940] Output: Specific guidance and feedback based on the problems are generated and sent to the user's device.

[1941] Step 8:

[1942] The device displays feedback received from the server to the user. It also uses wearable devices to guide the user in correcting their actions through vibrations and other means.

[1943] Input: Feedback data and instruction content.

[1944] Data processing: Display of feedback and vibration control.

[1945] Output: The feedback is displayed to the user, and guidance is provided for correcting the behavior.

[1946] Step 9:

[1947] When dancing with multiple people, each member launches the app and sets themselves to belong to the same group.

[1948] Input: Dance request information for each member.

[1949] Output: Information about each member is sent from the terminal to the server.

[1950] Step 10:

[1951] The server generates an integrated, original dance based on information from the entire group. It then determines individual choreography parts for each member and generates the corresponding choreography and music.

[1952] Input: Information for each member.

[1953] Data processing: Information analysis of the entire group, generation of integrated dance, and determination of individual choreography parts.

[1954] Output: The generated dance data is sent from the server to each terminal.

[1955] Step 11:

[1956] Each member's device displays their individual choreography part and music. Furthermore, it works in conjunction with wearable devices to collect real-time motion data from each member and send it to the server.

[1957] Input: Choreography data and motion data.

[1958] Data processing: Data display and collection and transmission of operational data.

[1959] Output: Operation data is sent to the server.

[1960] Step 12:

[1961] The server analyzes the operational data of multiple users and evaluates the performance of the group as a whole and each individual. Based on the evaluation results, it generates feedback and sends it to each terminal.

[1962] Input: Action data for each member.

[1963] Data processing: Analysis and evaluation of operational data, and generation of feedback.

[1964] Output: Feedback is sent to each device.

[1965] Step 13:

[1966] The terminal displays feedback to each member and supports performance correction. It also uses wearable devices to guide the correction process as needed.

[1967] Input: Feedback data.

[1968] Data processing: Controlling the display of feedback and correction guidelines.

[1969] Output: Feedback is displayed, and support is provided for correcting the behavior.

[1970] (Application Example 1)

[1971] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1972] While many factories currently utilize robots and automation technologies, efficiency can decrease if their operation patterns are not optimized. Furthermore, optimizing overall work efficiency is challenging when multiple robots and workers collaborate. These challenges need to be addressed because they lead to decreased productivity and increased costs.

[1973] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1974] In this invention, the server includes means for receiving information on the factory environment and robot characteristics from a user, means for using a generative AI model to generate operation patterns for the factory environment and robots based on the received information, and means for transmitting the generated operation patterns to the robots. This makes it possible to optimize the operation patterns of robots and workers in the factory and improve overall work efficiency.

[1975] A "user" is a person or organization that uses a system, and is the entity that inputs information or gives instructions for its actions.

[1976] "Dance style" refers to the form of body movements and choreography that are synchronized with a specific piece of music and rhythm, and is selected based on the user's preferences.

[1977] "Age" refers to the number of years calculated from the user's date of birth, and serves as a basis for adjusting the difficulty level of dances and movements.

[1978] "Physical fitness" refers to the user's athletic ability and physical capabilities, which affect the intensity and duration of dance and movement.

[1979] "Preferences" refer to information that indicates a user's likes and interests, and are used to customize the generated dance and movement patterns.

[1980] "Means of receiving information" refers to interfaces and devices that receive input data from users and analyze it.

[1981] A "generative AI model" refers to an artificial intelligence algorithm or framework that automatically generates optimal dance or movement patterns based on input data.

[1982] A "database" refers to a system that systematically stores existing data and information, and allows for efficient retrieval and utilization of that data.

[1983] An "original dance video" refers to video content that includes unique dance choreography generated based on user input.

[1984] "Music" refers to a combination of sounds used in conjunction with dance or movement, and is an integrated form of sound data containing various elements.

[1985] A "user terminal" refers to devices such as smartphones, tablets, and PCs that users directly operate to input information or receive feedback.

[1986] A "wearable device" refers to a device that a user wears on their body to collect motion data in real time and transmit it to a system.

[1987] "Real-time" refers to a state where data is acquired and processed almost instantly, and information is updated without delay.

[1988] A "motion pattern" refers to a sequence of movements optimized to achieve a specific purpose.

[1989] A "robot" is an automated mechanical device designed to perform a specified task or action.

[1990] "Feedback" refers to improvement suggestions and correction instructions provided to users or robots based on analyzed data.

[1991] "Factory environment" refers to the physical conditions, layout, temperature, humidity, and other factors of the site where the robot operates.

[1992] This invention is a system for optimizing the motion patterns of robots in a factory and improving overall work efficiency. The embodiments for carrying out this invention are described below.

[1993] Users input information about the factory environment and robot characteristics using devices such as smartphones, tablets, and PCs. For example, they input information such as the factory's temperature, humidity, layout, robot type, speed, and accuracy. This information is then transmitted from the device to the server.

[1994] The server analyzes the received information and uses a generative AI model to generate factory environment and robot operation patterns. The generative AI model automatically generates the optimal operation pattern based on the input data. This operation pattern is a sequence designed to allow the robot to operate with maximum efficiency.

[1995] The generated motion patterns are sent from the server to the robot. The robot performs the task according to the received motion pattern. Motion data from each robot is collected in real time and sent back to the server. The server analyzes this motion data and generates feedback to improve efficiency. This feedback is provided to the user terminal and the robot, allowing for modifications and improvements to the robot's movements.

[1996] For example, if a user specifies that the factory temperature is 22°C, humidity is 40%, layout is Type A, the robot is an arm type, and it operates at medium speed and high precision, the AI ​​model will generate an optimal motion pattern based on these conditions. The generated motion pattern is sent to the robot as a command sequence such as "move_arm_to_XYZ, pick_and_place, verify_and_adjust". The robot operates according to this command sequence, and the server analyzes the collected motion data and provides feedback with correction instructions as needed.

[1997] This series of processes optimizes the operation of robots in the factory, making it possible to improve overall work efficiency.

[1998] The hardware used includes user terminals (smartphones, tablets, PCs), servers, and factory robots. The software used includes generative AI models (implemented in Keras / TensorFlow), Python scripts for data analysis, and front-end applications for the user interface.

[1999] Example of a prompt:

[2000] "Factory temperature is 22°C, humidity is 40%, layout is Type A, robot is arm-type, medium speed, high precision."

[2001] This allows the system to generate optimal robot motion patterns suited to the factory environment, thereby improving work efficiency.

[2002] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[2003] Step 1:

[2004] Users input information about the factory environment and robot characteristics using devices such as smartphones, tablets, and PCs. Specifically, they input details such as the factory temperature, humidity, layout, robot type, speed, and accuracy, and then press the submit button. The input format is as follows: "Factory temperature: 22°C, humidity: 40%, layout: Type A, robot: arm type, medium speed, high accuracy."

[2005] Step 2:

[2006] The terminal formats the information about the factory environment and robot characteristics received from the user and sends it to the server. Here, the terminal converts the input data into JSON format and sends it to the server over the network. This is a data transmission operation, where the input is the user's input information and the output is the data transmission to the server.

[2007] Step 3:

[2008] The server analyzes the received information. Specifically, the server loads a generative AI model for analyzing factory environment and robot characteristic information, and supplies the received data to the generative AI model as input data. The AI ​​model generates the optimal operation pattern based on this data. The input is JSON data received from the terminal, and the output is the optimal operation pattern.

[2009] Step 4:

[2010] The server sends the generated motion patterns to the robots. These generated motion patterns are formatted as motion sequences for each robot and applied to specific robots within the factory. Specifically, the server sends the optimal motion pattern to the robot control unit. The input is the motion pattern generated from the AI ​​model, and the output is the transmission of the motion pattern to the robot.

[2011] Step 5:

[2012] The robots perform tasks according to motion patterns received from the server. Each robot executes the specified movements based on the received sequence. This allows the robots to efficiently perform specific work processes within the factory. The input is the motion pattern received from the server, and the output is the actual robot movement.

[2013] Step 6:

[2014] Robot motion data is collected in real time and transmitted to a server. The robot uses its built-in sensors and monitoring system to record motion data and transmit it to the server. Input is data obtained from the robot's motion sensors, and output is data transmission to the server.

[2015] Step 7:

[2016] The server analyzes the received robot motion data. Specifically, a data analysis module within the server analyzes the motion data and generates feedback necessary to improve the efficiency of factory work. The input is the motion data received from the robot, and the output is the feedback for efficiency improvement.

[2017] Step 8:

[2018] The server provides the generated feedback to the user terminal and the robot. The server sends the feedback to the user terminal, presenting the user with specific suggestions for improvement. Similarly, feedback is sent to the robot, allowing for modifications and adjustments to its operation. The input is the server's analysis results, and the output is the transmission of feedback to the user terminal and the robot.

[2019] By executing the above steps sequentially, the operation of factory robots is optimized, resulting in improved overall work efficiency.

[2020] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[2021] This invention is a system that generates original dances tailored to the user by allowing the user to communicate their dance style, image, emotions, age, physical condition, and preferences to an AI, and then supports their practice and instruction. In addition to this system, an emotion engine that recognizes the user's emotions is incorporated to achieve more advanced feedback and customization. It is configured as follows:

[2022] 1. Creating an original dance

[2023] User:

[2024] Users launch the app on their smartphones, tablets, PCs, or other devices and input information such as their dance style, image, emotions, age, physical condition, and preferences through an interface such as a chat window. Once they have finished inputting the information, they press the submit button.

[2025] Terminal:

[2026] The terminal receives information entered by the user, formats the data, and sends it to the server.

[2027] server:

[2028] The server analyzes the received data and searches its database for dance moves and music that are suitable for the specified dance style, conditions, and the user's emotions. Using an AI model, it integrates these elements to generate an original dance video and music. The generated video and music are then sent to the device.

[2029] Specific example

[2030] For example, if a user requests "a jazz dance style, a fun mood, intermediate level, suitable for young people in their 20s, and they like upbeat music," the device sends this information to the server. The server uses an AI model and emotion engine to generate choreography and music that reflect the elements of jazz dance and a "fun" atmosphere. The generated jazz dance video and music are sent to the user's device and presented to the user.

[2031] 2. Support for dance practice

[2032] User:

[2033] When practicing choreography, users wear AI-enabled wearable devices on their forehead, abdomen, both wrists, and both ankles. Next, they select a dance practice mode in the app and start playing the dance.

[2034] Terminal:

[2035] The device works in conjunction with a wearable device to collect user movement data in real time. It also uses the device's camera to record the user's dance. The collected data and video are sent to a server.

[2036] server:

[2037] The server analyzes the received data and video, comparing them to the generated original dance to identify problems. Furthermore, it uses an emotion engine to analyze the user's emotions during practice in real time and adaptively adjust the feedback. Based on the identified problems and the emotion analysis results, it generates feedback to specifically instruct the user on how to correct them and sends it to the terminal.

[2038] Terminal:

[2039] The device displays feedback to the user and provides support to correct actions according to the instructions. It also uses wearable devices such as vibration to guide the user in correcting their actions.

[2040] Specific example

[2041] The system uses a wearable device and camera worn by the user to acquire movement data from the jazz dance they are practicing. The data sent to the server is analyzed by AI and an emotion engine to identify problems such as "insufficient hip movement" and emotions such as "feeling frustrated." As a result, the device provides the user with specific feedback such as "move your hips a little more" and emotionally responsive advice such as "relax and enjoy yourself."

[2042] 3. Creating and teaching dances with multiple people

[2043] User:

[2044] If multiple people wish to dance together, each member must launch the app and set themselves to belong to the same group. All members then enter their dance request information and emotional information.

[2045] Terminal:

[2046] The information of each user is converted into a data packet and sent to the server along with the group ID.

[2047] server:

[2048] The server generates an integrated, original dance based on information from the entire group. It then determines individual choreography parts for each member and generates corresponding choreography and music for each. The generated data is then sent to the terminal.

[2049] Terminal:

[2050] Each member's device displays their individual choreography part and music. Wearable devices are used to collect real-time movement data from each member and send it to a server.

[2051] server:

[2052] The server analyzes the behavioral data of multiple individuals and evaluates the performance of the entire group and each individual. Furthermore, it uses an emotion engine to analyze the emotional state of the entire group and generates emotion-based feedback. Based on the evaluation results and emotion analysis results, it generates individual and group-wide feedback and sends it to the terminal.

[2053] Terminal:

[2054] The device displays feedback to each member, supporting performance improvements and emotional enhancement. It also uses wearable devices to guide behavioral corrections as needed.

[2055] Specific example

[2056] When a group of five requests a "spectacular and energetic dance for a wedding," information about each member is sent to the server. The server uses AI and an emotion engine to generate a cohesive dance, assigning individual choreography and music. The emotion engine analyzes each member's emotional state and provides feedback to ensure the entire group enjoys the performance together. Each member's movement data is analyzed, and improvement instructions are provided for the group as a whole and for individual members. Through this process, users can efficiently master the dance while also receiving emotional support.

[2057] The following describes the processing flow.

[2058] 1. Creating an original dance

[2059] Step 1:

[2060] User:

[2061] Launch the app and enter information such as dance style, image, emotions, age, physical condition, and preferences into the chat window.

[2062] Step 2:

[2063] Terminal:

[2064] The system receives the input information, formats it into a data packet, and sends it to the server.

[2065] Step 3:

[2066] server:

[2067] The system analyzes received data packets and searches a database for dance style, music genre, and emotional data based on the input information.

[2068] Step 4:

[2069] server:

[2070] Using AI models and an emotion engine, the system generates original dance videos and music based on search results.

[2071] Step 5:

[2072] server:

[2073] The generated original dance video and music are sent to the device.

[2074] Step 6:

[2075] Terminal:

[2076] The received original dance video and music are displayed and played on the user interface.

[2077] 2. Support for dance practice

[2078] Step 1:

[2079] User:

[2080] AI-enabled wearable devices are attached to the head, abdomen, both wrists, and both ankles.

[2081] Select dance practice mode in the app.

[2082] Step 2:

[2083] Terminal:

[2084] Wearable devices are connected via Bluetooth or other communication methods to acquire motion data and emotion data in real time.

[2085] The device's camera records the user's dance.

[2086] Step 3:

[2087] Terminal:

[2088] The collected motion data, emotion data, and recorded video are bundled into data packets and sent to the server.

[2089] Step 4:

[2090] server:

[2091] The received motion data, emotion data, and recorded video are analyzed using an AI model and emotion engine, and compared to the generated original dance.

[2092] Step 5:

[2093] server:

[2094] Identify operational problems, refine feedback based on real-time sentiment data, and generate specific corrective instructions and advice.

[2095] Step 6:

[2096] server:

[2097] The generated feedback information is sent to the device.

[2098] Step 7:

[2099] Terminal:

[2100] Display feedback information in the user interface.

[2101] Wearable devices are used to provide guidance through vibrations and other means.

[2102] 3. Creating and teaching dances with multiple people

[2103] Step 1:

[2104] User:

[2105] Each member launches the app and sets themselves to belong to the same group.

[2106] Enter your dance style, image, emotions, and performance preferences.

[2107] Step 2:

[2108] Terminal:

[2109] The input information of each member is converted into a data packet and sent to the server along with the group ID.

[2110] Step 3:

[2111] server:

[2112] Based on information from the entire group, it generates an original dance that conveys a sense of unity.

[2113] Assign choreography parts to each member.

[2114] Step 4:

[2115] server:

[2116] The generated dance video, music, and choreography parts are sent to the device.

[2117] Step 5:

[2118] Terminal:

[2119] The system displays the choreography part and music assigned to each member on their device.

[2120] The system connects to wearable devices to acquire motion and emotion data in real time and transmits it to a server.

[2121] Step 6:

[2122] server:

[2123] We analyze the behavioral and emotional data of multiple individuals to evaluate the performance of the group as a whole and individual members.

[2124] Step 7:

[2125] server:

[2126] It generates group-wide and individual feedback, further refines the feedback based on sentiment data, and sends it to the device.

[2127] Step 8:

[2128] Terminal:

[2129] Provide feedback to each member and offer suggestions for revisions.

[2130] Provide guidance via vibration from wearable devices as needed.

[2131] Through these steps, users can efficiently learn dance and receive support that takes their emotional state into consideration.

[2132] (Example 2)

[2133] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[2134] Conventional dance generation and practice support systems failed to adequately reflect users' individual preferences and emotions, and their motion corrections and feedback during practice were not adapted to the user's emotional state. Furthermore, when generating dance performances for multiple people, there were insufficient methods for coordinating the group's overall choreography with individual choreography. In addition, real-time motion data collection and analysis lacked integration with emotion analysis.

[2135] The identification processing performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for receiving information from the user regarding dance style, age, physical strength, preferences, and emotions; means for analyzing the received information and generating original dance videos and music using a generation AI model based on an existing dance database; and means for transmitting the generated original dance videos and music to the user terminal. This makes it possible to generate original dance videos and music that reflect individual preferences and emotions.

[2136] The server also includes means for acquiring motion data in real time through a wearable device worn by the user; means for analyzing the acquired motion data and identifying problems by comparing it with the generated original dance video; means for analyzing the user's emotions using an emotion engine and generating feedback based on that; means for providing guidance and feedback to the user based on the identified problems and emotions; and means for guiding the user's motion correction through the wearable device. This makes it possible for motion correction and feedback during practice to be adapted to the user's emotional state.

[2137] Furthermore, the server includes means for receiving individual dance styles, preferences, and emotional information from multiple users to generate an integrated dance performance as a group; means for assigning individual choreography parts to each member; means for acquiring movement data of each member in real time and analyzing the overall performance of the group; and means for generating feedback to the group as a whole and to individual members using a generative AI model and an emotion engine to support performance improvement. This makes it possible to generate and improve dance performances involving multiple people.

[2138] "Dance style" refers to the type or form of dance desired by the user, and examples include jazz dance, ballet, and hip hop.

[2139] "Age" refers to the user's age group and is information used to determine appropriate dance choreography and exercise intensity.

[2140] "Physical fitness" refers to the user's physical ability and endurance, and serves as a criterion for adjusting the difficulty and intensity of the dance.

[2141] "Preferences" refer to a user's likes and interests, and include preferences such as favorite music genres and dance styles.

[2142] "Emotions" refers to the emotional state a user experiences while dancing, and includes feelings such as happiness, sadness, and relaxation.

[2143] A "wearable device" is a device worn and used by a user to acquire movement data in real time. Examples include smartwatches and fitness trackers.

[2144] A "generative AI model" is a model that uses artificial intelligence to analyze and generate data, and is used to generate original dance videos and music based on specific inputs.

[2145] An "emotion engine" is a system that analyzes a user's emotional state and provides optimal feedback and guidance based on that analysis.

[2146] "Feedback" refers to information that provides suggestions and advice for improving a user's actions and practice methods.

[2147] A "database" is a system that stores information on dance moves and music, and is used to search for and retrieve appropriate data based on user input.

[2148] An "original dance video" is a video of a custom dance, generated using an AI model and tailored to the user's requests.

[2149] "Original music" refers to custom music generated using AI models, tailored to the user's requests.

[2150] This invention is a system that generates personalized, original dances based on the user's dance style, image, emotions, age, physical condition, and preferences communicated to the AI, and supports practice and instruction. The system incorporates an emotion engine that recognizes the user's emotions in real time and refle...

Claims

1. A means of receiving information from users regarding their dance style, age, physical fitness, and preferences, A means for analyzing the received information and generating original dance videos and music based on an existing dance database, A means for transmitting the generated original dance video and music to the user terminal, A system that includes this.

2. A means of acquiring motion data in real time through a wearable device worn by the user, A means for analyzing the acquired motion data and comparing it with the generated original dance video to identify problems, Based on the identified problems, a means of providing guidance and feedback to users, Means for guiding the user to correct their actions through the wearable device, The system according to claim 1, including the following:

3. A means of receiving individual dance style and preference information from multiple users and generating an integrated dance performance as a group, A means of assigning individual choreography parts to each member, A means of acquiring real-time activity data from each member and analyzing the overall performance of the group, A means of providing feedback to the entire group and individual members to support performance improvement, The system according to claim 1, including the following:

Images