System

Abstract

A system is provided.SOLUTION: A system comprising: means for generating music for music therapy using a generation AI; means for inputting information on symptoms and preferences of patients; means for transmitting the input information to the generation AI; means for receiving music generated by the generation AI and distributing the music to terminals; and means for storing the music in the terminals of the patients and reproducing the music.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In the past, music therapy required the use of commercially available sound sources, which meant paying copyright fees. Furthermore, selecting music appropriate for a patient's symptoms and preferences is a specialized task that requires advanced knowledge and experience, making it difficult. This has led to the issue of patients not receiving the full benefits of music therapy. [Means for solving the problem]

[0005] The present invention provides a system that uses generative AI to generate music for music therapy. This system first includes a means for inputting information about the patient's symptoms and preferences. The input information is sent to the generative AI, which then generates music for music therapy that includes 1 / f fluctuations and tones that induce specific brain waves. The generated music data is then distributed via a server to the patient's device, where it is saved and played back. This series of processes makes it possible to provide music therapy that is suited to each patient's symptoms while reducing the burden of copyright fees.

[0006] "Generative AI" is an artificial intelligence that automatically creates music based on the patient's symptoms and preferences.

[0007] "Music therapy music" is music designed to alleviate specific symptoms and contains elements that induce 1 / f fluctuations and specific brain waves.

[0008] "Information about the patient's symptoms and preferences" is data about the patient's specific health conditions and musical preferences.

[0009] "Device" refers to an electronic device used by a patient, including a smartphone or portable music player.

[0010] "Music data" refers to digital music files created by generative AI.

[0011] "1 / f fluctuation" is one of the rhythms that exists in nature and is a characteristic of sound that is said to have a relaxing effect.

[0012] "Electroencephalograms" refer to the rhythm of electrical activity generated in the cerebral cortex, and include alpha waves, beta waves, theta waves, and gamma waves. [Brief explanation of the drawings]

[0013] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0014] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0015] First, the terms used in the following description will be explained.

[0016] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0017] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0018] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0019] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0020] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0021] [First embodiment]

[0022] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0023] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0024] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0025] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0026] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0027] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0028] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0029] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0030] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0031] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0032] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0033] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0034] This invention is a system that uses generative AI to generate music therapy songs and provide effective music therapy to patients. In this embodiment of the invention, three main elements - a server, a terminal, and a user - operate according to a series of flows.

[0035] overview

[0036] The server is the central device that manages the process of generating music therapy music for each patient using generative AI. The device is responsible for inputting and transmitting patient information and receiving and playing the generated music. The users are the patients themselves, who receive treatment by listening to music therapy music.

[0037] Program processing

[0038] Enter and submit patient information

[0039] The device inputs the patient's symptoms and preferences. Using a device such as a smartphone or tablet, the user inputs symptoms (insomnia, stress, anxiety, etc.), preferred sounds (waves, babbling, rustling leaves, etc.), and the brain waves they want to induce (alpha waves, beta waves, theta waves, gamma waves). The device converts this data into a secure format and sends it to the server.

[0040] Server-side processing

[0041] The server sends a music generation request to the AI ​​based on the patient information received from the device. The server analyzes the received data and extracts parameters (symptoms, preferred sounds, brain waves) for the AI ​​to generate music. The server then constructs a music generation request based on these parameters and sends it to the AI.

[0042] Music Generation

[0043] The generation AI generates music for music therapy based on requests sent from the server. The generation AI combines 1 / f fluctuations and tones that induce specified brain waves to create music that is optimal for the patient's symptoms. The generated music data is encoded and sent back to the server.

[0044] Music distribution and playback

[0045] The server sends the music data received from the AI ​​to the device, which receives it and stores it in local storage. The user can then play the music on their smartphone or portable music player to engage in music therapy.

[0046] Specific examples

[0047] Example 1: Treating insomnia

[0048] 1. The user has insomnia and launches a dedicated application on their smartphone.

[0049] 2. Select "insomnia" on the device, "wave sound" as your preferred sound, and "alpha waves" as the brain waves you want to induce.

[0050] 3. The device sends this information to the server.

[0051] 4. The server sends a request to the generation AI, asking it to generate music based on "insomnia," "the sound of waves," and "alpha waves."

[0052] 5. The AI ​​generates music containing the pleasant sounds of waves and alpha waves and sends it back to the server.

[0053] 6. The server sends the generated music data to the device.

[0054] 7. Users can play the songs before going to bed at night to treat insomnia.

[0055] Example 2: Stress relief

[0056] 1. The user is stressed and uses a portable music player.

[0057] 2. Select "Stress" on the device, "Babbling Stream" as your preferred sound, and "Theta Waves" as the brain waves you want to induce.

[0058] 3. The device sends the information to the server, and the server sends a music generation request to the generation AI.

[0059] 4. The generative AI creates music that includes the relaxing sound of a stream and theta waves.

[0060] 5. The server sends the music data to the device, which stores it.

[0061] 6. Users can play the music during their work breaks to reduce stress.

[0062] In this way, the system of the present invention allows patients to effectively receive music therapy tailored to their individual symptoms, and reduces the burden of expensive copyright fees for hospitals and clinics.

[0063] The processing flow will be explained below.

[0064] Step 1:

[0065] The device inputs the patient's symptoms and preferences.

[0066] Action 1.1: The user launches the application on their device and selects the symptom type (insomnia, stress, anxiety, etc.).

[0067] Action 1.2: The user selects their preferred sound (waves, babbling, rustling leaves, etc.).

[0068] Action 1.3: The user specifies the brain waves (alpha waves, beta waves, theta waves, gamma waves) they want to induce.

[0069] Action 1.4: Once you have completed the input, press the send button.

[0070] Step 2:

[0071] The terminal transmits the input patient information to the server.

[0072] Action 2.1: Convert input data to JSON format.

[0073] Action 2.2: Send the data to the server using a secure communications protocol (e.g., HTTPS).

[0074] Action 2.3: Wait for confirmation from the server that the data was sent successfully.

[0075] Step 3:

[0076] Based on the patient information received by the server, a request to generate music is sent to the generation AI.

[0077] Operation 3.1: The server analyzes the received data and extracts the necessary parameters (symptoms, preferred sounds, brain waves).

[0078] Action 3.2: Construct a music generation request using the extracted parameters.

[0079] Action 3.3: Send a music generation request to the generation AI endpoint.

[0080] Action 3.4: Confirmation of successful music generation request submission is received.

[0081] Step 4:

[0082] The generation AI generates music based on the request sent.

[0083] Action 4.1: The generation AI determines the components of the song based on the received parameters.

[0084] Action 4.2: The generative AI designs music containing 1 / f fluctuations and tones that induce the specified brain waves.

[0085] Action 4.3: Generate the melody, rhythm, and harmony of the song and synthesize the final song.

[0086] Action 4.4: The generated music data is encoded and sent back to the server.

[0087] Step 5:

[0088] The server sends the music data received from the generation AI to the terminal.

[0089] Step 5.1: Check the format of the received music data (e.g. MP3, WAV).

[0090] Action 5.2: The song data is wrapped in JSON format again and sent to the device.

[0091] Action 5.3: Wait for confirmation that the music data has been successfully sent to the device.

[0092] Step 6:

[0093] The terminal receives the music data sent from the server and downloads it to local storage.

[0094] Action 6.1: Extract the song from the received JSON data.

[0095] Step 6.2: Save the song data to local storage.

[0096] Step 6.3: Verify that the song data was saved correctly.

[0097] Action 6.4: Notify the user that a song is available.

[0098] Step 7:

[0099] The effects of music therapy are sustained by having users listen to the music generated on a daily basis.

[0100] Action 7.1: A user plays a stored song on a smartphone or portable music player.

[0101] Action 7.2: The user listens to music regularly to achieve continuous therapeutic benefits.

[0102] Action 7.3: If necessary, go back to step 1 again to generate a new song.

[0103] Example 1

[0104] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0105] Existing music therapy systems are unable to generate music tailored to individual patients' symptoms and preferences, limiting their therapeutic effectiveness. Furthermore, it is difficult for patients to select music themselves, requiring specialized knowledge to achieve a certain level of therapeutic effectiveness. In response to these challenges, the present invention aims to provide a system that generates individual music therapy music tailored to a patient's symptoms and preferences and provides it effectively.

[0106] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0107] In this invention, the server includes a means for inputting information about the patient's symptoms and preferences, a means for sending a music generation request to the generation AI, and a means for receiving the generated music data and distributing it to the terminal. This makes it possible to dynamically generate music that corresponds to the symptoms and preferences of each patient and distribute it efficiently.

[0108] "Patient information" refers to information that is individually set, such as the patient's symptoms, preferences, and desired brain waves.

[0109] "Generative AI" refers to a system that uses artificial intelligence technology to dynamically generate music based on patient information.

[0110] "Music generation request" refers to a data request sent from the server to the generation AI for music generation based on patient information.

[0111] "Music data" refers to digital data of music for music therapy generated by generative AI.

[0112] "Terminal" refers to a device used by a patient to receive and play music data, such as a smartphone, tablet, or portable playback device.

[0113] This invention is a system that uses generative AI to generate music therapy songs and provide effective music therapy to patients. In this form of the invention, three main elements - a server, a terminal, and a user - operate according to a series of flows.

[0114] overview

[0115] The server is the central device that manages the process of generating music therapy music for each patient using generative AI. The device is responsible for inputting and transmitting patient information and receiving and playing the generated music. The users are the patients themselves, who receive treatment by listening to music therapy music.

[0116] Hardware and Software Configuration

[0117] Server: A server computer with high processing power is used, equipped with specialized software and the necessary databases to run the generative AI model.

[0118] Device: A mobile device such as a smartphone or tablet used by a patient. These devices have a dedicated application installed and provide data entry and music playback functions.

[0119] Generative AI: An AI model for generating music for music therapy. This AI model has the ability to generate music by combining 1 / f fluctuations and tones that induce specific brain waves.

[0120] Program processing

[0121] In this system, music therapy songs are generated and provided to patients through the following process.

[0122] Enter and submit patient information

[0123] 1. The user launches a dedicated application on their smartphone or tablet.

[0124] 2. The device displays an information input form to the user, which includes items such as symptoms (e.g., insomnia), preferred sounds (e.g., waves), and desired brain waves (e.g., alpha waves).

[0125] 3. The user enters information in each field, confirms it, and then presses the submit button.

[0126] 4. The device encrypts the entered data and sends it in a secure format to the server.

[0127] Server-side processing

[0128] 1. The server receives the data sent from the device.

[0129] 2. The server analyzes the received data and extracts the symptoms, preferred sounds, and brain wave parameters to be induced.

[0130] 3. The server constructs and sends a music generation request to the generation AI.

[0131] Music Generation

[0132] 1. The generation AI receives the request sent from the server.

[0133] 2. Based on the request, the generation AI generates music by combining 1 / f fluctuations and tones that induce the specified brain waves.

[0134] 3. The music data generated by the generation AI is encoded and sent back to the server.

[0135] Music distribution and playback

[0136] 1. The server sends the music data received from the generation AI to the device.

[0137] 2. The device saves the received music data in local storage.

[0138] 3. Users play music on their smartphones or portable music players and engage in music therapy.

[0139] Specific examples

[0140] Example 1: Treating insomnia

[0141] 1. The user has insomnia and launches a dedicated application on their smartphone.

[0142] 2. Select "insomnia" on the device, "wave sound" as your preferred sound, and "alpha waves" as the brain waves you want to induce.

[0143] 3. The device sends this information to the server.

[0144] 4. The server sends a request to the generation AI, asking it to generate music based on "insomnia," "the sound of waves," and "alpha waves."

[0145] 5. The AI ​​generates music containing the pleasant sounds of waves and alpha waves and sends it back to the server.

[0146] 6. The server sends the generated music data to the device.

[0147] 7. Users can play the songs before going to bed at night to treat insomnia.

[0148] Example 2: Stress relief

[0149] 1. The user is stressed and uses a portable music player.

[0150] 2. Select "Stress" on the device, "Babbling Stream" as your preferred sound, and "Theta Waves" as the brain waves you want to induce.

[0151] 3. The device sends the information to the server, and the server sends a music generation request to the generation AI.

[0152] 4. The generative AI creates music that includes the relaxing sounds of babbling and theta waves.

[0153] 5. The server sends the music data to the device, which stores it.

[0154] 6. Users can play the music during their work breaks to reduce stress.

[0155] Prompt Sentence Examples

[0156] 1. "Please create a piece of music that is effective in treating insomnia. I would like music that is based on the sound of ocean waves and induces alpha waves."

[0157] 2. "Please create a piece of music that will help reduce stress. I'd like it to use the sound of a stream to induce theta waves."

[0158] In this way, the system of the present invention allows patients to effectively receive music therapy tailored to their individual symptoms, and reduces the burden of expensive copyright fees for hospitals and clinics.

[0159] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0160] Step 1:

[0161] The user launches a dedicated application on their smartphone or tablet. The application displays a home screen and a "Start a new session" button. The input data is the user's operation, and the output is the display of the information input screen.

[0162] Step 2:

[0163] The terminal displays an information input form, which includes input fields for "symptoms," "preferred sound," and "desired brainwave induction." The input data is the patient's information, and the output is the creation of input fields and a submit button.

[0164] Step 3:

[0165] The user enters the following information into the information input form. For example, the user enters "insomnia" as the symptom, "wave sounds" as the preferred sound, and "alpha waves" as the brain waves to induce. The input data is the information entered by the user in each field, and the output is a confirmation screen for the entered information.

[0166] Step 4:

[0167] The terminal encrypts the data entered by the user. It uses the AES-256 encryption method to convert the patient information into a secure format. The input data is the information entered by the user, and the output is the encrypted data.

[0168] Step 5:

[0169] The device sends encrypted data to the server. The secure HTTPS protocol is used to transfer data safely. The input data is encrypted patient information, and the output is a confirmation of transmission to the server.

[0170] Step 6:

[0171] The server receives the data sent from the terminal. After receiving it, it decrypts the data using the AES-256 method. The input data is the encrypted information, and the output is the decrypted data.

[0172] Step 7:

[0173] The server parses the received data in JSON format and extracts the symptoms, preferred sounds, and brain wave parameters to be induced. The input data is the decoded information, and the output is the extracted parameters.

[0174] Step 8:

[0175] The server constructs a music generation request to send to the generation AI. The request includes the patient's symptoms, preferred sounds, and the brain wave data to be induced. The input data are the extracted parameters, and the output is a music generation request.

[0176] Step 9:

[0177] The server sends a music generation request to the generation AI as an HTTP POST request. The input data is the music generation request, and the output is a confirmation of the transmission to the generation AI.

[0178] Step 10:

[0179] The generation AI processes the request received from the server. The request contents include "insomnia," "sound of waves," and "alpha waves." The input data is the request from the server, and the output is confirmation that processing has started.

[0180] Step 11:

[0181] Based on the request, the generative AI generates music by combining 1 / f fluctuations and tones that induce the specified brain waves. The input data are the request parameters, and the output is the generated music data.

[0182] Step 12:

[0183] The music data generated by the generative AI is encoded in FLAC format. The input data is the generated music, and the output is the encoded music data.

[0184] Step 13:

[0185] The generation AI returns the encoded music data to the server. The input data is the encoded music data, and the output is a confirmation of transmission to the server.

[0186] Step 14:

[0187] The server sends the music data received from the generation AI to the device. It re-encrypts the data and transfers it securely. The input data is the encoded music data, and the output is a confirmation of transmission to the device.

[0188] Step 15:

[0189] The music data received by the device is saved to local storage in the specified directory. The input data is the encrypted music data, and the output is the saved data.

[0190] Step 16:

[0191] The user taps the "Play" button in the application to play a saved song. The input data is the saved music data, and the output is the music being played.

[0192] By explaining each processing step in detail, the specific operation of this system becomes clear. The generative AI model and prompt sentences play an important role.

[0193] (Application example 1)

[0194] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0195] In modern society, many people suffer from mental health problems such as stress, insomnia, and anxiety. To provide individually optimized music therapy for these problems, music customized for each patient is necessary. However, traditional music therapy has the challenge of creating music that corresponds to individual symptoms and preferences. There is also a need to provide an environment where patients can easily obtain music and play it at home or elsewhere.

[0196] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0197] In this invention, the server includes a means for generating music for music therapy using a generation AI, a means for inputting information about the patient's symptoms and preferences, and a means for transmitting the input information to the generation AI, thereby enabling the provision of music therapy within a virtual store.

[0198] "Generative AI" is artificial intelligence that generates music for music therapy based on the individual symptoms and preferences of each patient.

[0199] "Music therapy" is a therapeutic method that uses music to improve a patient's mental and physical health.

[0200] "Patient" refers to an individual receiving music therapy, including those with specific conditions (e.g., insomnia, stress, anxiety).

[0201] A "terminal" is an electronic device (e.g., smartphone, portable music player) used by the patient to input information and play the generated music.

[0202] A "virtual store" is an online virtual space that provides music therapy and related services via the Internet.

[0203] This invention provides a system that uses generation AI to generate music for music therapy tailored to individual symptoms. This system is primarily composed of four elements: a server, a terminal, a virtual store, and a user. This configuration allows music therapy to be provided effectively and easily.

[0204] The server plays a central role in generating music for music therapy using generative AI. First, the server receives information about the patient's symptoms and preferences sent from the device. Next, it analyzes this information and sends a music generation request to the generative AI model based on that information. This generative AI model is trained using machine learning libraries such as TensorFlow to generate music with appropriate tones and brainwave induction. The generated music data is then encoded by the server and sent back to the device.

[0205] The terminal is an electronic device used by the user to input information and receive and play the generated music. This terminal can be a smartphone or portable music player. The user operates the application through the terminal and inputs their symptoms, preferred sounds, and the brain waves they want to induce. This input information is sent to the server in a secure format. The music data received from the server is stored on the terminal, and the user receives music therapy by playing it.

[0206] The virtual store is an online virtual space that provides music therapy and related services via the Internet. Users can download music therapy applications within the virtual store and receive individually optimized music.

[0207] As a concrete example, consider the case where a user has symptoms of insomnia. First, the user launches a music therapy application on their smartphone and selects "insomnia," "wave sounds," and "alpha waves." This information is sent to the server, which then sends a request to the generation AI. The generation AI generates the optimal music and sends it back to the server. The generated music data is then sent to the device, and the user can play it before going to bed at night to achieve the therapeutic effect of insomnia.

[0208] The following are examples of prompt sentences:

[0209] "Send a request to the generative AI model: Symptom = insomnia, Favorite sound = ocean waves, EEG to induce = alpha waves. Please generate the optimal music therapy piece for the patient."

[0210] This allows the system to effectively provide music therapy tailored to each patient's individual symptoms and make it easily accessible through a virtual store.

[0211] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0212] Step 1:

[0213] The device accepts input from the user about their symptoms, preferred sounds, and the brain waves they want to induce. Examples of input information include "insomnia," "wave sounds," and "alpha waves." This information is then converted into a secure format, such as JSON.

[0214] Step 2:

[0215] The device sends the input information to the server using the HTTPS protocol, ensuring secure communication. The transmitted data includes symptoms, preferred sounds, and the brainwaves to be induced.

[0216] Step 3:

[0217] The server receives the information sent from the device. The received data is analyzed and the parameters required for the generative AI are extracted. The analysis results include symptoms, preferred sounds, and brainwave induction parameters. A request is then sent to the generative AI model.

[0218] Step 4:

[0219] The generative AI generates music based on parameters received from the server. The generative AI model (using TensorFlow) is instructed to generate music using a prompt. An example prompt is, "Symptoms = insomnia, Favorite sound = Wave sounds, Brain waves to induce = Alpha waves. Please generate music for music therapy that is optimal for the patient." The generative AI creates music by combining 1 / f fluctuations and sounds that induce specific brain waves, and generates music data.

[0220] Step 5:

[0221] The server receives the music data sent from the generation AI. This music data is encoded and in a format that can be played by the user. The server then transfers the received music data to the device.

[0222] Step 6:

[0223] The device receives the music data sent from the server and stores it in local storage. The saved music data can then be played using a dedicated application. The user operates the application to play the generated music and receive music therapy.

[0224] Step 7:

[0225] The user plays the music created using the device whenever necessary. This music playback provides music therapy benefits such as insomnia relief and stress reduction. The user can also acquire various other music therapy songs through the virtual store.

[0226] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0227] This invention provides a system that combines a generative AI and an emotion engine to provide more effective music therapy to patients. Specific program processing of this system and its embodiments are described below.

[0228] overview

[0229] The server is the central system that manages the process of generating music for individual patients using generative AI. In addition, the device is equipped with an emotion engine that recognizes the user's emotions, and this emotion data is also provided to the generative AI. The user is the patient themselves, and they receive treatment by listening to music optimized for their symptoms and emotions.

[0230] Program processing

[0231] Entering patient information and emotion data

[0232] The device inputs the patient's symptoms, preferences, and emotional data. The user uses a device such as a smartphone or tablet to input symptoms (insomnia, stress, anxiety, etc.), favorite sounds (sound of waves, babbling brook, rustling leaves, etc.), and emotional data obtained from heart rate and facial expressions. The emotion engine analyzes the user's facial expressions and voice in real time to generate emotional data. The device converts this data into JSON format and sends it to the server.

[0233] Server-side processing

[0234] The server sends a music generation request to the generation AI based on the patient information and emotional data received from the device. The server analyzes the received data and extracts parameters (symptoms, preferred sounds, brain waves, emotional information) for the generation AI to generate music. Based on these parameters, the server constructs a music generation request and sends it to the generation AI.

[0235] Music Generation

[0236] The generative AI generates music for music therapy based on requests and emotional data sent from the server. The generative AI combines 1 / f fluctuations and tones that induce specified brain waves to create music that is optimal for the patient's symptoms and emotions. The generated music data is encoded and sent back to the server.

[0237] Music distribution and playback

[0238] The server sends the music data received from the AI ​​to the device, which receives it and stores it in local storage. The user can then play the music on their smartphone or portable music player to engage in music therapy.

[0239] Specific examples

[0240] Example 1: Insomnia treatment and emotion recognition

[0241] 1. A user suffers from insomnia and launches a dedicated application on their smartphone.

[0242] 2. Select "insomnia" on the device, "wave sound" as your preferred sound, and "alpha waves" as the brain waves you want to induce.

[0243] 3. The emotion engine analyzes the user's facial expressions and heart rate to recognize when they are feeling anxious.

[0244] 4. The device converts this information into JSON format and sends it to the server.

[0245] 5. The server sends a request to the generation AI, asking it to generate music based on "insomnia," "the sound of waves," "alpha waves," and "the user's anxiety."

[0246] 6. The AI ​​generates music incorporating elements that alleviate anxiety, such as the soothing sounds of waves and alpha waves, and sends it back to the server.

[0247] 7. The server sends the generated music data to the device, which saves it.

[0248] 8. Users can play the songs before going to bed at night to treat insomnia.

[0249] Example 2: Stress reduction and real-time emotion recognition

[0250] 1. The user is stressed and uses a portable music player.

[0251] 2. Select "Stress" on the device, "Babbling Stream" as your preferred sound, and "Theta Waves" as the brain waves you want to induce.

[0252] 3. The emotion engine recognizes high stress levels from the user's tone of voice.

[0253] 4. The device sends the information and emotional data in JSON format to the server, and the server sends a music generation request to the generation AI.

[0254] 5. The generative AI generates music that incorporates elements that reduce stress, including the relaxing sound of a stream and theta waves.

[0255] 6. The server sends the music data to the device, which receives and saves it.

[0256] 7. Users can play the songs during their work breaks to reduce stress.

[0257] This system allows patients to receive optimal music therapy based on their individual symptoms and real-time emotional data, which is expected to improve therapeutic effectiveness. The introduction of an emotional engine makes it possible to more accurately grasp the patient's psychological state and provide effective music.

[0258] The processing flow will be explained below.

[0259] Step 1:

[0260] The device inputs the patient's symptoms, preferences, and emotional data.

[0261] Action 1.1: The user launches the application on their device and selects the symptom type (insomnia, stress, anxiety, etc.).

[0262] Action 1.2: The user selects their preferred sound (waves, babbling, rustling leaves, etc.).

[0263] Action 1.3: The user specifies the brain waves (alpha waves, beta waves, theta waves, gamma waves) they want to induce.

[0264] Action 1.4: The emotion engine analyzes the user's facial expressions, voice, and heart rate in real time to generate emotion data.

[0265] Action 1.5: Once all the information has been entered and parsed, the device presses the send button to send it to the server.

[0266] Step 2:

[0267] The terminal transmits the input patient information and emotion data to the server.

[0268] Action 2.1: Convert input data and emotion data into JSON format.

[0269] Action 2.2: Send the data to the server using a secure communications protocol (e.g., HTTPS).

[0270] Action 2.3: Wait for confirmation from the server that the data was sent successfully.

[0271] Step 3:

[0272] The server sends a request to the AI ​​to generate music based on the patient information and emotional data it receives.

[0273] Operation 3.1: The server analyzes the received data and extracts the necessary parameters (symptoms, preferred sounds, brain waves, emotions).

[0274] Action 3.2: Construct a music generation request using the extracted parameters.

[0275] Action 3.3: Send a music generation request to the generation AI endpoint.

[0276] Action 3.4: Confirmation of successful music generation request submission is received.

[0277] Step 4:

[0278] The generation AI generates music based on the request sent.

[0279] Action 4.1: The generation AI determines the components of the song based on the received parameters.

[0280] Action 4.2: The generative AI designs music containing 1 / f fluctuations and tones that induce the specified brain waves.

[0281] Action 4.3: Generate the melody, rhythm, and harmony of the song and synthesize the final song.

[0282] Action 4.4: The generated music data is encoded and sent back to the server.

[0283] Step 5:

[0284] The server sends the music data received from the generation AI to the terminal.

[0285] Step 5.1: Check the format of the received music data (e.g. MP3, WAV).

[0286] Action 5.2: The song data is wrapped in JSON format again and sent to the device.

[0287] Action 5.3: Wait for confirmation that the music data has been successfully sent to the device.

[0288] Step 6:

[0289] The terminal receives the music data sent from the server and downloads it to local storage.

[0290] Action 6.1: Extract the song from the received JSON data.

[0291] Step 6.2: Save the song data to local storage.

[0292] Step 6.3: Verify that the song data was saved correctly.

[0293] Action 6.4: Notify the user that a song is available.

[0294] Step 7:

[0295] The effects of music therapy are sustained by having users listen to the music generated on a daily basis.

[0296] Action 7.1: A user plays a stored song on a smartphone or portable music player.

[0297] Action 7.2: The user listens to music regularly to achieve continuous therapeutic benefits.

[0298] Action 7.3: If necessary, go back to step 1 again to generate a new song.

[0299] Example 2

[0300] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0301] Music therapy is an effective treatment for many illnesses and psychological problems, but providing music optimized for each patient's symptoms and emotions remains a challenge. Furthermore, while the therapeutic effect could be further improved if real-time emotional data about the patient could be reflected in the music generation, a system to achieve this has yet to be developed. To address these challenges, the present invention aims to provide a music therapy system that combines generative AI and an emotion engine, thereby providing more effective music therapy for patients.

[0302] The specification process by the specification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes a means for generating music for music therapy using a generation AI, a means for inputting information about the patient's symptoms and preferences, a means including an emotion engine that collects and analyzes emotional data of the patient in real time, a means for converting the input information and emotional data into JSON format and sending it to the generation AI, a means for receiving music data generated by the generation AI and distributing it to a terminal, and a means for saving and playing the music data on the patient's terminal. This makes it possible to provide optimal music for each patient based on their symptoms and emotions in real time.

[0303] "Generative AI" is a system that uses artificial intelligence technology to automatically generate new music and data.

[0304] "Music therapy music" is music created with the purpose of improving a specific symptom or emotional state.

[0305] "Patient symptoms" refers to any physical or psychological problem or medical condition that a patient is experiencing.

[0306] "Preference information" is data that indicates a patient's individual tastes and preferences for music and sounds.

[0307] "Emotional data" is information that indicates the patient's emotional state, collected through heart rate and facial expression analysis, etc.

[0308] The "emotion engine" is a system that analyzes emotional data collected in real time and determines the patient's emotional state.

[0309] The "JSON format" is a lightweight data description language format primarily used for data exchange.

[0310] "Terminal" means a device used by a patient to input information or play generated music, including smart devices and portable audio equipment.

[0311] "1 / f fluctuation" is a type of signal with specific rhythmic and noise characteristics that is widely found in nature and is said to have a relaxation effect.

[0312] "Brainwave-inducing tones" refer to sound characteristics designed to induce specific brainwave activity (e.g., alpha waves, theta waves, etc.).

[0313] "Network connectivity" refers to the infrastructure that allows data communication between digital devices.

[0314] "Internal storage" refers to data storage devices built into a digital device.

[0315] "Encoding" is the process of converting data into a particular format.

[0316] A "music library" is a database that refers to a collection of music or sound clips.

[0317] This invention realizes a system that provides more effective music therapy to patients by combining a generative AI and an emotion engine. Specific program processing of this system and its embodiments are described below.

[0318] System configuration

[0319] The system mainly consists of a server, a terminal, a generative AI model, and an emotion engine. The server is responsible for managing data and sending requests to the generative AI, while the terminal inputs patient information and plays the generated music.

[0320] Entering patient information and emotion data

[0321] The terminal inputs the patient's symptoms, preferences, and emotional data. The user uses a smart device (e.g., smartphone or tablet) to input symptoms (insomnia, stress, anxiety, etc.) and preferred sounds (sound of waves, babbling brook, rustling leaves, etc.). The emotion engine also analyzes the user's facial expressions and voice, collecting emotional data derived from heart rate and facial expressions in real time. The terminal converts this data into JSON format and sends it to the server.

[0322] Server-side processing

[0323] The server sends a music generation request to the AI ​​based on the patient information and emotional data received from the device. The server analyzes the received data and extracts parameters (symptoms, preferred sounds, emotional information, and desired brain waves) that the AI ​​will need to generate music. Based on these parameters, the server constructs and sends a request to the AI.

[0324] Music Generation

[0325] The generative AI generates music for music therapy based on requests and emotional data sent from the server. The generative AI combines 1 / f fluctuations and tones that induce specified brain waves to create music that is optimal for the patient's symptoms and emotions. The generated music data is encoded and sent back to the server.

[0326] Music distribution and playback

[0327] The server sends the music data received from the AI ​​to the device, which receives it and stores it in local storage. The user can then play the generated music on a smart device or portable audio device and engage in music therapy.

[0328] Specific examples

[0329] A specific example of operation is given below.

[0330] Example 1: Insomnia treatment and emotion recognition

[0331] 1. Assume that the user has insomnia and launches a dedicated application on their smartphone.

[0332] 2. On the device, select "insomnia," your preferred sound "wave sound," and "alpha waves" as the brain waves you want to induce.

[0333] 3. The emotion engine analyzes the user's facial expressions and heart rate to recognize that they are feeling anxious rather than relaxed.

[0334] 4. The device converts this information into JSON format and sends it to the server.

[0335] 5. The server sends a request to the generation AI, asking it to generate music based on "insomnia," "the sound of waves," "alpha waves," and "the user's anxiety."

[0336] 6. The AI ​​generates music incorporating elements that alleviate anxiety, such as the soothing sounds of waves and alpha waves, and sends it back to the server.

[0337] 7. The server sends the generated music data to the device, which saves it.

[0338] 8. Users can play the songs before going to bed at night to treat insomnia.

[0339] Example 2: Stress reduction and real-time emotion recognition

[0340] 1. If the user is stressed, use a portable sound device.

[0341] 2. The device selects "stress," the preferred sound "babbling brook," and "theta waves" as the brain waves to be induced.

[0342] 3. The emotion engine recognizes high stress levels from the user's tone of voice.

[0343] 4. The device sends the information and emotional data in JSON format to the server, and the server sends a music generation request to the generation AI.

[0344] 5. The generative AI generates music that incorporates elements that reduce stress, including the relaxing sound of a stream and theta waves.

[0345] 6. The server sends the music data to the device, which receives and saves it.

[0346] 7. Users can play the songs during their work breaks to reduce stress.

[0347] Examples of prompt statements

[0348] Below is an example of a prompt sentence that is input to a generative AI model.

[0349] Prompts to generate music to relieve anxiety with "Insomnia," "Wave sounds," and "Alpha waves":

[0350] Symptoms: Insomnia

[0351] Favorite sound: The sound of waves

[0352] Brain waves to induce: alpha waves

[0353] Emotional data: Anxiety

[0354] Prompts to generate music that promotes relaxation through "stress," "babbling," and "theta waves":

[0355] Symptoms: Stress

[0356] Favorite sound: A babbling brook

[0357] Brain waves to be induced: θ waves

[0358] Emotional data: High stress levels

[0359] This system enables patients to receive optimal music therapy based on their individual symptoms and real-time emotional data, which is expected to improve therapeutic effectiveness. In addition, the introduction of an emotional engine makes it possible to more accurately grasp the patient's psychological state and provide more effective music.

[0360] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0361] Step 1: Enter patient information and emotion data

[0362] The device inputs information about the patient's symptoms and preferences, as well as emotional data, through a dedicated application. The user launches the app and inputs their symptoms (e.g., insomnia, stress, anxiety, etc.) and preferred sounds (e.g., the sound of waves, babbling brook, rustling leaves, etc.). The emotion engine uses the camera and heart rate monitor to analyze the user's facial expressions and heart rate in real time and generates emotional data (e.g., anxiety, heart rate, etc.). The input data is converted into JSON format within the application. In concrete terms, the user selects information on the touchscreen, and the camera and sensors collect data.

[0363] input:

[0364] User-entered symptoms and preferred sounds

[0365] Real-time emotion data obtained from camera and heart rate monitor

[0366] output:

[0367] Patient information and emotion data converted into JSON format

[0368] Step 2: Send patient information and emotion data

[0369] The device sends the generated JSON data to the server. The device establishes a network connection and sends the data using an HTTP POST request. Specifically, the application sends the data to the server in the background.

[0370] input:

[0371] JSON formatted patient information and emotion data

[0372] output:

[0373] Patient information and emotion data sent to the server

[0374] Step 3: Processing the music generation request on the server side

[0375] The server analyzes the received JSON data and constructs a music generation request for the generation AI. The server analyzes the data and extracts parameters such as symptoms, preferred sounds, emotional data, and the brain waves to be induced. It creates a generation AI request based on these parameters and sends it to the generation AI. Specifically, the server accesses the database to obtain the necessary information and constructs the request.

[0376] input:

[0377] JSON data received by the server

[0378] output:

[0379] A song generation request sent to the generation AI

[0380] Step 4: Run the music generation

[0381] The generative AI generates music for music therapy based on requests from the server. The generative AI selects appropriate sound clips from its internal music library and synthesizes them by combining 1 / f fluctuations and tones that induce specific brain waves. The generated music data is encoded and sent back to the server. Specifically, the generative AI uses an algorithm to generate music and encode the data.

[0382] input:

[0383] A song generation request sent to the generation AI

[0384] output:

[0385] Encoded music data sent back to the server

[0386] Step 5: Receiving and distributing music data

[0387] The server sends the encoded music data received from the generation AI to the device. The server sends the music data via HTTP, and the device receives and analyzes it, saving it to its internal storage. Specifically, the server transfers the music data, and the device writes it to its storage.

[0388] input:

[0389] Encoded music data received from the generation AI

[0390] output:

[0391] Music data stored on the device

[0392] Step 6: Music Playback and Music Therapy

[0393] The user plays music stored on the device and performs music therapy. The user operates a dedicated application on a smart device or portable audio device to play music and enjoy the therapeutic effects. Specifically, the user operates the app to select a song and listen to it through the device's speakers or earphones.

[0394] input:

[0395] Music data stored on the device

[0396] output:

[0397] Music played and listened to by the user

[0398] The above processing steps enable personalized music therapy. By combining generative AI and an emotion engine, it is possible to provide optimal music that reflects the patient's real-time emotional state, which is expected to improve the effectiveness of treatment.

[0399] (Application example 2)

[0400] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0401] Conventional music therapy systems have difficulty recognizing a user's emotional state in real time and providing personalized music. This has resulted in the inability to provide music that is optimal for the user's emotions and symptoms, limiting the therapeutic effect. Furthermore, use on devices such as smartphones and portable music players has been limited, resulting in low versatility. To solve these problems, a system that can recognize a user's emotional data in real time and provide personalized music is needed.

[0402] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0403] In this invention, the server includes a means for generating music for music therapy using a generation AI, a means for inputting information on the patient's symptoms and preferences, a means for transmitting the input information and emotional data to the generation AI, a means for receiving the music data generated by the generation AI and distributing it to a terminal, a means for the terminal to recognize the user's emotional state in real time, and a means for saving and playing the music data on the patient's terminal. This makes it possible to provide optimal music in real time according to the user's emotional state and improve the effectiveness of music therapy.

[0404] "Generative AI" is a type of artificial intelligence that uses data and algorithms to automatically perform specific tasks and generate music for music therapy.

[0405] A "patient" is a person who has a particular symptom or emotional state and receives music therapy to improve or treat that condition.

[0406] "Symptoms" refer to the state of illness or discomfort experienced by a patient, and include anxiety, insomnia, stress, etc.

[0407] "Preferences" refer to the patient's individual preferences for types of music or sounds.

[0408] "Information" includes data about patient symptoms and preferences, as well as emotional data collected in real time.

[0409] "Emotional data" refers to data that indicates the user's emotional state, derived from factors such as heart rate, facial expression, and tone of voice.

[0410] "Terminal" refers to a device such as a smartphone or portable music player that is used by the patient.

[0411] "Music data" refers to digital data of music for music therapy generated by the generative AI.

[0412] "Distribution" is the process of sending music data generated by the generation AI from the server to the device.

[0413] "Real-time" means responding immediately to ongoing events and changes.

[0414] The system that realizes this application example is a music therapy system that combines a generative AI model and an emotion engine. The core of the system is a server that provides music therapy in cooperation with the patient's device. The specific program processing is as follows:

[0415] System Configuration

[0416] server

[0417] The server receives the information and emotional data sent from the patient's device and requests the generation AI to generate a song. The generation AI generates a song based on the parameters and sends the song data back to the server. The server then distributes the generated song data to the patient's device. A high-performance cloud server (e.g., AWS EC2, Google Cloud) is used as the server.

[0418] Terminal

[0419] The patient's device is a smartphone or portable music player, and the user inputs information about their symptoms and preferences, as well as emotional data collected in real time. This data is sent from the device to a server. The device is equipped with an emotion engine that analyzes heart rate, facial expressions, and vocal tone in real time to generate emotional data. The device has the ability to receive, store, and play the generated music data.

[0420] User

[0421] The user is a patient who operates a device to receive music therapy based on their symptoms and preferences. The user starts the application and enters the necessary information to begin interaction with the system.

[0422] Data processing and calculation used

[0423] Emotion data collection: The device analyzes the emotional data collected, such as heart rate, facial expressions, and voice tone, converts it into JSON format, and sends it to the server.

[0424] Music generation: The server sends a request to the generation AI based on the input information and emotional data, and sets the music generation parameters (e.g., treatment for insomnia, relaxation effect, etc.).

[0425] Music data distribution: The server receives the music data created by the generation AI and distributes it back to the device. The device stores this data in local storage and plays it for music therapy.

[0426] Specific examples

[0427] 1. Music therapy for stress management:

[0428] The user feels stressed and launches a dedicated application.

[0429] The device selects "stress," the preferred sound is "babbling brook," and the brain waves you want to induce are "theta waves."

[0430] The emotion engine collects user emotional data and recognizes high stress levels.

[0431] The device converts this information into JSON format and sends it to the server.

[0432] The server sends a request to the generation AI, asking it to generate music to reduce stress.

[0433] The AI ​​generates music that includes the relaxing sound of a stream and theta waves, and sends it back to the server.

[0434] The server distributes the generated music data to the terminal, and the user plays it to relax.

[0435] Prompt Sentence Examples

[0436] "The user is feeling stressed. Please generate music that will have a relaxing effect. Preferably music that contains the sounds of waves and theta waves."

[0437] Based on this prompt, the AI ​​generates music that soothes the user's emotions.

[0438] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0439] Program processing flow and specific explanation

[0440] Step 1

[0441] Terminal-based information and emotional data collection

[0442] Input: Patient symptoms (e.g., stress), preferred sound (e.g., babbling), desired brainwave induction (e.g., theta waves), and real-time user emotional data (heart rate, facial expression, and voice tone).

[0443] Data processing: The device converts this data into JSON format.

[0444] How it works: The emotion engine analyzes the user's emotional state and generates emotion data, for example by using the smartphone's camera and microphone to capture the user's facial expressions and voice tone.

[0445] Output: JSON format data (symptoms, preferences, EEG, emotion data).

[0446] Step 2

[0447] Sending data from the device to the server

[0448] Input: The JSON formatted data generated in step 1.

[0449] Data processing: Send data from the device to the server via an HTTP POST request.

[0450] How it works: A smartphone or portable music player connects to a server using a communications interface and sends JSON data.

[0451] Output: Patient information and emotion data sent to the server.

[0452] Step 3

[0453] Data analysis by the server and sending requests to the generation AI

[0454] Input: Patient information and emotion data received in step 2 (JSON format).

[0455] Data processing: The server analyzes the data and extracts parameters (symptoms, preferences, brain waves, emotional information) for the generation AI to generate music.

[0456] How it works: A data analysis program runs on the server and constructs a request for the generation AI. The server then sends the music generation request to the generation AI using an HTTP POST request.

[0457] Output: A song generation request (in JSON format) sent to the generation AI.

[0458] Step 4

[0459] Music generation using generative AI

[0460] Input: The song generation request (in JSON format) sent in step 3.

[0461] Data processing: Generative AI generates music based on requests, combining specified brain waves and emotionally soothing tones (e.g., babbling, theta waves, etc.).

[0462] How it works: A generative AI model uses algorithms to generate music based on data, for example using a deep learning model to generate personalized music.

[0463] Output: Generated song data (binary format).

[0464] Step 5

[0465] Sending the generated music data back to the server

[0466] Input: Music data (binary format) generated by the generation AI.

[0467] Data processing: The music data is sent back to the server as an HTTP response.

[0468] How it works: The server receives the music data and encodes / decodes it as needed.

[0469] Output: Song data received by the server.

[0470] Step 6

[0471] Distribution and storage of music data to devices

[0472] Input: Song data (binary format) received by the server in step 5.

[0473] Data processing: Sends music data from the server to the device using HTTP responses and streaming.

[0474] Operation: The device receives the music data and saves it to local storage. Example: A smartphone saves the received music file to its internal storage.

[0475] Output: Song data saved on your device.

[0476] Step 7

[0477] Playing music on a device

[0478] Input: Song data (binary format) saved on the device in step 6.

[0479] Data processing: Decodes the data into a suitable format for playback.

[0480] How it works: The device plays music, and the user listens to the music and receives music therapy. Example: A smartphone or portable music player plays music through a music playback application.

[0481] Output: The music played to the user.

[0482] Specific examples of operation

[0483] Example prompt: "The user is feeling stressed. Please generate music that will have a relaxing effect. Preferably music that contains the sounds of waves and theta waves."

[0484] Based on this prompt, the generative AI will go through the steps above to generate a song that will ease the user's emotions, and then play it back to provide music therapy.

[0485] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0486] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0487] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0488] [Second embodiment]

[0489] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0490] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0491] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0492] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0493] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0494] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0495] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0496] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0497] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0498] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0499] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0500] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0501] This invention is a system that uses generative AI to generate music therapy songs and provide effective music therapy to patients. In this embodiment of the invention, three main elements - a server, a terminal, and a user - operate according to a series of flows.

[0502] overview

[0503] The server is the central device that manages the process of generating music therapy music for each patient using generative AI. The device is responsible for inputting and transmitting patient information and receiving and playing the generated music. The users are the patients themselves, who receive treatment by listening to music therapy music.

[0504] Program processing

[0505] Enter and submit patient information

[0506] The device inputs the patient's symptoms and preferences. Using a device such as a smartphone or tablet, the user inputs symptoms (insomnia, stress, anxiety, etc.), preferred sounds (waves, babbling, rustling leaves, etc.), and the brain waves they want to induce (alpha waves, beta waves, theta waves, gamma waves). The device converts this data into a secure format and sends it to the server.

[0507] Server-side processing

[0508] The server sends a music generation request to the AI ​​based on the patient information received from the device. The server analyzes the received data and extracts parameters (symptoms, preferred sounds, brain waves) for the AI ​​to generate music. The server then constructs a music generation request based on these parameters and sends it to the AI.

[0509] Music Generation

[0510] The generation AI generates music for music therapy based on requests sent from the server. The generation AI combines 1 / f fluctuations and tones that induce specified brain waves to create music that is optimal for the patient's symptoms. The generated music data is encoded and sent back to the server.

[0511] Music distribution and playback

[0512] The server sends the music data received from the AI ​​to the device, which receives it and stores it in local storage. The user can then play the music on their smartphone or portable music player to engage in music therapy.

[0513] Specific examples

[0514] Example 1: Treating insomnia

[0515] 1. The user has insomnia and launches a dedicated application on their smartphone.

[0516] 2. Select "insomnia" on the device, "wave sound" as your preferred sound, and "alpha waves" as the brain waves you want to induce.

[0517] 3. The device sends this information to the server.

[0518] 4. The server sends a request to the generation AI, asking it to generate music based on "insomnia," "the sound of waves," and "alpha waves."

[0519] 5. The AI ​​generates music containing the pleasant sounds of waves and alpha waves and sends it back to the server.

[0520] 6. The server sends the generated music data to the device.

[0521] 7. Users can play the songs before going to bed at night to treat insomnia.

[0522] Example 2: Stress relief

[0523] 1. The user is stressed and uses a portable music player.

[0524] 2. Select "Stress" on the device, "Babbling Stream" as your preferred sound, and "Theta Waves" as the brain waves you want to induce.

[0525] 3. The device sends the information to the server, and the server sends a music generation request to the generation AI.

[0526] 4. The generative AI creates music that includes the relaxing sound of a stream and theta waves.

[0527] 5. The server sends the music data to the device, which stores it.

[0528] 6. Users can play the music during their work breaks to reduce stress.

[0529] In this way, the system of the present invention allows patients to effectively receive music therapy tailored to their individual symptoms, and reduces the burden of expensive copyright fees for hospitals and clinics.

[0530] The processing flow will be explained below.

[0531] Step 1:

[0532] The device inputs the patient's symptoms and preferences.

[0533] Action 1.1: The user launches the application on their device and selects the symptom type (insomnia, stress, anxiety, etc.).

[0534] Action 1.2: The user selects their preferred sound (waves, babbling, rustling leaves, etc.).

[0535] Action 1.3: The user specifies the brain waves (alpha waves, beta waves, theta waves, gamma waves) they want to induce.

[0536] Action 1.4: Once you have completed the input, press the send button.

[0537] Step 2:

[0538] The terminal transmits the input patient information to the server.

[0539] Action 2.1: Convert input data to JSON format.

[0540] Action 2.2: Send the data to the server using a secure communications protocol (e.g., HTTPS).

[0541] Action 2.3: Wait for confirmation from the server that the data was sent successfully.

[0542] Step 3:

[0543] Based on the patient information received by the server, a request to generate music is sent to the generation AI.

[0544] Operation 3.1: The server analyzes the received data and extracts the necessary parameters (symptoms, preferred sounds, brain waves).

[0545] Action 3.2: Construct a music generation request using the extracted parameters.

[0546] Action 3.3: Send a music generation request to the generation AI endpoint.

[0547] Action 3.4: Confirmation of successful music generation request submission is received.

[0548] Step 4:

[0549] The generation AI generates music based on the request sent.

[0550] Action 4.1: The generation AI determines the components of the song based on the received parameters.

[0551] Action 4.2: The generative AI designs music containing 1 / f fluctuations and tones that induce the specified brain waves.

[0552] Action 4.3: Generate the melody, rhythm, and harmony of the song and synthesize the final song.

[0553] Action 4.4: The generated music data is encoded and sent back to the server.

[0554] Step 5:

[0555] The server sends the music data received from the generation AI to the terminal.

[0556] Step 5.1: Check the format of the received music data (e.g. MP3, WAV).

[0557] Action 5.2: The song data is wrapped in JSON format again and sent to the device.

[0558] Action 5.3: Wait for confirmation that the music data has been successfully sent to the device.

[0559] Step 6:

[0560] The terminal receives the music data sent from the server and downloads it to local storage.

[0561] Action 6.1: Extract the song from the received JSON data.

[0562] Step 6.2: Save the song data to local storage.

[0563] Step 6.3: Verify that the song data was saved correctly.

[0564] Action 6.4: Notify the user that a song is available.

[0565] Step 7:

[0566] The effects of music therapy are sustained by having users listen to the music generated on a daily basis.

[0567] Action 7.1: A user plays a stored song on a smartphone or portable music player.

[0568] Action 7.2: The user listens to music regularly to achieve continuous therapeutic benefits.

[0569] Action 7.3: If necessary, go back to step 1 again to generate a new song.

[0570] Example 1

[0571] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0572] Existing music therapy systems are unable to generate music tailored to individual patients' symptoms and preferences, limiting their therapeutic effectiveness. Furthermore, it is difficult for patients to select music themselves, requiring specialized knowledge to achieve a certain level of therapeutic effectiveness. In response to these challenges, the present invention aims to provide a system that generates individual music therapy music tailored to a patient's symptoms and preferences and provides it effectively.

[0573] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0574] In this invention, the server includes a means for inputting information about the patient's symptoms and preferences, a means for sending a music generation request to the generation AI, and a means for receiving the generated music data and distributing it to the terminal. This makes it possible to dynamically generate music that corresponds to the symptoms and preferences of each patient and distribute it efficiently.

[0575] "Patient information" refers to information that is individually set, such as the patient's symptoms, preferences, and desired brain waves.

[0576] "Generative AI" refers to a system that uses artificial intelligence technology to dynamically generate music based on patient information.

[0577] "Music generation request" refers to a data request sent from the server to the generation AI for music generation based on patient information.

[0578] "Music data" refers to digital data of music for music therapy generated by generative AI.

[0579] "Terminal" refers to a device used by a patient to receive and play music data, such as a smartphone, tablet, or portable playback device.

[0580] This invention is a system that uses generative AI to generate music therapy songs and provide effective music therapy to patients. In this form of the invention, three main elements - a server, a terminal, and a user - operate according to a series of flows.

[0581] overview

[0582] The server is the central device that manages the process of generating music therapy music for each patient using generative AI. The device is responsible for inputting and transmitting patient information and receiving and playing the generated music. The users are the patients themselves, who receive treatment by listening to music therapy music.

[0583] Hardware and Software Configuration

[0584] Server: A server computer with high processing power is used, equipped with specialized software and the necessary databases to run the generative AI model.

[0585] Device: A mobile device such as a smartphone or tablet used by a patient. These devices have a dedicated application installed and provide data entry and music playback functions.

[0586] Generative AI: An AI model for generating music for music therapy. This AI model has the ability to generate music by combining 1 / f fluctuations and tones that induce specific brain waves.

[0587] Program processing

[0588] In this system, music therapy songs are generated and provided to patients through the following process.

[0589] Enter and submit patient information

[0590] 1. The user launches a dedicated application on their smartphone or tablet.

[0591] 2. The device displays an information input form to the user, which includes items such as symptoms (e.g., insomnia), preferred sounds (e.g., waves), and desired brain waves (e.g., alpha waves).

[0592] 3. The user enters information in each field, confirms it, and then presses the submit button.

[0593] 4. The device encrypts the entered data and sends it in a secure format to the server.

[0594] Server-side processing

[0595] 1. The server receives the data sent from the device.

[0596] 2. The server analyzes the received data and extracts the symptoms, preferred sounds, and brain wave parameters to be induced.

[0597] 3. The server constructs and sends a music generation request to the generation AI.

[0598] Music Generation

[0599] 1. The generation AI receives the request sent from the server.

[0600] 2. Based on the request, the generation AI generates music by combining 1 / f fluctuations and tones that induce the specified brain waves.

[0601] 3. The music data generated by the generation AI is encoded and sent back to the server.

[0602] Music distribution and playback

[0603] 1. The server sends the music data received from the generation AI to the device.

[0604] 2. The device saves the received music data in local storage.

[0605] 3. Users play music on their smartphones or portable music players and engage in music therapy.

[0606] Specific examples

[0607] Example 1: Treating insomnia

[0608] 1. The user has insomnia and launches a dedicated application on their smartphone.

[0609] 2. Select "insomnia" on the device, "wave sound" as your preferred sound, and "alpha waves" as the brain waves you want to induce.

[0610] 3. The device sends this information to the server.

[0611] 4. The server sends a request to the generation AI, asking it to generate music based on "insomnia," "the sound of waves," and "alpha waves."

[0612] 5. The AI ​​generates music containing the pleasant sounds of waves and alpha waves and sends it back to the server.

[0613] 6. The server sends the generated music data to the device.

[0614] 7. Users can play the songs before going to bed at night to treat insomnia.

[0615] Example 2: Stress relief

[0616] 1. The user is stressed and uses a portable music player.

[0617] 2. Select "Stress" on the device, "Babbling Stream" as your preferred sound, and "Theta Waves" as the brain waves you want to induce.

[0618] 3. The device sends the information to the server, and the server sends a music generation request to the generation AI.

[0619] 4. The generative AI creates music that includes the relaxing sounds of babbling and theta waves.

[0620] 5. The server sends the music data to the device, which stores it.

[0621] 6. Users can play the music during their work breaks to reduce stress.

[0622] Prompt Sentence Examples

[0623] 1. "Please create a piece of music that is effective in treating insomnia. I would like music that is based on the sound of ocean waves and induces alpha waves."

[0624] 2. "Please create a piece of music that will help reduce stress. I'd like it to use the sound of a stream to induce theta waves."

[0625] In this way, the system of the present invention allows patients to effectively receive music therapy tailored to their individual symptoms, and reduces the burden of expensive copyright fees for hospitals and clinics.

[0626] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0627] Step 1:

[0628] The user launches a dedicated application on their smartphone or tablet. The application displays a home screen and a "Start a new session" button. The input data is the user's operation, and the output is the display of the information input screen.

[0629] Step 2:

[0630] The terminal displays an information input form, which includes input fields for "symptoms," "preferred sound," and "desired brainwave induction." The input data is the patient's information, and the output is the creation of input fields and a submit button.

[0631] Step 3:

[0632] The user enters the following information into the information input form. For example, the user enters "insomnia" as the symptom, "wave sounds" as the preferred sound, and "alpha waves" as the brain waves to induce. The input data is the information entered by the user in each field, and the output is a confirmation screen for the entered information.

[0633] Step 4:

[0634] The terminal encrypts the data entered by the user. It uses the AES-256 encryption method to convert the patient information into a secure format. The input data is the information entered by the user, and the output is the encrypted data.

[0635] Step 5:

[0636] The device sends encrypted data to the server. The secure HTTPS protocol is used to transfer data safely. The input data is encrypted patient information, and the output is a confirmation of transmission to the server.

[0637] Step 6:

[0638] The server receives the data sent from the terminal. After receiving it, it decrypts the data using the AES-256 method. The input data is the encrypted information, and the output is the decrypted data.

[0639] Step 7:

[0640] The server parses the received data in JSON format and extracts the symptoms, preferred sounds, and brain wave parameters to be induced. The input data is the decoded information, and the output is the extracted parameters.

[0641] Step 8:

[0642] The server constructs a music generation request to send to the generation AI. The request includes the patient's symptoms, preferred sounds, and the brain wave data to be induced. The input data are the extracted parameters, and the output is a music generation request.

[0643] Step 9:

[0644] The server sends a music generation request to the generation AI as an HTTP POST request. The input data is the music generation request, and the output is a confirmation of the transmission to the generation AI.

[0645] Step 10:

[0646] The generation AI processes the request received from the server. The request contents include "insomnia," "sound of waves," and "alpha waves." The input data is the request from the server, and the output is confirmation that processing has started.

[0647] Step 11:

[0648] Based on the request, the generative AI generates music by combining 1 / f fluctuations and tones that induce the specified brain waves. The input data are the request parameters, and the output is the generated music data.

[0649] Step 12:

[0650] The music data generated by the generative AI is encoded in FLAC format. The input data is the generated music, and the output is the encoded music data.

[0651] Step 13:

[0652] The generation AI returns the encoded music data to the server. The input data is the encoded music data, and the output is a confirmation of transmission to the server.

[0653] Step 14:

[0654] The server sends the music data received from the generation AI to the device. It re-encrypts the data and transfers it securely. The input data is the encoded music data, and the output is a confirmation of transmission to the device.

[0655] Step 15:

[0656] The music data received by the device is saved to local storage in the specified directory. The input data is the encrypted music data, and the output is the saved data.

[0657] Step 16:

[0658] The user taps the "Play" button in the application to play a saved song. The input data is the saved music data, and the output is the music being played.

[0659] By explaining each processing step in detail, the specific operation of this system becomes clear. The generative AI model and prompt sentences play an important role.

[0660] (Application example 1)

[0661] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0662] In modern society, many people suffer from mental health problems such as stress, insomnia, and anxiety. To provide individually optimized music therapy for these problems, music customized for each patient is necessary. However, traditional music therapy has the challenge of creating music that corresponds to individual symptoms and preferences. There is also a need to provide an environment where patients can easily obtain music and play it at home or elsewhere.

[0663] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0664] In this invention, the server includes a means for generating music for music therapy using a generation AI, a means for inputting information about the patient's symptoms and preferences, and a means for transmitting the input information to the generation AI, thereby enabling the provision of music therapy within a virtual store.

[0665] "Generative AI" is artificial intelligence that generates music for music therapy based on the individual symptoms and preferences of each patient.

[0666] "Music therapy" is a therapeutic method that uses music to improve a patient's mental and physical health.

[0667] "Patient" refers to an individual receiving music therapy, including those with specific conditions (e.g., insomnia, stress, anxiety).

[0668] A "terminal" is an electronic device (e.g., smartphone, portable music player) used by the patient to input information and play the generated music.

[0669] A "virtual store" is an online virtual space that provides music therapy and related services via the Internet.

[0670] This invention provides a system that uses generation AI to generate music for music therapy tailored to individual symptoms. This system is primarily composed of four elements: a server, a terminal, a virtual store, and a user. This configuration allows music therapy to be provided effectively and easily.

[0671] The server plays a central role in generating music for music therapy using generative AI. First, the server receives information about the patient's symptoms and preferences sent from the device. Next, it analyzes this information and sends a music generation request to the generative AI model based on that information. This generative AI model is trained using machine learning libraries such as TensorFlow to generate music with appropriate tones and brainwave induction. The generated music data is then encoded by the server and sent back to the device.

[0672] The terminal is an electronic device used by the user to input information and receive and play the generated music. This terminal can be a smartphone or portable music player. The user operates the application through the terminal and inputs their symptoms, preferred sounds, and the brain waves they want to induce. This input information is sent to the server in a secure format. The music data received from the server is stored on the terminal, and the user receives music therapy by playing it.

[0673] The virtual store is an online virtual space that provides music therapy and related services via the Internet. Users can download music therapy applications within the virtual store and receive individually optimized music.

[0674] As a concrete example, consider the case where a user has symptoms of insomnia. First, the user launches a music therapy application on their smartphone and selects "insomnia," "wave sounds," and "alpha waves." This information is sent to the server, which then sends a request to the generation AI. The generation AI generates the optimal music and sends it back to the server. The generated music data is then sent to the device, and the user can play it before going to bed at night to achieve the therapeutic effect of insomnia.

[0675] The following are examples of prompt sentences:

[0676] "Send a request to the generative AI model: Symptom = insomnia, Favorite sound = ocean waves, EEG to induce = alpha waves. Please generate the optimal music therapy piece for the patient."

[0677] This allows the system to effectively provide music therapy tailored to each patient's individual symptoms and make it easily accessible through a virtual store.

[0678] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0679] Step 1:

[0680] The device accepts input from the user about their symptoms, preferred sounds, and the brain waves they want to induce. Examples of input information include "insomnia," "wave sounds," and "alpha waves." This information is then converted into a secure format, such as JSON.

[0681] Step 2:

[0682] The device sends the input information to the server using the HTTPS protocol, ensuring secure communication. The transmitted data includes symptoms, preferred sounds, and the brainwaves to be induced.

[0683] Step 3:

[0684] The server receives the information sent from the device. The received data is analyzed and the parameters required for the generative AI are extracted. The analysis results include symptoms, preferred sounds, and brainwave induction parameters. A request is then sent to the generative AI model.

[0685] Step 4:

[0686] The generative AI generates music based on parameters received from the server. The generative AI model (using TensorFlow) is instructed to generate music using a prompt. An example prompt is, "Symptoms = insomnia, Favorite sound = Wave sounds, Brain waves to induce = Alpha waves. Please generate music for music therapy that is optimal for the patient." The generative AI creates music by combining 1 / f fluctuations and sounds that induce specific brain waves, and generates music data.

[0687] Step 5:

[0688] The server receives the music data sent from the generation AI. This music data is encoded and in a format that can be played by the user. The server then transfers the received music data to the device.

[0689] Step 6:

[0690] The device receives the music data sent from the server and stores it in local storage. The saved music data can then be played using a dedicated application. The user operates the application to play the generated music and receive music therapy.

[0691] Step 7:

[0692] The user plays the music created using the device whenever necessary. This music playback provides music therapy benefits such as insomnia relief and stress reduction. The user can also acquire various other music therapy songs through the virtual store.

[0693] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0694] This invention provides a system that combines a generative AI and an emotion engine to provide more effective music therapy to patients. Specific program processing of this system and its embodiments are described below.

[0695] overview

[0696] The server is the central system that manages the process of generating music for individual patients using generative AI. In addition, the device is equipped with an emotion engine that recognizes the user's emotions, and this emotion data is also provided to the generative AI. The user is the patient themselves, and they receive treatment by listening to music optimized for their symptoms and emotions.

[0697] Program processing

[0698] Entering patient information and emotion data

[0699] The device inputs the patient's symptoms, preferences, and emotional data. The user uses a device such as a smartphone or tablet to input symptoms (insomnia, stress, anxiety, etc.), favorite sounds (sound of waves, babbling brook, rustling leaves, etc.), and emotional data obtained from heart rate and facial expressions. The emotion engine analyzes the user's facial expressions and voice in real time to generate emotional data. The device converts this data into JSON format and sends it to the server.

[0700] Server-side processing

[0701] The server sends a music generation request to the generation AI based on the patient information and emotional data received from the device. The server analyzes the received data and extracts parameters (symptoms, preferred sounds, brain waves, emotional information) for the generation AI to generate music. Based on these parameters, the server constructs a music generation request and sends it to the generation AI.

[0702] Music Generation

[0703] The generative AI generates music for music therapy based on requests and emotional data sent from the server. The generative AI combines 1 / f fluctuations and tones that induce specified brain waves to create music that is optimal for the patient's symptoms and emotions. The generated music data is encoded and sent back to the server.

[0704] Music distribution and playback

[0705] The server sends the music data received from the AI ​​to the device, which receives it and stores it in local storage. The user can then play the music on their smartphone or portable music player to engage in music therapy.

[0706] Specific examples

[0707] Example 1: Insomnia treatment and emotion recognition

[0708] 1. A user suffers from insomnia and launches a dedicated application on their smartphone.

[0709] 2. Select "insomnia" on the device, "wave sound" as your preferred sound, and "alpha waves" as the brain waves you want to induce.

[0710] 3. The emotion engine analyzes the user's facial expressions and heart rate to recognize when they are feeling anxious.

[0711] 4. The device converts this information into JSON format and sends it to the server.

[0712] 5. The server sends a request to the generation AI, asking it to generate music based on "insomnia," "the sound of waves," "alpha waves," and "the user's anxiety."

[0713] 6. The AI ​​generates music incorporating elements that alleviate anxiety, such as the soothing sounds of waves and alpha waves, and sends it back to the server.

[0714] 7. The server sends the generated music data to the device, which saves it.

[0715] 8. Users can play the songs before going to bed at night to treat insomnia.

[0716] Example 2: Stress reduction and real-time emotion recognition

[0717] 1. The user is stressed and uses a portable music player.

[0718] 2. Select "Stress" on the device, "Babbling Stream" as your preferred sound, and "Theta Waves" as the brain waves you want to induce.

[0719] 3. The emotion engine recognizes high stress levels from the user's tone of voice.

[0720] 4. The device sends the information and emotional data in JSON format to the server, and the server sends a music generation request to the generation AI.

[0721] 5. The generative AI generates music that incorporates elements that reduce stress, including the relaxing sound of a stream and theta waves.

[0722] 6. The server sends the music data to the device, which receives and saves it.

[0723] 7. Users can play the songs during their work breaks to reduce stress.

[0724] This system allows patients to receive optimal music therapy based on their individual symptoms and real-time emotional data, which is expected to improve therapeutic effectiveness. The introduction of an emotional engine makes it possible to more accurately grasp the patient's psychological state and provide effective music.

[0725] The processing flow will be explained below.

[0726] Step 1:

[0727] The device inputs the patient's symptoms, preferences, and emotional data.

[0728] Action 1.1: The user launches the application on their device and selects the symptom type (insomnia, stress, anxiety, etc.).

[0729] Action 1.2: The user selects their preferred sound (waves, babbling, rustling leaves, etc.).

[0730] Action 1.3: The user specifies the brain waves (alpha waves, beta waves, theta waves, gamma waves) they want to induce.

[0731] Action 1.4: The emotion engine analyzes the user's facial expressions, voice, and heart rate in real time to generate emotion data.

[0732] Action 1.5: Once all the information has been entered and parsed, the device presses the send button to send it to the server.

[0733] Step 2:

[0734] The terminal transmits the input patient information and emotion data to the server.

[0735] Action 2.1: Convert input data and emotion data into JSON format.

[0736] Action 2.2: Send the data to the server using a secure communications protocol (e.g., HTTPS).

[0737] Action 2.3: Wait for confirmation from the server that the data was sent successfully.

[0738] Step 3:

[0739] The server sends a request to the AI ​​to generate music based on the patient information and emotional data it receives.

[0740] Operation 3.1: The server analyzes the received data and extracts the necessary parameters (symptoms, preferred sounds, brain waves, emotions).

[0741] Action 3.2: Construct a music generation request using the extracted parameters.

[0742] Action 3.3: Send a music generation request to the generation AI endpoint.

[0743] Action 3.4: Confirmation of successful music generation request submission is received.

[0744] Step 4:

[0745] The generation AI generates music based on the request sent.

[0746] Action 4.1: The generation AI determines the components of the song based on the received parameters.

[0747] Action 4.2: The generative AI designs music containing 1 / f fluctuations and tones that induce the specified brain waves.

[0748] Action 4.3: Generate the melody, rhythm, and harmony of the song and synthesize the final song.

[0749] Action 4.4: The generated music data is encoded and sent back to the server.

[0750] Step 5:

[0751] The server sends the music data received from the generation AI to the terminal.

[0752] Step 5.1: Check the format of the received music data (e.g. MP3, WAV).

[0753] Action 5.2: The song data is wrapped in JSON format again and sent to the device.

[0754] Action 5.3: Wait for confirmation that the music data has been successfully sent to the device.

[0755] Step 6:

[0756] The terminal receives the music data sent from the server and downloads it to local storage.

[0757] Action 6.1: Extract the song from the received JSON data.

[0758] Step 6.2: Save the song data to local storage.

[0759] Step 6.3: Verify that the song data was saved correctly.

[0760] Action 6.4: Notify the user that a song is available.

[0761] Step 7:

[0762] The effects of music therapy are sustained by having users listen to the music generated on a daily basis.

[0763] Action 7.1: A user plays a stored song on a smartphone or portable music player.

[0764] Action 7.2: The user listens to music regularly to achieve continuous therapeutic benefits.

[0765] Action 7.3: If necessary, go back to step 1 again to generate a new song.

[0766] Example 2

[0767] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0768] Music therapy is an effective treatment for many illnesses and psychological problems, but providing music optimized for each patient's symptoms and emotions remains a challenge. Furthermore, while the therapeutic effect could be further improved if real-time emotional data about the patient could be reflected in the music generation, a system to achieve this has yet to be developed. To address these challenges, the present invention aims to provide a music therapy system that combines generative AI and an emotion engine, thereby providing more effective music therapy for patients.

[0769] The specification process by the specification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes a means for generating music for music therapy using a generation AI, a means for inputting information about the patient's symptoms and preferences, a means including an emotion engine that collects and analyzes emotional data of the patient in real time, a means for converting the input information and emotional data into JSON format and sending it to the generation AI, a means for receiving music data generated by the generation AI and distributing it to a terminal, and a means for saving and playing the music data on the patient's terminal. This makes it possible to provide optimal music for each patient based on their symptoms and emotions in real time.

[0770] "Generative AI" is a system that uses artificial intelligence technology to automatically generate new music and data.

[0771] "Music therapy music" is music created with the purpose of improving a specific symptom or emotional state.

[0772] "Patient symptoms" refers to any physical or psychological problem or medical condition that a patient is experiencing.

[0773] "Preference information" is data that indicates a patient's individual tastes and preferences for music and sounds.

[0774] "Emotional data" is information that indicates the patient's emotional state, collected through heart rate and facial expression analysis, etc.

[0775] The "emotion engine" is a system that analyzes emotional data collected in real time and determines the patient's emotional state.

[0776] The "JSON format" is a lightweight data description language format primarily used for data exchange.

[0777] "Terminal" means a device used by a patient to input information or play generated music, including smart devices and portable audio equipment.

[0778] "1 / f fluctuation" is a type of signal with specific rhythmic and noise characteristics that is widely found in nature and is said to have a relaxation effect.

[0779] "Brainwave-inducing tones" refer to sound characteristics designed to induce specific brainwave activity (e.g., alpha waves, theta waves, etc.).

[0780] "Network connectivity" refers to the infrastructure that allows data communication between digital devices.

[0781] "Internal storage" refers to data storage devices built into a digital device.

[0782] "Encoding" is the process of converting data into a particular format.

[0783] A "music library" is a database that refers to a collection of music or sound clips.

[0784] This invention realizes a system that provides more effective music therapy to patients by combining a generative AI and an emotion engine. Specific program processing of this system and its embodiments are described below.

[0785] System configuration

[0786] The system mainly consists of a server, a terminal, a generative AI model, and an emotion engine. The server is responsible for managing data and sending requests to the generative AI, while the terminal inputs patient information and plays the generated music.

[0787] Entering patient information and emotion data

[0788] The terminal inputs the patient's symptoms, preferences, and emotional data. The user uses a smart device (e.g., smartphone or tablet) to input symptoms (insomnia, stress, anxiety, etc.) and preferred sounds (sound of waves, babbling brook, rustling leaves, etc.). The emotion engine also analyzes the user's facial expressions and voice, collecting emotional data derived from heart rate and facial expressions in real time. The terminal converts this data into JSON format and sends it to the server.

[0789] Server-side processing

[0790] The server sends a music generation request to the AI ​​based on the patient information and emotional data received from the device. The server analyzes the received data and extracts parameters (symptoms, preferred sounds, emotional information, and desired brain waves) that the AI ​​will need to generate music. Based on these parameters, the server constructs and sends a request to the AI.

[0791] Music Generation

[0792] The generative AI generates music for music therapy based on requests and emotional data sent from the server. The generative AI combines 1 / f fluctuations and tones that induce specified brain waves to create music that is optimal for the patient's symptoms and emotions. The generated music data is encoded and sent back to the server.

[0793] Music distribution and playback

[0794] The server sends the music data received from the AI ​​to the device, which receives it and stores it in local storage. The user can then play the generated music on a smart device or portable audio device and engage in music therapy.

[0795] Specific examples

[0796] A specific example of operation is given below.

[0797] Example 1: Insomnia treatment and emotion recognition

[0798] 1. Assume that the user has insomnia and launches a dedicated application on their smartphone.

[0799] 2. On the device, select "insomnia," your preferred sound "wave sound," and "alpha waves" as the brain waves you want to induce.

[0800] 3. The emotion engine analyzes the user's facial expressions and heart rate to recognize that they are feeling anxious rather than relaxed.

[0801] 4. The device converts this information into JSON format and sends it to the server.

[0802] 5. The server sends a request to the generation AI, asking it to generate music based on "insomnia," "the sound of waves," "alpha waves," and "the user's anxiety."

[0803] 6. The AI ​​generates music incorporating elements that alleviate anxiety, such as the soothing sounds of waves and alpha waves, and sends it back to the server.

[0804] 7. The server sends the generated music data to the device, which saves it.

[0805] 8. Users can play the songs before going to bed at night to treat insomnia.

[0806] Example 2: Stress reduction and real-time emotion recognition

[0807] 1. If the user is stressed, use a portable sound device.

[0808] 2. The device selects "stress," the preferred sound "babbling brook," and "theta waves" as the brain waves to be induced.

[0809] 3. The emotion engine recognizes high stress levels from the user's tone of voice.

[0810] 4. The device sends the information and emotional data in JSON format to the server, and the server sends a music generation request to the generation AI.

[0811] 5. The generative AI generates music that incorporates elements that reduce stress, including the relaxing sound of a stream and theta waves.

[0812] 6. The server sends the music data to the device, which receives and saves it.

[0813] 7. Users can play the songs during their work breaks to reduce stress.

[0814] Examples of prompt statements

[0815] Below is an example of a prompt sentence that is input to a generative AI model.

[0816] Prompts to generate music to relieve anxiety with "Insomnia," "Wave sounds," and "Alpha waves":

[0817] Symptoms: Insomnia

[0818] Favorite sound: The sound of waves

[0819] Brain waves to induce: alpha waves

[0820] Emotional data: Anxiety

[0821] Prompts to generate music that promotes relaxation through "stress," "babbling," and "theta waves":

[0822] Symptoms: Stress

[0823] Favorite sound: A babbling brook

[0824] Brain waves to be induced: θ waves

[0825] Emotional data: High stress levels

[0826] This system enables patients to receive optimal music therapy based on their individual symptoms and real-time emotional data, which is expected to improve therapeutic effectiveness. In addition, the introduction of an emotional engine makes it possible to more accurately grasp the patient's psychological state and provide more effective music.

[0827] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0828] Step 1: Enter patient information and emotion data

[0829] The device inputs information about the patient's symptoms and preferences, as well as emotional data, through a dedicated application. The user launches the app and inputs their symptoms (e.g., insomnia, stress, anxiety, etc.) and preferred sounds (e.g., the sound of waves, babbling brook, rustling leaves, etc.). The emotion engine uses the camera and heart rate monitor to analyze the user's facial expressions and heart rate in real time and generates emotional data (e.g., anxiety, heart rate, etc.). The input data is converted into JSON format within the application. In concrete terms, the user selects information on the touchscreen, and the camera and sensors collect data.

[0830] input:

[0831] User-entered symptoms and preferred sounds

[0832] Real-time emotion data obtained from camera and heart rate monitor

[0833] output:

[0834] Patient information and emotion data converted into JSON format

[0835] Step 2: Send patient information and emotion data

[0836] The device sends the generated JSON data to the server. The device establishes a network connection and sends the data using an HTTP POST request. Specifically, the application sends the data to the server in the background.

[0837] input:

[0838] JSON formatted patient information and emotion data

[0839] output:

[0840] Patient information and emotion data sent to the server

[0841] Step 3: Processing the music generation request on the server side

[0842] The server analyzes the received JSON data and constructs a music generation request for the generation AI. The server analyzes the data and extracts parameters such as symptoms, preferred sounds, emotional data, and the brain waves to be induced. It creates a generation AI request based on these parameters and sends it to the generation AI. Specifically, the server accesses the database to obtain the necessary information and constructs the request.

[0843] input:

[0844] JSON data received by the server

[0845] output:

[0846] A song generation request sent to the generation AI

[0847] Step 4: Run the music generation

[0848] The generative AI generates music for music therapy based on requests from the server. The generative AI selects appropriate sound clips from its internal music library and synthesizes them by combining 1 / f fluctuations and tones that induce specific brain waves. The generated music data is encoded and sent back to the server. Specifically, the generative AI uses an algorithm to generate music and encode the data.

[0849] input:

[0850] A song generation request sent to the generation AI

[0851] output:

[0852] Encoded music data sent back to the server

[0853] Step 5: Receiving and distributing music data

[0854] The server sends the encoded music data received from the generation AI to the device. The server sends the music data via HTTP, and the device receives and analyzes it, saving it to its internal storage. Specifically, the server transfers the music data, and the device writes it to its storage.

[0855] input:

[0856] Encoded music data received from the generation AI

[0857] output:

[0858] Music data stored on the device

[0859] Step 6: Music Playback and Music Therapy

[0860] The user plays music stored on the device and performs music therapy. The user operates a dedicated application on a smart device or portable audio device to play music and enjoy the therapeutic effects. Specifically, the user operates the app to select a song and listen to it through the device's speakers or earphones.

[0861] input:

[0862] Music data stored on the device

[0863] output:

[0864] Music played and listened to by the user

[0865] The above processing steps enable personalized music therapy. By combining generative AI and an emotion engine, it is possible to provide optimal music that reflects the patient's real-time emotional state, which is expected to improve the effectiveness of treatment.

[0866] (Application example 2)

[0867] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0868] Conventional music therapy systems have difficulty recognizing a user's emotional state in real time and providing personalized music. This has resulted in the inability to provide music that is optimal for the user's emotions and symptoms, limiting the therapeutic effect. Furthermore, use on devices such as smartphones and portable music players has been limited, resulting in low versatility. To solve these problems, a system that can recognize a user's emotional data in real time and provide personalized music is needed.

[0869] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0870] In this invention, the server includes a means for generating music for music therapy using a generation AI, a means for inputting information on the patient's symptoms and preferences, a means for transmitting the input information and emotional data to the generation AI, a means for receiving the music data generated by the generation AI and distributing it to a terminal, a means for the terminal to recognize the user's emotional state in real time, and a means for saving and playing the music data on the patient's terminal. This makes it possible to provide optimal music in real time according to the user's emotional state and improve the effectiveness of music therapy.

[0871] "Generative AI" is a type of artificial intelligence that uses data and algorithms to automatically perform specific tasks and generate music for music therapy.

[0872] A "patient" is a person who has a particular symptom or emotional state and receives music therapy to improve or treat that condition.

[0873] "Symptoms" refer to the state of illness or discomfort experienced by a patient, and include anxiety, insomnia, stress, etc.

[0874] "Preferences" refer to the patient's individual preferences for types of music or sounds.

[0875] "Information" includes data about patient symptoms and preferences, as well as emotional data collected in real time.

[0876] "Emotional data" refers to data that indicates the user's emotional state, derived from factors such as heart rate, facial expression, and tone of voice.

[0877] "Terminal" refers to a device such as a smartphone or portable music player that is used by the patient.

[0878] "Music data" refers to digital data of music for music therapy generated by the generative AI.

[0879] "Distribution" is the process of sending music data generated by the generation AI from the server to the device.

[0880] "Real-time" means responding immediately to ongoing events and changes.

[0881] The system that realizes this application example is a music therapy system that combines a generative AI model and an emotion engine. The core of the system is a server that provides music therapy in cooperation with the patient's device. The specific program processing is as follows:

[0882] System Configuration

[0883] server

[0884] The server receives the information and emotional data sent from the patient's device and requests the generation AI to generate a song. The generation AI generates a song based on the parameters and sends the song data back to the server. The server then distributes the generated song data to the patient's device. A high-performance cloud server (e.g., AWS EC2, Google Cloud) is used as the server.

[0885] Terminal

[0886] The patient's device is a smartphone or portable music player, and the user inputs information about their symptoms and preferences, as well as emotional data collected in real time. This data is sent from the device to a server. The device is equipped with an emotion engine that analyzes heart rate, facial expressions, and vocal tone in real time to generate emotional data. The device has the ability to receive, store, and play the generated music data.

[0887] User

[0888] The user is a patient who operates a device to receive music therapy based on their symptoms and preferences. The user starts the application and enters the necessary information to begin interaction with the system.

[0889] Data processing and calculation used

[0890] Emotion data collection: The device analyzes the emotional data collected, such as heart rate, facial expressions, and voice tone, converts it into JSON format, and sends it to the server.

[0891] Music generation: The server sends a request to the generation AI based on the input information and emotional data, and sets the music generation parameters (e.g., treatment for insomnia, relaxation effect, etc.).

[0892] Music data distribution: The server receives the music data created by the generation AI and distributes it back to the device. The device stores this data in local storage and plays it for music therapy.

[0893] Specific examples

[0894] 1. Music therapy for stress management:

[0895] The user feels stressed and launches a dedicated application.

[0896] The device selects "stress," the preferred sound is "babbling brook," and the brain waves you want to induce are "theta waves."

[0897] The emotion engine collects user emotional data and recognizes high stress levels.

[0898] The device converts this information into JSON format and sends it to the server.

[0899] The server sends a request to the generation AI, asking it to generate music to reduce stress.

[0900] The AI ​​generates music that includes the relaxing sound of a stream and theta waves, and sends it back to the server.

[0901] The server distributes the generated music data to the terminal, and the user plays it to relax.

[0902] Prompt Sentence Examples

[0903] "The user is feeling stressed. Please generate music that will have a relaxing effect. Preferably music that contains the sounds of waves and theta waves."

[0904] Based on this prompt, the AI ​​generates music that soothes the user's emotions.

[0905] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0906] Program processing flow and specific explanation

[0907] Step 1

[0908] Terminal-based information and emotional data collection

[0909] Input: Patient symptoms (e.g., stress), preferred sound (e.g., babbling), desired brainwave induction (e.g., theta waves), and real-time user emotional data (heart rate, facial expression, and voice tone).

[0910] Data processing: The device converts this data into JSON format.

[0911] How it works: The emotion engine analyzes the user's emotional state and generates emotion data, for example by using the smartphone's camera and microphone to capture the user's facial expressions and voice tone.

[0912] Output: JSON format data (symptoms, preferences, EEG, emotion data).

[0913] Step 2

[0914] Sending data from the device to the server

[0915] Input: The JSON formatted data generated in step 1.

[0916] Data processing: Send data from the device to the server via an HTTP POST request.

[0917] How it works: A smartphone or portable music player connects to a server using a communications interface and sends JSON data.

[0918] Output: Patient information and emotion data sent to the server.

[0919] Step 3

[0920] Data analysis by the server and sending requests to the generation AI

[0921] Input: Patient information and emotion data received in step 2 (JSON format).

[0922] Data processing: The server analyzes the data and extracts parameters (symptoms, preferences, brain waves, emotional information) for the generation AI to generate music.

[0923] How it works: A data analysis program runs on the server and constructs a request for the generation AI. The server then sends the music generation request to the generation AI using an HTTP POST request.

[0924] Output: A song generation request (in JSON format) sent to the generation AI.

[0925] Step 4

[0926] Music generation using generative AI

[0927] Input: The song generation request (in JSON format) sent in step 3.

[0928] Data processing: Generative AI generates music based on requests, combining specified brain waves and emotionally soothing tones (e.g., babbling, theta waves, etc.).

[0929] How it works: A generative AI model uses algorithms to generate music based on data, for example using a deep learning model to generate personalized music.

[0930] Output: Generated song data (binary format).

[0931] Step 5

[0932] Sending the generated music data back to the server

[0933] Input: Music data (binary format) generated by the generation AI.

[0934] Data processing: The music data is sent back to the server as an HTTP response.

[0935] How it works: The server receives the music data and encodes / decodes it as needed.

[0936] Output: Song data received by the server.

[0937] Step 6

[0938] Distribution and storage of music data to devices

[0939] Input: Song data (binary format) received by the server in step 5.

[0940] Data processing: Sends music data from the server to the device using HTTP responses and streaming.

[0941] Operation: The device receives the music data and saves it to local storage. Example: A smartphone saves the received music file to its internal storage.

[0942] Output: Song data saved on your device.

[0943] Step 7

[0944] Playing music on a device

[0945] Input: Song data (binary format) saved on the device in step 6.

[0946] Data processing: Decodes the data into a suitable format for playback.

[0947] How it works: The device plays music, and the user listens to the music and receives music therapy. Example: A smartphone or portable music player plays music through a music playback application.

[0948] Output: The music played to the user.

[0949] Specific examples of operation

[0950] Example prompt: "The user is feeling stressed. Please generate music that will have a relaxing effect. Preferably music that contains the sounds of waves and theta waves."

[0951] Based on this prompt, the generative AI will go through the steps above to generate a song that will ease the user's emotions, and then play it back to provide music therapy.

[0952] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0953] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0954] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0955] [Third embodiment]

[0956] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0957] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0958] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0959] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0960] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0961] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0962] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0963] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0964] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0965] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0966] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0967] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0968] This invention is a system that uses generative AI to generate music therapy songs and provide effective music therapy to patients. In this embodiment of the invention, three main elements - a server, a terminal, and a user - operate according to a series of flows.

[0969] overview

[0970] The server is the central device that manages the process of generating music therapy music for each patient using generative AI. The device is responsible for inputting and transmitting patient information and receiving and playing the generated music. The users are the patients themselves, who receive treatment by listening to music therapy music.

[0971] Program processing

[0972] Enter and submit patient information

[0973] The device inputs the patient's symptoms and preferences. Using a device such as a smartphone or tablet, the user inputs symptoms (insomnia, stress, anxiety, etc.), preferred sounds (waves, babbling, rustling leaves, etc.), and the brain waves they want to induce (alpha waves, beta waves, theta waves, gamma waves). The device converts this data into a secure format and sends it to the server.

[0974] Server-side processing

[0975] The server sends a music generation request to the AI ​​based on the patient information received from the device. The server analyzes the received data and extracts parameters (symptoms, preferred sounds, brain waves) for the AI ​​to generate music. The server then constructs a music generation request based on these parameters and sends it to the AI.

[0976] Music Generation

[0977] The generation AI generates music for music therapy based on requests sent from the server. The generation AI combines 1 / f fluctuations and tones that induce specified brain waves to create music that is optimal for the patient's symptoms. The generated music data is encoded and sent back to the server.

[0978] Music distribution and playback

[0979] The server sends the music data received from the AI ​​to the device, which receives it and stores it in local storage. The user can then play the music on their smartphone or portable music player to engage in music therapy.

[0980] Specific examples

[0981] Example 1: Treating insomnia

[0982] 1. The user has insomnia and launches a dedicated application on their smartphone.

[0983] 2. Select "insomnia" on the device, "wave sound" as your preferred sound, and "alpha waves" as the brain waves you want to induce.

[0984] 3. The device sends this information to the server.

[0985] 4. The server sends a request to the generation AI, asking it to generate music based on "insomnia," "the sound of waves," and "alpha waves."

[0986] 5. The AI ​​generates music containing the pleasant sounds of waves and alpha waves and sends it back to the server.

[0987] 6. The server sends the generated music data to the device.

[0988] 7. Users can play the songs before going to bed at night to treat insomnia.

[0989] Example 2: Stress relief

[0990] 1. The user is stressed and uses a portable music player.

[0991] 2. Select "Stress" on the device, "Babbling Stream" as your preferred sound, and "Theta Waves" as the brain waves you want to induce.

[0992] 3. The device sends the information to the server, and the server sends a music generation request to the generation AI.

[0993] 4. The generative AI creates music that includes the relaxing sound of a stream and theta waves.

[0994] 5. The server sends the music data to the device, which stores it.

[0995] 6. Users can play the music during their work breaks to reduce stress.

[0996] In this way, the system of the present invention allows patients to effectively receive music therapy tailored to their individual symptoms, and reduces the burden of expensive copyright fees for hospitals and clinics.

[0997] The processing flow will be explained below.

[0998] Step 1:

[0999] The device inputs the patient's symptoms and preferences.

[1000] Action 1.1: The user launches the application on their device and selects the symptom type (insomnia, stress, anxiety, etc.).

[1001] Action 1.2: The user selects their preferred sound (waves, babbling, rustling leaves, etc.).

[1002] Action 1.3: The user specifies the brain waves (alpha waves, beta waves, theta waves, gamma waves) they want to induce.

[1003] Action 1.4: Once you have completed the input, press the send button.

[1004] Step 2:

[1005] The terminal transmits the input patient information to the server.

[1006] Action 2.1: Convert input data to JSON format.

[1007] Action 2.2: Send the data to the server using a secure communications protocol (e.g., HTTPS).

[1008] Action 2.3: Wait for confirmation from the server that the data was sent successfully.

[1009] Step 3:

[1010] Based on the patient information received by the server, a request to generate music is sent to the generation AI.

[1011] Operation 3.1: The server analyzes the received data and extracts the necessary parameters (symptoms, preferred sounds, brain waves).

[1012] Action 3.2: Construct a music generation request using the extracted parameters.

[1013] Action 3.3: Send a music generation request to the generation AI endpoint.

[1014] Action 3.4: Confirmation of successful music generation request submission is received.

[1015] Step 4:

[1016] The generation AI generates music based on the request sent.

[1017] Action 4.1: The generation AI determines the components of the song based on the received parameters.

[1018] Action 4.2: The generative AI designs music containing 1 / f fluctuations and tones that induce the specified brain waves.

[1019] Action 4.3: Generate the melody, rhythm, and harmony of the song and synthesize the final song.

[1020] Action 4.4: The generated music data is encoded and sent back to the server.

[1021] Step 5:

[1022] The server sends the music data received from the generation AI to the terminal.

[1023] Step 5.1: Check the format of the received music data (e.g. MP3, WAV).

[1024] Action 5.2: The song data is wrapped in JSON format again and sent to the device.

[1025] Action 5.3: Wait for confirmation that the music data has been successfully sent to the device.

[1026] Step 6:

[1027] The terminal receives the music data sent from the server and downloads it to local storage.

[1028] Action 6.1: Extract the song from the received JSON data.

[1029] Step 6.2: Save the song data to local storage.

[1030] Step 6.3: Verify that the song data was saved correctly.

[1031] Action 6.4: Notify the user that a song is available.

[1032] Step 7:

[1033] The effects of music therapy are sustained by having users listen to the music generated on a daily basis.

[1034] Action 7.1: A user plays a stored song on a smartphone or portable music player.

[1035] Action 7.2: The user listens to music regularly to achieve continuous therapeutic benefits.

[1036] Action 7.3: If necessary, go back to step 1 again to generate a new song.

[1037] Example 1

[1038] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1039] Existing music therapy systems are unable to generate music tailored to individual patients' symptoms and preferences, limiting their therapeutic effectiveness. Furthermore, it is difficult for patients to select music themselves, requiring specialized knowledge to achieve a certain level of therapeutic effectiveness. In response to these challenges, the present invention aims to provide a system that generates individual music therapy music tailored to a patient's symptoms and preferences and provides it effectively.

[1040] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1041] In this invention, the server includes a means for inputting information about the patient's symptoms and preferences, a means for sending a music generation request to the generation AI, and a means for receiving the generated music data and distributing it to the terminal. This makes it possible to dynamically generate music that corresponds to the symptoms and preferences of each patient and distribute it efficiently.

[1042] "Patient information" refers to information that is individually set, such as the patient's symptoms, preferences, and desired brain waves.

[1043] "Generative AI" refers to a system that uses artificial intelligence technology to dynamically generate music based on patient information.

[1044] "Music generation request" refers to a data request sent from the server to the generation AI for music generation based on patient information.

[1045] "Music data" refers to digital data of music for music therapy generated by generative AI.

[1046] "Terminal" refers to a device used by a patient to receive and play music data, such as a smartphone, tablet, or portable playback device.

[1047] This invention is a system that uses generative AI to generate music therapy songs and provide effective music therapy to patients. In this form of the invention, three main elements - a server, a terminal, and a user - operate according to a series of flows.

[1048] overview

[1049] The server is the central device that manages the process of generating music therapy music for each patient using generative AI. The device is responsible for inputting and transmitting patient information and receiving and playing the generated music. The users are the patients themselves, who receive treatment by listening to music therapy music.

[1050] Hardware and Software Configuration

[1051] Server: A server computer with high processing power is used, equipped with specialized software and the necessary databases to run the generative AI model.

[1052] Device: A mobile device such as a smartphone or tablet used by a patient. These devices have a dedicated application installed and provide data entry and music playback functions.

[1053] Generative AI: An AI model for generating music for music therapy. This AI model has the ability to generate music by combining 1 / f fluctuations and tones that induce specific brain waves.

[1054] Program processing

[1055] In this system, music therapy songs are generated and provided to patients through the following process.

[1056] Enter and submit patient information

[1057] 1. The user launches a dedicated application on their smartphone or tablet.

[1058] 2. The device displays an information input form to the user, which includes items such as symptoms (e.g., insomnia), preferred sounds (e.g., waves), and desired brain waves (e.g., alpha waves).

[1059] 3. The user enters information in each field, confirms it, and then presses the submit button.

[1060] 4. The device encrypts the entered data and sends it in a secure format to the server.

[1061] Server-side processing

[1062] 1. The server receives the data sent from the device.

[1063] 2. The server analyzes the received data and extracts the symptoms, preferred sounds, and brain wave parameters to be induced.

[1064] 3. The server constructs and sends a music generation request to the generation AI.

[1065] Music Generation

[1066] 1. The generation AI receives the request sent from the server.

[1067] 2. Based on the request, the generation AI generates music by combining 1 / f fluctuations and tones that induce the specified brain waves.

[1068] 3. The music data generated by the generation AI is encoded and sent back to the server.

[1069] Music distribution and playback

[1070] 1. The server sends the music data received from the generation AI to the device.

[1071] 2. The device saves the received music data in local storage.

[1072] 3. Users play music on their smartphones or portable music players and engage in music therapy.

[1073] Specific examples

[1074] Example 1: Treating insomnia

[1075] 1. The user has insomnia and launches a dedicated application on their smartphone.

[1076] 2. Select "insomnia" on the device, "wave sound" as your preferred sound, and "alpha waves" as the brain waves you want to induce.

[1077] 3. The device sends this information to the server.

[1078] 4. The server sends a request to the generation AI, asking it to generate music based on "insomnia," "the sound of waves," and "alpha waves."

[1079] 5. The AI ​​generates music containing the pleasant sounds of waves and alpha waves and sends it back to the server.

[1080] 6. The server sends the generated music data to the device.

[1081] 7. Users can play the songs before going to bed at night to treat insomnia.

[1082] Example 2: Stress relief

[1083] 1. The user is stressed and uses a portable music player.

[1084] 2. Select "Stress" on the device, "Babbling Stream" as your preferred sound, and "Theta Waves" as the brain waves you want to induce.

[1085] 3. The device sends the information to the server, and the server sends a music generation request to the generation AI.

[1086] 4. The generative AI creates music that includes the relaxing sounds of babbling and theta waves.

[1087] 5. The server sends the music data to the device, which stores it.

[1088] 6. Users can play the music during their work breaks to reduce stress.

[1089] Prompt Sentence Examples

[1090] 1. "Please create a piece of music that is effective in treating insomnia. I would like music that is based on the sound of ocean waves and induces alpha waves."

[1091] 2. "Please create a piece of music that will help reduce stress. I'd like it to use the sound of a stream to induce theta waves."

[1092] In this way, the system of the present invention allows patients to effectively receive music therapy tailored to their individual symptoms, and reduces the burden of expensive copyright fees for hospitals and clinics.

[1093] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1094] Step 1:

[1095] The user launches a dedicated application on their smartphone or tablet. The application displays a home screen and a "Start a new session" button. The input data is the user's operation, and the output is the display of the information input screen.

[1096] Step 2:

[1097] The terminal displays an information input form, which includes input fields for "symptoms," "preferred sound," and "desired brainwave induction." The input data is the patient's information, and the output is the creation of input fields and a submit button.

[1098] Step 3:

[1099] The user enters the following information into the information input form. For example, the user enters "insomnia" as the symptom, "wave sounds" as the preferred sound, and "alpha waves" as the brain waves to induce. The input data is the information entered by the user in each field, and the output is a confirmation screen for the entered information.

[1100] Step 4:

[1101] The terminal encrypts the data entered by the user. It uses the AES-256 encryption method to convert the patient information into a secure format. The input data is the information entered by the user, and the output is the encrypted data.

[1102] Step 5:

[1103] The device sends encrypted data to the server. The secure HTTPS protocol is used to transfer data safely. The input data is encrypted patient information, and the output is a confirmation of transmission to the server.

[1104] Step 6:

[1105] The server receives the data sent from the terminal. After receiving it, it decrypts the data using the AES-256 method. The input data is the encrypted information, and the output is the decrypted data.

[1106] Step 7:

[1107] The server parses the received data in JSON format and extracts the symptoms, preferred sounds, and brain wave parameters to be induced. The input data is the decoded information, and the output is the extracted parameters.

[1108] Step 8:

[1109] The server constructs a music generation request to send to the generation AI. The request includes the patient's symptoms, preferred sounds, and the brain wave data to be induced. The input data are the extracted parameters, and the output is a music generation request.

[1110] Step 9:

[1111] The server sends a music generation request to the generation AI as an HTTP POST request. The input data is the music generation request, and the output is a confirmation of the transmission to the generation AI.

[1112] Step 10:

[1113] The generation AI processes the request received from the server. The request contents include "insomnia," "sound of waves," and "alpha waves." The input data is the request from the server, and the output is confirmation that processing has started.

[1114] Step 11:

[1115] Based on the request, the generative AI generates music by combining 1 / f fluctuations and tones that induce the specified brain waves. The input data are the request parameters, and the output is the generated music data.

[1116] Step 12:

[1117] The music data generated by the generative AI is encoded in FLAC format. The input data is the generated music, and the output is the encoded music data.

[1118] Step 13:

[1119] The generation AI returns the encoded music data to the server. The input data is the encoded music data, and the output is a confirmation of transmission to the server.

[1120] Step 14:

[1121] The server sends the music data received from the generation AI to the device. It re-encrypts the data and transfers it securely. The input data is the encoded music data, and the output is a confirmation of transmission to the device.

[1122] Step 15:

[1123] The music data received by the device is saved to local storage in the specified directory. The input data is the encrypted music data, and the output is the saved data.

[1124] Step 16:

[1125] The user taps the "Play" button in the application to play a saved song. The input data is the saved music data, and the output is the music being played.

[1126] By explaining each processing step in detail, the specific operation of this system becomes clear. The generative AI model and prompt sentences play an important role.

[1127] (Application example 1)

[1128] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1129] In modern society, many people suffer from mental health problems such as stress, insomnia, and anxiety. To provide individually optimized music therapy for these problems, music customized for each patient is necessary. However, traditional music therapy has the challenge of creating music that corresponds to individual symptoms and preferences. There is also a need to provide an environment where patients can easily obtain music and play it at home or elsewhere.

[1130] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1131] In this invention, the server includes a means for generating music for music therapy using a generation AI, a means for inputting information about the patient's symptoms and preferences, and a means for transmitting the input information to the generation AI, thereby enabling the provision of music therapy within a virtual store.

[1132] "Generative AI" is artificial intelligence that generates music for music therapy based on the individual symptoms and preferences of each patient.

[1133] "Music therapy" is a therapeutic method that uses music to improve a patient's mental and physical health.

[1134] "Patient" refers to an individual receiving music therapy, including those with specific conditions (e.g., insomnia, stress, anxiety).

[1135] A "terminal" is an electronic device (e.g., smartphone, portable music player) used by the patient to input information and play the generated music.

[1136] A "virtual store" is an online virtual space that provides music therapy and related services via the Internet.

[1137] This invention provides a system that uses generation AI to generate music for music therapy tailored to individual symptoms. This system is primarily composed of four elements: a server, a terminal, a virtual store, and a user. This configuration allows music therapy to be provided effectively and easily.

[1138] The server plays a central role in generating music for music therapy using generative AI. First, the server receives information about the patient's symptoms and preferences sent from the device. Next, it analyzes this information and sends a music generation request to the generative AI model based on that information. This generative AI model is trained using machine learning libraries such as TensorFlow to generate music with appropriate tones and brainwave induction. The generated music data is then encoded by the server and sent back to the device.

[1139] The terminal is an electronic device used by the user to input information and receive and play the generated music. This terminal can be a smartphone or portable music player. The user operates the application through the terminal and inputs their symptoms, preferred sounds, and the brain waves they want to induce. This input information is sent to the server in a secure format. The music data received from the server is stored on the terminal, and the user receives music therapy by playing it.

[1140] The virtual store is an online virtual space that provides music therapy and related services via the Internet. Users can download music therapy applications within the virtual store and receive individually optimized music.

[1141] As a concrete example, consider the case where a user has symptoms of insomnia. First, the user launches a music therapy application on their smartphone and selects "insomnia," "wave sounds," and "alpha waves." This information is sent to the server, which then sends a request to the generation AI. The generation AI generates the optimal music and sends it back to the server. The generated music data is then sent to the device, and the user can play it before going to bed at night to achieve the therapeutic effect of insomnia.

[1142] The following are examples of prompt sentences:

[1143] "Send a request to the generative AI model: Symptom = insomnia, Favorite sound = ocean waves, EEG to induce = alpha waves. Please generate the optimal music therapy piece for the patient."

[1144] This allows the system to effectively provide music therapy tailored to each patient's individual symptoms and make it easily accessible through a virtual store.

[1145] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1146] Step 1:

[1147] The device accepts input from the user about their symptoms, preferred sounds, and the brain waves they want to induce. Examples of input information include "insomnia," "wave sounds," and "alpha waves." This information is then converted into a secure format, such as JSON.

[1148] Step 2:

[1149] The device sends the input information to the server using the HTTPS protocol, ensuring secure communication. The transmitted data includes symptoms, preferred sounds, and the brainwaves to be induced.

[1150] Step 3:

[1151] The server receives the information sent from the device. The received data is analyzed and the parameters required for the generative AI are extracted. The analysis results include symptoms, preferred sounds, and brainwave induction parameters. A request is then sent to the generative AI model.

[1152] Step 4:

[1153] The generative AI generates music based on parameters received from the server. The generative AI model (using TensorFlow) is instructed to generate music using a prompt. An example prompt is, "Symptoms = insomnia, Favorite sound = Wave sounds, Brain waves to induce = Alpha waves. Please generate music for music therapy that is optimal for the patient." The generative AI creates music by combining 1 / f fluctuations and sounds that induce specific brain waves, and generates music data.

[1154] Step 5:

[1155] The server receives the music data sent from the generation AI. This music data is encoded and in a format that can be played by the user. The server then transfers the received music data to the device.

[1156] Step 6:

[1157] The device receives the music data sent from the server and stores it in local storage. The saved music data can then be played using a dedicated application. The user operates the application to play the generated music and receive music therapy.

[1158] Step 7:

[1159] The user plays the music created using the device whenever necessary. This music playback provides music therapy benefits such as insomnia relief and stress reduction. The user can also acquire various other music therapy songs through the virtual store.

[1160] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1161] This invention provides a system that combines a generative AI and an emotion engine to provide more effective music therapy to patients. Specific program processing of this system and its embodiments are described below.

[1162] overview

[1163] The server is the central system that manages the process of generating music for individual patients using generative AI. In addition, the device is equipped with an emotion engine that recognizes the user's emotions, and this emotion data is also provided to the generative AI. The user is the patient themselves, and they receive treatment by listening to music optimized for their symptoms and emotions.

[1164] Program processing

[1165] Entering patient information and emotion data

[1166] The device inputs the patient's symptoms, preferences, and emotional data. The user uses a device such as a smartphone or tablet to input symptoms (insomnia, stress, anxiety, etc.), favorite sounds (sound of waves, babbling brook, rustling leaves, etc.), and emotional data obtained from heart rate and facial expressions. The emotion engine analyzes the user's facial expressions and voice in real time to generate emotional data. The device converts this data into JSON format and sends it to the server.

[1167] Server-side processing

[1168] The server sends a music generation request to the generation AI based on the patient information and emotional data received from the device. The server analyzes the received data and extracts parameters (symptoms, preferred sounds, brain waves, emotional information) for the generation AI to generate music. Based on these parameters, the server constructs a music generation request and sends it to the generation AI.

[1169] Music Generation

[1170] The generative AI generates music for music therapy based on requests and emotional data sent from the server. The generative AI combines 1 / f fluctuations and tones that induce specified brain waves to create music that is optimal for the patient's symptoms and emotions. The generated music data is encoded and sent back to the server.

[1171] Music distribution and playback

[1172] The server sends the music data received from the AI ​​to the device, which receives it and stores it in local storage. The user can then play the music on their smartphone or portable music player to engage in music therapy.

[1173] Specific examples

[1174] Example 1: Insomnia treatment and emotion recognition

[1175] 1. A user suffers from insomnia and launches a dedicated application on their smartphone.

[1176] 2. Select "insomnia" on the device, "wave sound" as your preferred sound, and "alpha waves" as the brain waves you want to induce.

[1177] 3. The emotion engine analyzes the user's facial expressions and heart rate to recognize when they are feeling anxious.

[1178] 4. The device converts this information into JSON format and sends it to the server.

[1179] 5. The server sends a request to the generation AI, asking it to generate music based on "insomnia," "the sound of waves," "alpha waves," and "the user's anxiety."

[1180] 6. The AI ​​generates music incorporating elements that alleviate anxiety, such as the soothing sounds of waves and alpha waves, and sends it back to the server.

[1181] 7. The server sends the generated music data to the device, which saves it.

[1182] 8. Users can play the songs before going to bed at night to treat insomnia.

[1183] Example 2: Stress reduction and real-time emotion recognition

[1184] 1. The user is stressed and uses a portable music player.

[1185] 2. Select "Stress" on the device, "Babbling Stream" as your preferred sound, and "Theta Waves" as the brain waves you want to induce.

[1186] 3. The emotion engine recognizes high stress levels from the user's tone of voice.

[1187] 4. The device sends the information and emotional data in JSON format to the server, and the server sends a music generation request to the generation AI.

[1188] 5. The generative AI generates music that incorporates elements that reduce stress, including the relaxing sound of a stream and theta waves.

[1189] 6. The server sends the music data to the device, which receives and saves it.

[1190] 7. Users can play the songs during their work breaks to reduce stress.

[1191] This system allows patients to receive optimal music therapy based on their individual symptoms and real-time emotional data, which is expected to improve therapeutic effectiveness. The introduction of an emotional engine makes it possible to more accurately grasp the patient's psychological state and provide effective music.

[1192] The processing flow will be explained below.

[1193] Step 1:

[1194] The device inputs the patient's symptoms, preferences, and emotional data.

[1195] Action 1.1: The user launches the application on their device and selects the symptom type (insomnia, stress, anxiety, etc.).

[1196] Action 1.2: The user selects their preferred sound (waves, babbling, rustling leaves, etc.).

[1197] Action 1.3: The user specifies the brain waves (alpha waves, beta waves, theta waves, gamma waves) they want to induce.

[1198] Action 1.4: The emotion engine analyzes the user's facial expressions, voice, and heart rate in real time to generate emotion data.

[1199] Action 1.5: Once all the information has been entered and parsed, the device presses the send button to send it to the server.

[1200] Step 2:

[1201] The terminal transmits the input patient information and emotion data to the server.

[1202] Action 2.1: Convert input data and emotion data into JSON format.

[1203] Action 2.2: Send the data to the server using a secure communications protocol (e.g., HTTPS).

[1204] Action 2.3: Wait for confirmation from the server that the data was sent successfully.

[1205] Step 3:

[1206] The server sends a request to the AI ​​to generate music based on the patient information and emotional data it receives.

[1207] Operation 3.1: The server analyzes the received data and extracts the necessary parameters (symptoms, preferred sounds, brain waves, emotions).

[1208] Action 3.2: Construct a music generation request using the extracted parameters.

[1209] Action 3.3: Send a music generation request to the generation AI endpoint.

[1210] Action 3.4: Confirmation of successful music generation request submission is received.

[1211] Step 4:

[1212] The generation AI generates music based on the request sent.

[1213] Action 4.1: The generation AI determines the components of the song based on the received parameters.

[1214] Action 4.2: The generative AI designs music containing 1 / f fluctuations and tones that induce the specified brain waves.

[1215] Action 4.3: Generate the melody, rhythm, and harmony of the song and synthesize the final song.

[1216] Action 4.4: The generated music data is encoded and sent back to the server.

[1217] Step 5:

[1218] The server sends the music data received from the generation AI to the terminal.

[1219] Step 5.1: Check the format of the received music data (e.g. MP3, WAV).

[1220] Action 5.2: The song data is wrapped in JSON format again and sent to the device.

[1221] Action 5.3: Wait for confirmation that the music data has been successfully sent to the device.

[1222] Step 6:

[1223] The terminal receives the music data sent from the server and downloads it to local storage.

[1224] Action 6.1: Extract the song from the received JSON data.

[1225] Step 6.2: Save the song data to local storage.

[1226] Step 6.3: Verify that the song data was saved correctly.

[1227] Action 6.4: Notify the user that a song is available.

[1228] Step 7:

[1229] The effects of music therapy are sustained by having users listen to the music generated on a daily basis.

[1230] Action 7.1: A user plays a stored song on a smartphone or portable music player.

[1231] Action 7.2: The user listens to music regularly to achieve continuous therapeutic benefits.

[1232] Action 7.3: If necessary, go back to step 1 again to generate a new song.

[1233] Example 2

[1234] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1235] Music therapy is an effective treatment for many illnesses and psychological problems, but providing music optimized for each patient's symptoms and emotions remains a challenge. Furthermore, while the therapeutic effect could be further improved if real-time emotional data about the patient could be reflected in the music generation, a system to achieve this has yet to be developed. To address these challenges, the present invention aims to provide a music therapy system that combines generative AI and an emotion engine, thereby providing more effective music therapy for patients.

[1236] The specification process by the specification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes a means for generating music for music therapy using a generation AI, a means for inputting information about the patient's symptoms and preferences, a means including an emotion engine that collects and analyzes emotional data of the patient in real time, a means for converting the input information and emotional data into JSON format and sending it to the generation AI, a means for receiving music data generated by the generation AI and distributing it to a terminal, and a means for saving and playing the music data on the patient's terminal. This makes it possible to provide optimal music for each patient based on their symptoms and emotions in real time.

[1237] "Generative AI" is a system that uses artificial intelligence technology to automatically generate new music and data.

[1238] "Music therapy music" is music created with the purpose of improving a specific symptom or emotional state.

[1239] "Patient symptoms" refers to any physical or psychological problem or medical condition that a patient is experiencing.

[1240] "Preference information" is data that indicates a patient's individual tastes and preferences for music and sounds.

[1241] "Emotional data" is information that indicates the patient's emotional state, collected through heart rate and facial expression analysis, etc.

[1242] The "emotion engine" is a system that analyzes emotional data collected in real time and determines the patient's emotional state.

[1243] The "JSON format" is a lightweight data description language format primarily used for data exchange.

[1244] "Terminal" means a device used by a patient to input information or play generated music, including smart devices and portable audio equipment.

[1245] "1 / f fluctuation" is a type of signal with specific rhythmic and noise characteristics that is widely found in nature and is said to have a relaxation effect.

[1246] "Brainwave-inducing tones" refer to sound characteristics designed to induce specific brainwave activity (e.g., alpha waves, theta waves, etc.).

[1247] "Network connectivity" refers to the infrastructure that allows data communication between digital devices.

[1248] "Internal storage" refers to data storage devices built into a digital device.

[1249] "Encoding" is the process of converting data into a particular format.

[1250] A "music library" is a database that refers to a collection of music or sound clips.

[1251] This invention realizes a system that provides more effective music therapy to patients by combining a generative AI and an emotion engine. Specific program processing of this system and its embodiments are described below.

[1252] System configuration

[1253] The system mainly consists of a server, a terminal, a generative AI model, and an emotion engine. The server is responsible for managing data and sending requests to the generative AI, while the terminal inputs patient information and plays the generated music.

[1254] Entering patient information and emotion data

[1255] The terminal inputs the patient's symptoms, preferences, and emotional data. The user uses a smart device (e.g., smartphone or tablet) to input symptoms (insomnia, stress, anxiety, etc.) and preferred sounds (sound of waves, babbling brook, rustling leaves, etc.). The emotion engine also analyzes the user's facial expressions and voice, collecting emotional data derived from heart rate and facial expressions in real time. The terminal converts this data into JSON format and sends it to the server.

[1256] Server-side processing

[1257] The server sends a music generation request to the AI ​​based on the patient information and emotional data received from the device. The server analyzes the received data and extracts parameters (symptoms, preferred sounds, emotional information, and desired brain waves) that the AI ​​will need to generate music. Based on these parameters, the server constructs and sends a request to the AI.

[1258] Music Generation

[1259] The generative AI generates music for music therapy based on requests and emotional data sent from the server. The generative AI combines 1 / f fluctuations and tones that induce specified brain waves to create music that is optimal for the patient's symptoms and emotions. The generated music data is encoded and sent back to the server.

[1260] Music distribution and playback

[1261] The server sends the music data received from the AI ​​to the device, which receives it and stores it in local storage. The user can then play the generated music on a smart device or portable audio device and engage in music therapy.

[1262] Specific examples

[1263] A specific example of operation is given below.

[1264] Example 1: Insomnia treatment and emotion recognition

[1265] 1. Assume that the user has insomnia and launches a dedicated application on their smartphone.

[1266] 2. On the device, select "insomnia," your preferred sound "wave sound," and "alpha waves" as the brain waves you want to induce.

[1267] 3. The emotion engine analyzes the user's facial expressions and heart rate to recognize that they are feeling anxious rather than relaxed.

[1268] 4. The device converts this information into JSON format and sends it to the server.

[1269] 5. The server sends a request to the generation AI, asking it to generate music based on "insomnia," "the sound of waves," "alpha waves," and "the user's anxiety."

[1270] 6. The AI ​​generates music incorporating elements that alleviate anxiety, such as the soothing sounds of waves and alpha waves, and sends it back to the server.

[1271] 7. The server sends the generated music data to the device, which saves it.

[1272] 8. Users can play the songs before going to bed at night to treat insomnia.

[1273] Example 2: Stress reduction and real-time emotion recognition

[1274] 1. If the user is stressed, use a portable sound device.

[1275] 2. The device selects "stress," the preferred sound "babbling brook," and "theta waves" as the brain waves to be induced.

[1276] 3. The emotion engine recognizes high stress levels from the user's tone of voice.

[1277] 4. The device sends the information and emotional data in JSON format to the server, and the server sends a music generation request to the generation AI.

[1278] 5. The generative AI generates music that incorporates elements that reduce stress, including the relaxing sound of a stream and theta waves.

[1279] 6. The server sends the music data to the device, which receives and saves it.

[1280] 7. Users can play the songs during their work breaks to reduce stress.

[1281] Examples of prompt statements

[1282] Below is an example of a prompt sentence that is input to a generative AI model.

[1283] Prompts to generate music to relieve anxiety with "Insomnia," "Wave sounds," and "Alpha waves":

[1284] Symptoms: Insomnia

[1285] Favorite sound: The sound of waves

[1286] Brain waves to induce: alpha waves

[1287] Emotional data: Anxiety

[1288] Prompts to generate music that promotes relaxation through "stress," "babbling," and "theta waves":

[1289] Symptoms: Stress

[1290] Favorite sound: A babbling brook

[1291] Brain waves to be induced: θ waves

[1292] Emotional data: High stress levels

[1293] This system enables patients to receive optimal music therapy based on their individual symptoms and real-time emotional data, which is expected to improve therapeutic effectiveness. In addition, the introduction of an emotional engine makes it possible to more accurately grasp the patient's psychological state and provide more effective music.

[1294] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1295] Step 1: Enter patient information and emotion data

[1296] The device inputs information about the patient's symptoms and preferences, as well as emotional data, through a dedicated application. The user launches the app and inputs their symptoms (e.g., insomnia, stress, anxiety, etc.) and preferred sounds (e.g., the sound of waves, babbling brook, rustling leaves, etc.). The emotion engine uses the camera and heart rate monitor to analyze the user's facial expressions and heart rate in real time and generates emotional data (e.g., anxiety, heart rate, etc.). The input data is converted into JSON format within the application. In concrete terms, the user selects information on the touchscreen, and the camera and sensors collect data.

[1297] input:

[1298] User-entered symptoms and preferred sounds

[1299] Real-time emotion data obtained from camera and heart rate monitor

[1300] output:

[1301] Patient information and emotion data converted into JSON format

[1302] Step 2: Send patient information and emotion data

[1303] The device sends the generated JSON data to the server. The device establishes a network connection and sends the data using an HTTP POST request. Specifically, the application sends the data to the server in the background.

[1304] input:

[1305] JSON formatted patient information and emotion data

[1306] output:

[1307] Patient information and emotion data sent to the server

[1308] Step 3: Processing the music generation request on the server side

[1309] The server analyzes the received JSON data and constructs a music generation request for the generation AI. The server analyzes the data and extracts parameters such as symptoms, preferred sounds, emotional data, and the brain waves to be induced. It creates a generation AI request based on these parameters and sends it to the generation AI. Specifically, the server accesses the database to obtain the necessary information and constructs the request.

[1310] input:

[1311] JSON data received by the server

[1312] output:

[1313] A song generation request sent to the generation AI

[1314] Step 4: Run the music generation

[1315] The generative AI generates music for music therapy based on requests from the server. The generative AI selects appropriate sound clips from its internal music library and synthesizes them by combining 1 / f fluctuations and tones that induce specific brain waves. The generated music data is encoded and sent back to the server. Specifically, the generative AI uses an algorithm to generate music and encode the data.

[1316] input:

[1317] A song generation request sent to the generation AI

[1318] output:

[1319] Encoded music data sent back to the server

[1320] Step 5: Receiving and distributing music data

[1321] The server sends the encoded music data received from the generation AI to the device. The server sends the music data via HTTP, and the device receives and analyzes it, saving it to its internal storage. Specifically, the server transfers the music data, and the device writes it to its storage.

[1322] input:

[1323] Encoded music data received from the generation AI

[1324] output:

[1325] Music data stored on the device

[1326] Step 6: Music Playback and Music Therapy

[1327] The user plays music stored on the device and performs music therapy. The user operates a dedicated application on a smart device or portable audio device to play music and enjoy the therapeutic effects. Specifically, the user operates the app to select a song and listen to it through the device's speakers or earphones.

[1328] input:

[1329] Music data stored on the device

[1330] output:

[1331] Music played and listened to by the user

[1332] The above processing steps enable personalized music therapy. By combining generative AI and an emotion engine, it is possible to provide optimal music that reflects the patient's real-time emotional state, which is expected to improve the effectiveness of treatment.

[1333] (Application example 2)

[1334] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1335] Conventional music therapy systems have difficulty recognizing a user's emotional state in real time and providing personalized music. This has resulted in the inability to provide music that is optimal for the user's emotions and symptoms, limiting the therapeutic effect. Furthermore, use on devices such as smartphones and portable music players has been limited, resulting in low versatility. To solve these problems, a system that can recognize a user's emotional data in real time and provide personalized music is needed.

[1336] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1337] In this invention, the server includes a means for generating music for music therapy using a generation AI, a means for inputting information on the patient's symptoms and preferences, a means for transmitting the input information and emotional data to the generation AI, a means for receiving the music data generated by the generation AI and distributing it to a terminal, a means for the terminal to recognize the user's emotional state in real time, and a means for saving and playing the music data on the patient's terminal. This makes it possible to provide optimal music in real time according to the user's emotional state and improve the effectiveness of music therapy.

[1338] "Generative AI" is a type of artificial intelligence that uses data and algorithms to automatically perform specific tasks and generate music for music therapy.

[1339] A "patient" is a person who has a particular symptom or emotional state and receives music therapy to improve or treat that condition.

[1340] "Symptoms" refer to the state of illness or discomfort experienced by a patient, and include anxiety, insomnia, stress, etc.

[1341] "Preferences" refer to the patient's individual preferences for types of music or sounds.

[1342] "Information" includes data about patient symptoms and preferences, as well as emotional data collected in real time.

[1343] "Emotional data" refers to data that indicates the user's emotional state, derived from factors such as heart rate, facial expression, and tone of voice.

[1344] "Terminal" refers to a device such as a smartphone or portable music player that is used by the patient.

[1345] "Music data" refers to digital data of music for music therapy generated by the generative AI.

[1346] "Distribution" is the process of sending music data generated by the generation AI from the server to the device.

[1347] "Real-time" means responding immediately to ongoing events and changes.

[1348] The system that realizes this application example is a music therapy system that combines a generative AI model and an emotion engine. The core of the system is a server that provides music therapy in cooperation with the patient's device. The specific program processing is as follows:

[1349] System Configuration

[1350] server

[1351] The server receives the information and emotional data sent from the patient's device and requests the generation AI to generate a song. The generation AI generates a song based on the parameters and sends the song data back to the server. The server then distributes the generated song data to the patient's device. A high-performance cloud server (e.g., AWS EC2, Google Cloud) is used as the server.

[1352] Terminal

[1353] The patient's device is a smartphone or portable music player, and the user inputs information about their symptoms and preferences, as well as emotional data collected in real time. This data is sent from the device to a server. The device is equipped with an emotion engine that analyzes heart rate, facial expressions, and vocal tone in real time to generate emotional data. The device has the ability to receive, store, and play the generated music data.

[1354] User

[1355] The user is a patient who operates a device to receive music therapy based on their symptoms and preferences. The user starts the application and enters the necessary information to begin interaction with the system.

[1356] Data processing and calculation used

[1357] Emotion data collection: The device analyzes the emotional data collected, such as heart rate, facial expressions, and voice tone, converts it into JSON format, and sends it to the server.

[1358] Music generation: The server sends a request to the generation AI based on the input information and emotional data, and sets the music generation parameters (e.g., treatment for insomnia, relaxation effect, etc.).

[1359] Music data distribution: The server receives the music data created by the generation AI and distributes it back to the device. The device stores this data in local storage and plays it for music therapy.

[1360] Specific examples

[1361] 1. Music therapy for stress management:

[1362] The user feels stressed and launches a dedicated application.

[1363] The device selects "stress," the preferred sound is "babbling brook," and the brain waves you want to induce are "theta waves."

[1364] The emotion engine collects user emotional data and recognizes high stress levels.

[1365] The device converts this information into JSON format and sends it to the server.

[1366] The server sends a request to the generation AI, asking it to generate music to reduce stress.

[1367] The AI ​​generates music that includes the relaxing sound of a stream and theta waves, and sends it back to the server.

[1368] The server distributes the generated music data to the terminal, and the user plays it to relax.

[1369] Prompt Sentence Examples

[1370] "The user is feeling stressed. Please generate music that will have a relaxing effect. Preferably music that contains the sounds of waves and theta waves."

[1371] Based on this prompt, the AI ​​generates music that soothes the user's emotions.

[1372] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1373] Program processing flow and specific explanation

[1374] Step 1

[1375] Terminal-based information and emotional data collection

[1376] Input: Patient symptoms (e.g., stress), preferred sound (e.g., babbling), desired brainwave induction (e.g., theta waves), and real-time user emotional data (heart rate, facial expression, and voice tone).

[1377] Data processing: The device converts this data into JSON format.

[1378] How it works: The emotion engine analyzes the user's emotional state and generates emotion data, for example by using the smartphone's camera and microphone to capture the user's facial expressions and voice tone.

[1379] Output: JSON format data (symptoms, preferences, EEG, emotion data).

[1380] Step 2

[1381] Sending data from the device to the server

[1382] Input: The JSON formatted data generated in step 1.

[1383] Data processing: Send data from the device to the server via an HTTP POST request.

[1384] How it works: A smartphone or portable music player connects to a server using a communications interface and sends JSON data.

[1385] Output: Patient information and emotion data sent to the server.

[1386] Step 3

[1387] Data analysis by the server and sending requests to the generation AI

[1388] Input: Patient information and emotion data received in step 2 (JSON format).

[1389] Data processing: The server analyzes the data and extracts parameters (symptoms, preferences, brain waves, emotional information) for the generation AI to generate music.

[1390] How it works: A data analysis program runs on the server and constructs a request for the generation AI. The server then sends the music generation request to the generation AI using an HTTP POST request.

[1391] Output: A song generation request (in JSON format) sent to the generation AI.

[1392] Step 4

[1393] Music generation using generative AI

[1394] Input: The song generation request (in JSON format) sent in step 3.

[1395] Data processing: Generative AI generates music based on requests, combining specified brain waves and emotionally soothing tones (e.g., babbling, theta waves, etc.).

[1396] How it works: A generative AI model uses algorithms to generate music based on data, for example using a deep learning model to generate personalized music.

[1397] Output: Generated song data (binary format).

[1398] Step 5

[1399] Sending the generated music data back to the server

[1400] Input: Music data (binary format) generated by the generation AI.

[1401] Data processing: The music data is sent back to the server as an HTTP response.

[1402] How it works: The server receives the music data and encodes / decodes it as needed.

[1403] Output: Song data received by the server.

[1404] Step 6

[1405] Distribution and storage of music data to devices

[1406] Input: Song data (binary format) received by the server in step 5.

[1407] Data processing: Sends music data from the server to the device using HTTP responses and streaming.

[1408] Operation: The device receives the music data and saves it to local storage. Example: A smartphone saves the received music file to its internal storage.

[1409] Output: Song data saved on your device.

[1410] Step 7

[1411] Playing music on a device

[1412] Input: Song data (binary format) saved on the device in step 6.

[1413] Data processing: Decodes the data into a suitable format for playback.

[1414] How it works: The device plays music, and the user listens to the music and receives music therapy. Example: A smartphone or portable music player plays music through a music playback application.

[1415] Output: The music played to the user.

[1416] Specific examples of operation

[1417] Example prompt: "The user is feeling stressed. Please generate music that will have a relaxing effect. Preferably music that contains the sounds of waves and theta waves."

[1418] Based on this prompt, the generative AI will go through the steps above to generate a song that will ease the user's emotions, and then play it back to provide music therapy.

[1419] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1420] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1421] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1422] [Fourth embodiment]

[1423] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1424] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1425] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1426] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1427] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1428] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1429] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1430] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1431] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1432] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1433] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1434] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1435] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1436] This invention is a system that uses generative AI to generate music therapy songs and provide effective music therapy to patients. In this embodiment of the invention, three main elements - a server, a terminal, and a user - operate according to a series of flows.

[1437] overview

[1438] The server is the central device that manages the process of generating music therapy music for each patient using generative AI. The device is responsible for inputting and transmitting patient information and receiving and playing the generated music. The users are the patients themselves, who receive treatment by listening to music therapy music.

[1439] Program processing

[1440] Enter and submit patient information

[1441] The device inputs the patient's symptoms and preferences. Using a device such as a smartphone or tablet, the user inputs symptoms (insomnia, stress, anxiety, etc.), preferred sounds (waves, babbling, rustling leaves, etc.), and the brain waves they want to induce (alpha waves, beta waves, theta waves, gamma waves). The device converts this data into a secure format and sends it to the server.

[1442] Server-side processing

[1443] The server sends a music generation request to the AI ​​based on the patient information received from the device. The server analyzes the received data and extracts parameters (symptoms, preferred sounds, brain waves) for the AI ​​to generate music. The server then constructs a music generation request based on these parameters and sends it to the AI.

[1444] Music Generation

[1445] The generation AI generates music for music therapy based on requests sent from the server. The generation AI combines 1 / f fluctuations and tones that induce specified brain waves to create music that is optimal for the patient's symptoms. The generated music data is encoded and sent back to the server.

[1446] Music distribution and playback

[1447] The server sends the music data received from the AI ​​to the device, which receives it and stores it in local storage. The user can then play the music on their smartphone or portable music player to engage in music therapy.

[1448] Specific examples

[1449] Example 1: Treating insomnia

[1450] 1. The user has insomnia and launches a dedicated application on their smartphone.

[1451] 2. Select "insomnia" on the device, "wave sound" as your preferred sound, and "alpha waves" as the brain waves you want to induce.

[1452] 3. The device sends this information to the server.

[1453] 4. The server sends a request to the generation AI, asking it to generate music based on "insomnia," "the sound of waves," and "alpha waves."

[1454] 5. The AI ​​generates music containing the pleasant sounds of waves and alpha waves and sends it back to the server.

[1455] 6. The server sends the generated music data to the device.

[1456] 7. Users can play the songs before going to bed at night to treat insomnia.

[1457] Example 2: Stress relief

[1458] 1. The user is stressed and uses a portable music player.

[1459] 2. Select "Stress" on the device, "Babbling Stream" as your preferred sound, and "Theta Waves" as the brain waves you want to induce.

[1460] 3. The device sends the information to the server, and the server sends a music generation request to the generation AI.

[1461] 4. The generative AI creates music that includes the relaxing sound of a stream and theta waves.

[1462] 5. The server sends the music data to the device, which stores it.

[1463] 6. Users can play the music during their work breaks to reduce stress.

[1464] In this way, the system of the present invention allows patients to effectively receive music therapy tailored to their individual symptoms, and reduces the burden of expensive copyright fees for hospitals and clinics.

[1465] The processing flow will be explained below.

[1466] Step 1:

[1467] The device inputs the patient's symptoms and preferences.

[1468] Action 1.1: The user launches the application on their device and selects the symptom type (insomnia, stress, anxiety, etc.).

[1469] Action 1.2: The user selects their preferred sound (waves, babbling, rustling leaves, etc.).

[1470] Action 1.3: The user specifies the brain waves (alpha waves, beta waves, theta waves, gamma waves) they want to induce.

[1471] Action 1.4: Once you have completed the input, press the send button.

[1472] Step 2:

[1473] The terminal transmits the input patient information to the server.

[1474] Action 2.1: Convert input data to JSON format.

[1475] Action 2.2: Send the data to the server using a secure communications protocol (e.g., HTTPS).

[1476] Action 2.3: Wait for confirmation from the server that the data was sent successfully.

[1477] Step 3:

[1478] Based on the patient information received by the server, a request to generate music is sent to the generation AI.

[1479] Operation 3.1: The server analyzes the received data and extracts the necessary parameters (symptoms, preferred sounds, brain waves).

[1480] Action 3.2: Construct a music generation request using the extracted parameters.

[1481] Action 3.3: Send a music generation request to the generation AI endpoint.

[1482] Action 3.4: Confirmation of successful music generation request submission is received.

[1483] Step 4:

[1484] The generation AI generates music based on the request sent.

[1485] Action 4.1: The generation AI determines the components of the song based on the received parameters.

[1486] Action 4.2: The generative AI designs music containing 1 / f fluctuations and tones that induce the specified brain waves.

[1487] Action 4.3: Generate the melody, rhythm, and harmony of the song and synthesize the final song.

[1488] Action 4.4: The generated music data is encoded and sent back to the server.

[1489] Step 5:

[1490] The server sends the music data received from the generation AI to the terminal.

[1491] Step 5.1: Check the format of the received music data (e.g. MP3, WAV).

[1492] Action 5.2: The song data is wrapped in JSON format again and sent to the device.

[1493] Action 5.3: Wait for confirmation that the music data has been successfully sent to the device.

[1494] Step 6:

[1495] The terminal receives the music data sent from the server and downloads it to local storage.

[1496] Action 6.1: Extract the song from the received JSON data.

[1497] Step 6.2: Save the song data to local storage.

[1498] Step 6.3: Verify that the song data was saved correctly.

[1499] Action 6.4: Notify the user that a song is available.

[1500] Step 7:

[1501] The effects of music therapy are sustained by having users listen to the music generated on a daily basis.

[1502] Action 7.1: A user plays a stored song on a smartphone or portable music player.

[1503] Action 7.2: The user listens to music regularly to achieve continuous therapeutic benefits.

[1504] Action 7.3: If necessary, go back to step 1 again to generate a new song.

[1505] Example 1

[1506] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1507] Existing music therapy systems are unable to generate music tailored to individual patients' symptoms and preferences, limiting their therapeutic effectiveness. Furthermore, it is difficult for patients to select music themselves, requiring specialized knowledge to achieve a certain level of therapeutic effectiveness. In response to these challenges, the present invention aims to provide a system that generates individual music therapy music tailored to a patient's symptoms and preferences and provides it effectively.

[1508] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1509] In this invention, the server includes a means for inputting information about the patient's symptoms and preferences, a means for sending a music generation request to the generation AI, and a means for receiving the generated music data and distributing it to the terminal. This makes it possible to dynamically generate music that corresponds to the symptoms and preferences of each patient and distribute it efficiently.

[1510] "Patient information" refers to information that is individually set, such as the patient's symptoms, preferences, and desired brain waves.

[1511] "Generative AI" refers to a system that uses artificial intelligence technology to dynamically generate music based on patient information.

[1512] "Music generation request" refers to a data request sent from the server to the generation AI for music generation based on patient information.

[1513] "Music data" refers to digital data of music for music therapy generated by generative AI.

[1514] "Terminal" refers to a device used by a patient to receive and play music data, such as a smartphone, tablet, or portable playback device.

[1515] This invention is a system that uses generative AI to generate music therapy songs and provide effective music therapy to patients. In this form of the invention, three main elements - a server, a terminal, and a user - operate according to a series of flows.

[1516] overview

[1517] The server is the central device that manages the process of generating music therapy music for each patient using generative AI. The device is responsible for inputting and transmitting patient information and receiving and playing the generated music. The users are the patients themselves, who receive treatment by listening to music therapy music.

[1518] Hardware and Software Configuration

[1519] Server: A server computer with high processing power is used, equipped with specialized software and the necessary databases to run the generative AI model.

[1520] Device: A mobile device such as a smartphone or tablet used by a patient. These devices have a dedicated application installed and provide data entry and music playback functions.

[1521] Generative AI: An AI model for generating music for music therapy. This AI model has the ability to generate music by combining 1 / f fluctuations and tones that induce specific brain waves.

[1522] Program processing

[1523] In this system, music therapy songs are generated and provided to patients through the following process.

[1524] Enter and submit patient information

[1525] 1. The user launches a dedicated application on their smartphone or tablet.

[1526] 2. The device displays an information input form to the user, which includes items such as symptoms (e.g., insomnia), preferred sounds (e.g., waves), and desired brain waves (e.g., alpha waves).

[1527] 3. The user enters information in each field, confirms it, and then presses the submit button.

[1528] 4. The device encrypts the entered data and sends it in a secure format to the server.

[1529] Server-side processing

[1530] 1. The server receives the data sent from the device.

[1531] 2. The server analyzes the received data and extracts the symptoms, preferred sounds, and brain wave parameters to be induced.

[1532] 3. The server constructs and sends a music generation request to the generation AI.

[1533] Music Generation

[1534] 1. The generation AI receives the request sent from the server.

[1535] 2. Based on the request, the generation AI generates music by combining 1 / f fluctuations and tones that induce the specified brain waves.

[1536] 3. The music data generated by the generation AI is encoded and sent back to the server.

[1537] Music distribution and playback

[1538] 1. The server sends the music data received from the generation AI to the device.

[1539] 2. The device saves the received music data in local storage.

[1540] 3. Users play music on their smartphones or portable music players and engage in music therapy.

[1541] Specific examples

[1542] Example 1: Treating insomnia

[1543] 1. The user has insomnia and launches a dedicated application on their smartphone.

[1544] 2. Select "insomnia" on the device, "wave sound" as your preferred sound, and "alpha waves" as the brain waves you want to induce.

[1545] 3. The device sends this information to the server.

[1546] 4. The server sends a request to the generation AI, asking it to generate music based on "insomnia," "the sound of waves," and "alpha waves."

[1547] 5. The AI ​​generates music containing the pleasant sounds of waves and alpha waves and sends it back to the server.

[1548] 6. The server sends the generated music data to the device.

[1549] 7. Users can play the songs before going to bed at night to treat insomnia.

[1550] Example 2: Stress relief

[1551] 1. The user is stressed and uses a portable music player.

[1552] 2. Select "Stress" on the device, "Babbling Stream" as your preferred sound, and "Theta Waves" as the brain waves you want to induce.

[1553] 3. The device sends the information to the server, and the server sends a music generation request to the generation AI.

[1554] 4. The generative AI creates music that includes the relaxing sounds of babbling and theta waves.

[1555] 5. The server sends the music data to the device, which stores it.

[1556] 6. Users can play the music during their work breaks to reduce stress.

[1557] Prompt Sentence Examples

[1558] 1. "Please create a piece of music that is effective in treating insomnia. I would like music that is based on the sound of ocean waves and induces alpha waves."

[1559] 2. "Please create a piece of music that will help reduce stress. I'd like it to use the sound of a stream to induce theta waves."

[1560] In this way, the system of the present invention allows patients to effectively receive music therapy tailored to their individual symptoms, and reduces the burden of expensive copyright fees for hospitals and clinics.

[1561] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1562] Step 1:

[1563] The user launches a dedicated application on their smartphone or tablet. The application displays a home screen and a "Start a new session" button. The input data is the user's operation, and the output is the display of the information input screen.

[1564] Step 2:

[1565] The terminal displays an information input form, which includes input fields for "symptoms," "preferred sound," and "desired brainwave induction." The input data is the patient's information, and the output is the creation of input fields and a submit button.

[1566] Step 3:

[1567] The user enters the following information into the information input form. For example, the user enters "insomnia" as the symptom, "wave sounds" as the preferred sound, and "alpha waves" as the brain waves to induce. The input data is the information entered by the user in each field, and the output is a confirmation screen for the entered information.

[1568] Step 4:

[1569] The terminal encrypts the data entered by the user. It uses the AES-256 encryption method to convert the patient information into a secure format. The input data is the information entered by the user, and the output is the encrypted data.

[1570] Step 5:

[1571] The device sends encrypted data to the server. The secure HTTPS protocol is used to transfer data safely. The input data is encrypted patient information, and the output is a confirmation of transmission to the server.

[1572] Step 6:

[1573] The server receives the data sent from the terminal. After receiving it, it decrypts the data using the AES-256 method. The input data is the encrypted information, and the output is the decrypted data.

[1574] Step 7:

[1575] The server parses the received data in JSON format and extracts the symptoms, preferred sounds, and brain wave parameters to be induced. The input data is the decoded information, and the output is the extracted parameters.

[1576] Step 8:

[1577] The server constructs a music generation request to send to the generation AI. The request includes the patient's symptoms, preferred sounds, and the brain wave data to be induced. The input data are the extracted parameters, and the output is a music generation request.

[1578] Step 9:

[1579] The server sends a music generation request to the generation AI as an HTTP POST request. The input data is the music generation request, and the output is a confirmation of the transmission to the generation AI.

[1580] Step 10:

[1581] The generation AI processes the request received from the server. The request contents include "insomnia," "sound of waves," and "alpha waves." The input data is the request from the server, and the output is confirmation that processing has started.

[1582] Step 11:

[1583] Based on the request, the generative AI generates music by combining 1 / f fluctuations and tones that induce the specified brain waves. The input data are the request parameters, and the output is the generated music data.

[1584] Step 12:

[1585] The music data generated by the generative AI is encoded in FLAC format. The input data is the generated music, and the output is the encoded music data.

[1586] Step 13:

[1587] The generation AI returns the encoded music data to the server. The input data is the encoded music data, and the output is a confirmation of transmission to the server.

[1588] Step 14:

[1589] The server sends the music data received from the generation AI to the device. It re-encrypts the data and transfers it securely. The input data is the encoded music data, and the output is a confirmation of transmission to the device.

[1590] Step 15:

[1591] The music data received by the device is saved to local storage in the specified directory. The input data is the encrypted music data, and the output is the saved data.

[1592] Step 16:

[1593] The user taps the "Play" button in the application to play a saved song. The input data is the saved music data, and the output is the music being played.

[1594] By explaining each processing step in detail, the specific operation of this system becomes clear. The generative AI model and prompt sentences play an important role.

[1595] (Application example 1)

[1596] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1597] In modern society, many people suffer from mental health problems such as stress, insomnia, and anxiety. To provide individually optimized music therapy for these problems, music customized for each patient is necessary. However, traditional music therapy has the challenge of creating music that corresponds to individual symptoms and preferences. There is also a need to provide an environment where patients can easily obtain music and play it at home or elsewhere.

[1598] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1599] In this invention, the server includes a means for generating music for music therapy using a generation AI, a means for inputting information about the patient's symptoms and preferences, and a means for transmitting the input information to the generation AI, thereby enabling the provision of music therapy within a virtual store.

[1600] "Generative AI" is artificial intelligence that generates music for music therapy based on the individual symptoms and preferences of each patient.

[1601] "Music therapy" is a therapeutic method that uses music to improve a patient's mental and physical health.

[1602] "Patient" refers to an individual receiving music therapy, including those with specific conditions (e.g., insomnia, stress, anxiety).

[1603] A "terminal" is an electronic device (e.g., smartphone, portable music player) used by the patient to input information and play the generated music.

[1604] A "virtual store" is an online virtual space that provides music therapy and related services via the Internet.

[1605] This invention provides a system that uses generation AI to generate music for music therapy tailored to individual symptoms. This system is primarily composed of four elements: a server, a terminal, a virtual store, and a user. This configuration allows music therapy to be provided effectively and easily.

[1606] The server plays a central role in generating music for music therapy using generative AI. First, the server receives information about the patient's symptoms and preferences sent from the device. Next, it analyzes this information and sends a music generation request to the generative AI model based on that information. This generative AI model is trained using machine learning libraries such as TensorFlow to generate music with appropriate tones and brainwave induction. The generated music data is then encoded by the server and sent back to the device.

[1607] The terminal is an electronic device used by the user to input information and receive and play the generated music. This terminal can be a smartphone or portable music player. The user operates the application through the terminal and inputs their symptoms, preferred sounds, and the brain waves they want to induce. This input information is sent to the server in a secure format. The music data received from the server is stored on the terminal, and the user receives music therapy by playing it.

[1608] The virtual store is an online virtual space that provides music therapy and related services via the Internet. Users can download music therapy applications within the virtual store and receive individually optimized music.

[1609] As a concrete example, consider the case where a user has symptoms of insomnia. First, the user launches a music therapy application on their smartphone and selects "insomnia," "wave sounds," and "alpha waves." This information is sent to the server, which then sends a request to the generation AI. The generation AI generates the optimal music and sends it back to the server. The generated music data is then sent to the device, and the user can play it before going to bed at night to achieve the therapeutic effect of insomnia.

[1610] The following are examples of prompt sentences:

[1611] "Send a request to the generative AI model: Symptom = insomnia, Favorite sound = ocean waves, EEG to induce = alpha waves. Please generate the optimal music therapy piece for the patient."

[1612] This allows the system to effectively provide music therapy tailored to each patient's individual symptoms and make it easily accessible through a virtual store.

[1613] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1614] Step 1:

[1615] The device accepts input from the user about their symptoms, preferred sounds, and the brain waves they want to induce. Examples of input information include "insomnia," "wave sounds," and "alpha waves." This information is then converted into a secure format, such as JSON.

[1616] Step 2:

[1617] The device sends the input information to the server using the HTTPS protocol, ensuring secure communication. The transmitted data includes symptoms, preferred sounds, and the brainwaves to be induced.

[1618] Step 3:

[1619] The server receives the information sent from the device. The received data is analyzed and the parameters required for the generative AI are extracted. The analysis results include symptoms, preferred sounds, and brainwave induction parameters. A request is then sent to the generative AI model.

[1620] Step 4:

[1621] The generative AI generates music based on parameters received from the server. The generative AI model (using TensorFlow) is instructed to generate music using a prompt. An example prompt is, "Symptoms = insomnia, Favorite sound = Wave sounds, Brain waves to induce = Alpha waves. Please generate music for music therapy that is optimal for the patient." The generative AI creates music by combining 1 / f fluctuations and sounds that induce specific brain waves, and generates music data.

[1622] Step 5:

[1623] The server receives the music data sent from the generation AI. This music data is encoded and in a format that can be played by the user. The server then transfers the received music data to the device.

[1624] Step 6:

[1625] The device receives the music data sent from the server and stores it in local storage. The saved music data can then be played using a dedicated application. The user operates the application to play the generated music and receive music therapy.

[1626] Step 7:

[1627] The user plays the music created using the device whenever necessary. This music playback provides music therapy benefits such as insomnia relief and stress reduction. The user can also acquire various other music therapy songs through the virtual store.

[1628] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1629] This invention provides a system that combines a generative AI and an emotion engine to provide more effective music therapy to patients. Specific program processing of this system and its embodiments are described below.

[1630] overview

[1631] The server is the central system that manages the process of generating music for individual patients using generative AI. In addition, the device is equipped with an emotion engine that recognizes the user's emotions, and this emotion data is also provided to the generative AI. The user is the patient themselves, and they receive treatment by listening to music optimized for their symptoms and emotions.

[1632] Program processing

[1633] Entering patient information and emotion data

[1634] The device inputs the patient's symptoms, preferences, and emotional data. The user uses a device such as a smartphone or tablet to input symptoms (insomnia, stress, anxiety, etc.), favorite sounds (sound of waves, babbling brook, rustling leaves, etc.), and emotional data obtained from heart rate and facial expressions. The emotion engine analyzes the user's facial expressions and voice in real time to generate emotional data. The device converts this data into JSON format and sends it to the server.

[1635] Server-side processing

[1636] The server sends a music generation request to the generation AI based on the patient information and emotional data received from the device. The server analyzes the received data and extracts parameters (symptoms, preferred sounds, brain waves, emotional information) for the generation AI to generate music. Based on these parameters, the server constructs a music generation request and sends it to the generation AI.

[1637] Music Generation

[1638] The generative AI generates music for music therapy based on requests and emotional data sent from the server. The generative AI combines 1 / f fluctuations and tones that induce specified brain waves to create music that is optimal for the patient's symptoms and emotions. The generated music data is encoded and sent back to the server.

[1639] Music distribution and playback

[1640] The server sends the music data received from the AI ​​to the device, which receives it and stores it in local storage. The user can then play the music on their smartphone or portable music player to engage in music therapy.

[1641] Specific examples

[1642] Example 1: Insomnia treatment and emotion recognition

[1643] 1. A user suffers from insomnia and launches a dedicated application on their smartphone.

[1644] 2. Select "insomnia" on the device, "wave sound" as your preferred sound, and "alpha waves" as the brain waves you want to induce.

[1645] 3. The emotion engine analyzes the user's facial expressions and heart rate to recognize when they are feeling anxious.

[1646] 4. The device converts this information into JSON format and sends it to the server.

[1647] 5. The server sends a request to the generation AI, asking it to generate music based on "insomnia," "the sound of waves," "alpha waves," and "the user's anxiety."

[1648] 6. The AI ​​generates music incorporating elements that alleviate anxiety, such as the soothing sounds of waves and alpha waves, and sends it back to the server.

[1649] 7. The server sends the generated music data to the device, which saves it.

[1650] 8. Users can play the songs before going to bed at night to treat insomnia.

[1651] Example 2: Stress reduction and real-time emotion recognition

[1652] 1. The user is stressed and uses a portable music player.

[1653] 2. Select "Stress" on the device, "Babbling Stream" as your preferred sound, and "Theta Waves" as the brain waves you want to induce.

[1654] 3. The emotion engine recognizes high stress levels from the user's tone of voice.

[1655] 4. The device sends the information and emotional data in JSON format to the server, and the server sends a music generation request to the generation AI.

[1656] 5. The generative AI generates music that incorporates elements that reduce stress, including the relaxing sound of a stream and theta waves.

[1657] 6. The server sends the music data to the device, which receives and saves it.

[1658] 7. Users can play the songs during their work breaks to reduce stress.

[1659] This system allows patients to receive optimal music therapy based on their individual symptoms and real-time emotional data, which is expected to improve therapeutic effectiveness. The introduction of an emotional engine makes it possible to more accurately grasp the patient's psychological state and provide effective music.

[1660] The processing flow will be explained below.

[1661] Step 1:

[1662] The device inputs the patient's symptoms, preferences, and emotional data.

[1663] Action 1.1: The user launches the application on their device and selects the symptom type (insomnia, stress, anxiety, etc.).

[1664] Action 1.2: The user selects their preferred sound (waves, babbling, rustling leaves, etc.).

[1665] Action 1.3: The user specifies the brain waves (alpha waves, beta waves, theta waves, gamma waves) they want to induce.

[1666] Action 1.4: The emotion engine analyzes the user's facial expressions, voice, and heart rate in real time to generate emotion data.

[1667] Action 1.5: Once all the information has been entered and parsed, the device presses the send button to send it to the server.

[1668] Step 2:

[1669] The terminal transmits the input patient information and emotion data to the server.

[1670] Action 2.1: Convert input data and emotion data into JSON format.

[1671] Action 2.2: Send the data to the server using a secure communications protocol (e.g., HTTPS).

[1672] Action 2.3: Wait for confirmation from the server that the data was sent successfully.

[1673] Step 3:

[1674] The server sends a request to the AI ​​to generate music based on the patient information and emotional data it receives.

[1675] Operation 3.1: The server analyzes the received data and extracts the necessary parameters (symptoms, preferred sounds, brain waves, emotions).

[1676] Action 3.2: Construct a music generation request using the extracted parameters.

[1677] Action 3.3: Send a music generation request to the generation AI endpoint.

[1678] Action 3.4: Confirmation of successful music generation request submission is received.

[1679] Step 4:

[1680] The generation AI generates music based on the request sent.

[1681] Action 4.1: The generation AI determines the components of the song based on the received parameters.

[1682] Action 4.2: The generative AI designs music containing 1 / f fluctuations and tones that induce the specified brain waves.

[1683] Action 4.3: Generate the melody, rhythm, and harmony of the song and synthesize the final song.

[1684] Action 4.4: The generated music data is encoded and sent back to the server.

[1685] Step 5:

[1686] The server sends the music data received from the generation AI to the terminal.

[1687] Step 5.1: Check the format of the received music data (e.g. MP3, WAV).

[1688] Action 5.2: The song data is wrapped in JSON format again and sent to the device.

[1689] Action 5.3: Wait for confirmation that the music data has been successfully sent to the device.

[1690] Step 6:

[1691] The terminal receives the music data sent from the server and downloads it to local storage.

[1692] Action 6.1: Extract the song from the received JSON data.

[1693] Step 6.2: Save the song data to local storage.

[1694] Step 6.3: Verify that the song data was saved correctly.

[1695] Action 6.4: Notify the user that a song is available.

[1696] Step 7:

[1697] The effects of music therapy are sustained by having users listen to the music generated on a daily basis.

[1698] Action 7.1: A user plays a stored song on a smartphone or portable music player.

[1699] Action 7.2: The user listens to music regularly to achieve continuous therapeutic benefits.

[1700] Action 7.3: If necessary, go back to step 1 again to generate a new song.

[1701] Example 2

[1702] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1703] Music therapy is an effective treatment for many illnesses and psychological problems, but providing music optimized for each patient's symptoms and emotions remains a challenge. Furthermore, while the therapeutic effect could be further improved if real-time emotional data about the patient could be reflected in the music generation, a system to achieve this has yet to be developed. To address these challenges, the present invention aims to provide a music therapy system that combines generative AI and an emotion engine, thereby providing more effective music therapy for patients.

[1704] The specification process by the specification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes a means for generating music for music therapy using a generation AI, a means for inputting information about the patient's symptoms and preferences, a means including an emotion engine that collects and analyzes emotional data of the patient in real time, a means for converting the input information and emotional data into JSON format and sending it to the generation AI, a means for receiving music data generated by the generation AI and distributing it to a terminal, and a means for saving and playing the music data on the patient's terminal. This makes it possible to provide optimal music for each patient based on their symptoms and emotions in real time.

[1705] "Generative AI" is a system that uses artificial intelligence technology to automatically generate new music and data.

[1706] "Music therapy music" is music created with the purpose of improving a specific symptom or emotional state.

[1707] "Patient symptoms" refers to any physical or psychological problem or medical condition that a patient is experiencing.

[1708] "Preference information" is data that indicates a patient's individual tastes and preferences for music and sounds.

[1709] "Emotional data" is information that indicates the patient's emotional state, collected through heart rate and facial expression analysis, etc.

[1710] The "emotion engine" is a system that analyzes emotional data collected in real time and determines the patient's emotional state.

[1711] The "JSON format" is a lightweight data description language format primarily used for data exchange.

[1712] "Terminal" means a device used by a patient to input information or play generated music, including smart devices and portable audio equipment.

[1713] "1 / f fluctuation" is a type of signal with specific rhythmic and noise characteristics that is widely found in nature and is said to have a relaxation effect.

[1714] "Brainwave-inducing tones" refer to sound characteristics designed to induce specific brainwave activity (e.g., alpha waves, theta waves, etc.).

[1715] "Network connectivity" refers to the infrastructure that allows data communication between digital devices.

[1716] "Internal storage" refers to data storage devices built into a digital device.

[1717] "Encoding" is the process of converting data into a particular format.

[1718] A "music library" is a database that refers to a collection of music or sound clips.

[1719] This invention realizes a system that provides more effective music therapy to patients by combining a generative AI and an emotion engine. Specific program processing of this system and its embodiments are described below.

[1720] System configuration

[1721] The system mainly consists of a server, a terminal, a generative AI model, and an emotion engine. The server is responsible for managing data and sending requests to the generative AI, while the terminal inputs patient information and plays the generated music.

[1722] Entering patient information and emotion data

[1723] The terminal inputs the patient's symptoms, preferences, and emotional data. The user uses a smart device (e.g., smartphone or tablet) to input symptoms (insomnia, stress, anxiety, etc.) and preferred sounds (sound of waves, babbling brook, rustling leaves, etc.). The emotion engine also analyzes the user's facial expressions and voice, collecting emotional data derived from heart rate and facial expressions in real time. The terminal converts this data into JSON format and sends it to the server.

[1724] Server-side processing

[1725] The server sends a music generation request to the AI ​​based on the patient information and emotional data received from the device. The server analyzes the received data and extracts parameters (symptoms, preferred sounds, emotional information, and desired brain waves) that the AI ​​will need to generate music. Based on these parameters, the server constructs and sends a request to the AI.

[1726] Music Generation

[1727] The generative AI generates music for music therapy based on requests and emotional data sent from the server. The generative AI combines 1 / f fluctuations and tones that induce specified brain waves to create music that is optimal for the patient's symptoms and emotions. The generated music data is encoded and sent back to the server.

[1728] Music distribution and playback

[1729] The server sends the music data received from the AI ​​to the device, which receives it and stores it in local storage. The user can then play the generated music on a smart device or portable audio device and engage in music therapy.

[1730] Specific examples

[1731] A specific example of operation is given below.

[1732] Example 1: Insomnia treatment and emotion recognition

[1733] 1. Assume that the user has insomnia and launches a dedicated application on their smartphone.

[1734] 2. On the device, select "insomnia," your preferred sound "wave sound," and "alpha waves" as the brain waves you want to induce.

[1735] 3. The emotion engine analyzes the user's facial expressions and heart rate to recognize that they are feeling anxious rather than relaxed.

[1736] 4. The device converts this information into JSON format and sends it to the server.

[1737] 5. The server sends a request to the generation AI, asking it to generate music based on "insomnia," "the sound of waves," "alpha waves," and "the user's anxiety."

[1738] 6. The AI ​​generates music incorporating elements that alleviate anxiety, such as the soothing sounds of waves and alpha waves, and sends it back to the server.

[1739] 7. The server sends the generated music data to the device, which saves it.

[1740] 8. Users can play the songs before going to bed at night to treat insomnia.

[1741] Example 2: Stress reduction and real-time emotion recognition

[1742] 1. If the user is stressed, use a portable sound device.

[1743] 2. The device selects "stress," the preferred sound "babbling brook," and "theta waves" as the brain waves to be induced.

[1744] 3. The emotion engine recognizes high stress levels from the user's tone of voice.

[1745] 4. The device sends the information and emotional data in JSON format to the server, and the server sends a music generation request to the generation AI.

[1746] 5. The generative AI generates music that incorporates elements that reduce stress, including the relaxing sound of a stream and theta waves.

[1747] 6. The server sends the music data to the device, which receives and saves it.

[1748] 7. Users can play the songs during their work breaks to reduce stress.

[1749] Examples of prompt statements

[1750] Below is an example of a prompt sentence that is input to a generative AI model.

[1751] Prompts to generate music to relieve anxiety with "Insomnia," "Wave sounds," and "Alpha waves":

[1752] Symptoms: Insomnia

[1753] Favorite sound: The sound of waves

[1754] Brain waves to induce: alpha waves

[1755] Emotional data: Anxiety

[1756] Prompts to generate music that promotes relaxation through "stress," "babbling," and "theta waves":

[1757] Symptoms: Stress

[1758] Favorite sound: A babbling brook

[1759] Brain waves to be induced: θ waves

[1760] Emotional data: High stress levels

[1761] This system enables patients to receive optimal music therapy based on their individual symptoms and real-time emotional data, which is expected to improve therapeutic effectiveness. In addition, the introduction of an emotional engine makes it possible to more accurately grasp the patient's psychological state and provide more effective music.

[1762] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1763] Step 1: Enter patient information and emotion data

[1764] The device inputs information about the patient's symptoms and preferences, as well as emotional data, through a dedicated application. The user launches the app and inputs their symptoms (e.g., insomnia, stress, anxiety, etc.) and preferred sounds (e.g., the sound of waves, babbling brook, rustling leaves, etc.). The emotion engine uses the camera and heart rate monitor to analyze the user's facial expressions and heart rate in real time and generates emotional data (e.g., anxiety, heart rate, etc.). The input data is converted into JSON format within the application. In concrete terms, the user selects information on the touchscreen, and the camera and sensors collect data.

[1765] input:

[1766] User-entered symptoms and preferred sounds

[1767] Real-time emotion data obtained from camera and heart rate monitor

[1768] output:

[1769] Patient information and emotion data converted into JSON format

[1770] Step 2: Send patient information and emotion data

[1771] The device sends the generated JSON data to the server. The device establishes a network connection and sends the data using an HTTP POST request. Specifically, the application sends the data to the server in the background.

[1772] input:

[1773] JSON formatted patient information and emotion data

[1774] output:

[1775] Patient information and emotion data sent to the server

[1776] Step 3: Processing the music generation request on the server side

[1777] The server analyzes the received JSON data and constructs a music generation request for the generation AI. The server analyzes the data and extracts parameters such as symptoms, preferred sounds, emotional data, and the brain waves to be induced. It creates a generation AI request based on these parameters and sends it to the generation AI. Specifically, the server accesses the database to obtain the necessary information and constructs the request.

[1778] input:

[1779] JSON data received by the server

[1780] output:

[1781] A song generation request sent to the generation AI

[1782] Step 4: Run the music generation

[1783] The generative AI generates music for music therapy based on requests from the server. The generative AI selects appropriate sound clips from its internal music library and synthesizes them by combining 1 / f fluctuations and tones that induce specific brain waves. The generated music data is encoded and sent back to the server. Specifically, the generative AI uses an algorithm to generate music and encode the data.

[1784] input:

[1785] A song generation request sent to the generation AI

[1786] output:

[1787] Encoded music data sent back to the server

[1788] Step 5: Receiving and distributing music data

[1789] The server sends the encoded music data received from the generation AI to the device. The server sends the music data via HTTP, and the device receives and analyzes it, saving it to its internal storage. Specifically, the server transfers the music data, and the device writes it to its storage.

[1790] input:

[1791] Encoded music data received from the generation AI

[1792] output:

[1793] Music data stored on the device

[1794] Step 6: Music Playback and Music Therapy

[1795] The user plays music stored on the device and performs music therapy. The user operates a dedicated application on a smart device or portable audio device to play music and enjoy the therapeutic effects. Specifically, the user operates the app to select a song and listen to it through the device's speakers or earphones.

[1796] input:

[1797] Music data stored on the device

[1798] output:

[1799] Music played and listened to by the user

[1800] The above processing steps enable personalized music therapy. By combining generative AI and an emotion engine, it is possible to provide optimal music that reflects the patient's real-time emotional state, which is expected to improve the effectiveness of treatment.

[1801] (Application example 2)

[1802] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1803] Conventional music therapy systems have difficulty recognizing a user's emotional state in real time and providing personalized music. This has resulted in the inability to provide music that is optimal for the user's emotions and symptoms, limiting the therapeutic effect. Furthermore, use on devices such as smartphones and portable music players has been limited, resulting in low versatility. To solve these problems, a system that can recognize a user's emotional data in real time and provide personalized music is needed.

[1804] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1805] In this invention, the server includes a means for generating music for music therapy using a generation AI, a means for inputting information on the patient's symptoms and preferences, a means for transmitting the input information and emotional data to the generation AI, a means for receiving the music data generated by the generation AI and distributing it to a terminal, a means for the terminal to recognize the user's emotional state in real time, and a means for saving and playing the music data on the patient's terminal. This makes it possible to provide optimal music in real time according to the user's emotional state and improve the effectiveness of music therapy.

[1806] "Generative AI" is a type of artificial intelligence that uses data and algorithms to automatically perform specific tasks and generate music for music therapy.

[1807] A "patient" is a person who has a particular symptom or emotional state and receives music therapy to improve or treat that condition.

[1808] "Symptoms" refer to the state of illness or discomfort experienced by a patient, and include anxiety, insomnia, stress, etc.

[1809] "Preferences" refer to the patient's individual preferences for types of music or sounds.

[1810] "Information" includes data about patient symptoms and preferences, as well as emotional data collected in real time.

[1811] "Emotional data" refers to data that indicates the user's emotional state, derived from factors such as heart rate, facial expression, and tone of voice.

[1812] "Terminal" refers to a device such as a smartphone or portable music player that is used by the patient.

[1813] "Music data" refers to digital data of music for music therapy generated by the generative AI.

[1814] "Distribution" is the process of sending music data generated by the generation AI from the server to the device.

[1815] "Real-time" means responding immediately to ongoing events and changes.

[1816] The system that realizes this application example is a music therapy system that combines a generative AI model and an emotion engine. The core of the system is a server that provides music therapy in cooperation with the patient's device. The specific program processing is as follows:

[1817] System Configuration

[1818] server

[1819] The server receives the information and emotional data sent from the patient's device and requests the generation AI to generate a song. The generation AI generates a song based on the parameters and sends the song data back to the server. The server then distributes the generated song data to the patient's device. A high-performance cloud server (e.g., AWS EC2, Google Cloud) is used as the server.

[1820] Terminal

[1821] The patient's device is a smartphone or portable music player, and the user inputs information about their symptoms and preferences, as well as emotional data collected in real time. This data is sent from the device to a server. The device is equipped with an emotion engine that analyzes heart rate, facial expressions, and vocal tone in real time to generate emotional data. The device has the ability to receive, store, and play the generated music data.

[1822] User

[1823] The user is a patient who operates a device to receive music therapy based on their symptoms and preferences. The user starts the application and enters the necessary information to begin interaction with the system.

[1824] Data processing and calculation used

[1825] Emotion data collection: The device analyzes the emotional data collected, such as heart rate, facial expressions, and voice tone, converts it into JSON format, and sends it to the server.

[1826] Music generation: The server sends a request to the generation AI based on the input information and emotional data, and sets the music generation parameters (e.g., treatment for insomnia, relaxation effect, etc.).

[1827] Music data distribution: The server receives the music data created by the generation AI and distributes it back to the device. The device stores this data in local storage and plays it for music therapy.

[1828] Specific examples

[1829] 1. Music therapy for stress management:

[1830] The user feels stressed and launches a dedicated application.

[1831] The device selects "stress," the preferred sound is "babbling brook," and the brain waves you want to induce are "theta waves."

[1832] The emotion engine collects user emotional data and recognizes high stress levels.

[1833] The device converts this information into JSON format and sends it to the server.

[1834] The server sends a request to the generation AI, asking it to generate music to reduce stress.

[1835] The AI ​​generates music that includes the relaxing sound of a stream and theta waves, and sends it back to the server.

[1836] The server distributes the generated music data to the terminal, and the user plays it to relax.

[1837] Prompt Sentence Examples

[1838] "The user is feeling stressed. Please generate music that will have a relaxing effect. Preferably music that contains the sounds of waves and theta waves."

[1839] Based on this prompt, the AI ​​generates music that soothes the user's emotions.

[1840] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1841] Program processing flow and specific explanation

[1842] Step 1

[1843] Terminal-based information and emotional data collection

[1844] Input: Patient symptoms (e.g., stress), preferred sound (e.g., babbling), desired brainwave induction (e.g., theta waves), and real-time user emotional data (heart rate, facial expression, and voice tone).

[1845] Data processing: The device converts this data into JSON format.

[1846] How it works: The emotion engine analyzes the user's emotional state and generates emotion data, for example by using the smartphone's camera and microphone to capture the user's facial expressions and voice tone.

[1847] Output: JSON format data (symptoms, preferences, EEG, emotion data).

[1848] Step 2

[1849] Sending data from the device to the server

[1850] Input: The JSON formatted data generated in step 1.

[1851] Data processing: Send data from the device to the server via an HTTP POST request.

[1852] How it works: A smartphone or portable music player connects to a server using a communications interface and sends JSON data.

[1853] Output: Patient information and emotion data sent to the server.

[1854] Step 3

[1855] Data analysis by the server and sending requests to the generation AI

[1856] Input: Patient information and emotion data received in step 2 (JSON format).

[1857] Data processing: The server analyzes the data and extracts parameters (symptoms, preferences, brain waves, emotional information) for the generation AI to generate music.

[1858] How it works: A data analysis program runs on the server and constructs a request for the generation AI. The server then sends the music generation request to the generation AI using an HTTP POST request.

[1859] Output: A song generation request (in JSON format) sent to the generation AI.

[1860] Step 4

[1861] Music generation using generative AI

[1862] Input: The song generation request (in JSON format) sent in step 3.

[1863] Data processing: Generative AI generates music based on requests, combining specified brain waves and emotionally soothing tones (e.g., babbling, theta waves, etc.).

[1864] How it works: A generative AI model uses algorithms to generate music based on data, for example using a deep learning model to generate personalized music.

[1865] Output: Generated song data (binary format).

[1866] Step 5

[1867] Sending the generated music data back to the server

[1868] Input: Music data (binary format) generated by the generation AI.

[1869] Data processing: The music data is sent back to the server as an HTTP response.

[1870] How it works: The server receives the music data and encodes / decodes it as needed.

[1871] Output: Song data received by the server.

[1872] Step 6

[1873] Distribution and storage of music data to devices

[1874] Input: Song data (binary format) received by the server in step 5.

[1875] Data processing: Sends music data from the server to the device using HTTP responses and streaming.

[1876] Operation: The device receives the music data and saves it to local storage. Example: A smartphone saves the received music file to its internal storage.

[1877] Output: Song data saved on your device.

[1878] Step 7

[1879] Playing music on a device

[1880] Input: Song data (binary format) saved on the device in step 6.

[1881] Data processing: Decodes the data into a suitable format for playback.

[1882] How it works: The device plays music, and the user listens to the music and receives music therapy. Example: A smartphone or portable music player plays music through a music playback application.

[1883] Output: The music played to the user.

[1884] Specific examples of operation

[1885] Example prompt: "The user is feeling stressed. Please generate music that will have a relaxing effect. Preferably music that contains the sounds of waves and theta waves."

[1886] Based on this prompt, the generative AI will go through the steps above to generate a song that will ease the user's emotions, and then play it back to provide music therapy.

[1887] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1888] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1889] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1890] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1891] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1892] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1893] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1894] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1895] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1896] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1897] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1898] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1899] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1900] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1901] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1902] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1903] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1904] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1905] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1906] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1907] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1908] The following is further disclosed regarding the above embodiment.

[1909] (Claim 1)

[1910] A means for generating music for music therapy using generative AI;

[1911] a means of inputting information about the patient's symptoms and preferences;

[1912] A means for transmitting the input information to a generation AI;

[1913] A means for receiving music data generated by the generation AI and distributing it to a terminal;

[1914] A means for storing and playing music data on a patient's device;

[1915] A system including:

[1916] (Claim 2)

[1917] The system according to claim 1, characterized in that the generation AI generates music that includes 1 / f fluctuations and tones that induce specific brain waves.

[1918] (Claim 3)

[1919] 2. The system of claim 1, wherein the patient's terminal is a smartphone or a portable music player.

[1920] "Example 1"

[1921] (Claim 1)

[1922] a means of inputting information about the patient's symptoms and preferences;

[1923] A means for transmitting the input information to a generation AI;

[1924] The generating AI has a means for receiving a request to generate music including 1 / f fluctuations and a tone that induces a specific brain wave;

[1925] A means for receiving the music data generated by the generation AI and distributing it to a terminal;

[1926] means for storing and playing music data on the patient's terminal;

[1927] A system including:

[1928] (Claim 2)

[1929] The system according to claim 1, characterized in that the generation AI dynamically generates music based on each patient's information.

[1930] (Claim 3)

[1931] 2. The system of claim 1, wherein the patient's terminal is a smartphone, a tablet, or a portable playback device.

[1932] "Application Example 1"

[1933] (Claim 1)

[1934] A means for generating music for music therapy using generative AI;

[1935] a means of inputting information about the patient's symptoms and preferences;

[1936] A means for transmitting the input information to a generation AI;

[1937] A means for receiving music data generated by the generation AI and distributing it to a terminal;

[1938] A means for storing and playing music data on a patient's device;

[1939] A means of providing music therapy within a virtual store;

[1940] A system including:

[1941] (Claim 2)

[1942] The system according to claim 1, characterized in that the generation AI generates music that includes 1 / f fluctuations and tones that induce specific brain waves.

[1943] (Claim 3)

[1944] 2. The system of claim 1, wherein the patient's terminal is a smartphone or a portable music player.

[1945] "Example 2: Combining Emotion Engines"

[1946] (Claim 1)

[1947] A means for generating music for music therapy using generative AI;

[1948] a means of inputting information about the patient's symptoms and preferences;

[1949] a means including an emotion engine for collecting and analyzing emotion data of a patient in real time;

[1950] A means for converting the input information and emotion data into JSON format and transmitting the JSON format to the generation AI;

[1951] A means for receiving music data generated by the generation AI and distributing it to a terminal;

[1952] A means for storing and playing music data on a patient's device;

[1953] A system including:

[1954] (Claim 2)

[1955] The system according to claim 1, characterized in that the generation AI generates music that includes 1 / f fluctuations and tones that induce specific brain waves.

[1956] (Claim 3)

[1957] 2. The system of claim 1, wherein the patient's terminal is a smart device or a portable audio device.

[1958] "Application example 2 when combining emotion engines"

[1959] (Claim 1)

[1960] A means for generating music for music therapy using generative AI;

[1961] a means of inputting information about the patient's symptoms and preferences;

[1962] means for transmitting the input information and emotion data to a generation AI;

[1963] A means for receiving music data generated by the generation AI and distributing it to a terminal;

[1964] means for the terminal to recognize the user's emotional state in real time;

[1965] A means for storing and playing music data on a patient's device;

[1966] A system including:

[1967] (Claim 2)

[1968] The system according to claim 1, characterized in that the generation AI generates music that includes 1 / f fluctuations and tones that induce specific brain waves.

[1969] (Claim 3)

[1970] 2. The system according to claim 1, wherein the patient's terminal is a general-purpose personal digital assistant or a portable music player. [Explanation of symbols]

[1971] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for generating music for music therapy using generative AI; a means of inputting information about the patient's symptoms and preferences; A means for transmitting the input information to a generation AI; A means for receiving music data generated by the generation AI and distributing it to a terminal; A means for storing and playing music data on a patient's device; A system including:

2. The system according to claim 1, characterized in that the generation AI generates music that includes 1 / f fluctuations and tones that induce specific brain waves.

3. The system according to claim 1, wherein the patient's terminal is a smartphone or a portable music player.

Images