System

Abstract

To provide a system.SOLUTION: A system includes: means for detecting an electric signal in a brain of a user from at least any one of a chip embedded in a head and a very small machinery and materials attached to an outside of a face; means for analyzing the detected electric signal and transmitting the electric signal to a feeling engine; means for the feeling engine to estimate a feeling state of the user from an analysis result of the electric signal; means for a generative AI model to generate music on the basis of a past listening history of the user and the feeling state of the user; and means for playing the generated music.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Currently, operating devices such as smartphones and computers requires physical contact with the device. However, operating devices can be difficult when you are unable to use your hands or when your hands are full. Furthermore, technology for generating and arranging music in real time to suit individual user preferences is still in its infancy. [Means for solving the problem]

[0005] This invention uses a chip implanted in the head or a tiny device attached to the outside of the face to input and output auditory and visual signals via electrical signals in the brain, monologue, bone conduction, sound waves, etc., making it possible to operate devices without using hands. It also uses generative AI to generate and arrange music based on the user's preferences and circumstances. This allows users to obtain information, make calls, use messaging apps, send emails, and more without touching anything, and enjoy their favorite music in real time. [Brief explanation of the drawings]

[0006] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 2 is a sequence diagram showing a flow of processing in the data processing system according to the first embodiment of the first form example. [Figure 12]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Embodiment 1. [Figure 13] FIG. 10 is a sequence diagram showing a processing flow of a data processing system in a second embodiment of the second form example. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 of Embodiment Example 2. [Figure 15] FIG. 10 is a sequence diagram showing the flow of processing in a data processing system according to a third embodiment of the third embodiment. [Figure 16] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 3 of Embodiment 3. [Figure 17] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the first embodiment of the first form example when an emotion engine is combined. [Figure 18] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Form Example 1 when an emotion engine is combined. [Figure 19] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the second embodiment of the second form example when an emotion engine is combined. [Figure 20] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 of Form Example 2 when an emotion engine is combined. [Figure 21] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the third embodiment of the third form example when an emotion engine is combined. [Figure 22] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 3 of Form Example 3 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0007] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0008] First, the terms used in the following description will be explained.

[0009] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, the processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (TENSOR PROCESSING UNIT (registered trademark)).

[0010] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0011] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0012] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0013] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0014] [First embodiment]

[0015] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0016] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0017] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0018] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0019] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0020] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0021] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0022] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0023] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0024] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0025] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0026] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0027] "Example 1"

[0028] One embodiment of the present invention uses a chip implanted in the head or a tiny device attached to the outside of the face. This chip or tiny device detects electrical signals in the brain, monologue, bone conduction, sound waves, etc., and analyzes this information. Specifically, an EEG analyzer analyzes brain waves and sends the results to a generative AI. The generative AI infers the user's intention based on the analysis results and performs appropriate operations. For example, if the user thinks, "Play music," the EEG analyzer captures that intention, and the generative AI controls the music playback.

[0029] "Example 2"

[0030] Furthermore, generative AI generates and arranges music according to the user's preferences and circumstances. Specifically, it generates and arranges music taking into account the user's past listening history and current situation (e.g., time, location, mood, etc.). For example, if a user wants to relax, generative AI will capture that intention and generate and arrange relaxing music.

[0031] "Example 3"

[0032] Furthermore, the system of the present invention retrieves information, makes calls, uses messenger apps, and sends emails via mobile networks. Specifically, the generative AI analyzes the user's intention and performs appropriate operations. For example, if the user thinks, "Send an email," the generative AI will grasp that intention and perform the operation to send an email.

[0033] The processing flow of each embodiment will be described below.

[0034] "Example 1"

[0035] Step 1: A chip is implanted in the user's head or a tiny device is attached to the outside of the face.

[0036] Step 2: A chip or tiny device detects electrical signals in the brain, speech, bone conduction, sound waves, etc.

[0037] Step 3: The detected information is sent to an EEG analysis device, which analyzes the EEG.

[0038] Step 4: Send the analysis results to the generative AI.

[0039] Step 5: The generative AI infers the user's intention based on the analysis results and performs the appropriate operation. For example, if the user thinks, "Play music," the generative AI will perform the operation to play music.

[0040] "Example 2"

[0041] Step 1: The generative AI takes into account the user's past listening history and current situation (e.g., time, location, mood, etc.).

[0042] Step 2: The generative AI generates and arranges music based on the information it has taken into account. For example, if the user wants to relax, the generative AI will capture that intention and generate and arrange relaxing music.

[0043] "Example 3"

[0044] Step 1: Generative AI analyzes user intent.

[0045] Step 2: The generative AI performs the appropriate action based on the analysis results. For example, if the user thinks, "Send an email," the generative AI will capture that intent and perform the action of sending an email.

[0046] Example 1

[0047] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0048] Conventional technology requires users to manually input information to operate devices, which is difficult to do, especially when users have physical limitations or their hands are full. It is also difficult to accurately read the user's intentions and perform appropriate operations. This reduces user convenience and makes it difficult to achieve efficient operation.

[0049] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0050] In this invention, the server includes means for detecting electrical signals in the brain, monologue, bone conduction, sound waves, etc. using a chip implanted in the head or a tiny device attached to the outside of the face, an electroencephalogram (EEG) analyzer for analyzing the detected electrical signals, means for transmitting the analysis results to a generative AI, means for the generative AI to infer the user's intentions based on the analysis results and perform appropriate operations, and means for conversing via a mobile line, obtaining necessary information, obtaining suggestions for the next action, playing music and arranging it to convey a user's favorite image to play new, perfect background music, talking to people, using a messenger app, sending emails, etc., all without touching anything. This enables the user to accurately read intentions and perform appropriate operations without manually operating the device.

[0051] A "headchip" is a small electronic device that is implanted in a user's head to detect electrical signals in the brain.

[0052] The "extremely small equipment attached to the outside of the face" is a small device that is attached to the outside of the user's face and detects electrical signals in the brain, monologue, bone conduction, sound waves, etc.

[0053] "Brain electrical signals" are signals that indicate electrical activity occurring in the user's brain.

[0054] "Soliloquy" refers to words that a user speaks to himself or herself and is detected as voice.

[0055] "Bone conduction" is the phenomenon in which sound waves travel through bones to the inner ear, thereby transmitting audio information.

[0056] "Sound waves" are vibrations that travel through the air and are perceived as sound.

[0057] An "EEG analysis device" is a device that analyzes electrical signals in the brain to read the user's intentions and thoughts.

[0058] "Generative AI" is an artificial intelligence system that infers the user's intentions based on analysis results and performs appropriate operations.

[0059] A "mobile line" is a communication line for sending and receiving data using wireless communication.

[0060] A "conversation" is a spoken communication between a user and another person or system.

[0061] "Acquisition of required information" is the process of searching for and acquiring the information the user needs.

[0062] "Getting next action suggestions" is the process of suggesting the next action to be taken based on the user's current situation and intentions.

[0063] "Playing music" is the process by which a user selects and plays the music they want to listen to.

[0064] "Personal image" refers to a visual or auditory image that the user likes.

[0065] "Arrangement" is the process of modifying existing music or images to suit the user's preferences.

[0066] "BGM" is an abbreviation for background music, and refers to music played as background music.

[0067] A "call" is a process in which a user communicates with another person by voice.

[0068] A "messenger app" is application software for sending and receiving text messages.

[0069] "Mail sending" is the process of creating and sending an email.

[0070] This invention is a system that uses a chip implanted in the user's head or a tiny device attached to the outside of the face to detect electrical signals in the brain, monologue, bone conduction, sound waves, etc., and analyzes this information. Specifically, the following hardware and software are used:

[0071] Hardware and software used

[0072] Hardware: Chips implanted in the head, tiny devices attached to the outside of the face, brain wave analyzers

[0073] Software: Generative AI

[0074] Data processing and calculation

[0075] 1. The user wears a chip implanted in the head or a tiny device attached to the outside of the face, which is then ready to detect electrical signals in the brain, speech, bone conduction, sound waves, etc.

[0076] 2. A chip or tiny device detects electrical signals in the user's brain in real time. For example, if a user thinks, "Play music," the corresponding electrical signals will be detected.

[0077] 3. The device transmits the detected electrical signal to the EEG analyzer, which then transmits this data via wireless communication or other means.

[0078] 4. The EEG analyzer analyzes the received electrical signals using sophisticated algorithms and pattern recognition to interpret the user's thoughts and intentions.

[0079] 5. The EEG analyzer sends the analysis results to the generative AI, which includes information about the user's intentions.

[0080] 6. The generative AI infers the user's intention based on the analysis results received from the EEG analyzer. For example, the intention may be "Play music."

[0081] 7. The generative AI performs the appropriate action based on the inferred intent. Specifically, it launches a music playback application and plays music.

[0082] Specific examples

[0083] For example, when a user wants to "play music," the specific operation is as follows.

[0084] 1. The user puts on the chip and thinks, "Play some music."

[0085] 2. The chip detects electrical signals corresponding to the thought.

[0086] 3. The device transmits the detected electrical signal to the EEG analysis device.

[0087] 4. The EEG analyzer analyzes the electrical signals and reads the intention to "play music."

[0088] 5. The EEG analysis device sends the analysis results to the generative AI.

[0089] 6. The generative AI infers the intent, "Play music."

[0090] 7. The generative AI launches a music playback application and plays music.

[0091] Prompt Sentence Examples

[0092] An example of a prompt sentence to input to the generative AI model is as follows:

[0093] When a user thinks, "Play music," the brainwave analysis device captures that intention, and the generative AI controls the music playback.

[0094] In this way, a system is realized that can read the user's intentions and perform appropriate operations.

[0095] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0096] Step 1:

[0097] The user wears a chip that is implanted in the head or a tiny device that is attached to the outside of the face.

[0098] Input: User wearing behavior

[0099] Output: With chip or tiny device attached

[0100] How it works: The user implants a chip in their head or wears a tiny device on the outside of their face, which is then ready to detect electrical signals in the brain, speech, bone conduction, sound waves, etc.

[0101] Step 2:

[0102] A chip or tiny device detects electrical signals in the brain.

[0103] Input: Electrical signals in the user's brain

[0104] Output: Detected electrical signal data

[0105] What it does: When a user thinks "play music," an electrical signal corresponding to that thought is detected by a chip or tiny piece of equipment.

[0106] Step 3:

[0107] The terminal transmits the detected electrical signal to an electroencephalogram analysis device.

[0108] Input: Detected electrical signal data

[0109] Output: Electrical signal data sent to EEG analyzer

[0110] Specific operation: The terminal receives the electrical signals detected by the chip or tiny device and transmits them to the EEG analysis device using wireless communication or other means.

[0111] Step 4:

[0112] An electroencephalogram (EEG) analyzer analyzes the electrical signals.

[0113] Input: Transmitted electrical signal data

[0114] Output: Parsed intent data

[0115] Specific operation: The EEG analyzer analyzes the received electrical signals and performs pattern recognition to read the user's thoughts and intentions. For example, the intention "play music" is analyzed.

[0116] Step 5:

[0117] The EEG analysis device sends the analysis results to the generative AI.

[0118] Input: Parsed intent data

[0119] Output: Intent data sent to the generative AI

[0120] How it works: The EEG analyzer sends the analysis results to the generative AI, which includes information about the user's intentions.

[0121] Step 6:

[0122] Generative AI infers the user's intentions based on the analysis results.

[0123] Input: Submitted intent data

[0124] Output: Inferred intent

[0125] Specific operation: The generative AI infers the user's intention based on the analysis results received from the EEG analysis device. For example, the intention may be "Play music."

[0126] Step 7:

[0127] The generative AI performs appropriate operations based on the inferred intent.

[0128] Input: Inferred intent

[0129] Output: The operation performed

[0130] Specific behavior: Based on the inferred intent, the generative AI launches a music playback application and performs the operation of playing music.

[0131] (Application example 1)

[0132] Next, a description will be given of Application Example 1 of Embodiment Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0133] Conventional autonomous vehicle operation requires the user to manually set the destination, making operation complicated. Although interfaces using brainwaves and voice exist, technology that accurately estimates the user's intentions and reflects them in the operation of an autonomous vehicle is still in its infancy. This has resulted in low user convenience and made intuitive operation difficult.

[0134] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0135] In this invention, the server includes means for detecting electrical signals in the brain, monologue, bone conduction, sound waves, etc. using a chip implanted in the head or a tiny device attached to the outside of the face, means for conversing with the generative AI via a communication line, obtaining necessary information, obtaining suggestions for the next action, playing music and arranging it to convey the user's favorite image to play the best new background music, talking to people, messaging apps, sending emails, etc., all without touching anything, and means for estimating the user's intentions and operating the self-driving vehicle, which allows the user to intuitively operate the self-driving vehicle.

[0136] A "headchip" is a small electronic device that is implanted in the user's head to detect electrical signals in the brain.

[0137] The "external facial equipment" is a small device that is worn on the outside of the user's face to detect electrical signals and sound waves in the brain.

[0138] "Electrical signals in the brain" are weak electrical fluctuations that occur when the brain is active, and by analyzing these, it is possible to infer the user's intentions and state.

[0139] "Muttering to oneself" refers to words that a user utters unconsciously in a low voice, and by detecting this, it is possible to understand the user's intentions.

[0140] "Bone conduction" is a phenomenon in which sound waves are transmitted to the inner ear through the skull, and is a technology that utilizes this phenomenon to obtain audio information.

[0141] "Sound waves" are vibrations that travel through the air, and by detecting them, audio information can be obtained.

[0142] "Generative AI" is an artificial intelligence system that generates appropriate responses and operations based on the user's intentions and requests.

[0143] "Communication lines" refer to network infrastructure for sending and receiving data, and include mobile lines and internet lines.

[0144] An "autonomous vehicle" is a vehicle that uses artificial intelligence and sensor technology to drive autonomously.

[0145] "Estimating user intent" means analyzing data such as brain waves and voice to determine what the user wants.

[0146] "Performing an operation" means controlling a device or system based on the user's intentions.

[0147] The following system configuration is used as an embodiment of this invention: The system is composed of a chip to be implanted in the head or a tiny device to be attached to the outside of the face, an EEG analyzer, a generative AI, a communication line, and a control system for an autonomous vehicle.

[0148] First, the user wears a chip implanted in the head or a tiny device attached to the outside of the face, which detects electrical signals in the brain, speech, bone conduction, sound waves, etc. This data is sent to an electroencephalogram (EEG) analyzer and analyzed to infer the user's intentions.

[0149] The analyzed data is sent via communication lines to a generative AI, which generates appropriate responses and operations based on the user's intentions. This generative AI also has the ability to generate and arrange music according to the user's preferences and circumstances.

[0150] The operation commands generated by the generative AI are then sent to the autonomous vehicle's control system, allowing the autonomous vehicle to set the destination and begin navigation simply by the user thinking, "Go to the destination."

[0151] Specific hardware includes Emotiv Epoc+ for the EEG analysis device, Google® Glass® for the smart glasses, and Tesla® Autopilot for the autonomous vehicle control system. Software includes Emotiv SDK for the EEG analysis library, OpenAI® GPT-4® for the generative AI model, and Tesla API for the autonomous vehicle control API.

[0152] As a concrete example, consider the case where a user puts on smart glasses and thinks, "I want to go home." In this case, the EEG analyzer captures the user's intention, and the generative AI generates the command "go home." The self-driving vehicle then sets "home" as the destination and begins navigation.

[0153] An example prompt might look like this:

[0154] "If a user thinks, 'I want to go home,' the EEG analyzer will capture that intention, the generative AI will generate the command 'go home,' and the self-driving vehicle will set the destination to 'home' and begin navigation."

[0155] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0156] Step 1:

[0157] The user wears a chip implanted in the head or a tiny device attached to the outside of the face, which detects electrical signals in the brain, speech, bone conduction, sound waves, etc. The input is the electrical signals and sound waves in the user's brain, and the output is these signal data.

[0158] Step 2:

[0159] The device sends the detected signal data to an EEG analysis device, which analyzes the data and infers the user's intention. The input is signal data, and the output is analyzed intention data. Specifically, the data is analyzed using an EEG analysis library (e.g., Emotiv SDK).

[0160] Step 3:

[0161] The server sends the analyzed intent data to the generative AI, which generates appropriate responses and operation commands based on the user's intent. The input is the intent data, and the output is the operation command. Specifically, the response is generated using a generative AI model (e.g., OpenAI GPT-4).

[0162] Step 4:

[0163] The server sends the generated operation command to the autonomous vehicle control system. The autonomous vehicle control system operates the vehicle based on the received command. The input is the operation command, and the output is the vehicle's behavior. The specific behavior is to control the vehicle using the autonomous vehicle control API (e.g., Tesla API).

[0164] Step 5:

[0165] If a user thinks, "I want to go home," the EEG analyzer captures that intention, the generative AI generates the command "go home," and the self-driving vehicle sets the destination to "home" and begins navigation. The input is the user's intention, and the output is setting the destination and starting navigation. Specifically, the navigation system sets the destination, and the vehicle begins moving automatically.

[0166] Example 2

[0167] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0168] Conventional music generation systems have had difficulty generating and arranging music in real time according to the user's preferences and circumstances. Furthermore, manual operation was required for the user to communicate their intentions to the system, which was inconvenient. Furthermore, there was a lack of means to effectively utilize the user's past listening history and current circumstances, making it difficult to provide optimal music for each individual user.

[0169] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0170] In this invention, the server includes means for collecting the user's past listening history and current situation (time, place, mood, etc.), means for transmitting the collected data to the server and storing it in a database, means for generating prompt sentences to be input to the generative AI model based on the stored data, means for inputting the prompt sentences to the generative AI model and generating and arranging music using music generation software, and means for transmitting the generated music to the user's terminal and playing it. This makes it possible to generate and arrange music in real time according to the user's preferences and situation, and provide optimal music without the user having to perform manual operations.

[0171] A "head chip or external facial device" is a small electronic device placed on the user's head or face to detect and transmit electrical signals and sound waves in the brain.

[0172] "Means for inputting / outputting information to the auditory and visual senses using electrical signals in the brain, monologue, bone conduction, sound waves, etc." refers to technology for inputting or outputting information to the auditory and visual senses using electrical signals and sound waves generated in the user's brain.

[0173] A "generative AI model" is an artificial intelligence model that generates and arranges music and other content based on user preferences and circumstances.

[0174] "Communication lines" refers to network infrastructure for sending and receiving data, including mobile lines and internet lines.

[0175] "Means for collecting a user's past listening history and current situation (time, location, mood, etc.)" refers to technology for collecting information such as the user's history of music they have listened to in the past, as well as their current time, location, mood, etc.

[0176] The "means for transmitting collected data to a server and storing the data in a database" refers to a technique for transmitting data collected from users to a server and storing the data in a database.

[0177] "Means for generating prompt sentences to be input to a generative AI model based on stored data" refers to a technology that generates instruction sentences (prompt sentences) to be input to a generative AI model based on user information stored in a database.

[0178] "Music generation software" means software for generating and arranging music based on instructions from a generative AI model.

[0179] "Means for transmitting the generated music to a user's device and playing it" refers to technology for transmitting music generated by a generative AI model to a user's device and playing the music on that device.

[0180] This invention is a system for generating and arranging music according to user preferences and circumstances. The system includes a user terminal, a server, a generative AI model, music generation software, and a database management system.

[0181] Hardware and software used

[0182] Hardware: Servers, user devices (smartphones, PCs, etc.)

[0183] Software: Generative AI models (e.g., GPT-3®), database management systems (e.g., MySQL®), music generation software

[0184] System Operation

[0185] 1. Collection of User Data

[0186] When a user uses a music streaming app, the device collects past listening history and current situation (time, location, mood, etc.).

[0187] Specifically, the device collects the history of songs played by the user, current GPS location information, and data obtained from the device's built-in sensors (e.g., heart rate, number of steps).

[0188] For example, if the user feels like relaxing in the evening, the terminal will record that information.

[0189] 2. Data transmission and storage

[0190] The terminal periodically transmits the collected data to the server.

[0191] The server stores the received data in a database management system (e.g. MySQL).

[0192] Specifically, the server stores the user ID, listening history, and current situation data in a database.

[0193] For example, the data for user ID "12345" may include information such as "nighttime" and "want to relax."

[0194] 3. Prompt generation

[0195] Based on the stored data, the server generates prompt sentences to be input into the generative AI model.

[0196] Specifically, the server obtains the user's current situation from a database and creates a prompt sentence based on that situation.

[0197] For example, a prompt sentence such as "User ID 12345 feels like relaxing, so please create some relaxing music" is generated.

[0198] 4. Music Creation and Arrangement

[0199] The server inputs prompts into a generative AI model (e.g., GPT-3), which then uses music generation software to generate and arrange music.

[0200] Specifically, the generative AI model analyzes the prompt and determines appropriate musical characteristics (such as tempo, melody, and instrumentation).

[0201] Music generation software then generates music based on these features.

[0202] For example, a piano piece with a slow tempo is generated as relaxing music.

[0203] 5. Distribution of generated music

[0204] The server transmits the generated music to the user's terminal.

[0205] The terminal then plays the received music for the user to listen to.

[0206] Specifically, the music player application on the device plays the generated music file.

[0207] For example, if the user feels like relaxing in the evening, the terminal will play a generated piano piece.

[0208] This system allows users to enjoy music in real time according to their preferences and circumstances.

[0209] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0210] Step 1:

[0211] User Data Collection

[0212] When a user uses a music streaming app, the device collects past listening history and current situation (time, location, mood, etc.).

[0213] Input: User playback history, GPS location, data from built-in sensors (heart rate, steps, etc.)

[0214] Data processing: Organize the collected data and identify the user's current situation.

[0215] Output: Organized user data (e.g. user ID, play history, current time, location, mood)

[0216] What it does: The device records the music history of the user, obtains the user's current GPS location, and collects heart rate and step counts from built-in sensors. For example, if the user feels like relaxing in the evening, it will record that information.

[0217] Step 2:

[0218] Data transmission and storage

[0219] The terminal periodically transmits the collected data to the server.

[0220] The server stores the received data in a database management system.

[0221] Input: Organized user data

[0222] Data processing: Converting data into a format suitable for storing in a database.

[0223] Output: User data stored in the database

[0224] Specific operation: The device sends the organized user data to the server, which then stores the data in a database. For example, for user ID "12345," the server stores information such as "evening time" and "want to relax."

[0225] Step 3:

[0226] Generate prompt statement

[0227] Based on the stored data, the server generates prompt sentences to be input into the generative AI model.

[0228] Input: User data stored in the database

[0229] Data processing: Analyze the user's current situation and create a prompt suitable for the generative AI model.

[0230] Output: Generated prompt statement

[0231] Specific operation: The server retrieves the user's current situation from the database and creates a prompt based on it. For example, it generates a prompt such as "User ID 12345 feels like relaxing, so please create some relaxing music."

[0232] Step 4:

[0233] Music Creation and Arrangement

[0234] The server inputs prompts into the generative AI model, which then uses music generation software to generate and arrange the music.

[0235] Input: Generated prompt statement

[0236] Data processing: The generative AI model analyzes the prompt and determines musical characteristics. Music generation software generates music based on these characteristics.

[0237] Output: Generated music file

[0238] How it works: The generative AI model analyzes the prompt and determines appropriate musical characteristics (tempo, melody, instrumentation, etc.). The music generation software then generates music based on these characteristics. For example, it might generate a piano piece with a slow tempo for relaxation.

[0239] Step 5:

[0240] Generated Music Distribution

[0241] The server transmits the generated music to the user's terminal.

[0242] The terminal then plays the received music for the user to listen to.

[0243] Input: Generated music file

[0244] Data processing: Convert music files into a format suitable for your device.

[0245] Output: Music files playable on your device

[0246] Specific operation: The server sends the generated music file to the user's device, and the device's music player app plays the music file. For example, if the user feels like relaxing at night, the device will play the generated piano music.

[0247] (Application example 2)

[0248] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0249] Conventional music playback systems have difficulty generating and arranging music based on the user's preferences and current situation, and are unable to respond to specific needs such as when users want to relax, concentrate, or exercise. Furthermore, there are problems with users having to go through the trouble of selecting music, and it takes time to find the perfect piece of music. Furthermore, there has been a lack of systems that can generate and arrange music taking into account the user's past listening history and current situation.

[0250] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0251] In this invention, the server includes a chip implanted in the head or a tiny device attached to the outside of the face, means for inputting and outputting to the auditory and visual senses using electrical signals in the brain, monologue, bone conduction, sound waves, etc., means for conversing with the generative AI via a mobile line, obtaining necessary information, obtaining suggestions for the next action, playing music and arranging it to convey one's favorite images to play the best new background music, talking to people, and sending messenger apps and emails, all without touching a thing, means for generating and arranging music taking into account the user's past listening history and current situation (time, place, mood, etc.), and means including a music streaming application installed on a smartphone. This makes it possible to generate and arrange optimal music according to the user's preferences and current situation.

[0252] A "headchip" is a small electronic device that is implanted in the user's head to detect and analyze electrical signals in the brain.

[0253] The "extremely small equipment attached to the outside of the face" is a small device that is attached to the outside of the user's face and uses sound waves and bone conduction to input / output auditory and visual information.

[0254] "Electrical signals in the brain" are signals that indicate electrical activity occurring in the user's brain, and by analyzing these, it is possible to understand the user's intentions and state.

[0255] "Soliloquy" refers to the words a user speaks to themselves, and by detecting this, it is possible to understand the user's intentions and state.

[0256] "Bone conduction" is a phenomenon in which sound waves are transmitted to the inner ear through the skull, and is a technology that utilizes this phenomenon to transmit audio information.

[0257] "Sound waves" are vibrations that travel through the air, and this technology utilizes them to transmit audio information.

[0258] "Generative AI" is an artificial intelligence technology that generates and arranges music and information according to the user's preferences and circumstances.

[0259] A "mobile line" is a wireless communication network used by mobile communication devices such as mobile phones and smartphones.

[0260] "Conversation" is the act of exchanging information between a user and a system through voice.

[0261] "Acquisition of necessary information" refers to the act of the system collecting and providing the information required by the user.

[0262] "Getting next action suggestions" is the act of the system suggesting the next action to be taken based on the user's current situation and intentions.

[0263] "Playing music" refers to the system playing music for the user.

[0264] "Communicate your preferred image, have it arranged, and play the best new background music" means that the user communicates their preferred musical image to the system, and the generative AI arranges the music based on that and plays the optimal background music.

[0265] "Talking to a person" means that a user converses with another person through voice.

[0266] "Messenger app or email sending" refers to the act of a user sending a message via a messaging application or email.

[0267] "User's past listening history" is a record of music that the user has listened to in the past.

[0268] "Current situation" refers to information that indicates the user's current state, such as time, location, and mood.

[0269] A "music streaming application" is software for playing music in real time over the Internet.

[0270] The system for implementing this invention is configured as follows: First, a chip to be implanted in the user's head or a tiny device to be attached to the outside of the face is required. This device detects electrical signals in the brain, monologue, bone conduction, sound waves, etc., and provides input / output to the auditory and visual senses.

[0271] Next, the generative AI talks to the user over the mobile network and obtains the necessary information. The generative AI generates and arranges music taking into account the user's past listening history and current situation (time, location, mood, etc.). This generative AI is implemented in a programming language such as Python (registered trademark) and works in conjunction with a music streaming application installed on the smartphone.

[0272] Specifically, it generates and arranges the most suitable music depending on the user's situation, such as whether they want to relax, concentrate, exercise, etc. For example, if a user wants to relax in the morning, the generative AI will select and play relaxing music from the user's past listening history.

[0273] The system's hardware includes a smartphone, a head-implanted chip, and a tiny external facial device, while the software includes a generative AI model, a music streaming application, and brainwave analysis software.

[0274] Data processing and calculation involves analyzing the electrical signals in the user's brain to understand the user's intentions and state. Next, generative AI generates and arranges music based on the user's past listening history and current situation. This allows for optimal music playback based on the user's preferences and current situation.

[0275] As a concrete example, by inputting the following prompt sentence into a generative AI model, music can be generated that meets the user's needs.

[0276] Example prompt sentence:

[0277] "If a user wants to relax in the morning, generate relaxing music based on their past listening history."

[0278] In this way, music can be generated and arranged according to the user's preferences and current situation.

[0279] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0280] Step 1:

[0281] The user wears a chip that is implanted in the head or a tiny device that is attached to the outside of the face.

[0282] Input: Electrical signals in the user's brain, monologue, bone conduction, sound waves, etc.

[0283] Output: Detected electrical signals and sound wave data

[0284] How it works: A chip or tiny device detects electrical signals and sound waves in the user's brain and records them as digital data.

[0285] Step 2:

[0286] The device analyzes the detected data and understands the user's intentions and status.

[0287] Input: Detected electrical signals or acoustic data

[0288] Output: Information about the user's intent and state

[0289] Specific operation: The device's built-in brain wave analysis software analyzes the detected electrical signals and sound wave data to understand the user's intentions and state, such as whether they want to relax, concentrate, or exercise.

[0290] Step 3:

[0291] The server obtains the user's past listening history.

[0292] Input: User's identity

[0293] Output: User's past listening history data

[0294] Specific operation: The server retrieves the past listening history from the database based on the user's identification information.

[0295] Step 4:

[0296] The server obtains the current situation (time, location, mood, etc.).

[0297] Input: Data about the user's current time, location, and mood

[0298] Output: Current status data

[0299] What it does: The server collects data about the user's current time, location, and mood from their smartphone and other sensors.

[0300] Step 5:

[0301] The server uses a generative AI model to generate and arrange music based on the user's past listening history and current situation.

[0302] Input: User's past listening history data, current situation data

[0303] Output: Generated music data

[0304] Specific operation: The generative AI model generates and arranges optimal music based on the user's past listening history and current situation. For example, it uses the prompt "If the user wants to relax in the morning, please generate relaxing music based on the user's past listening history."

[0305] Step 6:

[0306] The device plays the generated music.

[0307] Input: Generated music data

[0308] Output: Music being played

[0309] Specific operation: A music streaming application installed on the device plays the generated music data and allows the user to listen to it.

[0310] Step 7:

[0311] While listening to music, the user can converse with the generative AI as needed, which will suggest the next course of action and obtain necessary information.

[0312] Input: User voice input, generative AI suggestions and information

[0313] Output: Suggested next steps and required information

[0314] Specific operation: The user converses with the generative AI via voice to obtain suggestions for the next action and necessary information. For example, if the user says, "Tell me what to do next," the generative AI will make appropriate suggestions.

[0315] Example 3

[0316] Next, a description will be given of a third embodiment of the third embodiment. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0317] Conventional systems require users to perform physical operations when obtaining information or making calls or sending messages. This reduces user convenience, making operation particularly difficult when the user's hands are full or when the user has visual or hearing impairments. It is also difficult to accurately understand the user's intentions and perform appropriate operations. To solve these issues, a system that can analyze the user's intentions and automatically perform appropriate operations is needed.

[0318] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[0319] In this invention, the server includes means for acquiring user voice data and converting it into text using voice recognition software, means for analyzing the text data using a generative artificial intelligence model to understand the user's intention, and operation means for executing the generated procedures, thereby enabling the user to perform operations such as obtaining information, making calls, and sending messages using only voice input without performing any physical operations.

[0320] A "head chip" is a small electronic device that is implanted under the skin of the head and is used for communicating with the outside world and processing data.

[0321] "External facial attachment" refers to a small electronic device worn on the outside of the face for external communication and data processing.

[0322] "Electrical signals in the brain" are signals that indicate electrical activity emitted by the brain, and by analyzing these signals it is possible to read the state and intentions of the brain.

[0323] "Soliloquy" refers to the speech that a user speaks to themselves, and by analyzing this speech the intention of the user can be understood.

[0324] "Bone conduction" is a phenomenon in which sound waves are transmitted to the inner ear through bones, and is a technology that utilizes this phenomenon to transmit sound.

[0325] "Sound waves" are vibrations that travel through the air, and this is a technology that uses them to transmit sound.

[0326] "Auditory and visual input / output" refers to the means for providing audio and video to the user, and also to the means for obtaining audio and visual information from the user.

[0327] "Generative AI" is an AI technology that analyzes the user's intentions and performs appropriate operations.

[0328] A "communication line" is a network infrastructure for sending and receiving data, and includes mobile lines and internet lines.

[0329] "Speech recognition software" is software for converting voice data into text data.

[0330] A "generative artificial intelligence model" is an artificial intelligence model that analyzes a user's text data and understands their intentions.

[0331] The "operation means" is a means for executing the generated procedure, and is used to perform specific operations such as sending an email or starting a call.

[0332] This invention provides a system that enables a user to perform operations such as obtaining information, making calls, and sending messages only through voice input, without performing any physical operations. This system is implemented using the following hardware and software.

[0333] Hardware Configuration

[0334] 1. Chips implanted in the head or tiny devices attached to the outside of the face: These devices capture electrical signals and voices from the user's brain and communicate with an external server.

[0335] 2. Mobile terminal: A device that allows users to input voice data and transmits the voice data to the server.

[0336] Software Configuration

[0337] 1. Speech recognition software: Converts the user's voice data into text data using software such as the Google Speech-to-Text API.

[0338] 2. Generative AI models: Using models such as OpenAI's GPT-3, text data is analyzed to understand user intent.

[0339] 3. Email sending software: Uses an SMTP server or similar to send emails based on the generated instructions.

[0340] Data processing and calculation

[0341] The server receives the voice data sent by the user and converts it into text data using speech recognition software. It then uses a generative artificial intelligence model to analyze the text data and understand the user's intent. Based on the understood intent, it generates specific operating procedures and executes the operations using email sending software.

[0342] Specific examples

[0343] Consider the case where a user issues a verbal command such as "Send me the materials for tomorrow's meeting." In this case, the system operates as follows:

[0344] 1. The user speaks into the microphone on their smartphone, saying, "Send me the materials for tomorrow's meeting."

[0345] 2. The device records this voice data and sends it to the server.

[0346] 3. The server sends the received voice data to the Google Speech-to-Text API and obtains the text data, "Send me the materials for tomorrow's meeting."

[0347] 4. The server inputs the text data into a generative AI model and analyzes the user's intent. The generative AI model understands the instruction "Send the materials for tomorrow's meeting" as "Send an email."

[0348] 5. The generative AI model generates specific operational procedures such as:

[0349] To: example@example.com

[0350] Subject: Materials for tomorrow's meeting

[0351] Message: Hello. I have attached the materials for tomorrow's meeting below. Thank you in advance.

[0352] 6. The server sends an email using the SMTP server to execute the generated operation procedure. The email is sent to the specified destination, completing the user's instructions.

[0353] Prompt Sentence Examples

[0354] The user says, "Send me the materials for tomorrow's meeting." Use the following information to generate a procedure for sending an email.

[0355] To: example@example.com

[0356] Subject: Materials for tomorrow's meeting

[0357] Message: Hello. I have attached the materials for tomorrow's meeting below. Thank you in advance.

[0358] In this way, a system that analyzes the user's intention and performs an appropriate operation is realized. The flow of the identification process in the third embodiment will be described with reference to FIG.

[0359] Step 1:

[0360] The user issues commands by voice into the microphone on the mobile device, for example, "Send me the materials for tomorrow's meeting."

[0361] Input: User's voice data

[0362] Output: Audio data recorded on the device

[0363] Specific operation: The user gives voice instructions and the device records the voice.

[0364] Step 2:

[0365] The terminal transmits the recorded voice data to the server.

[0366] Input: Audio data recorded on the device

[0367] Output: Audio data sent to the server

[0368] Specific operation: The device sends voice data to the server.

[0369] Step 3:

[0370] The server sends the received voice data to speech recognition software (e.g., Google Speech-to-Text API) and converts it into text data.

[0371] Input: Audio data sent to the server

[0372] Output: Text data generated by speech recognition software

[0373] Specific operation: The server sends the voice data to the voice recognition software and obtains the text data.

[0374] Step 4:

[0375] The server inputs the acquired text data into a generative artificial intelligence model (e.g., OpenAI's GPT-3) and analyzes the user's intent.

[0376] Input: Text data generated by speech recognition software

[0377] Output: User intent analyzed by a generative AI model

[0378] Specific operation: The server inputs text data into a generative artificial intelligence model and analyzes the user's intention.

[0379] Step 5:

[0380] The generative AI model generates specific operational procedures based on the user's intentions, such as procedures for sending an email.

[0381] Input: User intent analyzed by a generative artificial intelligence model

[0382] Output: Generated specific operating instructions

[0383] Specific operation: A generative artificial intelligence model generates operating procedures based on the user's intentions.

[0384] Step 6:

[0385] The server sends an email using email sending software (e.g., an SMTP server) to execute the generated operation procedure.

[0386] Input: Generated specific operating instructions

[0387] Output: Email sent

[0388] Specific operation: The server uses email sending software to send the email to the specified recipient.

[0389] In this way, a system is realized that analyzes the user's intention and performs appropriate operations.

[0390] (Application example 3)

[0391] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0392] Conventional systems require users to manually operate the system when obtaining information or sending emails, resulting in low convenience. It is also difficult to accurately analyze the user's intentions and perform appropriate operations. Furthermore, when using wearable devices such as smart glasses, the display of information and operation are not intuitive, which poses a challenge.

[0393] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[0394] In this invention, the server includes a chip implanted in the head or a tiny device attached to the outside of the face, a means for inputting and outputting auditory and visual information using electrical signals in the brain, monologue, bone conduction, sound waves, etc., a means for conversing with generative AI via a mobile line, acquiring necessary information, acquiring suggestions for the next action, playing music and arranging it to create new, optimal background music based on user preferences, talking to people, using a messaging app, sending emails, etc., all without touching a device, a means for analyzing user intent and sending emails or acquiring information, and a means for displaying information on smart glasses. This allows the user to analyze intent and perform appropriate operations without manual operation, enabling intuitive information display and operation using smart glasses.

[0395] A "headchip" is a small electronic device that is implanted in a user's head to detect electrical signals in the brain and communicate with external devices.

[0396] "External facial attachment" refers to a small device that is attached to the outside of the user's face and is used for inputting and outputting auditory and visual information.

[0397] "Electrical signals in the brain" refer to electrical activity that occurs in the user's brain, and by analyzing these signals it is possible to read the user's intentions and thoughts.

[0398] "Soliloquy" refers to the words a user speaks to themselves, and analyzing this is a means of understanding the user's intentions.

[0399] "Bone conduction" refers to the phenomenon in which sound waves are transmitted to the inner ear through the skull, and is a technology that utilizes this phenomenon to transmit audio information to the user.

[0400] "Sound waves" are vibrations that travel through the air and are a means of transmitting audio information to the user.

[0401] "Generative AI" is an artificial intelligence system that analyzes user intentions and performs appropriate operations.

[0402] A "mobile line" is a communication line used to connect to the Internet via a mobile phone or smart device.

[0403] A "conversation" is a voice or text communication between a user and a generative AI.

[0404] "Obtaining necessary information" refers to the act of obtaining the information the user desires from the Internet or a database.

[0405] "Next action suggestion" is a function in which generative AI suggests the next action to be taken based on the user's current situation and intentions.

[0406] "Music playback" is a function in which generative AI selects and plays the music the user wants to listen to.

[0407] "Tell us your favorite image, let us arrange it and play the best new background music" is a function in which generative AI arranges music based on the user's preferences and image, and plays the optimal background music.

[0408] "Talk to people" is a function that allows a user to make voice calls with other people.

[0409] "Messenger apps and email sending" refers to the function that allows users to send messages or emails using messenger apps.

[0410] "Means of analyzing user intent and sending emails or obtaining information" refers to a function in which generative AI analyzes the user's intent and sends emails or obtains information based on that intent.

[0411] "Means for displaying on smart glasses" refers to a function that uses smart glasses to visually display acquired information or generated content to a user.

[0412] The system for implementing this invention includes a chip implanted in the head or a tiny device attached to the outside of the face, generative AI, a mobile network, and smart glasses. Specific embodiments of each element are described below.

[0413] A chip implanted in the head or a tiny device attached to the outside of the face

[0414] A chip implanted in the user's head or a tiny device attached to the outside of the face detects electrical signals in the brain, monologue, bone conduction, sound waves, etc., and transmits this information to a generative AI, which can then analyze the user's intentions and thoughts in real time.

[0415] Generative AI

[0416] Generative AI analyzes the user's intent and performs the appropriate operation. For example, if the user thinks, "Send an email," the generative AI will grasp that intent and perform the operation to send an email. Similarly, if the user thinks, "Tell me the latest news," the generative AI will retrieve the latest news and display it on the smart glasses. Generative AI uses OpenAI's API to analyze the user's intent.

[0417] Mobile line

[0418] The mobile line is a means of communication with the generative AI via the internet, allowing it to obtain necessary information in real time and provide it to the user.

[0419] Smart Glasses

[0420] Smart glasses are devices that visually display information obtained and content generated by generative AI to users. For example, if a user thinks, "Tell me the latest news," the news obtained by generative AI will be displayed on the smart glasses.

[0421] Specific examples

[0422] If a user thinks, "Send an email," the generative AI will analyze the user's intent using the following prompt:

[0423] Example prompt sentence:

[0424] Analyze user intent and take appropriate action: Send an email

[0425] The generative AI inputs this prompt into the OpenAI API and analyzes the user's intent. It then sends an email using SMTP. If the user chooses "Tell me the latest news," the generative AI retrieves the latest news using the news API and displays it on the smart glasses.

[0426] In this way, the user can analyze their intentions and perform appropriate operations without having to perform manual operations, enabling intuitive information display and operation using smart glasses.

[0427] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[0428] Step 1:

[0429] The user has an intent.

[0430] Users have intentions such as "send an email" or "tell me the latest news." These intentions are expressed as electrical signals in the brain or as monologue.

[0431] Input: User's brain electrical signals, monologue

[0432] Output: User intent (e.g., send email, get news)

[0433] Step 2:

[0434] The device detects the user's intention.

[0435] A chip implanted in the head or a tiny device attached to the outside of the face detects electrical signals in the user's brain and their speech and converts them into digital signals.

[0436] Input: User's brain electrical signals, monologue

[0437] Output: Digital signal

[0438] Step 3:

[0439] The device sends a digital signal to the generative AI.

[0440] The device uses a mobile network to transmit the detected digital signal to the generative AI.

[0441] Input: Digital signal

[0442] Output: Sending data to generative AI

[0443] Step 4:

[0444] Generative AI analyzes user intent.

[0445] Generative AI uses the OpenAI API to analyze digital signals and understand user intent, such as "send an email."

[0446] Input: Digital signal

[0447] Output: User intent (e.g., send email, get news)

[0448] Specific operation: Enter a prompt into the OpenAI API and obtain the analysis results.

[0449] Step 5:

[0450] Generative AI determines the appropriate action.

[0451] Generative AI determines the appropriate operation based on the analyzed user intent, such as sending an email or retrieving news.

[0452] Input: User intent

[0453] Output: Operation instructions (e.g., send email, get news)

[0454] Step 6:

[0455] The generative AI executes the operation.

[0456] The generative AI executes the determined action, for example, sending an email using SMTP or getting the latest news using a news API.

[0457] Input: Operation instructions

[0458] Output: Operation result (e.g., email sent, news retrieved)

[0459] Specific operations: Connect to an SMTP server to send email, send a request to the news API to retrieve news.

[0460] Step 7:

[0461] The terminal displays the operation results on the smart glasses.

[0462] The device receives the operation results from the generative AI and displays them on the smart glasses. For example, it displays the acquired news on the smart glasses.

[0463] Input: Operation result

[0464] Output: Display on smart glasses

[0465] Specific operation: The operation results are displayed on the smart glasses display.

[0466] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0467] "Example 1"

[0468] The invention incorporating the emotion engine has the ability to recognize the user's emotions and generate and arrange music accordingly. Specifically, it analyzes the user's brain waves and sends the results to the emotion engine. The emotion engine infers the user's emotions from the brain wave analysis results and generates and arranges music according to those emotions. For example, when the user is feeling happy, the emotion engine generates and arranges bright and cheerful music. Conversely, when the user is sad, the emotion engine generates and arranges calm music.

[0469] "Example 2"

[0470] The emotion engine also suggests the next action based on the user's emotions. Specifically, it analyzes the user's brain waves and sends the results to the emotion engine. The emotion engine infers the user's emotions from the brain wave analysis results and suggests the next action based on those emotions. For example, if the user is feeling stressed, the emotion engine might suggest playing relaxation music or taking deep breaths.

[0471] "Example 3"

[0472] Furthermore, the emotion engine has the ability to adjust the content of messages according to the user's emotions. Specifically, when a user sends a message, the emotion engine considers the user's current emotions and suggests expressions that are appropriate for that emotion. For example, when a user is feeling happy, the emotion engine will suggest cheerful expressions. Conversely, when a user is sad, the emotion engine will suggest calm expressions.

[0473] The processing flow of each embodiment will be described below.

[0474] "Example 1"

[0475] Step 1: The user's brain waves are analyzed. This analysis is done by a chip implanted in the brain or a small device attached to the face.

[0476] Step 2: The analysis results are sent to the emotion engine via wireless communication.

[0477] Step 3: The emotion engine estimates the user's emotions from the results of EEG analysis.

[0478] Step 4: The emotion engine generates and arranges music according to the estimated emotions.

[0479] "Example 2"

[0480] Step 1: The user's brain waves are analyzed. This analysis is done by a chip implanted in the brain or a small device attached to the face.

[0481] Step 2: The analysis results are sent to the emotion engine via wireless communication.

[0482] Step 3: The emotion engine estimates the user's emotions from the results of EEG analysis.

[0483] Step 4: The emotion engine suggests the next action based on the estimated emotion.

[0484] "Example 3"

[0485] Step 1: When a user sends a message, the emotion engine takes into account the user's current emotion.

[0486] Step 2: The emotion engine suggests expressions that fit the emotion.

[0487] Example 1

[0488] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0489] Conventional voice recognition systems and EEG analysis systems have had difficulty accurately grasping a user's intentions and emotions and providing appropriate operations and services based on them. Furthermore, there are only a limited number of systems that allow users to operate them without touching anything, which makes them less convenient. Furthermore, there have been no systems with the ability to generate and arrange music based on the user's emotions. To solve these issues, a new system is needed.

[0490] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0491] In this invention, the server includes means for detecting electrical signals in the brain, monologue, bone conduction, sound waves, etc. using a chip implanted in the head or a tiny device attached to the outside of the face, means for analyzing brain waves using an electroencephalogram analyzer, means for transmitting the analysis results to a generative AI, means for the generative AI to infer the user's intentions based on the analysis results and perform appropriate operations, means for recognizing the user's emotions using an emotion engine and generating and arranging music according to those emotions, and means for conversing via a mobile line, obtaining necessary information, obtaining suggestions for the next action, playing music, expressing a favorite image and arranging it to play new, perfect background music, talking to people, using a messenger app, sending emails, etc., all without touching anything. This makes it possible to accurately grasp the user's intentions and emotions and provide appropriate operations and services based on them.

[0492] A "headchip" is a small electronic device that is implanted in a user's head to detect electrical signals in the brain.

[0493] The "external facial device" is a small device that is worn on the outside of the user's face to detect electrical signals and sound waves in the brain.

[0494] "Brain electrical signals" are signals that indicate electrical activity occurring in the user's brain.

[0495] "Soliloquy" is a voice that a user speaks to himself or herself.

[0496] "Bone conduction" is the phenomenon in which sound waves are transmitted to the inner ear through bones.

[0497] "Sound waves" are sound vibrations that travel through the air.

[0498] An "electroencephalogram (EEG) analysis device" is a device that analyzes electrical signals in the brain and infers a user's intentions and emotions.

[0499] "Generative AI" is an artificial intelligence system that infers the user's intentions based on analysis results and performs appropriate operations.

[0500] The "Emotion Engine" is a system that recognizes the user's emotions and generates and arranges music according to those emotions.

[0501] A "mobile line" is a communication line for sending and receiving data using wireless communication.

[0502] A "conversation" is a spoken communication between a user and a system.

[0503] "Acquisition of necessary information" means that the system collects and provides the information the user requests.

[0504] "Obtaining suggestions for the next action" means that the system suggests the next action to be taken based on the user's situation.

[0505] "Playing music" means that the system provides music to the user through a playback device.

[0506] "Personal image" refers to the image of specific music or video that the user likes.

[0507] "Arrangement" means changing existing music or video to suit the user's preferences.

[0508] "BGM" is an abbreviation for background music and refers to background music.

[0509] A "call" is a user communicating with another person by voice.

[0510] A "messenger app" is an application for sending and receiving text messages.

[0511] "Send mail" means to send an e-mail.

[0512] The present invention is a system that uses a chip implanted in the user's head or a tiny device attached to the outside of the face to detect electrical signals in the user's brain, monologue, bone conduction, sound waves, etc., and analyzes this information. Specific embodiments of this system are described below.

[0513] Hardware and software used

[0514] 1. Hardware

[0515] Chip or tiny device: A small device implanted in the user's head or worn on the outside of the face that detects electrical signals or sound waves in the brain.

[0516] Brain wave analysis device: This device analyzes detected brain waves and generates data to infer the user's intentions and emotions.

[0517] 2. Software

[0518] Generative AI: As an example, we will use OpenAI GPT-4. This AI infers the user's intentions based on data sent from an EEG analyzer and performs appropriate operations.

[0519] Emotion Engine: This engine estimates the user's emotions from the results of EEG analysis and generates and arranges music according to those emotions.

[0520] Data processing and calculation

[0521] 1. Detection of EEG and audio signals

[0522] The device detects electrical signals in the user's brain, monologue, bone conduction, sound waves, etc. in real time.

[0523] Specific actions include the user thinking "play music" or talking to themselves.

[0524] 2. Analysis of detection data

[0525] The data detected by the terminal is sent to an electroencephalogram analysis device.

[0526] The electroencephalogram analysis device analyzes the received data and extracts information to estimate the user's intentions and emotions.

[0527] As a specific operation, the electroencephalogram analysis device analyzes the electroencephalogram pattern and detects the intention of "play music."

[0528] 3. Sending analysis results

[0529] The EEG analysis device sends the analysis results to the generative AI.

[0530] The server receives the analysis results and proceeds to the next processing step.

[0531] Specifically, the analysis result is sent to the server as "music playback intent present."

[0532] 4. Intention estimation and operation execution

[0533] The server uses generative AI to infer the user's intentions based on the analysis results.

[0534] The server performs the appropriate action based on the inferred intent.

[0535] Specifically, the server starts a music playback application and plays music.

[0536] 5. Emotion Recognition and Music Generation

[0537] The server uses an emotion engine to estimate the user's emotions from the analysis results.

[0538] The server generates and arranges music according to the estimated emotions.

[0539] Specifically, if the user feels joy, the server generates, arranges, and plays cheerful, upbeat music.

[0540] Specific examples

[0541] 1. Users think "Play music"

[0542] The terminal detects brain waves and transmits them to an EEG analysis device.

[0543] The EEG analysis device determines that there is an intention to play music and sends this to the server.

[0544] The server uses generative AI to confirm the intent and launches the music playback application.

[0545] The server plays the music.

[0546] 2. Users are delighted

[0547] The terminal detects brain waves and transmits them to an EEG analysis device.

[0548] The EEG analysis device determines that "emotions of joy are present" and sends the result to the server.

[0549] The server uses an emotion engine to generate and arrange bright and cheerful music.

[0550] The server plays the generated music.

[0551] Prompt Sentence Examples

[0552] "If the user wants to play music, explain how you detect that intent and control the music playback."

[0553] "Please explain the process of generating and arranging music based on the user's emotions."

[0554] This system analyzes the user's brain waves and emotions in real time and performs appropriate operations and generates music based on that information, thereby improving the user experience.

[0555] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0556] Step 1:

[0557] The user wears a chip implanted in the head or a tiny device attached to the outside of the face.

[0558] Input: Electrical signals in the user's brain, self-talk, bone conduction, sound waves, etc.

[0559] Specific actions: The user thinks "play music" or talks to themselves.

[0560] Output: Detected brainwaves and audio signals.

[0561] Step 2:

[0562] The data detected by the terminal is sent to an electroencephalogram analysis device.

[0563] Input: Detected brainwaves and audio signals.

[0564] Specific operation: The device transmits data in real time to an EEG analysis device.

[0565] Output: Data sent to EEG analyzer.

[0566] Step 3:

[0567] The electroencephalogram analysis device analyzes the received data and extracts information to estimate the user's intentions and emotions.

[0568] Input: Brainwave and audio signals sent from the device.

[0569] Specific operation: The EEG analyzer analyzes brainwave patterns and detects the intention of "play music."

[0570] Output: Analysis results (information about user intent and emotions).

[0571] Step 4:

[0572] The EEG analysis device sends the analysis results to the generative AI.

[0573] Input: Analysis results (information about the user's intent and emotions).

[0574] Specific operation: The analysis results are sent to the generative AI.

[0575] Output: Analysis results sent to the generative AI.

[0576] Step 5:

[0577] The server uses generative AI to infer the user's intentions based on the analysis results.

[0578] Input: Analysis results sent to the generative AI.

[0579] Specific operation: The generative AI infers the user's intention based on the analysis results.

[0580] Output: Inferred user intent.

[0581] Step 6:

[0582] The server performs the appropriate action based on the inferred intent.

[0583] Input: Inferred user intent.

[0584] Specific operation: The server starts a music playback application and plays music.

[0585] Output: The operation performed (playing music).

[0586] Step 7:

[0587] The server uses an emotion engine to estimate the user's emotions from the analysis results.

[0588] Input: Analysis results sent to the generative AI.

[0589] Specific operation: The emotion engine estimates the user's emotions based on the analysis results.

[0590] Output: Estimated user sentiment.

[0591] Step 8:

[0592] The server generates and arranges music according to the estimated emotions.

[0593] Input: Estimated user sentiment.

[0594] Specific operation: The server uses the emotion engine to generate and arrange bright and cheerful music.

[0595] Output: Generated and arranged music.

[0596] Step 9:

[0597] The server plays the generated music.

[0598] Input: Generated and arranged music.

[0599] Specific operation: The server plays music through a music playback application.

[0600] Output: The played music.

[0601] (Application example 1)

[0602] Next, a description will be given of Application Example 1 of Embodiment Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0603] In traditional brick-and-mortar shopping experiences, it was difficult to grasp a user's emotions and intentions in real time and provide appropriate product suggestions and navigation. Furthermore, when a user was searching for a specific product or was in a specific emotional state, there was a lack of a way to suggest appropriate products or services. This limited the user's shopping experience and led to problems such as reduced satisfaction.

[0604] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0605] In this invention, the server includes a means for detecting electrical signals in the brain, monologue, bone conduction, sound waves, etc. using a chip implanted in the head or a tiny device attached to the outside of the face, a means for conversing with the generative AI via a mobile line, obtaining necessary information, obtaining suggestions for the next action, playing music and arranging it to convey your favorite images to play the best new background music, talking to people, using messenger apps, sending emails, etc., all without touching anything, and a means for analyzing the user's brainwaves and emotions to make appropriate product suggestions and in-store navigation in a physical store. This makes it possible to understand the user's emotions and intentions in real time and make product suggestions and navigation accordingly.

[0606] A "head chip" is a small electronic device that is implanted under the skin of the head to detect electrical signals in the brain.

[0607] The "external facial microdevice" is a small device worn on the outside of the face to detect electrical signals and sound waves in the brain.

[0608] "Electrical signals in the brain" are weak electrical fluctuations that occur in association with neural activity in the brain.

[0609] "Soliloquy" refers to words spoken by a user to themselves and is detected as audio.

[0610] "Bone conduction" is the phenomenon in which sound waves are transmitted to the inner ear through the skull.

[0611] "Sound waves" are sound vibrations that travel through the air.

[0612] "Generative AI" is artificial intelligence that analyzes the user's intentions and emotions and performs appropriate operations and suggestions based on that.

[0613] A "mobile line" is a communication line used by mobile communication devices such as mobile phones and smartphones.

[0614] A "conversation" is the act of exchanging information between a user and a system through voice or text.

[0615] "Obtaining necessary information" means that the system collects and provides the information the user requests.

[0616] "Getting next action suggestions" means that the system suggests the next action to be taken based on the user's current situation and intentions.

[0617] "Playing music" means that the system plays music to the user.

[0618] "Convey your favorite image, arrange it, and play the best new background music" means generating and arranging music based on the user's preferences and emotions, and providing the best background music.

[0619] "Talking to a person" means that a user communicates with another person by voice.

[0620] "Messenger apps and emailing" refers to users sending text messages and emails.

[0621] "Means for analyzing brain waves" refers to technology that analyzes electrical signals in the brain and infers the user's intentions and emotions.

[0622] "Means for generating and arranging music according to the user's preferences and circumstances" refers to technology that creates and adjusts music based on the user's individual preferences and current emotional state.

[0623] "Means for making appropriate product suggestions and in-store navigation within a physical store" refers to technology that analyzes the user's brain waves and emotions and, based on that, suggests the most suitable products within the physical store or provides guidance within the store.

[0624] The system for implementing this invention uses a chip implanted in the head or a tiny device attached to the outside of the face to detect electrical signals in the brain, monologue, bone conduction, sound waves, etc. This data is transmitted to a mobile device via wireless communication such as Bluetooth. The mobile device receives this data and transmits it to the generative AI.

[0625] Generative AI analyzes the received data and infers the user's intentions and emotions. Based on the analysis results, generative AI performs appropriate operations and makes suggestions to the user. For example, if a user thinks, "Play music," generative AI will grasp that intention and operate the music playback. It is also possible to generate and arrange music according to the user's emotions.

[0626] Furthermore, to improve the shopping experience in physical stores, the system analyzes users' brainwaves and emotions to provide appropriate product suggestions and in-store navigation. If a user is looking for a specific product or is in a certain emotional state, the system will suggest products and services that suit them.

[0627] Hardware and software used

[0628] Hardware:

[0629] Brainwave detection device (Bluetooth compatible)

[0630] Mobile devices (smartphones and tablets)

[0631] Displays in physical stores and smartphones

[0632] software:

[0633] Python

[0634] Keras (loading and predicting emotion estimation model)

[0635] Scikit-learn (data preprocessing)

[0636] Bluetooth library (acquiring EEG data)

[0637] Data processing and calculation

[0638] 1. Acquiring brainwave data: The brainwave detection device detects the user's brainwaves and transmits them to the mobile device via Bluetooth.

[0639] 2. Data preprocessing: Standardize the EEG data received by the mobile device.

[0640] 3. Emotion estimation: Using standardized data, generative AI estimates the user's emotions.

[0641] 4. Product Recommendation: Based on the estimated emotions, appropriate products are suggested in-store.

[0642] Specific examples

[0643] A user enters a physical store and wears a brainwave detection device. The device detects the user's brainwaves and transmits the data via Bluetooth to a mobile device. The mobile device analyzes the data and determines that the user is happy. Recommendations such as "brightly colored clothes" or "fun books" are then displayed on displays in the store and on smartphones.

[0644] Prompt Sentence Examples

[0645] A user enters a physical store and wears a brainwave detection device. The device detects the user's brainwaves and transmits the data via Bluetooth. The application analyzes the data and determines that the user is happy. In-store displays and smartphones display suggestions like "brightly colored clothes" and "fun books."

[0646] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0647] Step 1:

[0648] A user wears a brainwave detection device.

[0649] Input: User's brainwaves

[0650] Output: EEG data

[0651] How it works: The user wears an EEG detection device on their head, which detects electrical signals in the brain.

[0652] Step 2:

[0653] The brainwave detection device transmits the brainwave data to a mobile device via Bluetooth.

[0654] Input: EEG data

[0655] Output: EEG data sent to a mobile device

[0656] Specific operation: The brainwave detection device transmits the detected brainwave data to the mobile device via Bluetooth communication.

[0657] Step 3:

[0658] Standardize the EEG data received by the mobile device.

[0659] Input: EEG data sent to a mobile device

[0660] Output: Normalized EEG data

[0661] Specific operation: The EEG data received by the mobile device is standardized using Scikit-learn's StandardScaler.

[0662] Step 4:

[0663] The mobile device sends standardized brainwave data to the generative AI.

[0664] Input: Normalized EEG data

[0665] Output: EEG data sent to the generative AI

[0666] How it works: The mobile device sends standardized EEG data to the generative AI.

[0667] Step 5:

[0668] Generative AI analyzes brainwave data and infers the user's emotions.

[0669] Input: EEG data sent to the generative AI

[0670] Output: Estimated emotion data

[0671] Specific operation: The generative AI uses a Keras model to analyze EEG data and estimate the user's emotions.

[0672] Step 6:

[0673] Generative AI suggests appropriate products in physical stores based on the estimated emotions.

[0674] Input: Estimated emotion data

[0675] Output: Product proposal data

[0676] How it works: Based on the estimated emotions, the generative AI suggests appropriate products on displays in physical stores and on smartphones.

[0677] Step 7:

[0678] Product suggestions are displayed to users on displays in physical stores and on smartphones.

[0679] Input: Product proposal data

[0680] Output: Product suggestions displayed to the user

[0681] Specific operation: Product suggestion data received from generative AI is displayed to users on displays in physical stores or on smartphones.

[0682] Example 2

[0683] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0684] Conventional music generation systems and behavior suggestion systems have had difficulty providing optimal music and behaviors based on the user's emotions and circumstances. Furthermore, there were no systems that could analyze the user's brainwave data in real time and make appropriate suggestions based on the results. This prevented the system from improving the user's quality of life.

[0685] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0686] In this invention, the server includes means for collecting the user's brain wave data and transmitting it to the server, means for analyzing the brain wave data and transmitting it to the emotion engine, means for the emotion engine to estimate the user's emotion from the brain wave analysis results and suggest the next action, means for the generative AI model to generate and arrange music taking into account the user's past listening history and current situation, and means for transmitting the generated music to the user's terminal and playing it. This makes it possible to provide optimal music and actions in real time according to the user's emotion and situation.

[0687] A "head implant" is a device that is implanted in the user's head to collect electrical signals from the brain.

[0688] The "small device attached to the outside of the face" is a small device that is attached to the outside of the user's face and collects electrical signals and sounds from the brain.

[0689] "Brain electrical signals" are signals that indicate electrical activity generated by the user's brain.

[0690] "Voice" refers to words or sounds made by the user and transmitted as sound waves.

[0691] "Bone conduction" is the phenomenon in which sound waves are transmitted to the inner ear through bones.

[0692] "Sound waves" are vibrations that travel through the air and are perceived as sound.

[0693] A "generative AI model" is an artificial intelligence model that generates and arranges music according to the user's preferences and circumstances.

[0694] A "communication line" is a network infrastructure for sending and receiving data.

[0695] The "emotion engine" is a system that estimates a user's emotions from the results of brain wave analysis and suggests actions based on those emotions.

[0696] "User terminal" refers to a device that is directly operated by a user, including smartphones, tablets, PCs, etc.

[0697] "EEG data" refers to data on electrical signals collected from the user's brain.

[0698] "Analysis" is the process of processing collected data and extracting meaningful information.

[0699] "Action suggestions" are suggestions for the next action to be taken based on the user's emotions and situation.

[0700] "Listening history" is a record of music or audio that a user has listened to in the past.

[0701] "Playback" means outputting the generated music or audio so that the user can hear it.

[0702] This invention is a system that collects and analyzes a user's brain wave data, generates and arranges music based on the results, and suggests the next action. Specific embodiments for implementing this system are described below.

[0703] Hardware and software used

[0704] Hardware: EEG sensor, server, user device (smartphone, tablet, PC, etc.)

[0705] Software: Generative AI models, emotion engines, data analysis software

[0706] System configuration

[0707] 1. Collecting user's brainwave data

[0708] The user wears an EEG sensor, which collects electrical signals from the user's brain in real time.

[0709] The device receives the brainwave data collected from the brainwave sensor and transmits it to the server.

[0710] 2. Analysis of EEG data

[0711] The server uses data analysis software to analyze the received EEG data, identifying the user's EEG patterns and sending them to the emotion engine.

[0712] 3. Emotion estimation and action suggestion

[0713] The emotion engine estimates the user's emotions from the results of EEG analysis. For example, if the user is feeling stressed, the emotion engine will suggest playing relaxation music or taking deep breaths.

[0714] 4. Music Creation and Arrangement

[0715] The server uses a generative AI model to generate and arrange music, taking into account the user's past listening history and current situation (time, location, mood, etc.). The generated music is customized to the user's preferences.

[0716] 5. Sending and playing music to the user's device

[0717] The server sends the generated music to the user's device, which then plays the music, providing a relaxing environment for the user.

[0718] Specific examples

[0719] User Situation: User wants to relax after work.

[0720] Brainwave data collection: The user wears an EEG sensor and the device collects EEG data.

[0721] Sending EEG data to the server: The device sends the collected EEG data to the server.

[0722] Analysis of EEG data: The server analyzes the EEG data and sends it to the emotion engine.

[0723] Emotion estimation and action suggestion: The emotion engine estimates that the user is feeling stressed and suggests playing relaxation music.

[0724] Music generation and arrangement: A generative AI model generates relaxing music and sends it to the user's device.

[0725] Sending and playing music to the user's device: The device plays the generated music, providing a relaxing environment for the user.

[0726] Prompt Sentence Examples

[0727] "What kind of music would a generative AI model generate if a user wanted to relax?"

[0728] "Please explain the process by which the emotion engine analyzes the user's brainwave data and suggests the next action."

[0729] This system aims to improve the quality of life of users by providing them with the most appropriate music and behaviors in real time based on their emotions and circumstances.

[0730] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0731] Step 1:

[0732] Collecting user brainwave data

[0733] The user wears a brainwave sensor.

[0734] Input: User's EEG signal.

[0735] The device collects brainwave data in real time from an EEG sensor.

[0736] Output: Collected EEG data.

[0737] How it works: When the user feels like they want to relax, they wear the brainwave sensor and the device starts collecting data.

[0738] Step 2:

[0739] Sending EEG data to the server

[0740] The device transmits the collected brainwave data to a server.

[0741] Input: Collected EEG data.

[0742] Output: EEG data sent to the server.

[0743] Specific operation: The device uses Wi-Fi or mobile data to send brainwave data to the server.

[0744] Step 3:

[0745] Analysis of EEG data

[0746] The server uses data analysis software to analyze the received brainwave data.

[0747] Input: EEG data sent to the server.

[0748] Output: Analyzed EEG data.

[0749] How it works: The server analyzes the EEG data and identifies the user's EEG patterns.

[0750] Step 4:

[0751] Emotion estimation and action suggestions

[0752] The server sends the analysis results to the emotion engine.

[0753] Input: Analyzed EEG data.

[0754] The emotion engine estimates the user's emotions from the results of brain wave analysis.

[0755] Output: Estimated user sentiment and action suggestions.

[0756] Specific behavior: The emotion engine infers that the user is feeling stressed and suggests playing relaxation music and taking deep breaths.

[0757] Step 5:

[0758] Music Creation and Arrangement

[0759] The server uses a generative AI model to generate and arrange music, taking into account the user's past listening history and current situation (time, location, mood, etc.).

[0760] Input: User's past listening history, current situation, and estimated emotion.

[0761] Output: Generated and arranged music.

[0762] What it does: A generative AI model generates relaxing music and arranges it to suit the user's preferences.

[0763] Step 6:

[0764] Sending and playing music to user devices

[0765] The server transmits the generated music to the user terminal.

[0766] Input: Generated and arranged music.

[0767] Output: Music sent to the user device.

[0768] The device plays the received music.

[0769] Specific behavior: The device plays music, providing a relaxing environment for the user.

[0770] (Application example 2)

[0771] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0772] Conventional music generation systems and emotion analysis systems can generate and arrange music according to the user's preferences and circumstances, but it has been difficult to generate and arrange music in real time according to the emotions and preferences of customers in physical stores. Furthermore, there has been no system that can analyze a customer's brain waves and suggest optimal music based on the results. This has led to the issue of not being able to improve the customer experience in physical stores.

[0773] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0774] In this invention, the server includes a chip implanted in the head or a tiny device attached to the outside of the face, a means for inputting and outputting audio and visual information using electrical signals in the brain, monologue, bone conduction, sound waves, etc., a means for conversing with a generative AI via a mobile line, obtaining necessary information, obtaining suggestions for the next action, playing music and arranging it to convey personal images and create new, optimal background music, talking to people, messaging apps, sending emails, etc., all without touching a single device, a means for generating and arranging music in real time according to the emotions and preferences of customers in physical stores, and a means for analyzing customers' brainwaves and having an emotion engine suggest optimal music based on the results. This makes it possible to generate and arrange music according to customers' emotions and preferences in physical stores, improving the customer experience.

[0775] A "head chip" is a small electronic device that is implanted in the user's head to capture electrical signals in the brain.

[0776] The "extrinsic equipment attached to the outside of the face" is a small device that is attached to the outside of the user's face and is used to acquire electrical signals and sound waves in the brain.

[0777] "Electrical signals in the brain" are electrical signals based on brain waves and neural activity, and are data used to analyze the user's emotions and state.

[0778] "Soliloquy" is what a user says to themselves and is captured in the system as voice input.

[0779] "Bone conduction" is the phenomenon in which sound waves travel through bones to the inner ear, and is a means of transmitting auditory information.

[0780] "Sound waves" are vibrations that travel through the air and are used as auditory information.

[0781] "Auditory and visual input / output means" refers to devices and technologies that provide audio and video to users and obtain audio and visual information from users.

[0782] "Generative AI" is artificial intelligence that generates and arranges music and other content based on the user's preferences and circumstances.

[0783] A "mobile line" is a communication line used to connect to the Internet via a mobile phone or smartphone.

[0784] "Conversation" is the act of exchanging information between a user and a system through voice.

[0785] "Obtaining necessary information" refers to the act of the system collecting and providing the information requested by the user.

[0786] "Getting next action suggestions" means that the system suggests the next action to be taken based on the user's current state and emotions.

[0787] "Playing music" refers to the system playing music for the user.

[0788] "Communicate your preferred image, arrange it, and play the best new background music" means that the user can communicate their preferred musical style and atmosphere to the system and play music generated based on that.

[0789] "Talking to a person" means communicating with another person by voice via the system.

[0790] "Messenger apps and emailing" refers to the act of using applications to send text messages or emails.

[0791] A "physical store" is a commercial establishment that exists in a physical location and that customers can visit in person.

[0792] "A means for generating and arranging in-store music in real time according to customer emotions and preferences" is a technology that generates and arranges music played in-store on the spot based on customer emotions and preferences.

[0793] "A means of analyzing the customer's brain waves and having the emotion engine suggest the most suitable music based on the results" is a technology that analyzes the customer's brain wave data and has the emotion engine select and suggest the most suitable music based on the results.

[0794] The system for implementing this invention has the following configuration: First, a chip is implanted in the user's head or a tiny device is attached to the outside of the face. This allows electrical signals in the brain, monologue, bone conduction, sound waves, etc. to be acquired and input / output to the auditory and visual senses.

[0795] Next, via generative AI and mobile network, it will converse with the user, obtain necessary information, and suggest the next course of action. It can also play music, convey images based on the user's preferences, and arrange them to play the best new background music. It can also talk to people, use messaging apps, and send emails.

[0796] In physical stores, it provides a means to generate and arrange music in real time according to the customer's emotions and preferences. The system analyzes the customer's brainwaves and uses the emotion engine to suggest the most suitable music. This system can improve the customer experience in physical stores.

[0797] Hardware and software used

[0798] Hardware: EEG sensor, smartphone

[0799] Software: Python, Keras (loading and predicting emotion estimation model), music generation module, emotion engine module

[0800] Data processing and calculation

[0801] The server acquires EEG data in real time from the EEG sensor and standardizes the data. The standardized data is then used to predict the user's emotions using an emotion estimation model. The emotion engine then generates optimal music based on the predicted emotions and plays it in the store.

[0802] Specific examples

[0803] For example, if a customer in a physical store feels like they want to relax, the brainwave sensor will detect that emotion and generate relaxing music to play in the store. If the customer is feeling stressed, the emotion engine will suggest deep breathing and play relaxation music.

[0804] Prompt Sentence Examples

[0805] Get the user's brainwave data, estimate their emotions, and generate music according to those emotions. For example, if the user wants to relax, generate relaxing music.

[0806] In this way, music can be generated and arranged according to the user's emotions and preferences, improving the customer experience in physical stores.

[0807] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0808] Step 1:

[0809] Acquire the user's brainwave data. The brainwave sensor is used to collect the user's brainwave data in real time. The input is raw data from the brainwave sensor, and the output is an array of brainwave data.

[0810] Step 2:

[0811] Preprocessing of EEG data is performed. The server normalizes the acquired EEG data to standardize it. The input is the array of EEG data obtained in step 1, and the output is the standardized EEG data.

[0812] Step 3:

[0813] Estimate emotions. The server inputs the standardized EEG data into an emotion estimation model (using Keras) to predict the user's emotions. The input is the standardized EEG data obtained in step 2, and the output is the predicted emotion label.

[0814] Step 4:

[0815] Generate music. The server uses an emotion engine to generate optimal music based on the predicted emotion. The input is the emotion label obtained in step 3, and the output is the generated music data.

[0816] Step 5:

[0817] Play the music. The terminal receives the generated music data and plays it in the physical store. The input is the music data obtained in step 4, and the output is the music played in the store.

[0818] Step 6:

[0819] Suggest next actions: The server uses an emotion engine to suggest next actions based on the user's current emotion. The input is the emotion label obtained in step 3, and the output is a list of suggested actions.

[0820] Step 7:

[0821] Notify the user of the proposed actions. The terminal notifies the user of the proposed actions received from the server. The input is the list of actions obtained in step 6, and the output is the notification to the user.

[0822] In this way, a series of processes is realized: acquiring the user's brain wave data, estimating their emotions, generating and playing music according to those emotions, and suggesting the next action.

[0823] Example 3

[0824] Next, a description will be given of a third embodiment of the third embodiment. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0825] Conventional systems require manual input and operation when users want to perform intended operations or send messages, which is inefficient. Furthermore, they are unable to adjust messages based on the user's emotions, which reduces the quality of communication. Furthermore, there is a lack of technology that can accurately analyze the user's intentions and emotions and adjust appropriate operations and messages based on them.

[0826] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[0827] In this invention, the server includes means for analyzing the user's intentions and performing appropriate operations, means for analyzing the user's emotions and adjusting the content of messages, and means for conversing with the generative AI via a mobile line, acquiring necessary information, obtaining suggestions for the next action, playing music and arranging it to convey the user's favorite image to play the best new background music, talking to people, using a messenger app, sending emails, etc., all without touching anything. This allows the user to perform intended operations and send messages without manual input or operation, and further enables messages to be adjusted according to the user's emotions.

[0828] A "headchip" is a small electronic device that is implanted in the user's head to detect and analyze electrical signals in the brain.

[0829] The "extrinsic facial device" is a small device that is worn on the outside of the user's face and uses sound waves and bone conduction to input and output information to the senses of hearing and vision.

[0830] "Brain electrical signals" are signals that indicate electrical activity occurring in the user's brain, and by analyzing these signals, data can be used to understand the user's intentions and emotions.

[0831] "Soliloquy" refers to the words a user speaks to themselves, and by analyzing this data it is possible to understand the user's intentions and emotions.

[0832] "Bone conduction" is a phenomenon in which sound waves are transmitted to the inner ear through the skull, and is a technology that utilizes this phenomenon to transmit auditory information.

[0833] "Sound waves" are vibrations that travel through the air and are a means of transmitting auditory information.

[0834] "Generative AI" is an artificial intelligence technology that analyzes a user's intentions and emotions and generates appropriate operations and messages.

[0835] A "mobile line" is a communication line for sending and receiving data using wireless communication.

[0836] An "emotion engine" is a technology that analyzes a user's emotions and adjusts the content of messages according to those emotions.

[0837] "Means for analyzing intent" refers to technology that analyzes user input and electrical signals in the brain to understand the user's intent.

[0838] "Means for performing operations" refers to the technology for executing specific operations based on the analyzed user's intentions.

[0839] The "means for adjusting the content of a message" is a technique for appropriately changing the content of a message based on the analyzed user's emotions.

[0840] MODE FOR CARRYING OUT THE INVENTION

[0841] The present invention is a system that analyzes the user's intentions and emotions and adjusts appropriate operations and messages. Specific embodiments of this system will be described below.

[0842] 1. System Configuration

[0843] The system consists of the following main components:

[0844] A chip implanted in the head or a tiny device attached to the outside of the face

[0845] A means of inputting / outputting to the auditory and visual senses using electrical signals in the brain, talking to oneself, bone conduction, sound waves, etc.

[0846] Generative AI

[0847] Mobile line

[0848] Emotion Engine

[0849] 2. Hardware and Software Use

[0850] The server analyzes the user's intent using a generative AI model (e.g., a natural language processing model). Specifically, OpenAI's GPT-4 is used. The emotion engine uses a sentiment analysis tool such as IBM's Watson (registered trademark) Tone Analyzer.

[0851] The device retrieves information, makes calls, uses messaging apps, sends emails, etc. via a mobile network. Based on the user's instructions, the device receives instructions from the generative AI and performs specific operations.

[0852] 3. Program Processing

[0853] The server analyzes the user's intent and takes appropriate action. For example, if a user wants to "send an email to a friend," the generative AI will grasp that intent and perform the operation to send the email. The emotion engine analyzes the user's emotions and adjusts the content of the message. For example, if a user says, "I had a lot of fun today," the emotion engine will determine that the user is feeling happy and suggest a more cheerful expression.

[0854] 4. Specific Examples

[0855] Consider the case where a user says, "I want to send an email to my friend and tell him I had a great day."

[0856] 1. The user uses the voice input function on their smartphone and says, "I want to send an email to my friend. I want to tell her I had a great day."

[0857] 2. The server converts the voice data into text and uses a generative AI model (GPT-4) to analyze the intent to "send an email."

[0858] 3. The server creates a template for sending email and sets the initial settings for the recipient, subject, and body of the email.

[0859] 4. The server uses the emotion engine to analyze that the user is feeling happy because he said, "I had a lot of fun today."

[0860] 5. Based on the results of the emotion engine, the server suggests a positive expression such as, "I had a great time today! See you again!"

[0861] 6. The device sends an email to the friend via mobile using the message received from the server. The user confirms that the email has been sent.

[0862] Prompt Sentence Examples

[0863] "Send a text to your friends. Tell them I had a great time today."

[0864] "Send an email to my boss. I want to update him on the progress of the project."

[0865] In this way, the system allows the server, terminal, and user to work together to adjust appropriate operations and messages according to the user's intentions and emotions. The flow of the identification process in the third embodiment will be described with reference to FIG.

[0866] Step 1:

[0867] The user inputs an intent through the device, for example using voice input or text input. The user says, "I want to send an email to a friend."

[0868] Input: User voice or text input

[0869] Output: Audio data or text data

[0870] Step 2:

[0871] The server converts the voice data into text using speech recognition software.

[0872] Input: Audio data

[0873] Output: Text data

[0874] Step 3:

[0875] The server uses a generative AI model (e.g., a natural language processing model) to analyze the user's intent. The server analyzes the text data and understands the intent to "send an email."

[0876] Input: Text data

[0877] Output: Analysis result (user intent)

[0878] Step 4:

[0879] The server determines the specific operation based on the analysis results. The server generates the necessary information for sending the email (recipient, subject, body, etc.). For example, the server creates a "template for sending an email to a friend."

[0880] Input: Analysis result (user intent)

[0881] Output: Email template

[0882] Step 5:

[0883] The server uses an emotion engine to analyze the user's emotions. The server infers the user's current emotions from the user's input and past data. For example, if the user says, "I had a lot of fun today," the server determines that the user is feeling happy.

[0884] Input: User input data, historical data

[0885] Output: Emotion analysis results

[0886] Step 6:

[0887] The server adjusts the content of the message based on the analysis results of the emotion engine. The server suggests expressions that match the user's emotions. For example, if the user is feeling happy, the server will suggest a cheerful expression such as "I had a great time today! See you again!"

[0888] Input: Sentiment analysis results, email template

[0889] Output: Adjusted message

[0890] Step 7:

[0891] The device actually sends the email using the adjusted message received from the server. The device sends the email via a mobile network. The user confirms that the email has been sent to their friend.

[0892] Input: Adjusted message

[0893] Output: Email sent

[0894] In this way, the system works in cooperation with the server, terminal, and user to adjust appropriate operations and messages according to the user's intentions and emotions.

[0895] (Application example 3)

[0896] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0897] Conventional food delivery systems require users to manually place orders, which is inconvenient, especially when they are busy or busy. They also fail to provide appropriate messages based on the user's emotions, which can lead to a poor user experience. There is a need to solve these problems and provide a more intuitive and emotionally sensitive food delivery experience.

[0898] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means. In this invention, the server includes a chip implanted in the head or a tiny device attached to the outside of the face; means for inputting / outputting to the auditory and visual senses using electrical signals in the brain, monologue, bone conduction, sound waves, etc.; means for conversing with a generative AI via a mobile line, obtaining necessary information, obtaining suggestions for the next action, playing music and arranging it to create new, optimal background music based on user preferences, talking to people, and sending emails without touching a device; means for analyzing the user's intentions and performing ordering operations; and means for analyzing the user's emotions and adjusting the content of messages. This allows users to intuitively order food delivery without using their hands and receive appropriate messages based on their emotions.

[0899] A "headchip" is a small electronic device that is implanted in the user's head to detect electrical signals in the brain and communicate with external devices.

[0900] The "external facial device" is a small electronic device worn on the outside of the user's face that detects electrical signals and sound waves in the brain and communicates with external devices.

[0901] "Brain electrical signals" are signals that indicate electrical activity generated by the user's brain.

[0902] "Soliloquy" refers to words that a user speaks to themselves.

[0903] "Bone conduction" is the phenomenon in which sound waves are transmitted to the inner ear through bones.

[0904] "Sound waves" are sound vibrations that travel through the air.

[0905] "Auditory and visual input / output" refers to the means by which information is provided to the user's auditory and visual senses, or by which information is received from the user.

[0906] "Generative AI" is artificial intelligence that analyzes a user's intentions and emotions and generates appropriate operations and messages.

[0907] A "mobile line" is a communication line used by mobile communication devices such as mobile phones and smartphones.

[0908] "Conversation" is the act of exchanging information between a user and a system through voice.

[0909] "Obtaining necessary information" refers to the act of the system collecting and providing the information requested by the user.

[0910] "Getting next action suggestions" is the act of the system suggesting the next action to be taken based on the user's current situation and intentions.

[0911] "Playing music" refers to the system playing music for the user.

[0912] "Background music" is music that plays in accordance with the user's activities and situations.

[0913] A "call" is an act of a user having a voice conversation with another person.

[0914] A "messenger app" is an application for sending and receiving text messages, images, and voice messages.

[0915] "Sending email" refers to the act of sending an email.

[0916] The "means for analyzing intent and performing ordering operations" refers to a means for analyzing a user's intent and placing a food delivery order based on that intent.

[0917] The "means for analyzing emotions and adjusting the content of a message" is a means for analyzing the emotions of a user and adjusting the content of a message in accordance with those emotions.

[0918] A system for implementing the present invention has the following configuration.

[0919] Hardware Configuration

[0920] 1. A chip implanted in the head or a tiny device attached to the outside of the face:

[0921] It is a small electronic device that is implanted in the user's head or worn externally on the face, which detects electrical signals and sound waves in the brain and communicates with an external device.

[0922] 2. Smartphone:

[0923] This is a device that uses mobile lines to communicate with generative AI.

[0924] Software Configuration

[0925] 1. Generative AI:

[0926] It is an artificial intelligence that analyzes the user's intentions and emotions and generates appropriate operations and messages. Specifically, it has the functions of voice recognition, intention analysis, and emotion analysis.

[0927] 2. Emotion Engine:

[0928] This software analyzes the user's emotions and adjusts the content of the message accordingly.

[0929] Data processing and calculation

[0930] 1. Speech Recognition:

[0931] It uses the smartphone's microphone to convert the user's speech into text, specifically using the SpeechRecognition library.

[0932] 2. Intent Analysis:

[0933] Generative AI analyzes what users say and understands their intent, specifically using the Transformers library.

[0934] 3. Emotion analysis:

[0935] Analyze the sentiment of a user's speech using the TextBlob library.

[0936] Specific examples

[0937] If a user says, "I want to order a pizza," the following process occurs:

[0938] 1. Speech Recognition:

[0939] It uses the smartphone's microphone to convert the user's speech into text.

[0940] 2. Intent Analysis:

[0941] The generative AI analyzes the intention of "I want to order pizza" and performs the pizza ordering operation.

[0942] 3. Emotion analysis:

[0943] The emotion engine analyzes the user's emotions and if the user is happy, it generates a message saying, "Your delicious pizza will be arriving soon!"

[0944] Prompt Sentence Examples

[0945] If a user says, "I want to order a pizza," the generative AI analyzes their intent and performs the pizza ordering operation. The emotion engine analyzes the user's emotions, and if the user is happy, it suggests the message, "Your delicious pizza will be delivered soon!"

[0946] In this way, users can intuitively order food delivery hands-free and receive emotionally appropriate messages.

[0947] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[0948] Step 1:

[0949] A user says, "I want to order a pizza." The input is the user's voice, and the output is voice data. Specifically, the smartphone's microphone captures the user's voice.

[0950] Step 2:

[0951] The device performs speech recognition. The input is the speech data obtained in step 1, and the output is text data. Specifically, the device converts the speech data into text using the SpeechRecognition library.

[0952] Step 3:

[0953] The server uses generative AI to perform intent analysis. The input is the text data obtained in step 2, and the output is data indicating the user's intent. Specifically, it uses the Transformers library to analyze the text data and extracts the intent, "I want to order pizza."

[0954] Step 4:

[0955] The server performs the ordering operation based on the user's intent. The input is the user's intent data obtained in step 3, and the output is data indicating that the order has been completed. Specifically, the server calls the API of a food delivery service and places an order for pizza.

[0956] Step 5:

[0957] The server performs emotion analysis using an emotion engine. The input is the text data obtained in step 2, and the output is data indicating the user's emotion. Specifically, the TextBlob library is used to analyze the text data and determine the user's emotion.

[0958] Step 6:

[0959] The server generates a message according to the emotion. The input is the user's emotion data obtained in step 5, and the output is a message according to the emotion. Specifically, if the user is happy, the server generates a message saying, "Your delicious pizza will be delivered soon!"

[0960] Step 7:

[0961] The device notifies the user of the generated message. The input is the message obtained in step 6, and the output is the notification displayed to the user. Specifically, the message is displayed using the smartphone's notification function.

[0962] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0963] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0964] Another example of generative AI is Gemini (registered trademark) (Internet search engine). <url: https: gemini.google.com ?hl="ja">) are mentioned.

[0965] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0966] [Second embodiment]

[0967] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0968] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0969] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0970] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0971] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0972] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0973] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0974] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0975] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0976] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0977] In the smart glasses 214, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0978] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0979] "Example 1"

[0980] One embodiment of the present invention uses a chip implanted in the head or a tiny device attached to the outside of the face. This chip or tiny device detects electrical signals in the brain, monologue, bone conduction, sound waves, etc., and analyzes this information. Specifically, an EEG analyzer analyzes brain waves and sends the results to a generative AI. The generative AI infers the user's intention based on the analysis results and performs appropriate operations. For example, if the user thinks, "Play music," the EEG analyzer captures that intention, and the generative AI controls the music playback.

[0981] "Example 2"

[0982] Furthermore, generative AI generates and arranges music according to the user's preferences and circumstances. Specifically, it generates and arranges music taking into account the user's past listening history and current situation (e.g., time, location, mood, etc.). For example, if a user wants to relax, generative AI will capture that intention and generate and arrange relaxing music.

[0983] "Example 3"

[0984] Furthermore, the system of the present invention retrieves information, makes calls, uses messenger apps, and sends emails via mobile networks. Specifically, the generative AI analyzes the user's intention and performs appropriate operations. For example, if the user thinks, "Send an email," the generative AI will grasp that intention and perform the operation to send an email.

[0985] The processing flow of each embodiment will be described below.

[0986] "Example 1"

[0987] Step 1: A chip is implanted in the user's head or a tiny device is attached to the outside of the face.

[0988] Step 2: A chip or tiny device detects electrical signals in the brain, speech, bone conduction, sound waves, etc.

[0989] Step 3: The detected information is sent to an EEG analysis device, which analyzes the EEG.

[0990] Step 4: Send the analysis results to the generative AI.

[0991] Step 5: The generative AI infers the user's intention based on the analysis results and performs the appropriate operation. For example, if the user thinks, "Play music," the generative AI will perform the operation to play music.

[0992] "Example 2"

[0993] Step 1: The generative AI takes into account the user's past listening history and current situation (e.g., time, location, mood, etc.).

[0994] Step 2: The generative AI generates and arranges music based on the information it has taken into account. For example, if the user wants to relax, the generative AI will capture that intention and generate and arrange relaxing music.

[0995] "Example 3"

[0996] Step 1: Generative AI analyzes user intent.

[0997] Step 2: The generative AI performs the appropriate action based on the analysis results. For example, if the user thinks, "Send an email," the generative AI will capture that intent and perform the action of sending an email.

[0998] Example 1

[0999] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1000] Conventional technology requires users to manually input information to operate devices, which is difficult to do, especially when users have physical limitations or their hands are full. It is also difficult to accurately read the user's intentions and perform appropriate operations. This reduces user convenience and makes it difficult to achieve efficient operation.

[1001] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1002] In this invention, the server includes means for detecting electrical signals in the brain, monologue, bone conduction, sound waves, etc. using a chip implanted in the head or a tiny device attached to the outside of the face, an electroencephalogram (EEG) analyzer for analyzing the detected electrical signals, means for transmitting the analysis results to a generative AI, means for the generative AI to infer the user's intentions based on the analysis results and perform appropriate operations, and means for conversing via a mobile line, obtaining necessary information, obtaining suggestions for the next action, playing music and arranging it to convey a user's favorite image to play new, perfect background music, talking to people, using a messenger app, sending emails, etc., all without touching anything. This enables the user to accurately read intentions and perform appropriate operations without manually operating the device.

[1003] A "headchip" is a small electronic device that is implanted in a user's head to detect electrical signals in the brain.

[1004] The "extrinsic equipment attached to the outside of the face" is a small device that is attached to the outside of the user's face and detects electrical signals in the brain, monologue, bone conduction, sound waves, etc.

[1005] "Brain electrical signals" are signals that indicate electrical activity occurring in the user's brain.

[1006] "Soliloquy" refers to words that a user speaks to himself or herself and is detected as voice.

[1007] "Bone conduction" is the phenomenon in which sound waves travel through bones to the inner ear, thereby transmitting audio information.

[1008] "Sound waves" are vibrations that travel through the air and are perceived as sound.

[1009] An "EEG analysis device" is a device that analyzes electrical signals in the brain to read the user's intentions and thoughts.

[1010] "Generative AI" is an artificial intelligence system that infers the user's intentions based on analysis results and performs appropriate operations.

[1011] A "mobile line" is a communication line for sending and receiving data using wireless communication.

[1012] A "conversation" is a spoken communication between a user and another person or system.

[1013] "Acquisition of required information" is the process of searching for and acquiring the information the user needs.

[1014] "Getting next action suggestions" is the process of suggesting the next action to be taken based on the user's current situation and intentions.

[1015] "Playing music" is the process by which a user selects and plays the music they want to listen to.

[1016] "Personal image" refers to a visual or auditory image that the user likes.

[1017] "Arrangement" is the process of modifying existing music or images to suit the user's preferences.

[1018] "BGM" is an abbreviation for background music, and refers to music played as background music.

[1019] A "call" is a process in which a user communicates with another person by voice.

[1020] A "messenger app" is application software for sending and receiving text messages.

[1021] "Mail sending" is the process of creating and sending an email.

[1022] This invention is a system that uses a chip implanted in the user's head or a tiny device attached to the outside of the face to detect electrical signals in the brain, monologue, bone conduction, sound waves, etc., and analyzes this information. Specifically, the following hardware and software are used:

[1023] Hardware and software used

[1024] Hardware: Chips implanted in the head, tiny devices attached to the outside of the face, brain wave analyzers

[1025] Software: Generative AI

[1026] Data processing and calculation

[1027] 1. The user wears a chip implanted in the head or a tiny device attached to the outside of the face, which is then ready to detect electrical signals in the brain, speech, bone conduction, sound waves, etc.

[1028] 2. A chip or tiny device detects electrical signals in the user's brain in real time. For example, if a user thinks, "Play music," the corresponding electrical signals will be detected.

[1029] 3. The device transmits the detected electrical signal to the EEG analyzer, which then transmits this data via wireless communication or other means.

[1030] 4. The EEG analyzer analyzes the received electrical signals using sophisticated algorithms and pattern recognition to interpret the user's thoughts and intentions.

[1031] 5. The EEG analyzer sends the analysis results to the generative AI, which includes information about the user's intentions.

[1032] 6. The generative AI infers the user's intention based on the analysis results received from the EEG analyzer. For example, the intention may be "Play music."

[1033] 7. The generative AI performs the appropriate action based on the inferred intent. Specifically, it launches a music playback application and plays music.

[1034] Specific examples

[1035] For example, when a user wants to "play music," the specific operation is as follows.

[1036] 1. The user puts on the chip and thinks, "Play some music."

[1037] 2. The chip detects electrical signals corresponding to the thought.

[1038] 3. The device transmits the detected electrical signal to the EEG analysis device.

[1039] 4. The EEG analyzer analyzes the electrical signals and reads the intention to "play music."

[1040] 5. The EEG analysis device sends the analysis results to the generative AI.

[1041] 6. The generative AI infers the intent, "Play music."

[1042] 7. The generative AI launches a music playback application and plays music.

[1043] Prompt Sentence Examples

[1044] An example of a prompt sentence to input to the generative AI model is as follows:

[1045] When a user thinks, "Play music," the brainwave analysis device captures that intention, and the generative AI controls the music playback.

[1046] In this way, a system is realized that can read the user's intentions and perform appropriate operations.

[1047] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1048] Step 1:

[1049] The user wears a chip that is implanted in the head or a tiny device that is attached to the outside of the face.

[1050] Input: User wearing behavior

[1051] Output: With chip or tiny device attached

[1052] How it works: The user implants a chip in their head or wears a tiny device on the outside of their face, which is then ready to detect electrical signals in the brain, speech, bone conduction, sound waves, etc.

[1053] Step 2:

[1054] A chip or tiny device detects electrical signals in the brain.

[1055] Input: Electrical signals in the user's brain

[1056] Output: Detected electrical signal data

[1057] What it does: When a user thinks "play music," an electrical signal corresponding to that thought is detected by a chip or tiny piece of equipment.

[1058] Step 3:

[1059] The terminal transmits the detected electrical signal to an electroencephalogram analysis device.

[1060] Input: Detected electrical signal data

[1061] Output: Electrical signal data sent to EEG analyzer

[1062] Specific operation: The terminal receives the electrical signals detected by the chip or tiny device and transmits them to the EEG analysis device using wireless communication or other means.

[1063] Step 4:

[1064] An electroencephalogram (EEG) analyzer analyzes the electrical signals.

[1065] Input: Transmitted electrical signal data

[1066] Output: Parsed intent data

[1067] Specific operation: The EEG analyzer analyzes the received electrical signals and performs pattern recognition to read the user's thoughts and intentions. For example, the intention "play music" is analyzed.

[1068] Step 5:

[1069] The EEG analysis device sends the analysis results to the generative AI.

[1070] Input: Parsed intent data

[1071] Output: Intent data sent to the generative AI

[1072] How it works: The EEG analyzer sends the analysis results to the generative AI, which includes information about the user's intentions.

[1073] Step 6:

[1074] Generative AI infers the user's intentions based on the analysis results.

[1075] Input: Submitted intent data

[1076] Output: Inferred intent

[1077] Specific operation: The generative AI infers the user's intention based on the analysis results received from the EEG analysis device. For example, the intention may be "Play music."

[1078] Step 7:

[1079] The generative AI performs appropriate operations based on the inferred intent.

[1080] Input: Inferred intent

[1081] Output: The operation performed

[1082] Specific behavior: Based on the inferred intent, the generative AI launches a music playback application and performs the operation of playing music.

[1083] (Application example 1)

[1084] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1085] Conventional autonomous vehicle operation requires the user to manually set the destination, making operation complicated. Although interfaces using brainwaves and voice exist, technology that accurately estimates the user's intentions and reflects them in the operation of an autonomous vehicle is still in its infancy. This has resulted in low user convenience and made intuitive operation difficult.

[1086] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1087] In this invention, the server includes means for detecting electrical signals in the brain, monologue, bone conduction, sound waves, etc. using a chip implanted in the head or a tiny device attached to the outside of the face, means for conversing with the generative AI via a communication line, obtaining necessary information, obtaining suggestions for the next action, playing music and arranging it to convey the user's favorite image to play the best new background music, talking to people, messaging apps, sending emails, etc., all without touching anything, and means for estimating the user's intentions and operating the self-driving vehicle, which allows the user to intuitively operate the self-driving vehicle.

[1088] A "headchip" is a small electronic device that is implanted in the user's head to detect electrical signals in the brain.

[1089] The "external facial equipment" is a small device that is worn on the outside of the user's face to detect electrical signals and sound waves in the brain.

[1090] "Electrical signals in the brain" are weak electrical fluctuations that occur when the brain is active, and by analyzing these, it is possible to infer the user's intentions and state.

[1091] "Muttering to oneself" refers to words that a user utters unconsciously in a low voice, and by detecting this, it is possible to understand the user's intentions.

[1092] "Bone conduction" is a phenomenon in which sound waves are transmitted to the inner ear through the skull, and is a technology that utilizes this phenomenon to obtain audio information.

[1093] "Sound waves" are vibrations that travel through the air, and by detecting them, audio information can be obtained.

[1094] "Generative AI" is an artificial intelligence system that generates appropriate responses and operations based on the user's intentions and requests.

[1095] "Communication lines" refer to network infrastructure for sending and receiving data, and include mobile lines and internet lines.

[1096] An "autonomous vehicle" is a vehicle that uses artificial intelligence and sensor technology to drive autonomously.

[1097] "Estimating user intent" means analyzing data such as brain waves and voice to determine what the user wants.

[1098] "Performing an operation" means controlling a device or system based on the user's intentions.

[1099] The following system configuration is used as an embodiment of this invention: The system is composed of a chip to be implanted in the head or a tiny device to be attached to the outside of the face, an EEG analyzer, a generative AI, a communication line, and a control system for an autonomous vehicle.

[1100] First, the user wears a chip implanted in the head or a tiny device attached to the outside of the face, which detects electrical signals in the brain, speech, bone conduction, sound waves, etc. This data is sent to an electroencephalogram (EEG) analyzer and analyzed to infer the user's intentions.

[1101] The analyzed data is sent via communication lines to a generative AI, which generates appropriate responses and operations based on the user's intentions. This generative AI also has the ability to generate and arrange music according to the user's preferences and circumstances.

[1102] The operation commands generated by the generative AI are then sent to the autonomous vehicle's control system, allowing the autonomous vehicle to set the destination and begin navigation simply by the user thinking, "Go to the destination."

[1103] Specific hardware features include the Emotiv Epoc+ brainwave analyzer, Google Glass smart glasses, and Tesla Autopilot for the autonomous vehicle control system. Software features include the Emotiv SDK brainwave analysis library, OpenAI GPT-4 generative AI model, and Tesla API for the autonomous vehicle control API.

[1104] As a concrete example, consider the case where a user puts on smart glasses and thinks, "I want to go home." In this case, the EEG analyzer captures the user's intention, and the generative AI generates the command "go home." The self-driving vehicle then sets "home" as the destination and begins navigation.

[1105] An example prompt might look like this:

[1106] "If a user thinks, 'I want to go home,' the EEG analyzer will capture that intention, the generative AI will generate the command 'go home,' and the self-driving vehicle will set the destination to 'home' and begin navigation."

[1107] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1108] Step 1:

[1109] The user wears a chip implanted in the head or a tiny device attached to the outside of the face, which detects electrical signals in the brain, speech, bone conduction, sound waves, etc. The input is the electrical signals and sound waves in the user's brain, and the output is these signal data.

[1110] Step 2:

[1111] The device sends the detected signal data to an EEG analysis device, which analyzes the data and infers the user's intention. The input is signal data, and the output is analyzed intention data. Specifically, the data is analyzed using an EEG analysis library (e.g., Emotiv SDK).

[1112] Step 3:

[1113] The server sends the analyzed intent data to the generative AI, which generates appropriate responses and operation commands based on the user's intent. The input is the intent data, and the output is the operation command. Specifically, the response is generated using a generative AI model (e.g., OpenAI GPT-4).

[1114] Step 4:

[1115] The server sends the generated operation command to the autonomous vehicle control system. The autonomous vehicle control system operates the vehicle based on the received command. The input is the operation command, and the output is the vehicle's behavior. The specific operation is to control the vehicle using the autonomous vehicle control API (e.g., Tesla API).

[1116] Step 5:

[1117] If a user thinks, "I want to go home," the EEG analyzer captures that intention, the generative AI generates the command "go home," and the self-driving vehicle sets the destination to "home" and begins navigation. The input is the user's intention, and the output is setting the destination and starting navigation. Specifically, the navigation system sets the destination, and the vehicle begins moving automatically.

[1118] Example 2

[1119] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1120] Conventional music generation systems have had difficulty generating and arranging music in real time according to the user's preferences and circumstances. Furthermore, manual operation was required for the user to communicate their intentions to the system, which was inconvenient. Furthermore, there was a lack of means to effectively utilize the user's past listening history and current circumstances, making it difficult to provide optimal music for each individual user.

[1121] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1122] In this invention, the server includes means for collecting the user's past listening history and current situation (time, place, mood, etc.), means for transmitting the collected data to the server and storing it in a database, means for generating prompt sentences to be input to the generative AI model based on the stored data, means for inputting the prompt sentences to the generative AI model and generating and arranging music using music generation software, and means for transmitting the generated music to the user's terminal and playing it. This makes it possible to generate and arrange music in real time according to the user's preferences and situation, and provide optimal music without the user having to perform manual operations.

[1123] A "head chip or external facial device" is a small electronic device placed on the user's head or face to detect and transmit electrical signals and sound waves in the brain.

[1124] "Means for inputting / outputting information to the auditory and visual senses using electrical signals in the brain, monologue, bone conduction, sound waves, etc." refers to technology for inputting or outputting information to the auditory and visual senses using electrical signals and sound waves generated in the user's brain.

[1125] A "generative AI model" is an artificial intelligence model that generates and arranges music and other content based on user preferences and circumstances.

[1126] "Communication lines" refers to network infrastructure for sending and receiving data, including mobile lines and internet lines.

[1127] "Means for collecting a user's past listening history and current situation (time, location, mood, etc.)" refers to technology for collecting information such as the user's history of music they have listened to in the past, as well as their current time, location, mood, etc.

[1128] "Means for transmitting collected data to a server and storing the data in a database" refers to a technique for transmitting data collected from users to a server and storing the data in a database.

[1129] "Means for generating prompt sentences to be input to a generative AI model based on stored data" refers to a technology that generates instruction sentences (prompt sentences) to be input to a generative AI model based on user information stored in a database.

[1130] "Music generation software" means software for generating and arranging music based on instructions from a generative AI model.

[1131] "Means for transmitting the generated music to a user's device and playing it" refers to technology for transmitting music generated by a generative AI model to a user's device and playing the music on that device.

[1132] This invention is a system for generating and arranging music according to user preferences and circumstances. The system includes a user terminal, a server, a generative AI model, music generation software, and a database management system.

[1133] Hardware and software used

[1134] Hardware: Servers, user devices (smartphones, PCs, etc.)

[1135] Software: Generative AI models (e.g., GPT-3), database management systems (e.g., MySQL), music generation software

[1136] System Operation

[1137] 1. Collection of User Data

[1138] When a user uses a music streaming app, the device collects past listening history and current situation (time, location, mood, etc.).

[1139] Specifically, the device collects the history of songs played by the user, current GPS location information, and data obtained from the device's built-in sensors (e.g., heart rate, number of steps).

[1140] For example, if the user feels like relaxing in the evening, the terminal will record that information.

[1141] 2. Data transmission and storage

[1142] The terminal periodically transmits the collected data to the server.

[1143] The server stores the received data in a database management system (e.g. MySQL).

[1144] Specifically, the server stores the user ID, listening history, and current situation data in a database.

[1145] For example, the data for user ID "12345" may include information such as "nighttime" and "want to relax."

[1146] 3. Prompt generation

[1147] Based on the stored data, the server generates prompt sentences to be input into the generative AI model.

[1148] Specifically, the server obtains the user's current situation from a database and creates a prompt sentence based on that situation.

[1149] For example, a prompt sentence such as "User ID 12345 feels like relaxing, so please create some relaxing music" is generated.

[1150] 4. Music Creation and Arrangement

[1151] The server inputs prompts into a generative AI model (e.g., GPT-3), which then uses music generation software to generate and arrange music.

[1152] Specifically, the generative AI model analyzes the prompt and determines appropriate musical characteristics (such as tempo, melody, and instrumentation).

[1153] Music generation software then generates music based on these features.

[1154] For example, a piano piece with a slow tempo is generated as relaxing music.

[1155] 5. Distribution of generated music

[1156] The server transmits the generated music to the user's terminal.

[1157] The terminal then plays the received music for the user to listen to.

[1158] Specifically, the music player application on the device plays the generated music file.

[1159] For example, if the user feels like relaxing in the evening, the terminal will play a generated piano piece.

[1160] This system allows users to enjoy music in real time according to their preferences and circumstances.

[1161] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1162] Step 1:

[1163] User Data Collection

[1164] When a user uses a music streaming app, the device collects past listening history and current situation (time, location, mood, etc.).

[1165] Input: User playback history, GPS location, data from built-in sensors (heart rate, steps, etc.)

[1166] Data processing: Organize the collected data and identify the user's current situation.

[1167] Output: Organized user data (e.g. user ID, play history, current time, location, mood)

[1168] What it does: The device records the music history of the user, obtains the user's current GPS location, and collects heart rate and step counts from built-in sensors. For example, if the user feels like relaxing in the evening, it will record that information.

[1169] Step 2:

[1170] Data transmission and storage

[1171] The terminal periodically transmits the collected data to the server.

[1172] The server stores the received data in a database management system.

[1173] Input: Organized user data

[1174] Data processing: Converting data into a format suitable for storing in a database.

[1175] Output: User data stored in the database

[1176] Specific operation: The device sends the organized user data to the server, which then stores the data in a database. For example, the data for user ID "12345" stores information such as "evening time" and "want to relax."

[1177] Step 3:

[1178] Generate prompt statement

[1179] Based on the stored data, the server generates prompt sentences to be input into the generative AI model.

[1180] Input: User data stored in the database

[1181] Data processing: Analyze the user's current situation and create a prompt suitable for the generative AI model.

[1182] Output: Generated prompt statement

[1183] Specific operation: The server retrieves the user's current situation from the database and creates a prompt based on it. For example, it generates a prompt such as "User ID 12345 feels like relaxing, so please create some relaxing music."

[1184] Step 4:

[1185] Music Creation and Arrangement

[1186] The server inputs prompts into the generative AI model, which then uses music generation software to generate and arrange the music.

[1187] Input: Generated prompt statement

[1188] Data processing: The generative AI model analyzes the prompt and determines musical characteristics. Music generation software generates music based on these characteristics.

[1189] Output: Generated music file

[1190] How it works: The generative AI model analyzes the prompt and determines appropriate musical characteristics (tempo, melody, instrumentation, etc.). The music generation software then generates music based on these characteristics. For example, it might generate a piano piece with a slow tempo for relaxation.

[1191] Step 5:

[1192] Generated Music Distribution

[1193] The server transmits the generated music to the user's terminal.

[1194] The terminal then plays the received music for the user to listen to.

[1195] Input: Generated music file

[1196] Data processing: Convert music files into a format suitable for your device.

[1197] Output: Music files playable on your device

[1198] Specific operation: The server sends the generated music file to the user's device, and the device's music player app plays the music file. For example, if the user feels like relaxing at night, the device will play the generated piano music.

[1199] (Application example 2)

[1200] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1201] Conventional music playback systems have difficulty generating and arranging music based on the user's preferences and current situation, and are unable to respond to specific needs such as when users want to relax, concentrate, or exercise. Furthermore, there are problems with users having to go through the trouble of selecting music, and it takes time to find the perfect piece of music. Furthermore, there has been a lack of systems that can generate and arrange music taking into account the user's past listening history and current situation.

[1202] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1203] In this invention, the server includes a chip implanted in the head or a tiny device attached to the outside of the face, means for inputting and outputting to the auditory and visual senses using electrical signals in the brain, monologue, bone conduction, sound waves, etc., means for conversing with the generative AI via a mobile line, obtaining necessary information, obtaining suggestions for the next action, playing music and arranging it to convey one's favorite images to play the best new background music, talking to people, and sending messenger apps and emails, all without touching a thing, means for generating and arranging music taking into account the user's past listening history and current situation (time, place, mood, etc.), and means including a music streaming application installed on a smartphone. This makes it possible to generate and arrange optimal music according to the user's preferences and current situation.

[1204] A "headchip" is a small electronic device that is implanted in the user's head to detect and analyze electrical signals in the brain.

[1205] The "extremely small equipment attached to the outside of the face" is a small device that is attached to the outside of the user's face and uses sound waves and bone conduction to input / output auditory and visual information.

[1206] "Electrical signals in the brain" are signals that indicate electrical activity occurring in the user's brain, and by analyzing these, it is possible to understand the user's intentions and state.

[1207] "Soliloquy" refers to the words a user speaks to themselves, and by detecting this, it is possible to understand the user's intentions and state.

[1208] "Bone conduction" is a phenomenon in which sound waves are transmitted to the inner ear through the skull, and is a technology that utilizes this phenomenon to transmit audio information.

[1209] "Sound waves" are vibrations that travel through the air, and this technology utilizes them to transmit audio information.

[1210] "Generative AI" is an artificial intelligence technology that generates and arranges music and information according to the user's preferences and circumstances.

[1211] A "mobile line" is a wireless communication network used by mobile communication devices such as mobile phones and smartphones.

[1212] "Conversation" is the act of exchanging information between a user and a system through voice.

[1213] "Acquisition of necessary information" refers to the act of the system collecting and providing the information required by the user.

[1214] "Getting next action suggestions" is the act of the system suggesting the next action to be taken based on the user's current situation and intentions.

[1215] "Playing music" refers to the system playing music for the user.

[1216] "Communicate your preferred image, have it arranged, and play the best new background music" means that the user communicates their preferred musical image to the system, and the generative AI arranges the music based on that and plays the optimal background music.

[1217] "Talking to a person" means that a user converses with another person through voice.

[1218] "Messenger app or email sending" refers to the act of a user sending a message via a messaging application or email.

[1219] "User's past listening history" is a record of music that the user has listened to in the past.

[1220] "Current situation" refers to information that indicates the user's current state, such as time, location, and mood.

[1221] A "music streaming application" is software for playing music in real time over the Internet.

[1222] The system for implementing this invention is configured as follows: First, a chip to be implanted in the user's head or a tiny device to be attached to the outside of the face is required. This device detects electrical signals in the brain, monologue, bone conduction, sound waves, etc., and provides input / output to the auditory and visual senses.

[1223] Next, the generative AI talks to the user over the mobile network and obtains the necessary information. The generative AI generates and arranges music taking into account the user's past listening history and current situation (time, location, mood, etc.). This generative AI is implemented in programming languages ​​such as Python and works in conjunction with music streaming applications installed on smartphones.

[1224] Specifically, it generates and arranges the most suitable music depending on the user's situation, such as whether they want to relax, concentrate, exercise, etc. For example, if a user wants to relax in the morning, the generative AI will select and play relaxing music from the user's past listening history.

[1225] The system's hardware includes a smartphone, a head-implanted chip, and a tiny external facial device, while the software includes a generative AI model, a music streaming application, and brainwave analysis software.

[1226] Data processing and calculation involves analyzing the electrical signals in the user's brain to understand the user's intentions and state. Next, generative AI generates and arranges music based on the user's past listening history and current situation. This allows for optimal music playback based on the user's preferences and current situation.

[1227] As a concrete example, by inputting the following prompt sentence into a generative AI model, music can be generated that meets the user's needs.

[1228] Example prompt sentence:

[1229] "If a user wants to relax in the morning, generate relaxing music based on their past listening history."

[1230] In this way, music can be generated and arranged according to the user's preferences and current situation.

[1231] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1232] Step 1:

[1233] The user wears a chip that is implanted in the head or a tiny device that is attached to the outside of the face.

[1234] Input: Electrical signals in the user's brain, monologue, bone conduction, sound waves, etc.

[1235] Output: Detected electrical signals and sound wave data

[1236] How it works: A chip or tiny device detects electrical signals and sound waves in the user's brain and records them as digital data.

[1237] Step 2:

[1238] The device analyzes the detected data and understands the user's intentions and status.

[1239] Input: Detected electrical signals or acoustic data

[1240] Output: Information about the user's intent and state

[1241] Specific operation: The device's built-in brain wave analysis software analyzes the detected electrical signals and sound wave data to understand the user's intentions and state, such as whether they want to relax, concentrate, or exercise.

[1242] Step 3:

[1243] The server obtains the user's past listening history.

[1244] Input: User identification information

[1245] Output: User's past listening history data

[1246] Specific operation: The server retrieves the past listening history from the database based on the user's identification information.

[1247] Step 4:

[1248] The server obtains the current situation (time, location, mood, etc.).

[1249] Input: Data about the user's current time, location, and mood

[1250] Output: Current status data

[1251] What it does: The server collects data about the user's current time, location, and mood from their smartphone and other sensors.

[1252] Step 5:

[1253] The server uses a generative AI model to generate and arrange music based on the user's past listening history and current situation.

[1254] Input: User's past listening history data, current situation data

[1255] Output: Generated music data

[1256] Specific operation: The generative AI model generates and arranges optimal music based on the user's past listening history and current situation. For example, it uses the prompt "If the user wants to relax in the morning, please generate relaxing music based on the user's past listening history."

[1257] Step 6:

[1258] The device plays the generated music.

[1259] Input: Generated music data

[1260] Output: Music being played

[1261] Specific operation: A music streaming application installed on the device plays the generated music data and allows the user to listen to it.

[1262] Step 7:

[1263] While listening to music, the user can converse with the generative AI as needed, which will suggest the next course of action and obtain necessary information.

[1264] Input: User voice input, generative AI suggestions and information

[1265] Output: Suggested next steps and required information

[1266] Specific operation: The user converses with the generative AI via voice to obtain suggestions for the next action and necessary information. For example, if the user says, "Tell me what to do next," the generative AI will make appropriate suggestions.

[1267] Example 3

[1268] Next, a description will be given of Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1269] Conventional systems require users to perform physical operations when obtaining information or making calls or sending messages. This reduces user convenience, making operation particularly difficult when the user's hands are full or when the user has visual or hearing impairments. It is also difficult to accurately understand the user's intentions and perform appropriate operations. To solve these issues, a system that can analyze the user's intentions and automatically perform appropriate operations is needed.

[1270] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[1271] In this invention, the server includes means for acquiring user voice data and converting it into text using voice recognition software, means for analyzing the text data using a generative artificial intelligence model to understand the user's intention, and operation means for executing the generated procedures, thereby enabling the user to perform operations such as obtaining information, making calls, and sending messages using only voice input without performing any physical operations.

[1272] A "head chip" is a small electronic device that is implanted under the skin of the head and is used for communicating with the outside world and processing data.

[1273] "External facial attachment" refers to a small electronic device worn on the outside of the face for external communication and data processing.

[1274] "Electrical signals in the brain" are signals that indicate electrical activity emitted by the brain, and by analyzing these signals it is possible to read the state and intentions of the brain.

[1275] "Soliloquy" refers to the speech that a user speaks to themselves, and by analyzing this speech the intention of the user can be understood.

[1276] "Bone conduction" is a phenomenon in which sound waves are transmitted to the inner ear through bones, and is a technology that utilizes this phenomenon to transmit sound.

[1277] "Sound waves" are vibrations that travel through the air, and this is a technology that uses them to transmit sound.

[1278] "Auditory and visual input / output" refers to the means for providing audio and video to the user, and also to the means for obtaining audio and visual information from the user.

[1279] "Generative AI" is an AI technology that analyzes the user's intentions and performs appropriate operations.

[1280] A "communication line" is a network infrastructure for sending and receiving data, and includes mobile lines and internet lines.

[1281] "Speech recognition software" is software for converting voice data into text data.

[1282] A "generative artificial intelligence model" is an artificial intelligence model that analyzes a user's text data and understands their intentions.

[1283] The "operation means" is a means for executing the generated procedure, and is used to perform specific operations such as sending an email or starting a call.

[1284] This invention provides a system that enables a user to perform operations such as obtaining information, making calls, and sending messages only through voice input, without performing any physical operations. This system is implemented using the following hardware and software.

[1285] Hardware Configuration

[1286] 1. Chips implanted in the head or tiny devices attached to the outside of the face: These devices capture electrical signals and voices from the user's brain and communicate with an external server.

[1287] 2. Mobile terminal: A device that allows users to input voice data and transmits the voice data to the server.

[1288] Software Configuration

[1289] 1. Speech recognition software: Converts the user's voice data into text data using software such as the Google Speech-to-Text API.

[1290] 2. Generative AI models: Using models such as OpenAI's GPT-3, text data is analyzed to understand user intent.

[1291] 3. Email sending software: Uses an SMTP server or similar to send emails based on the generated instructions.

[1292] Data processing and calculation

[1293] The server receives the voice data sent by the user and converts it into text data using speech recognition software. It then uses a generative artificial intelligence model to analyze the text data and understand the user's intent. Based on the understood intent, it generates specific operating procedures and executes the operations using email sending software.

[1294] Specific examples

[1295] Consider the case where a user issues a verbal command such as "Send me the materials for tomorrow's meeting." In this case, the system operates as follows:

[1296] 1. The user speaks into the microphone on their smartphone, saying, "Send me the materials for tomorrow's meeting."

[1297] 2. The device records this voice data and sends it to the server.

[1298] 3. The server sends the received voice data to the Google Speech-to-Text API and obtains the text data, "Send me the materials for tomorrow's meeting."

[1299] 4. The server inputs the text data into a generative AI model and analyzes the user's intent. The generative AI model understands the instruction "Send the materials for tomorrow's meeting" as "Send an email."

[1300] 5. The generative AI model generates specific operational procedures such as:

[1301] To: example@example.com

[1302] Subject: Materials for tomorrow's meeting

[1303] Message: Hello. I have attached the materials for tomorrow's meeting below. Thank you in advance.

[1304] 6. The server sends an email using the SMTP server to execute the generated operation procedure. The email is sent to the specified destination, completing the user's instructions.

[1305] Prompt Sentence Examples

[1306] The user says, "Send me the materials for tomorrow's meeting." Use the following information to generate a procedure for sending an email.

[1307] To: example@example.com

[1308] Subject: Materials for tomorrow's meeting

[1309] Message: Hello. I have attached the materials for tomorrow's meeting below. Thank you in advance.

[1310] In this way, a system that analyzes the user's intention and performs an appropriate operation is realized. The flow of the identification process in the third embodiment will be described with reference to FIG.

[1311] Step 1:

[1312] The user issues commands by voice into the microphone on the mobile device, for example, "Send me the materials for tomorrow's meeting."

[1313] Input: User's voice data

[1314] Output: Audio data recorded on the device

[1315] Specific operation: The user gives voice instructions and the device records the voice.

[1316] Step 2:

[1317] The terminal transmits the recorded voice data to the server.

[1318] Input: Audio data recorded on the device

[1319] Output: Audio data sent to the server

[1320] Specific operation: The device sends voice data to the server.

[1321] Step 3:

[1322] The server sends the received voice data to speech recognition software (e.g., Google Speech-to-Text API) and converts it into text data.

[1323] Input: Audio data sent to the server

[1324] Output: Text data generated by speech recognition software

[1325] Specific operation: The server sends the voice data to the voice recognition software and obtains the text data.

[1326] Step 4:

[1327] The server inputs the acquired text data into a generative artificial intelligence model (e.g., OpenAI's GPT-3) and analyzes the user's intent.

[1328] Input: Text data generated by speech recognition software

[1329] Output: User intent analyzed by a generative AI model

[1330] Specific operation: The server inputs text data into a generative artificial intelligence model and analyzes the user's intention.

[1331] Step 5:

[1332] The generative AI model generates specific operational procedures based on the user's intentions, such as procedures for sending an email.

[1333] Input: User intent analyzed by a generative artificial intelligence model

[1334] Output: Generated specific operating instructions

[1335] Specific operation: A generative artificial intelligence model generates operating procedures based on the user's intentions.

[1336] Step 6:

[1337] The server sends an email using email sending software (e.g., an SMTP server) to execute the generated operation procedure.

[1338] Input: Generated specific operating instructions

[1339] Output: Email sent

[1340] Specific operation: The server uses email sending software to send the email to the specified recipient.

[1341] In this way, a system is realized that analyzes the user's intention and performs appropriate operations.

[1342] (Application example 3)

[1343] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1344] Conventional systems require users to manually operate the system when obtaining information or sending emails, resulting in low convenience. It is also difficult to accurately analyze the user's intentions and perform appropriate operations. Furthermore, when using wearable devices such as smart glasses, the display of information and operation are not intuitive, which poses a challenge.

[1345] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[1346] In this invention, the server includes a chip implanted in the head or a tiny device attached to the outside of the face, a means for inputting and outputting auditory and visual information using electrical signals in the brain, monologue, bone conduction, sound waves, etc., a means for conversing with generative AI via a mobile line, acquiring necessary information, acquiring suggestions for the next action, playing music and arranging it to create new, optimal background music based on user preferences, talking to people, using a messaging app, sending emails, etc., all without touching a device, a means for analyzing user intent and sending emails or acquiring information, and a means for displaying information on smart glasses. This allows the user to analyze intent and perform appropriate operations without manual operation, enabling intuitive information display and operation using smart glasses.

[1347] A "headchip" is a small electronic device that is implanted in a user's head to detect electrical signals in the brain and communicate with external devices.

[1348] "External facial attachment" refers to a small device that is attached to the outside of the user's face and is used for inputting and outputting auditory and visual information.

[1349] "Electrical signals in the brain" refer to electrical activity that occurs in the user's brain, and by analyzing these signals it is possible to read the user's intentions and thoughts.

[1350] "Soliloquy" refers to the words a user speaks to themselves, and analyzing this is a means of understanding the user's intentions.

[1351] "Bone conduction" refers to the phenomenon in which sound waves are transmitted to the inner ear through the skull, and is a technology that utilizes this phenomenon to transmit audio information to the user.

[1352] "Sound waves" are vibrations that travel through the air and are a means of transmitting audio information to the user.

[1353] "Generative AI" is an artificial intelligence system that analyzes user intentions and performs appropriate operations.

[1354] A "mobile line" is a communication line used to connect to the Internet via a mobile phone or smart device.

[1355] A "conversation" is a voice or text communication between a user and a generative AI.

[1356] "Obtaining necessary information" refers to the act of obtaining the information the user desires from the Internet or a database.

[1357] "Next action suggestion" is a function in which generative AI suggests the next action to be taken based on the user's current situation and intentions.

[1358] "Music playback" is a function in which generative AI selects and plays the music the user wants to listen to.

[1359] "Tell us your favorite image, let us arrange it and play the best new background music" is a function in which generative AI arranges music based on the user's preferences and image, and plays the optimal background music.

[1360] "Talk to people" is a function that allows a user to make voice calls with other people.

[1361] "Messenger apps and email sending" refers to the function that allows users to send messages or emails using messenger apps.

[1362] "Means of analyzing user intent and sending emails or obtaining information" refers to a function in which generative AI analyzes the user's intent and sends emails or obtains information based on that intent.

[1363] "Means for displaying on smart glasses" refers to a function that uses the smart glasses to visually display acquired information or generated content to a user.

[1364] The system for implementing this invention includes a chip implanted in the head or a tiny device attached to the outside of the face, generative AI, a mobile network, and smart glasses. Specific embodiments of each element are described below.

[1365] A chip implanted in the head or a tiny device attached to the outside of the face

[1366] A chip implanted in the user's head or a tiny device attached to the outside of the face detects electrical signals in the brain, monologue, bone conduction, sound waves, etc., and transmits this information to a generative AI, which can then analyze the user's intentions and thoughts in real time.

[1367] Generative AI

[1368] Generative AI analyzes the user's intent and performs the appropriate operation. For example, if the user thinks, "Send an email," the generative AI will grasp that intent and perform the operation to send an email. Similarly, if the user thinks, "Tell me the latest news," the generative AI will retrieve the latest news and display it on the smart glasses. Generative AI uses OpenAI's API to analyze the user's intent.

[1369] Mobile line

[1370] The mobile line is a means of communication with the generative AI via the internet, allowing it to obtain necessary information in real time and provide it to the user.

[1371] Smart Glasses

[1372] Smart glasses are devices that visually display information obtained and content generated by generative AI to users. For example, if a user thinks, "Tell me the latest news," the news obtained by generative AI will be displayed on the smart glasses.

[1373] Specific examples

[1374] If a user thinks, "Send an email," the generative AI will analyze the user's intent using the following prompt:

[1375] Example prompt sentence:

[1376] Analyze user intent and take appropriate action: Send an email

[1377] The generative AI inputs this prompt into the OpenAI API and analyzes the user's intent. It then sends an email using SMTP. If the user chooses "Tell me the latest news," the generative AI retrieves the latest news using the news API and displays it on the smart glasses.

[1378] In this way, the user can analyze their intentions and perform appropriate operations without having to perform manual operations, enabling intuitive information display and operation using smart glasses.

[1379] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[1380] Step 1:

[1381] The user has an intent.

[1382] Users have intentions such as "send an email" or "tell me the latest news." These intentions are expressed as electrical signals in the brain or as monologue.

[1383] Input: User's brain electrical signals, monologue

[1384] Output: User intent (e.g., send email, get news)

[1385] Step 2:

[1386] The device detects the user's intention.

[1387] A chip implanted in the head or a tiny device attached to the outside of the face detects electrical signals in the user's brain and their speech and converts them into digital signals.

[1388] Input: User's brain electrical signals, monologue

[1389] Output: Digital signal

[1390] Step 3:

[1391] The device sends a digital signal to the generative AI.

[1392] The device uses a mobile network to transmit the detected digital signal to the generative AI.

[1393] Input: Digital signal

[1394] Output: Sending data to generative AI

[1395] Step 4:

[1396] Generative AI analyzes user intent.

[1397] Generative AI uses the OpenAI API to analyze digital signals and understand user intent, such as "send an email."

[1398] Input: Digital signal

[1399] Output: User intent (e.g., send email, get news)

[1400] Specific operation: Enter a prompt into the OpenAI API and obtain the analysis results.

[1401] Step 5:

[1402] Generative AI determines the appropriate action.

[1403] Generative AI determines the appropriate operation based on the analyzed user intent, such as sending an email or retrieving news.

[1404] Input: User intent

[1405] Output: Operation instructions (e.g., send email, get news)

[1406] Step 6:

[1407] The generative AI executes the operation.

[1408] The generative AI executes the determined action, for example, sending an email using SMTP or getting the latest news using a news API.

[1409] Input: Operation instructions

[1410] Output: Operation result (e.g., email sent, news retrieved)

[1411] Specific operations: Connect to an SMTP server to send email, send a request to the news API to retrieve news.

[1412] Step 7:

[1413] The terminal displays the operation results on the smart glasses.

[1414] The device receives the operation results from the generative AI and displays them on the smart glasses. For example, it displays the acquired news on the smart glasses.

[1415] Input: Operation result

[1416] Output: Display on smart glasses

[1417] Specific operation: The operation results are displayed on the smart glasses display.

[1418] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1419] "Example 1"

[1420] The invention incorporating the emotion engine has the ability to recognize the user's emotions and generate and arrange music accordingly. Specifically, it analyzes the user's brain waves and sends the results to the emotion engine. The emotion engine infers the user's emotions from the brain wave analysis results and generates and arranges music according to those emotions. For example, when the user is feeling happy, the emotion engine generates and arranges bright and cheerful music. Conversely, when the user is sad, the emotion engine generates and arranges calm music.

[1421] "Example 2"

[1422] The emotion engine also suggests the next action based on the user's emotions. Specifically, it analyzes the user's brain waves and sends the results to the emotion engine. The emotion engine infers the user's emotions from the brain wave analysis results and suggests the next action based on those emotions. For example, if the user is feeling stressed, the emotion engine might suggest playing relaxation music or taking deep breaths.

[1423] "Example 3"

[1424] Furthermore, the emotion engine has the ability to adjust the content of messages according to the user's emotions. Specifically, when a user sends a message, the emotion engine considers the user's current emotions and suggests expressions that are appropriate for that emotion. For example, when a user is feeling happy, the emotion engine will suggest cheerful expressions. Conversely, when a user is sad, the emotion engine will suggest calm expressions.

[1425] The processing flow of each embodiment will be described below.

[1426] "Example 1"

[1427] Step 1: The user's brain waves are analyzed. This analysis is done by a chip implanted in the brain or a small device attached to the face.

[1428] Step 2: The analysis results are sent to the emotion engine via wireless communication.

[1429] Step 3: The emotion engine estimates the user's emotions from the results of EEG analysis.

[1430] Step 4: The emotion engine generates and arranges music according to the estimated emotions.

[1431] "Example 2"

[1432] Step 1: The user's brain waves are analyzed. This analysis is done by a chip implanted in the brain or a small device attached to the face.

[1433] Step 2: The analysis results are sent to the emotion engine via wireless communication.

[1434] Step 3: The emotion engine estimates the user's emotions from the results of EEG analysis.

[1435] Step 4: The emotion engine suggests the next action based on the estimated emotion.

[1436] "Example 3"

[1437] Step 1: When a user sends a message, the emotion engine takes into account the user's current emotion.

[1438] Step 2: The emotion engine suggests expressions that fit the emotion.

[1439] Example 1

[1440] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1441] Conventional voice recognition systems and EEG analysis systems have had difficulty accurately grasping a user's intentions and emotions and providing appropriate operations and services based on them. Furthermore, there are only a limited number of systems that allow users to operate them without touching anything, which makes them less convenient. Furthermore, there have been no systems with the ability to generate and arrange music based on the user's emotions. To solve these issues, a new system is needed.

[1442] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1443] In this invention, the server includes means for detecting electrical signals in the brain, monologue, bone conduction, sound waves, etc. using a chip implanted in the head or a tiny device attached to the outside of the face, means for analyzing brain waves using an electroencephalogram analyzer, means for transmitting the analysis results to a generative AI, means for the generative AI to infer the user's intentions based on the analysis results and perform appropriate operations, means for recognizing the user's emotions using an emotion engine and generating and arranging music according to those emotions, and means for conversing via a mobile line, obtaining necessary information, obtaining suggestions for the next action, playing music, expressing a favorite image and arranging it to play new, perfect background music, talking to people, using a messenger app, sending emails, etc., all without touching anything. This makes it possible to accurately grasp the user's intentions and emotions and provide appropriate operations and services based on them.

[1444] A "headchip" is a small electronic device that is implanted in a user's head to detect electrical signals in the brain.

[1445] The "external facial device" is a small device that is worn on the outside of the user's face to detect electrical signals and sound waves in the brain.

[1446] "Brain electrical signals" are signals that indicate electrical activity occurring in the user's brain.

[1447] "Soliloquy" is a voice that a user speaks to himself or herself.

[1448] "Bone conduction" is the phenomenon in which sound waves are transmitted to the inner ear through bones.

[1449] "Sound waves" are sound vibrations that travel through the air.

[1450] An "electroencephalogram (EEG) analysis device" is a device that analyzes electrical signals in the brain and infers a user's intentions and emotions.

[1451] "Generative AI" is an artificial intelligence system that infers the user's intentions based on analysis results and performs appropriate operations.

[1452] The "Emotion Engine" is a system that recognizes the user's emotions and generates and arranges music according to those emotions.

[1453] A "mobile line" is a communication line for sending and receiving data using wireless communication.

[1454] A "conversation" is a spoken communication between a user and a system.

[1455] "Acquisition of necessary information" means that the system collects and provides the information the user requests.

[1456] "Obtaining suggestions for the next action" means that the system suggests the next action to be taken based on the user's situation.

[1457] "Playing music" means that the system provides music to the user through a playback device.

[1458] "Personal image" refers to the image of specific music or video that the user likes.

[1459] "Arrangement" means changing existing music or video to suit the user's preferences.

[1460] "BGM" is an abbreviation for background music and refers to background music.

[1461] A "call" is a user communicating with another person by voice.

[1462] A "messenger app" is an application for sending and receiving text messages.

[1463] "Send mail" means to send an e-mail.

[1464] The present invention is a system that uses a chip implanted in the user's head or a tiny device attached to the outside of the face to detect electrical signals in the user's brain, monologue, bone conduction, sound waves, etc., and analyzes this information. Specific embodiments of this system are described below.

[1465] Hardware and software used

[1466] 1. Hardware

[1467] Chip or tiny device: A small device implanted in the user's head or worn on the outside of the face that detects electrical signals or sound waves in the brain.

[1468] Brain wave analysis device: This device analyzes detected brain waves and generates data to infer the user's intentions and emotions.

[1469] 2. Software

[1470] Generative AI: As an example, we will use OpenAI GPT-4. This AI infers the user's intentions based on data sent from an EEG analyzer and performs appropriate operations.

[1471] Emotion Engine: This engine estimates the user's emotions from the results of EEG analysis and generates and arranges music according to those emotions.

[1472] Data processing and calculation

[1473] 1. Detection of EEG and audio signals

[1474] The device detects electrical signals in the user's brain, monologue, bone conduction, sound waves, etc. in real time.

[1475] Specific actions include the user thinking "play music" or talking to themselves.

[1476] 2. Analysis of detection data

[1477] The data detected by the terminal is sent to an electroencephalogram analysis device.

[1478] The electroencephalogram analysis device analyzes the received data and extracts information to estimate the user's intentions and emotions.

[1479] As a specific operation, the electroencephalogram analysis device analyzes the electroencephalogram pattern and detects the intention of "play music."

[1480] 3. Sending analysis results

[1481] The EEG analysis device sends the analysis results to the generative AI.

[1482] The server receives the analysis results and proceeds to the next processing step.

[1483] Specifically, the analysis result is sent to the server as "music playback intent present."

[1484] 4. Intention estimation and operation execution

[1485] The server uses generative AI to infer the user's intentions based on the analysis results.

[1486] The server performs the appropriate action based on the inferred intent.

[1487] Specifically, the server starts a music playback application and plays music.

[1488] 5. Emotion Recognition and Music Generation

[1489] The server uses an emotion engine to estimate the user's emotions from the analysis results.

[1490] The server generates and arranges music according to the estimated emotions.

[1491] Specifically, if the user feels joy, the server generates, arranges, and plays cheerful, upbeat music.

[1492] Specific examples

[1493] 1. Users think "Play music"

[1494] The terminal detects brain waves and transmits them to an EEG analysis device.

[1495] The EEG analysis device determines that there is an intention to play music and sends this to the server.

[1496] The server uses generative AI to confirm the intent and launches the music playback application.

[1497] The server plays the music.

[1498] 2. Users are delighted

[1499] The terminal detects brain waves and transmits them to an EEG analysis device.

[1500] The EEG analysis device determines that "emotions of joy are present" and sends the result to the server.

[1501] The server uses an emotion engine to generate and arrange bright and cheerful music.

[1502] The server plays the generated music.

[1503] Prompt Sentence Examples

[1504] "If the user wants to play music, explain how you detect that intent and control the music playback."

[1505] "Please explain the process of generating and arranging music based on the user's emotions."

[1506] This system analyzes the user's brain waves and emotions in real time and performs appropriate operations and generates music based on that information, thereby improving the user experience.

[1507] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1508] Step 1:

[1509] The user wears a chip implanted in the head or a tiny device attached to the outside of the face.

[1510] Input: Electrical signals in the user's brain, self-talk, bone conduction, sound waves, etc.

[1511] Specific actions: The user thinks "play music" or talks to themselves.

[1512] Output: Detected brainwaves and audio signals.

[1513] Step 2:

[1514] The data detected by the terminal is sent to an electroencephalogram analysis device.

[1515] Input: Detected brainwaves and audio signals.

[1516] Specific operation: The device transmits data in real time to an EEG analysis device.

[1517] Output: Data sent to EEG analyzer.

[1518] Step 3:

[1519] The electroencephalogram analysis device analyzes the received data and extracts information to estimate the user's intentions and emotions.

[1520] Input: Brainwave and audio signals sent from the device.

[1521] Specific operation: The EEG analyzer analyzes brainwave patterns and detects the intention of "play music."

[1522] Output: Analysis results (information about user intent and emotions).

[1523] Step 4:

[1524] The EEG analysis device sends the analysis results to the generative AI.

[1525] Input: Analysis results (information about the user's intent and emotions).

[1526] Specific operation: The analysis results are sent to the generative AI.

[1527] Output: Analysis results sent to the generative AI.

[1528] Step 5:

[1529] The server uses generative AI to infer the user's intentions based on the analysis results.

[1530] Input: Analysis results sent to the generative AI.

[1531] Specific operation: The generative AI infers the user's intention based on the analysis results.

[1532] Output: Inferred user intent.

[1533] Step 6:

[1534] The server performs the appropriate action based on the inferred intent.

[1535] Input: Inferred user intent.

[1536] Specific operation: The server starts a music playback application and plays music.

[1537] Output: The operation performed (playing music).

[1538] Step 7:

[1539] The server uses an emotion engine to estimate the user's emotions from the analysis results.

[1540] Input: Analysis results sent to the generative AI.

[1541] Specific operation: The emotion engine estimates the user's emotions based on the analysis results.

[1542] Output: Estimated user sentiment.

[1543] Step 8:

[1544] The server generates and arranges music according to the estimated emotions.

[1545] Input: Estimated user sentiment.

[1546] Specific operation: The server uses the emotion engine to generate and arrange bright and cheerful music.

[1547] Output: Generated and arranged music.

[1548] Step 9:

[1549] The server plays the generated music.

[1550] Input: Generated and arranged music.

[1551] Specific operation: The server plays music through a music playback application.

[1552] Output: The played music.

[1553] (Application example 1)

[1554] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1555] In traditional brick-and-mortar shopping experiences, it was difficult to grasp a user's emotions and intentions in real time and provide appropriate product suggestions and navigation. Furthermore, when a user was searching for a specific product or was in a specific emotional state, there was a lack of a way to suggest appropriate products or services. This limited the user's shopping experience and led to problems such as reduced satisfaction.

[1556] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1557] In this invention, the server includes a means for detecting electrical signals in the brain, monologue, bone conduction, sound waves, etc. using a chip implanted in the head or a tiny device attached to the outside of the face, a means for conversing with the generative AI via a mobile line, obtaining necessary information, obtaining suggestions for the next action, playing music and arranging it to convey your favorite images to play the best new background music, talking to people, using messenger apps, sending emails, etc., all without touching anything, and a means for analyzing the user's brainwaves and emotions to make appropriate product suggestions and in-store navigation in a physical store. This makes it possible to understand the user's emotions and intentions in real time and make product suggestions and navigation accordingly.

[1558] A "head chip" is a small electronic device that is implanted under the skin of the head to detect electrical signals in the brain.

[1559] The "external facial microdevice" is a small device worn on the outside of the face to detect electrical signals and sound waves in the brain.

[1560] "Electrical signals in the brain" are weak electrical fluctuations that occur in association with neural activity in the brain.

[1561] "Soliloquy" refers to words spoken by a user to themselves and is detected as audio.

[1562] "Bone conduction" is the phenomenon in which sound waves are transmitted to the inner ear through the skull.

[1563] "Sound waves" are sound vibrations that travel through the air.

[1564] "Generative AI" is artificial intelligence that analyzes the user's intentions and emotions and performs appropriate operations and suggestions based on that.

[1565] A "mobile line" is a communication line used by mobile communication devices such as mobile phones and smartphones.

[1566] A "conversation" is the act of exchanging information between a user and a system through voice or text.

[1567] "Obtaining necessary information" means that the system collects and provides the information the user requests.

[1568] "Getting next action suggestions" means that the system suggests the next action to be taken based on the user's current situation and intentions.

[1569] "Playing music" means that the system plays music to the user.

[1570] "Convey your favorite image, arrange it, and play the best new background music" means generating and arranging music based on the user's preferences and emotions, and providing the best background music.

[1571] "Talking to a person" means that a user communicates with another person by voice.

[1572] "Messenger apps and emailing" refers to users sending text messages and emails.

[1573] "Means for analyzing brain waves" refers to technology that analyzes electrical signals in the brain and infers the user's intentions and emotions.

[1574] "Means for generating and arranging music according to the user's preferences and circumstances" refers to technology that creates and adjusts music based on the user's individual preferences and current emotional state.

[1575] "Means for making appropriate product suggestions and in-store navigation within a physical store" refers to technology that analyzes the user's brain waves and emotions and, based on that, suggests the most suitable products within the physical store or provides guidance within the store.

[1576] The system for implementing this invention uses a chip implanted in the head or a tiny device attached to the outside of the face to detect electrical signals in the brain, monologue, bone conduction, sound waves, etc. This data is transmitted to a mobile device via wireless communication such as Bluetooth. The mobile device receives this data and transmits it to the generative AI.

[1577] Generative AI analyzes the received data and infers the user's intentions and emotions. Based on the analysis results, generative AI performs appropriate operations and makes suggestions to the user. For example, if a user thinks, "Play music," generative AI will grasp that intention and operate the music playback. It is also possible to generate and arrange music according to the user's emotions.

[1578] Furthermore, to improve the shopping experience in physical stores, the system analyzes users' brainwaves and emotions to provide appropriate product suggestions and in-store navigation. If a user is looking for a specific product or is in a certain emotional state, the system will suggest products and services that suit them.

[1579] Hardware and software used

[1580] Hardware:

[1581] Brainwave detection device (Bluetooth compatible)

[1582] Mobile devices (smartphones and tablets)

[1583] Displays in physical stores and smartphones

[1584] software:

[1585] Python

[1586] Keras (loading and predicting emotion estimation model)

[1587] Scikit-learn (data preprocessing)

[1588] Bluetooth library (acquiring EEG data)

[1589] Data processing and calculation

[1590] 1. Acquiring brainwave data: The brainwave detection device detects the user's brainwaves and transmits them to the mobile device via Bluetooth.

[1591] 2. Data preprocessing: Standardize the EEG data received by the mobile device.

[1592] 3. Emotion estimation: Using standardized data, generative AI estimates the user's emotions.

[1593] 4. Product Recommendation: Based on the estimated emotions, appropriate products are suggested in-store.

[1594] Specific examples

[1595] A user enters a physical store and wears a brainwave detection device. The device detects the user's brainwaves and transmits the data via Bluetooth to a mobile device. The mobile device analyzes the data and determines that the user is happy. Recommendations such as "brightly colored clothes" or "fun books" are then displayed on displays in the store and on smartphones.

[1596] Prompt Sentence Examples

[1597] A user enters a physical store and wears a brainwave detection device. The device detects the user's brainwaves and transmits the data via Bluetooth. The application analyzes the data and determines that the user is happy. In-store displays and smartphones display suggestions like "brightly colored clothes" and "fun books."

[1598] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1599] Step 1:

[1600] A user wears a brainwave detection device.

[1601] Input: User's brainwaves

[1602] Output: EEG data

[1603] How it works: The user wears an EEG detection device on their head, which detects electrical signals in the brain.

[1604] Step 2:

[1605] The brainwave detection device transmits the brainwave data to a mobile device via Bluetooth.

[1606] Input: EEG data

[1607] Output: EEG data sent to a mobile device

[1608] Specific operation: The brainwave detection device transmits the detected brainwave data to the mobile device via Bluetooth communication.

[1609] Step 3:

[1610] Standardize the EEG data received by the mobile device.

[1611] Input: EEG data sent to a mobile device

[1612] Output: Normalized EEG data

[1613] Specific operation: The EEG data received by the mobile device is standardized using Scikit-learn's StandardScaler.

[1614] Step 4:

[1615] The mobile device sends standardized brainwave data to the generative AI.

[1616] Input: Normalized EEG data

[1617] Output: EEG data sent to the generative AI

[1618] How it works: The mobile device sends standardized EEG data to the generative AI.

[1619] Step 5:

[1620] Generative AI analyzes brainwave data and infers the user's emotions.

[1621] Input: EEG data sent to the generative AI

[1622] Output: Estimated emotion data

[1623] Specific operation: The generative AI uses a Keras model to analyze EEG data and estimate the user's emotions.

[1624] Step 6:

[1625] Generative AI suggests appropriate products in physical stores based on the estimated emotions.

[1626] Input: Estimated emotion data

[1627] Output: Product proposal data

[1628] How it works: Based on the estimated emotions, the generative AI suggests appropriate products on displays in physical stores and on smartphones.

[1629] Step 7:

[1630] Product suggestions are displayed to users on displays in physical stores and on smartphones.

[1631] Input: Product proposal data

[1632] Output: Product suggestions displayed to the user

[1633] Specific operation: Product suggestion data received from generative AI is displayed to users on displays in physical stores or on smartphones.

[1634] Example 2

[1635] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1636] Conventional music generation systems and behavior suggestion systems have had difficulty providing optimal music and behaviors based on the user's emotions and circumstances. Furthermore, there were no systems that could analyze the user's brainwave data in real time and make appropriate suggestions based on the results. This prevented the system from improving the user's quality of life.

[1637] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1638] In this invention, the server includes means for collecting the user's brain wave data and transmitting it to the server, means for analyzing the brain wave data and transmitting it to the emotion engine, means for the emotion engine to estimate the user's emotion from the brain wave analysis results and suggest the next action, means for the generative AI model to generate and arrange music taking into account the user's past listening history and current situation, and means for transmitting the generated music to the user's terminal and playing it. This makes it possible to provide optimal music and actions in real time according to the user's emotion and situation.

[1639] A "head implant" is a device that is implanted in the user's head to collect electrical signals from the brain.

[1640] The "small device attached to the outside of the face" is a small device that is attached to the outside of the user's face and collects electrical signals and sounds from the brain.

[1641] "Brain electrical signals" are signals that indicate electrical activity generated by the user's brain.

[1642] "Voice" refers to words or sounds made by the user and transmitted as sound waves.

[1643] "Bone conduction" is the phenomenon in which sound waves are transmitted to the inner ear through bones.

[1644] "Sound waves" are vibrations that travel through the air and are perceived as sound.

[1645] A "generative AI model" is an artificial intelligence model that generates and arranges music according to the user's preferences and circumstances.

[1646] A "communication line" is a network infrastructure for sending and receiving data.

[1647] The "emotion engine" is a system that estimates a user's emotions from the results of brain wave analysis and suggests actions based on those emotions.

[1648] "User terminal" refers to a device that is directly operated by a user, including smartphones, tablets, PCs, etc.

[1649] "EEG data" refers to data on electrical signals collected from the user's brain.

[1650] "Analysis" is the process of processing collected data and extracting meaningful information.

[1651] "Action suggestions" are suggestions for the next action to be taken based on the user's emotions and situation.

[1652] "Listening history" is a record of music or audio that a user has listened to in the past.

[1653] "Playback" means outputting the generated music or audio so that the user can hear it.

[1654] This invention is a system that collects and analyzes a user's brain wave data, generates and arranges music based on the results, and suggests the next action. Specific embodiments for implementing this system are described below.

[1655] Hardware and software used

[1656] Hardware: EEG sensor, server, user device (smartphone, tablet, PC, etc.)

[1657] Software: Generative AI models, emotion engines, data analysis software

[1658] System configuration

[1659] 1. Collecting user's brainwave data

[1660] The user wears an EEG sensor, which collects electrical signals from the user's brain in real time.

[1661] The device receives the brainwave data collected from the brainwave sensor and transmits it to the server.

[1662] 2. Analysis of EEG data

[1663] The server uses data analysis software to analyze the received EEG data, identifying the user's EEG patterns and sending them to the emotion engine.

[1664] 3. Emotion estimation and action suggestion

[1665] The emotion engine estimates the user's emotions from the results of EEG analysis. For example, if the user is feeling stressed, the emotion engine will suggest playing relaxation music or taking deep breaths.

[1666] 4. Music Creation and Arrangement

[1667] The server uses a generative AI model to generate and arrange music, taking into account the user's past listening history and current situation (time, location, mood, etc.). The generated music is customized to the user's preferences.

[1668] 5. Sending and playing music to the user's device

[1669] The server sends the generated music to the user's device, which then plays the music, providing a relaxing environment for the user.

[1670] Specific examples

[1671] User Situation: User wants to relax after work.

[1672] Brainwave data collection: The user wears an EEG sensor and the device collects EEG data.

[1673] Sending EEG data to the server: The device sends the collected EEG data to the server.

[1674] Analysis of EEG data: The server analyzes the EEG data and sends it to the emotion engine.

[1675] Emotion estimation and action suggestion: The emotion engine estimates that the user is feeling stressed and suggests playing relaxation music.

[1676] Music generation and arrangement: A generative AI model generates relaxing music and sends it to the user's device.

[1677] Sending and playing music to the user's device: The device plays the generated music, providing a relaxing environment for the user.

[1678] Prompt Sentence Examples

[1679] "What kind of music would a generative AI model generate if a user wanted to relax?"

[1680] "Please explain the process by which the emotion engine analyzes the user's brainwave data and suggests the next action."

[1681] This system aims to improve the quality of life of users by providing them with the most appropriate music and behaviors in real time based on their emotions and circumstances.

[1682] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1683] Step 1:

[1684] Collecting user brainwave data

[1685] The user wears a brainwave sensor.

[1686] Input: User's EEG signal.

[1687] The device collects brainwave data in real time from an EEG sensor.

[1688] Output: Collected EEG data.

[1689] How it works: When the user feels like they want to relax, they wear the brainwave sensor and the device starts collecting data.

[1690] Step 2:

[1691] Sending EEG data to the server

[1692] The device transmits the collected brainwave data to a server.

[1693] Input: Collected EEG data.

[1694] Output: EEG data sent to the server.

[1695] Specific operation: The device uses Wi-Fi or mobile data to send brainwave data to the server.

[1696] Step 3:

[1697] Analysis of EEG data

[1698] The server uses data analysis software to analyze the received brainwave data.

[1699] Input: EEG data sent to the server.

[1700] Output: Analyzed EEG data.

[1701] How it works: The server analyzes the EEG data and identifies the user's EEG patterns.

[1702] Step 4:

[1703] Emotion estimation and action suggestions

[1704] The server sends the analysis results to the emotion engine.

[1705] Input: Analyzed EEG data.

[1706] The emotion engine estimates the user's emotions from the results of brain wave analysis.

[1707] Output: Estimated user sentiment and action suggestions.

[1708] Specific behavior: The emotion engine infers that the user is feeling stressed and suggests playing relaxation music and taking deep breaths.

[1709] Step 5:

[1710] Music Creation and Arrangement

[1711] The server uses a generative AI model to generate and arrange music, taking into account the user's past listening history and current situation (time, location, mood, etc.).

[1712] Input: User's past listening history, current situation, and estimated emotion.

[1713] Output: Generated and arranged music.

[1714] What it does: A generative AI model generates relaxing music and arranges it to suit the user's preferences.

[1715] Step 6:

[1716] Sending and playing music to user devices

[1717] The server transmits the generated music to the user terminal.

[1718] Input: Generated and arranged music.

[1719] Output: Music sent to the user device.

[1720] The device plays the received music.

[1721] Specific behavior: The device plays music, providing a relaxing environment for the user.

[1722] (Application example 2)

[1723] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1724] Conventional music generation systems and emotion analysis systems can generate and arrange music according to the user's preferences and circumstances, but it has been difficult to generate and arrange music in real time according to the emotions and preferences of customers in physical stores. Furthermore, there has been no system that can analyze a customer's brain waves and suggest optimal music based on the results. This has led to the issue of not being able to improve the customer experience in physical stores.

[1725] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1726] In this invention, the server includes a chip implanted in the head or a tiny device attached to the outside of the face, a means for inputting and outputting audio and visual information using electrical signals in the brain, monologue, bone conduction, sound waves, etc., a means for conversing with a generative AI via a mobile line, obtaining necessary information, obtaining suggestions for the next action, playing music and arranging it to convey personal images and create new, optimal background music, talking to people, messaging apps, sending emails, etc., all without touching a single device, a means for generating and arranging music in real time according to the emotions and preferences of customers in physical stores, and a means for analyzing customers' brainwaves and having an emotion engine suggest optimal music based on the results. This makes it possible to generate and arrange music according to customers' emotions and preferences in physical stores, improving the customer experience.

[1727] A "head chip" is a small electronic device that is implanted in the user's head to capture electrical signals in the brain.

[1728] The "extrinsic equipment attached to the outside of the face" is a small device that is attached to the outside of the user's face and is used to acquire electrical signals and sound waves in the brain.

[1729] "Electrical signals in the brain" are electrical signals based on brain waves and neural activity, and are data used to analyze the user's emotions and state.

[1730] "Soliloquy" is what a user says to themselves and is captured in the system as voice input.

[1731] "Bone conduction" is the phenomenon in which sound waves travel through bones to the inner ear, and is a means of transmitting auditory information.

[1732] "Sound waves" are vibrations that travel through the air and are used as auditory information.

[1733] "Auditory and visual input / output means" refers to devices and technologies that provide audio and video to users and obtain audio and visual information from users.

[1734] "Generative AI" is artificial intelligence that generates and arranges music and other content based on the user's preferences and circumstances.

[1735] A "mobile line" is a communication line used to connect to the Internet via a mobile phone or smartphone.

[1736] "Conversation" is the act of exchanging information between a user and a system through voice.

[1737] "Obtaining necessary information" refers to the act of the system collecting and providing the information requested by the user.

[1738] "Getting next action suggestions" means that the system suggests the next action to be taken based on the user's current state and emotions.

[1739] "Playing music" refers to the system playing music for the user.

[1740] "Communicate your preferred image, arrange it, and play the best new background music" means that the user can communicate their preferred musical style and atmosphere to the system and play music generated based on that.

[1741] "Talking to a person" means communicating with another person by voice via the system.

[1742] "Messenger apps and emailing" refers to the act of using applications to send text messages or emails.

[1743] A "physical store" is a commercial establishment that exists in a physical location and that customers can visit in person.

[1744] "A means for generating and arranging in-store music in real time according to customer emotions and preferences" is a technology that generates and arranges music played in-store on the spot based on customer emotions and preferences.

[1745] "A means of analyzing the customer's brain waves and having the emotion engine suggest the most suitable music based on the results" is a technology that analyzes the customer's brain wave data and has the emotion engine select and suggest the most suitable music based on the results.

[1746] The system for implementing this invention has the following configuration: First, a chip is implanted in the user's head or a tiny device is attached to the outside of the face. This allows electrical signals in the brain, monologue, bone conduction, sound waves, etc. to be acquired and input / output to the auditory and visual senses.

[1747] Next, via generative AI and mobile network, it will converse with the user, obtain necessary information, and suggest the next course of action. It can also play music, convey images based on the user's preferences, and arrange them to play the best new background music. It can also talk to people, use messaging apps, and send emails.

[1748] In physical stores, it provides a means to generate and arrange in-store music in real time according to the customer's emotions and preferences. The system analyzes the customer's brainwaves and uses the emotion engine to suggest the most suitable music. This system can improve the customer experience in physical stores.

[1749] Hardware and software used

[1750] Hardware: EEG sensor, smartphone

[1751] Software: Python, Keras (loading and predicting emotion estimation model), music generation module, emotion engine module

[1752] Data processing and calculation

[1753] The server acquires EEG data in real time from the EEG sensor and standardizes the data. The standardized data is then used to predict the user's emotions using an emotion estimation model. The emotion engine then generates optimal music based on the predicted emotions and plays it in the store.

[1754] Specific examples

[1755] For example, if a customer in a physical store feels like they want to relax, the brainwave sensor will detect that emotion and generate relaxing music to play in the store. If the customer is feeling stressed, the emotion engine will suggest deep breathing and play relaxation music.

[1756] Prompt Sentence Examples

[1757] Get the user's brainwave data, estimate their emotions, and generate music according to those emotions. For example, if the user wants to relax, generate relaxing music.

[1758] In this way, music can be generated and arranged according to the user's emotions and preferences, improving the customer experience in physical stores.

[1759] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1760] Step 1:

[1761] Acquire the user's brainwave data. The brainwave sensor is used to collect the user's brainwave data in real time. The input is raw data from the brainwave sensor, and the output is an array of brainwave data.

[1762] Step 2:

[1763] Preprocessing of EEG data is performed. The server normalizes the acquired EEG data to standardize it. The input is the array of EEG data obtained in step 1, and the output is the standardized EEG data.

[1764] Step 3:

[1765] Estimate emotions. The server inputs the standardized EEG data into an emotion estimation model (using Keras) to predict the user's emotions. The input is the standardized EEG data obtained in step 2, and the output is the predicted emotion label.

[1766] Step 4:

[1767] Generate music. The server uses an emotion engine to generate optimal music based on the predicted emotion. The input is the emotion label obtained in step 3, and the output is the generated music data.

[1768] Step 5:

[1769] Play the music. The terminal receives the generated music data and plays it in the physical store. The input is the music data obtained in step 4, and the output is the music played in the store.

[1770] Step 6:

[1771] Suggest next actions: The server uses an emotion engine to suggest next actions based on the user's current emotion. The input is the emotion label obtained in step 3, and the output is a list of suggested actions.

[1772] Step 7:

[1773] Notify the user of the proposed actions. The terminal notifies the user of the proposed actions received from the server. The input is the list of actions obtained in step 6, and the output is the notification to the user.

[1774] In this way, a series of processes is realized: acquiring the user's brain wave data, estimating their emotions, generating and playing music according to those emotions, and suggesting the next action.

[1775] Example 3

[1776] Next, a description will be given of Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1777] Conventional systems require manual input and operation when users want to perform intended operations or send messages, which is inefficient. Furthermore, they are unable to adjust messages based on the user's emotions, which reduces the quality of communication. Furthermore, there is a lack of technology that can accurately analyze the user's intentions and emotions and adjust appropriate operations and messages based on them.

[1778] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[1779] In this invention, the server includes means for analyzing the user's intentions and performing appropriate operations, means for analyzing the user's emotions and adjusting the content of messages, and means for conversing with the generative AI via a mobile line, acquiring necessary information, obtaining suggestions for the next action, playing music and arranging it to convey the user's favorite image to play the best new background music, talking to people, using a messenger app, sending emails, etc., all without touching anything. This allows the user to perform intended operations and send messages without manual input or operation, and further enables messages to be adjusted according to the user's emotions.

[1780] A "headchip" is a small electronic device that is implanted in the user's head to detect and analyze electrical signals in the brain.

[1781] The "extrinsic equipment attached to the outside of the face" is a small device that is attached to the outside of the user's face and uses sound waves and bone conduction to input / output auditory and visual information.

[1782] "Brain electrical signals" are signals that indicate electrical activity occurring in the user's brain, and by analyzing these signals, data can be used to understand the user's intentions and emotions.

[1783] "Soliloquy" refers to the words a user speaks to themselves, and by analyzing this data it is possible to understand the user's intentions and emotions.

[1784] "Bone conduction" is a phenomenon in which sound waves are transmitted to the inner ear through the skull, and is a technology that utilizes this phenomenon to transmit auditory information.

[1785] "Sound waves" are vibrations that travel through the air and are a means of transmitting auditory information.

[1786] "Generative AI" is an artificial intelligence technology that analyzes a user's intentions and emotions and generates appropriate operations and messages.

[1787] A "mobile line" is a communication line for sending and receiving data using wireless communication.

[1788] An "emotion engine" is a technology that analyzes a user's emotions and adjusts the content of messages according to those emotions.

[1789] "Means for analyzing intent" refers to technology that analyzes user input and electrical signals in the brain to understand the user's intent.

[1790] "Means for performing operations" refers to the technology for executing specific operations based on the analyzed user's intentions.

[1791] The "means for adjusting the content of a message" is a technique for appropriately changing the content of a message based on the analyzed user's emotions.

[1792] MODE FOR CARRYING OUT THE INVENTION

[1793] The present invention is a system that analyzes the user's intentions and emotions and adjusts appropriate operations and messages. Specific embodiments of this system will be described below.

[1794] 1. System Configuration

[1795] The system consists of the following main components:

[1796] A chip implanted in the head or a tiny device attached to the outside of the face

[1797] A means of inputting / outputting to the auditory and visual senses using electrical signals in the brain, talking to oneself, bone conduction, sound waves, etc.

[1798] Generative AI

[1799] Mobile line

[1800] Emotion Engine

[1801] 2. Hardware and Software Use

[1802] The server analyzes the user's intent using a generative AI model (e.g., a natural language processing model), such as OpenAI's GPT-4. The emotion engine uses a sentiment analysis tool such as IBM's Watson Tone Analyzer.

[1803] The device retrieves information, makes calls, uses messaging apps, sends emails, etc. via a mobile network. Based on the user's instructions, the device receives instructions from the generative AI and performs specific operations.

[1804] 3. Program Processing

[1805] The server analyzes the user's intent and takes appropriate action. For example, if a user wants to "send an email to a friend," the generative AI will grasp that intent and perform the email sending operation. The emotion engine analyzes the user's emotions and adjusts the content of the message. For example, if a user says, "I had a lot of fun today," the emotion engine will determine that the user is feeling happy and suggest a more cheerful expression.

[1806] 4. Specific Examples

[1807] Consider the case where a user says, "I want to send an email to my friend and tell him I had a great day."

[1808] 1. The user uses the voice input function on their smartphone and says, "I want to send an email to my friend. I want to tell her I had a great day."

[1809] 2. The server converts the voice data into text and uses a generative AI model (GPT-4) to analyze the intent to "send an email."

[1810] 3. The server creates a template for sending email and sets the initial settings for the recipient, subject, and body of the email.

[1811] 4. The server uses the emotion engine to analyze that the user is feeling happy because he said, "I had a lot of fun today."

[1812] 5. Based on the results of the emotion engine, the server suggests a positive expression such as, "I had a great time today! See you again!"

[1813] 6. The device sends an email to the friend via mobile using the message received from the server. The user confirms that the email has been sent.

[1814] Prompt Sentence Examples

[1815] "Send a text to your friends. Tell them I had a great time today."

[1816] "Send an email to my boss. I want to update him on the progress of the project."

[1817] In this way, the system allows the server, terminal, and user to work together to adjust appropriate operations and messages according to the user's intentions and emotions. The flow of the identification process in the third embodiment will be described with reference to FIG.

[1818] Step 1:

[1819] The user inputs an intent through the device, for example using voice input or text input. The user says, "I want to send an email to a friend."

[1820] Input: User voice or text input

[1821] Output: Audio data or text data

[1822] Step 2:

[1823] The server converts the voice data into text using speech recognition software.

[1824] Input: Audio data

[1825] Output: Text data

[1826] Step 3:

[1827] The server uses a generative AI model (e.g., a natural language processing model) to analyze the user's intent. The server analyzes the text data and understands the intent to "send an email."

[1828] Input: Text data

[1829] Output: Analysis result (user intent)

[1830] Step 4:

[1831] The server determines the specific operation based on the analysis results. The server generates the necessary information for sending the email (recipient, subject, body, etc.). For example, the server creates a "template for sending an email to a friend."

[1832] Input: Analysis result (user intent)

[1833] Output: Email template

[1834] Step 5:

[1835] The server uses an emotion engine to analyze the user's emotions. The server infers the user's current emotions from the user's input and past data. For example, if the user says, "I had a lot of fun today," the server determines that the user is feeling happy.

[1836] Input: User input data, historical data

[1837] Output: Emotion analysis results

[1838] Step 6:

[1839] The server adjusts the content of the message based on the analysis results of the emotion engine. The server suggests expressions that match the user's emotions. For example, if the user is feeling happy, the server will suggest a cheerful expression such as "I had a great time today! See you again!"

[1840] Input: Sentiment analysis results, email template

[1841] Output: Adjusted message

[1842] Step 7:

[1843] The device actually sends the email using the adjusted message received from the server. The device sends the email via a mobile network. The user confirms that the email has been sent to their friend.

[1844] Input: Adjusted message

[1845] Output: Email sent

[1846] In this way, the system works in cooperation with the server, terminal, and user to adjust appropriate operations and messages according to the user's intentions and emotions.

[1847] (Application example 3)

[1848] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1849] Conventional food delivery systems require users to manually place orders, which is inconvenient, especially when they are busy or busy. They also fail to provide appropriate messages based on the user's emotions, which can lead to a poor user experience. There is a need to solve these problems and provide a more intuitive and emotionally sensitive food delivery experience.

[1850] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means. In this invention, the server includes a chip implanted in the head or a tiny device attached to the outside of the face; means for inputting / outputting to the auditory and visual senses using electrical signals in the brain, monologue, bone conduction, sound waves, etc.; means for conversing with a generative AI via a mobile line, obtaining necessary information, obtaining suggestions for the next action, playing music and arranging it to create new, optimal background music based on user preferences, talking to people, and sending emails without touching a device; means for analyzing the user's intentions and performing ordering operations; and means for analyzing the user's emotions and adjusting the content of messages. This allows users to intuitively order food delivery without using their hands and receive appropriate messages based on their emotions.

[1851] A "headchip" is a small electronic device that is implanted in the user's head to detect electrical signals in the brain and communicate with external devices.

[1852] The "external facial device" is a small electronic device worn on the outside of the user's face that detects electrical signals and sound waves in the brain and communicates with external devices.

[1853] "Brain electrical signals" are signals that indicate electrical activity generated by the user's brain.

[1854] "Soliloquy" refers to words that a user speaks to themselves.

[1855] "Bone conduction" is the phenomenon in which sound waves are transmitted to the inner ear through bones.

[1856] "Sound waves" are sound vibrations that travel through the air.

[1857] "Auditory and visual input / output" refers to the means by which information is provided to the user's auditory and visual senses, or by which information is received from the user.

[1858] "Generative AI" is artificial intelligence that analyzes a user's intentions and emotions and generates appropriate operations and messages.

[1859] A "mobile line" is a communication line used by mobile communication devices such as mobile phones and smartphones.

[1860] "Conversation" is the act of exchanging information between a user and a system through voice.

[1861] "Obtaining necessary information" refers to the act of the system collecting and providing the information requested by the user.

[1862] "Getting next action suggestions" is the act of the system suggesting the next action to be taken based on the user's current situation and intentions.

[1863] "Playing music" refers to the system playing music for the user.

[1864] "Background music" is music that plays in accordance with the user's activities and situations.

[1865] A "call" is an act of a user having a voice conversation with another person.

[1866] A "messenger app" is an application for sending and receiving text messages, images, and voice messages.

[1867] "Sending email" refers to the act of sending an email.

[1868] The "means for analyzing intent and performing ordering operations" refers to a means for analyzing a user's intent and placing a food delivery order based on that intent.

[1869] The "means for analyzing emotions and adjusting the content of a message" is a means for analyzing the emotions of a user and adjusting the content of a message in accordance with those emotions.

[1870] A system for implementing the present invention has the following configuration.

[1871] Hardware Configuration

[1872] 1. A chip implanted in the head or a tiny device attached to the outside of the face:

[1873] It is a small electronic device that is implanted in the user's head or worn externally on the face, which detects electrical signals and sound waves in the brain and communicates with an external device.

[1874] 2. Smartphone:

[1875] This is a device that uses mobile lines to communicate with generative AI.

[1876] Software Configuration

[1877] 1. Generative AI:

[1878] It is an artificial intelligence that analyzes the user's intentions and emotions and generates appropriate operations and messages. Specifically, it has the functions of voice recognition, intention analysis, and emotion analysis.

[1879] 2. Emotion Engine:

[1880] This software analyzes the user's emotions and adjusts the content of the message accordingly.

[1881] Data processing and calculation

[1882] 1. Speech Recognition:

[1883] It uses the smartphone's microphone to convert the user's speech into text, specifically using the SpeechRecognition library.

[1884] 2. Intent Analysis:

[1885] Generative AI analyzes what users say and understands their intent, specifically using the Transformers library.

[1886] 3. Emotion analysis:

[1887] Analyze the sentiment of a user's speech using the TextBlob library.

[1888] Specific examples

[1889] If a user says, "I want to order a pizza," the following process occurs:

[1890] 1. Speech Recognition:

[1891] It uses the smartphone's microphone to convert the user's speech into text.

[1892] 2. Intent Analysis:

[1893] The generative AI analyzes the intention of "I want to order pizza" and performs the pizza ordering operation.

[1894] 3. Emotion analysis:

[1895] The emotion engine analyzes the user's emotions and if the user is happy, it generates a message saying, "Your delicious pizza will be arriving soon!"

[1896] Prompt Sentence Examples

[1897] If a user says, "I want to order a pizza," the generative AI analyzes their intent and performs the pizza ordering operation. The emotion engine analyzes the user's emotions, and if the user is happy, it suggests the message, "Your delicious pizza will be delivered soon!"

[1898] In this way, users can intuitively order food delivery without using their hands and receive appropriate messages based on their emotions.

[1899] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[1900] Step 1:

[1901] A user says, "I want to order a pizza." The input is the user's voice, and the output is voice data. Specifically, the smartphone's microphone captures the user's voice.

[1902] Step 2:

[1903] The device performs speech recognition. The input is the speech data obtained in step 1, and the output is text data. Specifically, the device converts the speech data into text using the SpeechRecognition library.

[1904] Step 3:

[1905] The server uses generative AI to perform intent analysis. The input is the text data obtained in step 2, and the output is data indicating the user's intent. Specifically, it uses the Transformers library to analyze the text data and extracts the intent, "I want to order pizza."

[1906] Step 4:

[1907] The server performs the ordering operation based on the user's intent. The input is the user's intent data obtained in step 3, and the output is data indicating that the order has been completed. The specific operation is to call the API of a food delivery service and place an order for pizza.

[1908] Step 5:

[1909] The server performs emotion analysis using an emotion engine. The input is the text data obtained in step 2, and the output is data indicating the user's emotion. Specifically, the TextBlob library is used to analyze the text data and determine the user's emotion.

[1910] Step 6:

[1911] The server generates a message according to the emotion. The input is the user's emotion data obtained in step 5, and the output is a message according to the emotion. Specifically, if the user is happy, the server generates a message saying, "Your delicious pizza will be delivered soon!"

[1912] Step 7:

[1913] The device notifies the user of the generated message. The input is the message obtained in step 6, and the output is the notification displayed to the user. Specifically, the message is displayed using the smartphone's notification function.

[1914] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1915] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1916] Another example of generative AI is Gemini (internet search engine). <url: https: gemini.google.com ?hl="ja">) are mentioned.

[1917] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1918] [Third embodiment]

[1919] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1920] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1921] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1922] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1923] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1924] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1925] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1926] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1927] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1928] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1929] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1930] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[1931] "Example 1"

[1932] One embodiment of the present invention uses a chip implanted in the head or a tiny device attached to the outside of the face. This chip or tiny device detects electrical signals in the brain, monologue, bone conduction, sound waves, etc., and analyzes this information. Specifically, an EEG analyzer analyzes brain waves and sends the results to a generative AI. The generative AI infers the user's intention based on the analysis results and performs appropriate operations. For example, if the user thinks, "Play music," the EEG analyzer captures that intention, and the generative AI controls the music playback.

[1933] "Example 2"

[1934] Furthermore, generative AI generates and arranges music according to the user's preferences and circumstances. Specifically, it generates and arranges music taking into account the user's past listening history and current situation (e.g., time, location, mood, etc.). For example, if a user wants to relax, generative AI will capture that intention and generate and arrange relaxing music.

[1935] "Example 3"

[1936] Furthermore, the system of the present invention retrieves information, makes calls, uses messenger apps, and sends emails via mobile networks. Specifically, the generative AI analyzes the user's intention and performs appropriate operations. For example, if the user thinks, "Send an email," the generative AI will grasp that intention and perform the operation to send an email.

[1937] The processing flow of each embodiment will be described below.

[1938] "Example 1"

[1939] Step 1: A chip is implanted in the user's head or a tiny device is attached to the outside of the face.

[1940] Step 2: A chip or tiny device detects electrical signals in the brain, speech, bone conduction, sound waves, etc.

[1941] Step 3: The detected information is sent to an EEG analysis device, which analyzes the EEG.

[1942] Step 4: Send the analysis results to the generative AI.

[1943] Step 5: The generative AI infers the user's intention based on the analysis results and performs the appropriate operation. For example, if the user thinks, "Play music," the generative AI will perform the operation to play music.

[1944] "Example 2"

[1945] Step 1: The generative AI takes into account the user's past listening history and current situation (e.g., time, location, mood, etc.).

[1946] Step 2: The generative AI generates and arranges music based on the information it has taken into account. For example, if the user wants to relax, the generative AI will capture that intention and generate and arrange relaxing music.

[1947] "Example 3"

[1948] Step 1: Generative AI analyzes user intent.

[1949] Step 2: The generative AI performs the appropriate action based on the analysis results. For example, if the user thinks, "Send an email," the generative AI will capture that intent and perform the action of sending an email.

[1950] Example 1

[1951] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1952] Conventional technology requires users to manually input information to operate devices, which is difficult to do, especially when users have physical limitations or their hands are full. It is also difficult to accurately read the user's intentions and perform appropriate operations. This reduces user convenience and makes it difficult to achieve efficient operation.

[1953] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1954] In this invention, the server includes means for detecting electrical signals in the brain, monologue, bone conduction, sound waves, etc. using a chip implanted in the head or a tiny device attached to the outside of the face, an electroencephalogram (EEG) analyzer for analyzing the detected electrical signals, means for transmitting the analysis results to a generative AI, means for the generative AI to infer the user's intentions based on the analysis results and perform appropriate operations, and means for conversing via a mobile line, obtaining necessary information, obtaining suggestions for the next action, playing music and arranging it to convey a user's favorite image to play new, perfect background music, talking to people, using a messenger app, sending emails, etc., all without touching anything. This enables the user to accurately read intentions and perform appropriate operations without manually operating the device.

[1955] A "headchip" is a small electronic device that is implanted in a user's head to detect electrical signals in the brain.

[1956] The "extremely small equipment attached to the outside of the face" is a small device that is attached to the outside of the user's face and detects electrical signals in the brain, monologue, bone conduction, sound waves, etc.

[1957] "Brain electrical signals" are signals that indicate electrical activity occurring in the user's brain.

[1958] "Soliloquy" refers to words that a user speaks to himself or herself and is detected as voice.

[1959] "Bone conduction" is the phenomenon in which sound waves travel through bones to the inner ear, thereby transmitting audio information.

[1960] "Sound waves" are vibrations that travel through the air and are perceived as sound.

[1961] An "EEG analysis device" is a device that analyzes electrical signals in the brain to read the user's intentions and thoughts.

[1962] "Generative AI" is an artificial intelligence system that infers the user's intentions based on analysis results and performs appropriate operations.

[1963] A "mobile line" is a communication line for sending and receiving data using wireless communication.

[1964] A "conversation" is a spoken communication between a user and another person or system.

[1965] "Acquisition of required information" is the process of searching for and acquiring the information the user needs.

[1966] "Getting next action suggestions" is the process of suggesting the next action to be taken based on the user's current situation and intentions.

[1967] "Playing music" is the process by which a user selects and plays the music they want to listen to.

[1968] "Personal image" refers to a visual or auditory image that the user likes.

[1969] "Arrangement" is the process of modifying existing music or images to suit the user's preferences.

[1970] "BGM" is an abbreviation for background music, and refers to music played as background music.

[1971] A "call" is a process in which a user communicates with another person by voice.

[1972] A "messenger app" is application software for sending and receiving text messages.

[1973] "Mail sending" is the process of creating and sending an email.

[1974] This invention is a system that uses a chip implanted in the user's head or a tiny device attached to the outside of the face to detect electrical signals in the brain, monologue, bone conduction, sound waves, etc., and analyzes this information. Specifically, the following hardware and software are used:

[1975] Hardware and software used

[1976] Hardware: Chips implanted in the head, tiny devices attached to the outside of the face, brain wave analyzers

[1977] Software: Generative AI

[1978] Data processing and calculation

[1979] 1. The user wears a chip implanted in the head or a tiny device attached to the outside of the face, which is then ready to detect electrical signals in the brain, speech, bone conduction, sound waves, etc.

[1980] 2. A chip or tiny device detects electrical signals in the user's brain in real time. For example, if a user thinks, "Play music," the corresponding electrical signals will be detected.

[1981] 3. The device transmits the detected electrical signals to the EEG analyzer, which then transmits this data via wireless communication or other means.

[1982] 4. The EEG analyzer analyzes the received electrical signals using sophisticated algorithms and pattern recognition to interpret the user's thoughts and intentions.

[1983] 5. The EEG analysis device sends the analysis results to the generative AI, which includes information about the user's intentions.

[1984] 6. The generative AI infers the user's intention based on the analysis results received from the EEG analyzer. For example, the intention may be "Play music."

[1985] 7. The generative AI performs the appropriate action based on the inferred intent. Specifically, it launches a music playback application and plays music.

[1986] Specific examples

[1987] For example, when a user wants to "play music," the specific operation is as follows.

[1988] 1. The user puts in the chip and thinks, "Play some music."

[1989] 2. The chip detects electrical signals corresponding to the thought.

[1990] 3. The device transmits the detected electrical signal to the EEG analysis device.

[1991] 4. The EEG analyzer analyzes the electrical signals and reads the intention to "play music."

[1992] 5. The EEG analysis device sends the analysis results to the generative AI.

[1993] 6. The generative AI infers the intent, "Play music."

[1994] 7. The generative AI launches a music playback application and plays music.

[1995] Prompt Sentence Examples

[1996] An example of a prompt sentence to input to the generative AI model is as follows:

[1997] When a user thinks, "Play music," the brainwave analysis device captures that intention, and the generative AI controls the music playback.

[1998] In this way, a system is realized that can read the user's intentions and perform appropriate operations.

[1999] The flow of the identification process in the first embodiment will be described with reference to FIG.

[2000] Step 1:

[2001] The user wears a chip that is implanted in the head or a tiny device that is attached to the outside of the face.

[2002] Input: User wearing behavior

[2003] Output: With chip or tiny device attached

[2004] How it works: The user implants a chip in their head or wears a tiny device on the outside of their face, which is then ready to detect electrical signals in the brain, speech, bone conduction, sound waves, etc.

[2005] Step 2:

[2006] A chip or tiny device detects electrical signals in the brain.

[2007] Input: Electrical signals in the user's brain

[2008] Output: Detected electrical signal data

[2009] What it does: When a user thinks "play music," an electrical signal corresponding to that thought is detected by a chip or tiny piece of equipment.

[2010] Step 3:

[2011] The terminal transmits the detected electrical signal to an electroencephalogram analysis device.

[2012] Input: Detected electrical signal data

[2013] Output: Electrical signal data sent to EEG analyzer

[2014] Specific operation: The terminal receives the electrical signals detected by the chip or tiny device and transmits them to the EEG analysis device using wireless communication or other means.

[2015] Step 4:

[2016] An electroencephalogram (EEG) analyzer analyzes the electrical signals.

[2017] Input: Transmitted electrical signal data

[2018] Output: Parsed intent data

[2019] Specific operation: The EEG analyzer analyzes the received electrical signals and performs pattern recognition to read the user's thoughts and intentions. For example, the intention "play music" is analyzed.

[2020] Step 5:

[2021] The EEG analysis device sends the analysis results to the generative AI.

[2022] Input: Parsed intent data

[2023] Output: Intent data sent to the generative AI

[2024] How it works: The EEG analyzer sends the analysis results to the generative AI, which includes information about the user's intentions.

[2025] Step 6:

[2026] Generative AI infers the user's intentions based on the analysis results.

[2027] Input: Submitted intent data

[2028] Output: Inferred intent

[2029] Specific operation: The generative AI infers the user's intention based on the analysis results received from the EEG analysis device. For example, the intention may be "Play music."

[2030] Step 7:

[2031] The generative AI performs appropriate operations based on the inferred intent.

[2032] Input: Inferred intent

[2033] Output: The operation performed

[2034] Specific behavior: Based on the inferred intent, the generative AI launches a music playback application and performs the operation of playing music.

[2035] (Application example 1)

[2036] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[2037] Conventional autonomous vehicle operation requires the user to manually set the destination, making operation complicated. Although interfaces using brainwaves and voice exist, technology that accurately estimates the user's intentions and reflects them in the operation of an autonomous vehicle is still in its infancy. This has resulted in low user convenience and made intuitive operation difficult.

[2038] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[2039] In this invention, the server includes means for detecting electrical signals in the brain, monologue, bone conduction, sound waves, etc. using a chip implanted in the head or a tiny device attached to the outside of the face, means for conversing with the generative AI via a communication line, obtaining necessary information, obtaining suggestions for the next action, playing music and arranging it to convey the user's favorite image to play the best new background music, talking to people, messaging apps, sending emails, etc., all without touching anything, and means for estimating the user's intentions and operating the self-driving vehicle, which allows the user to intuitively operate the self-driving vehicle.

[2040] A "headchip" is a small electronic device that is implanted in the user's head to detect electrical signals in the brain.

[2041] The "external facial equipment" is a small device that is worn on the outside of the user's face to detect electrical signals and sound waves in the brain.

[2042] "Electrical signals in the brain" are weak electrical fluctuations that occur when the brain is active, and by analyzing these, it is possible to infer the user's intentions and state.

[2043] "Muttering to oneself" refers to words that a user utters unconsciously in a low voice, and by detecting this, it is possible to understand the user's intentions.

[2044] "Bone conduction" is a phenomenon in which sound waves are transmitted to the inner ear through the skull, and is a technology that utilizes this phenomenon to obtain audio information.

[2045] "Sound waves" are vibrations that travel through the air, and by detecting them, audio information can be obtained.

[2046] "Generative AI" is an artificial intelligence system that generates appropriate responses and operations based on the user's intentions and requests.

[2047] "Communication lines" refer to network infrastructure for sending and receiving data, and include mobile lines and internet lines.

[2048] An "autonomous vehicle" is a vehicle that uses artificial intelligence and sensor technology to drive autonomously.

[2049] "Estimating user intent" means analyzing data such as brain waves and voice to determine what the user wants.

[2050] "Performing an operation" means controlling a device or system based on the user's intentions.

[2051] The following system configuration is used as an embodiment of this invention: The system is composed of a chip to be implanted in the head or a tiny device to be attached to the outside of the face, an EEG analyzer, a generative AI, a communication line, and a control system for an autonomous vehicle.

[2052] First, the user wears a chip implanted in the head or a tiny device attached to the outside of the face, which detects electrical signals in the brain, speech, bone conduction, sound waves, etc. This data is sent to an electroencephalogram (EEG) analyzer and analyzed to infer the user's intentions.

[2053] The analyzed data is sent via communication lines to a generative AI, which generates appropriate responses and operations based on the user's intentions. This generative AI also has the ability to generate and arrange music according to the user's preferences and circumstances.

[2054] The operation commands generated by the generative AI are then sent to the autonomous vehicle's control system, allowing the autonomous vehicle to set the destination and begin navigation simply by the user thinking, "Go to the destination."

[2055] Specific hardware features include the Emotiv Epoc+ brainwave analyzer, Google Glass smart glasses, and Tesla Autopilot for the autonomous vehicle control system. Software features include the Emotiv SDK brainwave analysis library, OpenAI GPT-4 generative AI model, and Tesla API for the autonomous vehicle control API.

[2056] As a concrete example, consider the case where a user puts on smart glasses and thinks, "I want to go home." In this case, the EEG analyzer captures the user's intention, and the generative AI generates the command "go home." The self-driving vehicle then sets "home" as the destination and begins navigation.

[2057] An example prompt might look like this:

[2058] "If a user thinks, 'I want to go home,' the EEG analyzer will capture that intention, the generative AI will generate the command 'go home,' and the self-driving vehicle will set the destination to 'home' and begin navigation."

[2059] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[2060] Step 1:

[2061] The user wears a chip implanted in the head or a tiny device attached to the outside of the face, which detects electrical signals in the brain, speech, bone conduction, sound waves, etc. The input is the electrical signals and sound waves in the user's brain, and the output is these signal data.

[2062] Step 2:

[2063] The device sends the detected signal data to an EEG analysis device, which analyzes the data and infers the user's intention. The input is signal data, and the output is analyzed intention data. Specifically, the data is analyzed using an EEG analysis library (e.g., Emotiv SDK).

[2064] Step 3:

[2065] The server sends the analyzed intent data to the generative AI, which generates appropriate responses and operation commands based on the user's intent. The input is the intent data, and the output is the operation command. Specifically, the response is generated using a generative AI model (e.g., OpenAI GPT-4).

[2066] Step 4:

[2067] The server sends the generated operation command to the autonomous vehicle control system. The autonomous vehicle control system operates the vehicle based on the received command. The input is the operation command, and the output is the vehicle's behavior. The specific operation is to control the vehicle using the autonomous vehicle control API (e.g., Tesla API).

[2068] Step 5:

[2069] If a user thinks, "I want to go home," the EEG analyzer captures that intention, the generative AI generates the command "go home," and the self-driving vehicle sets the destination to "home" and begins navigation. The input is the user's intention, and the output is setting the destination and starting navigation. Specifically, the navigation system sets the destination, and the vehicle begins moving automatically.

[2070] Example 2

[2071] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[2072] Conventional music generation systems have had difficulty generating and arranging music in real time according to the user's preferences and circumstances. Furthermore, manual operation was required for the user to communicate their intentions to the system, which was inconvenient. Furthermore, there was a lack of means to effectively utilize the user's past listening history and current circumstances, making it difficult to provide optimal music for each individual user.

[2073] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[2074] In this invention, the server includes means for collecting the user's past listening history and current situation (time, place, mood, etc.), means for transmitting the collected data to the server and storing it in a database, means for generating prompt sentences to be input to the generative AI model based on the stored data, means for inputting the prompt sentences to the generative AI model and generating and arranging music using music generation software, and means for transmitting the generated music to the user's terminal and playing it. This makes it possible to generate and arrange music in real time according to the user's preferences and situation, and provide optimal music without the user having to perform manual operations.

[2075] A "head chip or external facial device" is a small electronic device placed on the user's head or face to detect and transmit electrical signals and sound waves in the brain.

[2076] "Means for inputting / outputting information to the auditory and visual senses using electrical signals in the brain, monologue, bone conduction, sound waves, etc." refers to technology for inputting or outputting information to the auditory and visual senses using electrical signals and sound waves generated in the user's brain.

[2077] A "generative AI model" is an artificial intelligence model that generates and arranges music and other content based on user preferences and circumstances.

[2078] "Communication lines" refers to network infrastructure for sending and receiving data, including mobile lines and internet lines.

[2079] "Means for collecting a user's past listening history and current situation (time, location, mood, etc.)" refers to technology for collecting information such as the user's history of music they have listened to in the past, as well as their current time, location, mood, etc.

[2080] "Means for transmitting collected data to a server and storing the data in a database" refers to a technique for transmitting data collected from users to a server and storing the data in a database.

[2081] "Means for generating prompt sentences to be input to a generative AI model based on stored data" refers to a technology that generates instruction sentences (prompt sentences) to be input to a generative AI model based on user information stored in a database.

[2082] "Music generation software" means software for generating and arranging music based on instructions from a generative AI model.

[2083] "Means for transmitting the generated music to a user's device and playing it" refers to technology for transmitting music generated by a generative AI model to a user's device and playing the music on that device.

[2084] This invention is a system for generating and arranging music according to user preferences and circumstances. The system includes a user terminal, a server, a generative AI model, music generation software, and a database management system.

[2085] Hardware and software used

[2086] Hardware: Servers, user devices (smartphones, PCs, etc.)

[2087] Software: Generative AI models (e.g., GPT-3), database management systems (e.g., MySQL), music generation software

[2088] System Operation

[2089] 1. Collection of User Data

[2090] When a user uses a music streaming app, the device collects past listening history and current situation (time, location, mood, etc.).

[2091] Specifically, the device collects the history of songs played by the user, current GPS location information, and data obtained from the device's built-in sensors (e.g., heart rate, number of steps).

[2092] For example, if the user feels like relaxing in the evening, the terminal will record that information.

[2093] 2. Data transmission and storage

[2094] The terminal periodically transmits the collected data to the server.

[2095] The server stores the received data in a database management system (e.g. MySQL).

[2096] Specifically, the server stores the user ID, listening history, and current situation data in a database.

[2097] For example, the data for user ID "12345" may include information such as "nighttime" and "want to relax."

[2098] 3. Prompt generation

[2099] Based on the stored data, the server generates prompt sentences to be input into the generative AI model.

[2100] Specifically, the server obtains the user's current situation from a database and creates a prompt sentence based on that situation.

[2101] For example, a prompt sentence such as "User ID 12345 feels like relaxing, so please create some relaxing music" is generated.

[2102] 4. Music Creation and Arrangement

[2103] The server inputs prompts into a generative AI model (e.g., GPT-3), which then uses music generation software to generate and arrange music.

[2104] Specifically, the generative AI model analyzes the prompt and determines appropriate musical characteristics (such as tempo, melody, and instrumentation).

[2105] Music generation software then generates music based on these features.

[2106] For example, a piano piece with a slow tempo is generated as relaxing music.

[2107] 5. Distribution of generated music

[2108] The server transmits the generated music to the user's terminal.

[2109] The terminal then plays the received music for the user to listen to.

[2110] Specifically, the music player application on the device plays the generated music file.

[2111] For example, if the user feels like relaxing in the evening, the terminal will play a generated piano piece.

[2112] This system allows users to enjoy music in real time according to their preferences and circumstances.

[2113] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2114] Step 1:

[2115] User Data Collection

[2116] When a user uses a music streaming app, the device collects past listening history and current situation (time, location, mood, etc.).

[2117] Input: User playback history, GPS location, data from built-in sensors (heart rate, steps, etc.)

[2118] Data processing: Organize the collected data and identify the user's current situation.

[2119] Output: Organized user data (e.g. user ID, play history, current time, location, mood)

[2120] What it does: The device records the music history of the user, obtains the user's current GPS location, and collects heart rate and step counts from built-in sensors. For example, if the user feels like relaxing in the evening, it will record that information.

[2121] Step 2:

[2122] Data transmission and storage

[2123] The terminal periodically transmits the collected data to the server.

[2124] The server stores the received data in a database management system.

[2125] Input: Organized user data

[2126] Data processing: Converting data into a format suitable for storing in a database.

[2127] Output: User data stored in the database

[2128] Specific operation: The device sends the organized user data to the server, which then stores the data in a database. For example, for user ID "12345," the server stores information such as "evening time" and "want to relax."

[2129] Step 3:

[2130] Generate prompt statement

[2131] Based on the stored data, the server generates prompt sentences to be input into the generative AI model.

[2132] Input: User data stored in the database

[2133] Data processing: Analyze the user's current situation and create a prompt suitable for the generative AI model.

[2134] Output: Generated prompt statement

[2135] Specific operation: The server retrieves the user's current situation from the database and creates a prompt based on it. For example, it generates a prompt such as "User ID 12345 feels like relaxing, so please create some relaxing music."

[2136] Step 4:

[2137] Music Creation and Arrangement

[2138] The server inputs prompts into the generative AI model, which then uses music generation software to generate and arrange the music.

[2139] Input: Generated prompt statement

[2140] Data processing: The generative AI model analyzes the prompt and determines musical characteristics. Music generation software generates music based on these characteristics.

[2141] Output: Generated music file

[2142] How it works: The generative AI model analyzes the prompt and determines appropriate musical characteristics (tempo, melody, instrumentation, etc.). The music generation software then generates music based on these characteristics. For example, it might generate a piano piece with a slow tempo for relaxation.

[2143] Step 5:

[2144] Generated Music Distribution

[2145] The server transmits the generated music to the user's terminal.

[2146] The terminal then plays the received music for the user to listen to.

[2147] Input: Generated music file

[2148] Data processing: Convert music files into a format suitable for your device.

[2149] Output: Music files playable on your device

[2150] Specific operation: The server sends the generated music file to the user's device, and the device's music player app plays the music file. For example, if the user feels like relaxing at night, the device will play the generated piano music.

[2151] (Application example 2)

[2152] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[2153] Conventional music playback systems have difficulty generating and arranging music based on the user's preferences and current situation, and are unable to respond to specific needs such as when users want to relax, concentrate, or exercise. Furthermore, there are problems with users having to go through the trouble of selecting music, and it takes time to find the perfect piece of music. Furthermore, there has been a lack of systems that can generate and arrange music taking into account the user's past listening history and current situation.

[2154] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[2155] In this invention, the server includes a chip implanted in the head or a tiny device attached to the outside of the face, means for inputting and outputting to the auditory and visual senses using electrical signals in the brain, monologue, bone conduction, sound waves, etc., means for conversing with the generative AI via a mobile line, obtaining necessary information, obtaining suggestions for the next action, playing music and arranging it to convey one's favorite images to play the best new background music, talking to people, and sending messeng...

Claims

1. A means for detecting electrical signals in the user's brain from at least one of a chip implanted in the head and a tiny device attached to the outside of the face; A means for analyzing the detected electrical signal and transmitting it to an emotion engine; a means for estimating an emotional state of the user from an analysis result of the electrical signal by the emotion engine; A means for generating music based on the user's past listening history and the user's emotional state by a generative AI model; means for playing the generated music; A system including:

2. The transmitting means analyzes the user's brain waves from the electrical signals. The system of claim 1 .

3. The generating means uses the generative AI model to arrange the music based on the preferences and the emotional state of the user. The system of claim 1 .

Images