System

Abstract

To provide a system.SOLUTION: A system includes: means for acquiring brain wave data of a user in real time; means for analyzing the acquired brain wave data to extract intention of the user; means for analyzing a state of feeling of the user; and means for executing output using a generative AI, based on the extracted intention of the user, and a prompt text according to the analyzed state of feeling of the user.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional brain-machine interfaces (BMIs) have faced a dilemma between accuracy and invasiveness, limiting their widespread adoption and limiting the provision of new services using BMIs. [Means for solving the problem]

[0005] This invention provides a means for generating minimally invasive, highly accurate BMIs by combining EEG and generative AI, and a means for providing a platform for popularizing this BMI throughout society. Furthermore, it also provides a means for providing new services, such as intuitive communication services that utilize brain activity, metaverse interfaces, health management services, and biometric authentication services, through this platform. This will enable the popularization of BMIs and the provision of new services. [Brief explanation of the drawings]

[0006] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 2 is a sequence diagram showing a flow of processing in the data processing system according to the first embodiment of the first form example. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Embodiment 1. [Figure 13] FIG. 10 is a sequence diagram showing a processing flow of a data processing system in a second embodiment of the second form example. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 of Embodiment 2. [Figure 15]FIG. 10 is a sequence diagram showing the flow of processing in a data processing system according to a third embodiment of the third embodiment. [Figure 16] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 3 of Embodiment 3. [Figure 17] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the first embodiment of the first form example when an emotion engine is combined. [Figure 18] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Form Example 1 when an emotion engine is combined. [Figure 19] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the second embodiment of the second form example when an emotion engine is combined. [Figure 20] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 of Form Example 2 when an emotion engine is combined. [Figure 21] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the third embodiment of the third form example when an emotion engine is combined. [Figure 22] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 3 of Form Example 3 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0007] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0008] First, the terms used in the following description will be explained.

[0009] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be one arithmetic device or a combination of multiple arithmetic devices. Also, the processor may be one type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), Examples include a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (TENSOR PROCESSING UNIT (registered trademark)).

[0010] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0011] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0012] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0013] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0014] [First embodiment]

[0015] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0016] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0017] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0018] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0019] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0020] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0021] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0022] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0023] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0024] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0025] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0026] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0027] "Example 1"

[0028] One embodiment of the present invention is a system that generates a brain-machine interface (BMI) that combines brain waves and generative AI. This system measures the user's brain waves in real time.

[0029] The brainwave data is analyzed by a system and the results are input into a generative AI. The generative AI understands the user's intention based on this brainwave data and generates output based on that. For example, when a user thinks, "I want to drink coffee," the generative AI analyzes their brainwaves, understands their intention, and can take action such as "making coffee."

[0030] "Example 2"

[0031] One embodiment of the present invention is a platform for popularizing BMI in society. This platform has a function that allows users to manage their own EEG data and share it with third parties as needed. For example, users can manage their own EEG data on the platform and share it with medical institutions and research institutions, thereby enabling more accurate medical services and research results.

[0032] "Example 3"

[0033] Furthermore, one embodiment of the present invention provides new services through the platform. These services include an intuitive communication service that utilizes brain activity, a metaverse interface, a health management service, and a biometric authentication service. For example, the intuitive communication service analyzes a user's brain waves to understand their emotions and intentions and communicate based on that information. The metaverse interface analyzes a user's brain waves to control their behavior in the VR or AR world. The health management service analyzes a user's brain waves to understand their stress level, sleep state, and other factors and proposes health management based on that information. The biometric authentication service analyzes a user's brain wave patterns to perform personal authentication.

[0034] The processing flow of each embodiment will be described below.

[0035] "Example 1"

[0036] Step 1: The user has a specific thought or intention.

[0037] Step 2: The system captures and analyzes the user's brainwaves in real time.

[0038] Step 3: Input the analysis results into the generative AI.

[0039] Step 4: The generative AI understands the user's intention based on the EEG data and generates an output based on that. For example, it takes an action such as "make coffee."

[0040] "Example 2"

[0041] Step 1: The user accesses the platform and uploads their EEG data.

[0042] Step 2: The platform manages the user's EEG data and shares it with third parties as necessary. For example, sharing it with medical institutions or research institutions will enable more accurate medical services and research results.

[0043] "Example 3"

[0044] Step 1: The user selects a specific service through the platform, such as an intuitive communication service, a metaverse interface, a health management service, or a biometric authentication service.

[0045] Step 2: The selected service analyzes the user's brainwaves and provides services based on them. For example, an intuitive communication service analyzes the user's brainwaves to understand their emotions and intentions, and communicates based on that.

[0046] Example 1

[0047] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0048] Conventional brain-machine interfaces (BMIs) have low accuracy in analyzing the user's brain waves, making it difficult to understand the user's intentions in real time. Furthermore, these systems are highly invasive and place a heavy burden on the user. Furthermore, it is not easy to manage and share brain wave data, which has hindered their widespread adoption in society.

[0049] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0050] In this invention, the server includes: means for collecting a user's brain waves in real time; means for analyzing the collected brain wave data; means for inputting the analyzed brain wave data into a generative AI model; means for the generative AI model to understand the user's intention and generate output based on that understanding; means for feeding back the generated output to the user; means for providing a platform for popularizing the brain-machine interface (BMI) in society; and means for providing intuitive communication services, virtual space interfaces, health management services, and biometric authentication services that utilize brain activity through the platform. This enables highly accurate and minimally invasive analysis of a user's brain waves and understanding of intentions in real time. Furthermore, it facilitates the management and sharing of brain wave data, promoting its widespread use in society.

[0051] "User" refers to an individual who uses a Brain-Machine Interface (BMI).

[0052] "Eeg" refers to the electrical signals generated by the user's brain.

[0053] "Real-time" refers to data being processed and analyzed as it is generated.

[0054] "Means for collection" refers to devices and methods for detecting the user's brain waves and acquiring them as data.

[0055] "Means for analyzing" refers to a method or device for processing collected EEG data and inferring the user's intentions.

[0056] A "generative AI model" refers to an artificial intelligence model that understands the user's intent based on input data and generates output based on that.

[0057] "Means of input" refers to the method or device for providing analyzed EEG data to a generative AI model.

[0058] "Output" refers to the results or suggestions that a generative AI model generates by understanding the user's intent.

[0059] "Feedback means" refers to the method or device used to communicate the generated output to the user.

[0060] "Platform" refers to the systems and services that form the foundation for popularizing brain-machine interfaces (BMIs) in society.

[0061] "Communication service" refers to a service that uses brain waves to enable users to intuitively exchange information with others.

[0062] A "virtual space interface" refers to an interface that allows users to operate and move within a virtual space using their brain waves.

[0063] "Health management service" refers to a service that uses a user's brainwave data to monitor and manage their health condition.

[0064] "Biometric authentication service" refers to a service that uses a user's brainwave data to authenticate an individual.

[0065] This invention is a brain-machine interface (BMI) system that analyzes a user's brain waves in real time and inputs the results into a generative AI model to understand the user's intentions and generate output based on those intentions. This system is implemented using the following hardware and software.

[0066] Hardware and Software Configuration

[0067] Brainwave sensor

[0068] The user wears an EEG sensor on their head, which detects the user's brain waves in real time and collects the data as electrical signals.

[0069] server

[0070] The server receives the EEG data sent from the EEG sensor, which is then sent to the device for analysis.

[0071] Terminal

[0072] The terminal analyzes the brain wave data sent from the server. The analysis uses an electroencephalogram analysis library using Python (registered trademark). The terminal then sends the analysis results to the server.

[0073] Generative AI Model

[0074] The server inputs the analyzed EEG data into a generative AI model (e.g., OpenAI's GPT-4®). The generative AI model understands the user's intent based on the input data and generates output based on that understanding.

[0075] feedback

[0076] The generated output is sent to the device via the server, and the device provides it as feedback to the user, who can then take the suggested action.

[0077] Specific examples

[0078] For example, consider the case where a user wants to "listen to music." The EEG sensor collects the user's EEG data, and the device analyzes the data. If the analysis result indicates the user's intent to "listen to music," the following prompt sentence is input to the generative AI model:

[0079] The user's EEG data has detected the intent "I want to listen to music." Please suggest an action to play appropriate music.

[0080] Based on this prompt, the generative AI model generates the output "play music" and sends it back to the user via their device. The user can then play the music according to the suggested action.

[0081] In this way, the system can analyze the user's brain waves with high accuracy and minimal invasiveness, enabling them to understand their intentions in real time. It also makes it easier to manage and share brain wave data, promoting its widespread adoption in society.

[0082] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0083] Step 1:

[0084] The user wears an EEG sensor on their head. The EEG sensor detects the user's brain waves in real time and collects data as electrical signals. The collected EEG data is then sent to a server.

[0085] Input: User's brainwaves

[0086] Output: EEG data (electrical signal)

[0087] Step 2:

[0088] The server receives the EEG data transmitted from the EEG sensor, which is then sent to the device for analysis.

[0089] Input: EEG data (electrical signal)

[0090] Output: EEG data (transferred from server to device)

[0091] Step 3:

[0092] The device receives the EEG data sent from the server. The device analyzes the EEG data using a Python-based EEG analysis library. During the analysis, specific EEG patterns are detected and the user's intentions are inferred. The analysis results are then sent to the server.

[0093] Input: EEG data (transferred from the server)

[0094] Output: Analysis results (user intent)

[0095] Step 4:

[0096] The server receives the analysis results sent from the device, generates a prompt sentence based on the analysis results, and inputs it into the generative AI model.

[0097] Input: Analysis result (user intent)

[0098] Output: prompt statement

[0099] Step 5:

[0100] The generative AI model receives the prompt sent from the server. The generative AI model understands the user's intent based on the prompt and generates output based on that. The generated output is then sent to the server.

[0101] Input: prompt statement

[0102] Output: Output

[0103] Step 6:

[0104] The server receives the output sent from the generative AI model and sends the output to the device.

[0105] Input: Output (transferred from generative AI model)

[0106] Output: Output (transfer from server to device)

[0107] Step 7:

[0108] The terminal receives the output sent from the server, and feeds the output back to the user, who can then take the suggested action.

[0109] Input: Output (transfer from server)

[0110] Output: Feedback to the user

[0111] (Application example 1)

[0112] Next, a description will be given of Application Example 1 of Embodiment Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0113] Conventional factory robot control methods mainly involve manual operation or program control, which poses the problem of difficulty in intuitive operation. Furthermore, in order to improve work efficiency, a more intuitive and rapid control method is required. Furthermore, brain-machine interfaces (BMIs) that use brain waves are being increasingly applied in the fields of medicine and entertainment, but have not yet been fully utilized in factory automation or robot control. To solve these issues, a system that can intuitively control factory robots using brain waves is needed.

[0114] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0115] In this invention, the server includes: means for generating a minimally invasive, highly accurate brain-machine interface (BMI) that combines brainwaves and generative AI; means for providing a platform for popularizing the BMI throughout society; means for providing intuitive communication services, metaverse interfaces, health management services, and biometric authentication services that utilize brain activity, all of which are provided through the platform; means for factory workers to wear head-mounted displays and control factory robots using brainwaves; means for analyzing the user's brainwave data in real time and inputting the results into generative AI to understand the user's intentions and generate output based on those intentions; and means for controlling factory robots based on the user's intentions, thereby enabling factory workers to intuitively control factory robots using brainwaves.

[0116] "Electroencephalograms" are electrical signals generated by neural activity in the brain.

[0117] "Generative AI" is a type of artificial intelligence that generates new data and output based on input data.

[0118] A brain-machine interface (BMI) is an interface that uses brain waves to control machines and computers.

[0119] "Minimally invasive" refers to minimal strain or damage to the body.

[0120] "High precision" means very accurate.

[0121] A "platform" is a system or environment that serves as the foundation for providing specific services or functions.

[0122] An "intuitive communication service" is a service that allows users to operate and communicate intuitively.

[0123] A "metaverse interface" is an interface for performing activities and operations in a virtual space.

[0124] "Health management service" refers to a service for managing and improving the user's health condition.

[0125] "Biometric authentication service" refers to a service that authenticates individuals using biometric information such as fingerprints or facial recognition.

[0126] A "factory worker" is a worker engaged in production activities in a factory.

[0127] A "head-mounted display" is a display device that is worn on the head.

[0128] A "factory robot" is a robot used to automate production work in a factory.

[0129] "Real-time" refers to processing or reaction occurring immediately, without delay.

[0130] "User intent" refers to what the user wants to do or what they are thinking.

[0131] "Output" is the result or action that is produced by a system or device.

[0132] To implement the present invention, the following system configuration and program are required.

[0133] System Configuration

[0134] 1. Hardware

[0135] Head-mounted display (HMD): Equipped with a built-in brainwave sensor, it collects brainwave data in real time.

[0136] Factory robot: A robot that performs actions based on the user's intentions.

[0137] Server: Analyzes EEG data and runs generative AI models.

[0138] 2. Software

[0139] BrainwaveReader: A module for collecting EEG data.

[0140] Generative AI Model: A generative AI model that analyzes brainwave data and understands user intent.

[0141] RobotController: A module for controlling a robot based on the user's intentions.

[0142] Program processing explanation

[0143] The server first collects brainwave data from a head-mounted display (HMD). The BrainwaveReader module analyzes this data in real time and inputs it into a generative AI model (GenerativeAIModel). The generative AI model interprets the user's intention based on the brainwave data and passes that intention to the RobotController. The RobotController controls the factory robot based on the interpreted intention and performs the corresponding action.

[0144] Specific examples

[0145] For example, if a factory worker thinks, "I'm going to assemble parts," the following prompt sentence is input into the generative AI model:

[0146] Example prompt sentence:

[0147] Analysis of the user's brainwave data reveals that the user is thinking "assemble parts." Based on this intent, instruct the robot to assemble the parts.

[0148] Based on this prompt, the generative AI model interprets the user's intention and instructs the RobotController to "assemble the parts," which causes the factory robot to automatically begin assembling the parts.

[0149] This system allows factory workers to intuitively control factory robots using their brain waves, greatly improving work efficiency.

[0150] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0151] Step 1:

[0152] The user wears a head-mounted display (HMD). The HMD has a built-in brainwave sensor that collects the user's brainwave data in real time. The input is the user's brainwaves, and the output is the collected brainwave data.

[0153] Step 2:

[0154] The server uses the BrainwaveReader module to receive EEG data transmitted from the HMD. BrainwaveReader analyzes this data in real time, removing noise and extracting features. The input is EEG data from the HMD, and the output is the analyzed EEG data.

[0155] Step 3:

[0156] The server inputs the analyzed EEG data into the GenerativeAIModel, which is a generative AI model for interpreting the user's intentions based on EEG data. The input is the analyzed EEG data, and the output is data that indicates the user's intentions.

[0157] Step 4:

[0158] The server generates a prompt based on the results of its interpretation of the user's intent. This prompt specifically describes the user's intent. The input is data that indicates the user's intent, and the output is the prompt.

[0159] Step 5:

[0160] The server passes the generated prompt sentence to the RobotController, which generates specific instructions for the factory robot based on this prompt sentence. The input is the prompt sentence, and the output is the instruction for the robot.

[0161] Step 6:

[0162] The RobotController sends instructions to the factory robots, which then perform the corresponding actions based on these instructions. The input is the instruction to the robot, and the output is the robot's action.

[0163] Step 7:

[0164] The user checks the operation of a factory robot through an HMD. The HMD displays the robot's operating status in real time, allowing the user to check whether the robot is operating as intended. The input is the robot's operating status, and the output is the image displayed on the HMD.

[0165] Example 2

[0166] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0167] Conventional brain-machine interfaces (BMIs) lacked a platform for effectively managing users' EEG data and safely sharing it with third parties. Furthermore, real-time analysis of EEG data and integration with generative AI were insufficient, making it difficult to accurately understand the user's intentions and generate appropriate output. Furthermore, data security was not ensured due to insufficient encryption of EEG data and management of access rights.

[0168] The specific processing by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes a means for uploading EEG data acquired by a user using a dedicated measuring device from a terminal to the server, a means for the server to store the EEG data received in a database, a means for the user to access the platform and manage the data, a means for the user to set up data sharing with a third party, and a means for the server to encrypt the data and grant access rights to the third party. This allows the user to effectively manage their own EEG data and safely share it with a third party. Furthermore, by linking with generative AI, the system can accurately understand the user's intentions and generate appropriate output.

[0169] A "brain-machine interface (BMI)" is an interface that uses brain waves to communicate the user's intentions to a machine.

[0170] "Generative AI" is artificial intelligence that analyzes a user's brainwave data and generates appropriate output based on the results.

[0171] The "platform" is a system that allows users to manage their EEG data and share it with third parties as needed.

[0172] "Electroencephalogram data" refers to data obtained by measuring the user's brain activity, and is obtained using an electroencephalogram measuring device.

[0173] A "measuring device" is a device for measuring a user's brain waves in real time.

[0174] A "terminal" is a device such as a computer or smartphone that allows a user to upload EEG data acquired using a measurement device to a server.

[0175] A "server" is a computer system for storing and managing brain wave data received from a user.

[0176] The "database" is a system for efficiently storing and accessing the EEG data received by the server.

[0177] "Encryption" is a technology that converts data into a format that cannot be deciphered by third parties.

[0178] "Access authority" is a function that sets the rights that allow specific users or third parties to access data.

[0179] "Output" refers to the output generated by the generative AI based on the results of analyzing the user's brainwave data.

[0180] MODE FOR CARRYING OUT THE INVENTION

[0181] This invention provides a system that provides a platform for users to manage EEG data acquired using a dedicated measuring device and share it with third parties as needed. Specific embodiments of this system are described below.

[0182] Hardware and Software Configuration

[0183] The user acquires EEG data using a dedicated EEG measuring device, which measures the user's EEG in real time and transmits the data to a device (such as a PC or smartphone).

[0184] The device uploads the acquired EEG data to a server via a dedicated application, which converts the data into an appropriate format (e.g., CSV or JSON) and transmits it securely to the server using the HTTPS protocol.

[0185] The server stores the received brainwave data in a database (for example, MySQL (registered trademark) or PostgreSQL). The database organizes the data for each user and sets an index to enable efficient access.

[0186] Users access the platform through a web browser or mobile application. The platform authenticates users (e.g., via OAuth or JWT) and provides an interface where users can view, edit, and delete their own EEG data.

[0187] Data Sharing and Encryption

[0188] If a user wants to share data with a third party, they select the third party (e.g., a medical institution or research institution) they want to share with on the platform, set the scope and period of data to be shared, and submit a sharing request.

[0189] The server receives the user's sharing request, encrypts the data to be shared, and grants access privileges to the designated third party, generating a secure access link to the shared data.

[0190] Specific examples

[0191] For example, if a user wants to share their EEG data with a medical institution, they follow the steps below.

[0192] 1. The user wears the EEG sensor and launches the dedicated application.

[0193] 2. The device converts the acquired EEG data into CSV format and uploads it to the server using HTTPS.

[0194] 3. The server stores the received data in a MySQL database.

[0195] 4. The user logs into the platform using a web browser and checks their own EEG data.

[0196] 5. The user selects a medical institution and sets the scope and period of data to be shared.

[0197] 6. The server encrypts the data with AES-256 and grants access privileges to the medical institution.

[0198] Example prompts to input to a generative AI model:

[0199] "Please explain the steps a user takes to use the EEG sensor to obtain EEG data and share it with a medical institution."

[0200] By inputting this prompt into the generative AI model, the model outputs specific steps for the user to obtain EEG data and share it with a medical institution.

[0201] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0202] Step 1:

[0203] The user acquires EEG data using a dedicated EEG measurement device. The user wears the measurement device and launches a dedicated application. The measurement device measures the user's EEG in real time and transmits the data to a terminal. The input is the user's EEG, and the output is the EEG data transmitted to the terminal.

[0204] Step 2:

[0205] The device uploads the EEG data to the server. The device converts the acquired EEG data into CSV format using a dedicated application and sends it to the server using the HTTPS protocol. The input is the EEG data sent from the measurement device, and the output is the CSV-formatted EEG data uploaded to the server.

[0206] Step 3:

[0207] The server stores the received EEG data in a database. The server imports the received CSV format EEG data into the database and organizes the data for each user. The input is the CSV format EEG data sent from the device, and the output is the EEG data stored in the database.

[0208] Step 4:

[0209] Users access the platform and manage their data. They log in to the platform through a web browser or mobile application to view, edit, and delete their own EEG data. The input is the user's login information and the EEG data stored in the database, and the output is the EEG data that the user viewed, edited, or deleted.

[0210] Step 5:

[0211] The user sets up data sharing with third parties. The user selects the third parties with whom they want to share data on the platform and sets the scope and period of data to be shared. The input is the user's sharing setting information, and the output is a sharing request sent to the server.

[0212] Step 6:

[0213] The server encrypts the data and grants access permissions to the third party. The server receives the user's sharing request and encrypts the data to be shared with AES-256. The server grants access permissions to the specified third party and generates a secure access link to the shared data. The input is the user's sharing request and the EEG data stored in the database, and the output is the encrypted data and the access permissions granted to the third party.

[0214] (Application example 2)

[0215] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0216] Conventional brain-machine interfaces (BMIs) lacked a platform for effectively managing users' EEG data and sharing it with third parties as needed. Furthermore, there was insufficient management of EEG data security and sharing history, making it difficult to smoothly share data with medical and research institutions. Furthermore, there was a lack of functionality to collect users' EEG data in real time, encrypt it, and upload it to the cloud, which meant that data security and convenience were not ensured.

[0217] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for generating a minimally invasive, highly accurate brain-machine interface (BMI) that combines electroencephalograms and generative AI; means for providing a platform for popularizing the BMI throughout society; means for providing intuitive communication services using brain activity, a metaverse interface, health management services, and biometric authentication services provided through the platform; means for collecting a user's electroencephalogram data in real time, encrypting it, and uploading it to the cloud; and means for sharing the data with medical institutions and research institutions and managing the data sharing history. This makes it possible to safely and efficiently manage a user's electroencephalogram data and share it with third parties as needed.

[0218] "Electroencephalograms" are electrical signals generated by neural activity in the brain.

[0219] "Generative AI" is a type of artificial intelligence, a system that has the ability to generate new data and information based on input data.

[0220] "Minimally invasive" means causing less strain and damage to the body.

[0221] "High accuracy" means very accurate.

[0222] A "brain-machine interface (BMI)" is an interface that uses brain waves to operate machines and computers.

[0223] A "platform" is a system or environment that serves as the foundation for providing specific functions or services.

[0224] An "intuitive communication service" is one that allows users to exchange information in a natural way.

[0225] A "metaverse interface" is an interface that allows users to operate and experience things within a virtual space.

[0226] A "health management service" is a service for monitoring and managing a user's health condition.

[0227] A "biometric authentication service" is a service that verifies a user's identity using their biometric information.

[0228] "Real-time" means that data or information is processed immediately.

[0229] "Encryption" means transforming data using a specific algorithm into a form that is not easily understandable to third parties.

[0230] The "cloud" is a collection of computer resources and services provided over the Internet.

[0231] "Sharing history" is a record of who shared data, when, and how.

[0232] A "medical institution" is a facility that provides medical services, such as a hospital or clinic.

[0233] A "research institution" is an organization or facility for conducting scientific research.

[0234] The following system configuration will be described as an embodiment of the present invention.

[0235] System Configuration

[0236] The system collects users' brainwave data in real time, encrypts it, and uploads it to the cloud. It also has the ability to share the data with medical and research institutions and manage the data sharing history.

[0237] Hardware used

[0238] Smartphone: A device that allows users to manage EEG data and upload it to the cloud.

[0239] Brainwave sensor: A device used to collect a user's brainwave data (e.g., Muse, Emotiv).

[0240] Software used

[0241] Python: A programming language used to perform the main operations of a program.

[0242] NeuroKit2: A library for simulating and analyzing EEG data.

[0243] Requests: A library for sending HTTP requests.

[0244] Cryptography: A library for performing data encryption.

[0245] Processing flow

[0246] 1. EEG data collection: EEG sensors are used to collect users' EEG data in real time, which is then stored on the smartphone.

[0247] 2. Data encryption: The collected EEG data is encrypted using the Cryptography library, which ensures the security of the data.

[0248] 3. Data Upload: The encrypted data is uploaded to the cloud using the Requests library, where it is securely stored.

[0249] 4. Data sharing: Users can share data with medical institutions or research institutes as needed. Sharing history is stored locally, allowing users to see who accessed what data and when.

[0250] Specific examples

[0251] The user wears the EEG sensor and launches the NeuroGuard app. The app collects EEG data, encrypts it, and uploads it to the cloud. Medical institutions can then retrieve the data from the cloud with the user's permission and use it for diagnosis.

[0252] Prompt Sentence Examples

[0253] The user wears the EEG sensor and launches the NeuroGuard app. The app collects EEG data, encrypts it, and uploads it to the cloud. With the user's permission, medical institutions can retrieve the data from the cloud and use it for diagnosis.

[0254] This system allows users' brainwave data to be managed safely and efficiently, and to be shared with third parties if necessary.

[0255] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0256] Step 1:

[0257] The user wears the EEG sensor and launches the NeuroGuard app on their smartphone. The EEG sensor collects the user's brainwave data in real time and transmits it to the smartphone. The input is the brainwave data from the EEG sensor, and the output is the raw brainwave data stored on the smartphone.

[0258] Step 2:

[0259] The device (smartphone) analyzes the collected EEG data using the NeuroKit2 library. This analysis extracts features from the EEG data. The input is raw EEG data, and the output is the features of the analyzed EEG data. Specifically, it calls NeuroKit2 functions to analyze the EEG data and extract features.

[0260] Step 3:

[0261] The device encrypts the analyzed EEG data using the Cryptography library. The input is the characteristics of the analyzed EEG data, and the output is the encrypted EEG data. Specifically, the device generates an encryption key and uses the key to encrypt the EEG data.

[0262] Step 4:

[0263] The device uses the Requests library to upload encrypted EEG data to the cloud. The input is encrypted EEG data, and the output is encrypted data stored on the cloud. Specifically, the device sends an HTTP POST request to upload the data to the cloud server.

[0264] Step 5:

[0265] The server manages the encrypted data stored on the cloud and shares it with medical institutions and research institutions as needed. The input is the encrypted data on the cloud, and the output is the shared data. Specifically, with the user's permission, the server decrypts the data and provides it to a designated third party.

[0266] Step 6:

[0267] The server manages the data sharing history and records who accessed which data and when. The input is the data access request information, and the output is a log of the sharing history. Specifically, it generates an access log and stores it locally or on the cloud.

[0268] In this way, the user's brain wave data can be managed safely and efficiently and shared with third parties as needed.

[0269] Example 3

[0270] Next, a description will be given of a third embodiment of the third embodiment. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0271] Conventional brain-machine interfaces (BMIs) have had difficulty effectively analyzing a user's brainwave data and identifying emotions and intentions in real time. Furthermore, there has been a lack of a platform for providing services based on this data, such as intuitive communication, health management, and biometric authentication. Furthermore, there is a lack of functionality for users to manage their own brainwave data and share it with third parties as needed. To solve these issues, a system that can analyze brainwave data with high precision and provide a variety of services based on that data is needed.

[0272] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[0273] In this invention, the server includes: means for a user to wear an EEG measuring device and connect it to a terminal; means for the terminal to collect EEG data from the EEG measuring device and transmit it to the server; means for the server to receive and preprocess the EEG data; means for the server to analyze the preprocessed data and identify emotions and intentions; means for the server to transmit the analysis results to the terminal; means for the terminal to display the analysis results and provide feedback to the user; means for providing a platform for popularizing the system in society; and means for providing intuitive communication services, virtual reality interfaces, health management services, and biometric authentication services that utilize brain activity, which are provided through the platform. This makes it possible to analyze users' EEG data with high accuracy and provide a variety of services based on the analysis results in real time.

[0274] "User" refers to an individual who wears an EEG measurement device and uses the system.

[0275] "Electroencephalogram measuring device" refers to a device for measuring a user's electroencephalogram data.

[0276] "Terminal" refers to an electronic device that transmits data collected from an EEG measuring device to a server and displays the analysis results to the user.

[0277] "Server" refers to a computer system that receives EEG data, performs pre-processing and analysis, and transmits the results to a terminal.

[0278] "Preprocessing" refers to the cleansing of data before analysis, such as removing noise and filtering EEG data.

[0279] "Analysis" refers to data processing to identify the user's emotions and intentions based on preprocessed EEG data.

[0280] "Generative AI model" refers to a machine learning model used to analyze EEG data and identify a user's emotions and intent.

[0281] The "platform" refers to the infrastructure for disseminating the system to society, allowing users to manage their EEG data, and share it with third parties as needed.

[0282] An "intuitive communication service" is a service that understands a user's emotions and intentions based on their brainwave data and communicates based on that.

[0283] A "virtual reality interface" refers to an interface that controls behavior in virtual reality or augmented reality worlds based on the user's brainwave data.

[0284] "Health management service" refers to a service that uses a user's brainwave data to understand their stress level, sleep state, etc., and makes health management suggestions based on that information.

[0285] "Biometric authentication service" refers to a service that analyzes a user's brain wave patterns to authenticate them.

[0286] This invention begins with the user wearing an EEG device and connecting it to a terminal. The user wears the EEG device (e.g., EEG device) on their head and connects it to a terminal (e.g., smartphone or PC) using Bluetooth or a USB cable.

[0287] The device collects EEG data in real time from the EEG device using a dedicated application. The collected data is then sent to a server via the internet. The server receives the EEG data and performs preprocessing. This preprocessing includes noise removal and filtering, and data cleansing is performed using the Python MNE library.

[0288] The preprocessed data is input to a machine learning model (e.g., using TENSORFLOW (registered trademark) or PyTorch) on the server to analyze emotions and intent. The analysis results are used to identify the user's emotions (e.g., joy, sadness, surprise, etc.) and intent (e.g., the intention to perform a specific action).

[0289] The analysis results are sent from the server to the device. The device then displays the analysis results received from the server to the user. For example, a chat app might display a message saying "The user is happy." The metaverse interface also controls behavior in the VR or AR world based on the user's intentions.

[0290] As a concrete example, consider an intuitive communication service. A user wears an EEG device and connects it to a smartphone. The smartphone collects brainwave data from the EEG device in real time and sends it to a server. The server preprocesses the received data and analyzes emotions using TensorFlow. If the analysis shows that the user is "happy," that information is reflected in the smartphone's chat app, and a message saying "the user is happy" is sent to the other party.

[0291] Example prompt sentence:

[0292] The user wears an EEG device and connects it to a smartphone. The smartphone collects brainwave data in real time and sends it to a server. The server preprocesses the data and analyzes emotions using TensorFlow. If the analysis shows that the user is "happy," the server reflects this information in a chat app and sends the other party a message saying "the user is happy."

[0293] In this way, a system is realized in which the server, the terminal, and the user work together to provide new services using electroencephalogram data. The flow of the identification process in the third embodiment will be described with reference to FIG.

[0294] Step 1:

[0295] The user wears the EEG measurement device and connects it to the terminal.

[0296] Specifically, the user wears an EEG device on their head and connects it to a device (e.g., a smartphone or PC) using Bluetooth or a USB cable. The input is the user's brainwaves, and the output is the EEG device connected to the device.

[0297] Step 2:

[0298] The terminal collects brain wave data from the EEG measuring device and transmits it to the server.

[0299] Specifically, the terminal uses a dedicated application to collect brainwave data from the EEG device in real time. The collected data is then sent to a server via the Internet. The input is the brainwave data from the EEG measurement device, and the output is the brainwave data sent to the server.

[0300] Step 3:

[0301] The server receives the EEG data and performs preprocessing.

[0302] Specifically, the server receives EEG data sent from the device. The received data undergoes preprocessing such as noise removal and filtering. The data is cleansed using the Python MNE library. The input is the EEG data sent from the device, and the output is the preprocessed EEG data.

[0303] Step 4:

[0304] The server analyzes the pre-processed data to identify emotions and intent.

[0305] Specifically, the server inputs the preprocessed data into a machine learning model (using, for example, TensorFlow or PyTorch). The model identifies the user's emotions and intentions from the EEG data. The input is the preprocessed EEG data, and the output is the analysis result, which is information on emotions and intentions.

[0306] Step 5:

[0307] The server sends the analysis results to the device.

[0308] Specifically, the server sends the analysis results to the device. The analysis results include the user's emotions (e.g., joy, sadness, surprise, etc.) and intentions (e.g., the intention to perform a specific action). The input is the analysis results, and the output is the analysis results sent to the device.

[0309] Step 6:

[0310] The device displays the analysis results and provides feedback to the user.

[0311] Specifically, the device displays the analysis results received from the server to the user. For example, in a chat app, it displays a message saying "The user is happy." In addition, the metaverse interface controls behavior in the VR or AR world based on the user's intentions. The input is the analysis results received from the server, and the output is the feedback displayed to the user.

[0312] (Application example 3)

[0313] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0314] Conventional brain-machine interfaces (BMIs) have limited technology for analyzing users' brain waves, making it difficult to perform high-precision analysis in real time. They also lacked intuitive means for navigation and interaction within the metaverse, making it difficult to execute actions that reflect the user's intentions and emotions. Furthermore, the functionality for managing and sharing brainwave data was inadequate, resulting in low user convenience.

[0315] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[0316] In this invention, the server includes: means for generating a minimally invasive, highly accurate brain-machine interface (BMI) that combines brain waves and generative AI; means for providing a platform for popularizing the BMI throughout society; means for providing intuitive communication services, metaverse interfaces, health management services, and biometric authentication services that utilize brain activity through the platform; means for analyzing a user's brain waves in real time and controlling navigation and interaction within the metaverse; and means for executing actions within the metaverse based on the user's intentions and emotions. This enables users to intuitively navigate and interact within the metaverse using their brain waves.

[0317] "Electroencephalograms" are records of the brain's electrical activity and are used to analyze a user's intentions and emotions.

[0318] "Generative AI" is a type of artificial intelligence, a technology that generates new information and output based on input data.

[0319] A "brain-machine interface (BMI)" is an interface that analyzes brain waves to control machines and computers.

[0320] A "platform" is a system that serves as the foundation for providing specific services and functions.

[0321] An "intuitive communication service" is a service that analyzes the user's brain waves to understand their emotions and intentions, and then communicates based on that.

[0322] A "metaverse interface" is an interface that allows users to interact in virtual reality and augmented reality worlds.

[0323] The "health management service" analyzes the user's brain waves to determine their stress level and sleep state, and then makes health management suggestions based on that information.

[0324] "Biometric authentication service" is a service that analyzes a user's brain wave patterns to authenticate them.

[0325] "Navigation" refers to giving instructions and performing operations to allow a user to move within a virtual space.

[0326] "Interaction" refers to a user interacting with other objects or characters in a virtual space.

[0327] "Intent" is the user's intention to perform a particular action or operation.

[0328] "Emotion" refers to the user's psychological state or feelings.

[0329] An "action" is a movement or reaction that is performed in a virtual space based on the user's intentions and emotions.

[0330] A system for implementing this invention includes means for generating a minimally invasive, highly accurate brain-machine interface (BMI) that combines brain waves and generative AI, means for providing a platform for popularizing the BMI throughout society, means for providing intuitive communication services, metaverse interfaces, health management services, and biometric authentication services that utilize brain activity provided through the platform, means for analyzing a user's brain waves in real time and controlling navigation and interaction within the metaverse, and means for executing actions within the metaverse based on the user's intentions and emotions.

[0331] System configuration

[0332] The system uses the following hardware and software:

[0333] Hardware: EEG sensor, head-mounted display

[0334] Software: Python, EEG sensor API, Metaverse controller API

[0335] Program processing explanation

[0336] The server analyzes the brainwave data acquired from the EEG sensor in real time and inputs it into the generative AI. The generative AI analyzes the user's intentions and emotions based on the input brainwave data and generates output based on that. Specifically, if the user wants to move forward, the brainwave sensor detects that intention and the avatar moves forward. Also, if the user feels happy, that emotion is detected and the avatar smiles.

[0337] Specific examples

[0338] For example, if a user wants to move forward in the Metaverse, the EEG sensor will detect that intention and the avatar will move forward. If the user feels happy, the sensor will detect that emotion and the avatar will smile.

[0339] Prompt Sentence Examples

[0340] An example of a prompt to input to a generative AI model is as follows:

[0341] If the user wants to move forward, the EEG sensor should detect that intention and generate a program that makes the avatar move forward. Also, if the user feels happy, the sensor should detect that emotion and make the avatar smile.

[0342] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[0343] Step 1:

[0344] The server acquires the brain wave data from the brain wave sensor.

[0345] Input: Raw data from EEG sensor

[0346] Output: Acquired EEG data

[0347] Specific operation: The EEG sensor measures the user's brain waves in real time and sends the data to the server.

[0348] Step 2:

[0349] The server preprocesses the acquired electroencephalogram data.

[0350] Input: Acquired EEG data

[0351] Output: Preprocessed EEG data

[0352] Specific operation: Perform preprocessing such as noise removal and filtering, and convert the data into a format suitable for analysis.

[0353] Step 3:

[0354] The server inputs the preprocessed EEG data into the generative AI.

[0355] Input: Preprocessed EEG data

[0356] Output: Data input to the generative AI

[0357] Specific operation: Preprocessed EEG data is input into a generative AI model to analyze the user's intentions and emotions.

[0358] Step 4:

[0359] Generative AI analyzes the user's intentions and emotions based on input brainwave data.

[0360] Input: Data entered into the generative AI

[0361] Output: Analyzed intent and emotion

[0362] How it works: A generative AI model analyzes EEG data to determine the user's intentions and emotions.

[0363] Step 5:

[0364] The server determines actions within the metaverse based on the analysis results.

[0365] Input: Parsed intent and emotion

[0366] Output: Actions taken within the metaverse

[0367] Specific actions: If the user's intention is "move forward," the avatar will move forward; if the emotion is "happy," the avatar will smile.

[0368] Step 6:

[0369] The server sends the determined action to the metaverse controller.

[0370] Input: Actions performed within the metaverse

[0371] Output: Action instructions sent to the metaverse controller

[0372] Specific action: The determined action is sent through the metaverse controller's API, instructing the avatar to actually perform the action.

[0373] Step 7:

[0374] The metaverse controller controls the avatar based on the received action instructions.

[0375] Input: Action instructions sent to the Metaverse Controller

[0376] Output: Avatar movement

[0377] Specific behavior: The Metaverse Controller controls the avatar and performs actions based on the user's intentions and emotions, such as making the avatar move forward or smile.

[0378] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0379] "Example 1"

[0380] In one embodiment of the present invention, a brain-machine interface (BMI) analyzes the user's brain waves in real time and inputs the results into a generative AI. Furthermore, by combining an emotion engine, the user's emotional state is also analyzed simultaneously. Based on the analysis results, the system understands the user's intentions and emotional state and generates output based on that. For example, if a user feels joy and thinks, "I'm going to make coffee," the system recognizes that joy and takes appropriate action to make coffee.

[0381] "Example 2"

[0382] In another embodiment of the present invention, the platform has a function that allows users to manage their own brain wave data and emotional state and share it with third parties as needed. For example, sharing brain wave data when a user is feeling joy with medical institutions or research institutions can lead to more accurate medical services and research results.

[0383] "Example 3"

[0384] In a further embodiment of the present invention, the services provided through the platform have a function of providing services that utilize the user's emotional state. For example, an intuitive communication service analyzes the user's brain waves to understand their emotions and intentions and communicates based on them. If the user is feeling happy, the service will recognize that feeling and provide communication tailored to that emotion.

[0385] The processing flow of each embodiment will be described below.

[0386] "Example 1"

[0387] Step 1: The user thinks about a specific action, for example, "make coffee."

[0388] Step 2: The brain-machine interface (BMI) analyzes the user's brain waves in real time.

[0389] Step 3: The emotion engine analyzes the user's emotional state. In this example, it analyzes that the user has the emotion of joy.

[0390] Step 4: Based on the analysis results, understand the user's intention and emotional state and generate an output based on that. In this example, it triggers an action to make coffee that matches the emotion of joy.

[0391] "Example 2"

[0392] Step 1: The user experiences a specific emotion, for example, joy.

[0393] Step 2: The platform manages the user's brainwave data and emotional state.

[0394] Step 3: The user optionally shares their EEG data and emotional state with third parties, in this example, medical or research institutions.

[0395] "Example 3"

[0396] Step 1: A user uses a service through the platform, for example, an intuitive communication service.

[0397] Step 2: The service analyzes the user's brainwaves to understand their emotions and intentions.

[0398] Step 3: The service uses the analysis results to provide communication tailored to the user's emotional state. In this example, if the user is feeling happy, the service provides communication tailored to that emotion.

[0399] Example 1

[0400] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0401] Conventional brain-machine interfaces (BMIs) only analyzed the user's brain waves, but were unable to take the user's emotional state into account. This made it difficult to accurately understand the user's intentions and generate appropriate output. Furthermore, there was a lack of a platform for managing the user's brainwave data and sharing it with third parties. This hindered the widespread adoption of BMIs in society, making it difficult to provide intuitive communication services, health management services, and biometric authentication services.

[0402] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0403] In this invention, the server includes means for acquiring a user's brain waves in real time, means for analyzing the acquired brain wave data and extracting the user's intention, means for analyzing the user's emotional state, means for generating and inputting prompt sentences to the generative AI based on the analysis results, and means for executing the output generated by the generative AI. This enables the system to accurately understand the user's intention and emotional state and generate appropriate output. Furthermore, a platform for disseminating this system throughout society can be provided, allowing for intuitive communication services, virtual space interfaces, health management services, and biometric authentication services that utilize brain activity.

[0404] "User" refers to an individual who uses the System.

[0405] "Electroencephalograms" refer to electrical signals generated by neural activity in the brain.

[0406] "Real-time" refers to processing data as it is generated.

[0407] "Means of acquisition" refers to the devices and methods used to collect data.

[0408] "Means for analyzing" refers to devices and methods for analyzing acquired data and extracting meaningful information.

[0409] "Intent" refers to information that indicates what a user wants to do or what they are thinking.

[0410] "Emotional state" refers to information that indicates a user's emotions or mood.

[0411] "Generative AI" refers to artificial intelligence that generates new information or output based on given data.

[0412] A "prompt sentence" refers to an input sentence used to give instructions or ask questions to a generative AI.

[0413] "Output" refers to the results or responses that a generative AI generates based on a prompt.

[0414] "Means of execution" refers to the devices and methods used to convert the output generated by generative AI into concrete actions or operations.

[0415] "Platform" refers to the infrastructure for operating the system and providing services to users.

[0416] An "intuitive communication service" is a service that allows users to exchange information with others in a natural way.

[0417] "Virtual space interface" refers to an interface that allows users to operate and experience things within a virtual environment.

[0418] "Health management service" refers to a service for monitoring and managing a user's health status.

[0419] "Biometric authentication service" refers to a service that verifies a user's identity using their biometric information.

[0420] This invention is a system that analyzes a user's brain waves in real time and inputs the results into a generative AI to understand the user's intentions and emotional state and generate output based on that. This system includes an EEG measurement device for acquiring the user's brain waves, software for analyzing the EEG data, an emotion engine for analyzing the emotional state, the generative AI, and hardware for executing the generated output.

[0421] Hardware and software used

[0422] 1. EEG measurement device:

[0423] The user wears a specially designed electroencephalogram (EEG) measuring device, which captures the user's brainwaves in real time.

[0424] 2. EEG analysis software:

[0425] The device uses EEG analysis software (e.g., OpenBCI) to analyze the EEG data acquired from the EEG measurement device. This software analyzes the EEG data and extracts the user's intentions.

[0426] 3. Emotion Engine:

[0427] The device uses an emotion engine to analyze the user's emotional state, which obtains emotional data through facial recognition cameras and voice analysis to understand the user's emotional state.

[0428] 4. Generative AI:

[0429] Based on the analysis results, the server generates and inputs a prompt to the generative AI (e.g., OpenAI GPT-4), which then generates an appropriate output based on the prompt.

[0430] 5. Output Execution Hardware:

[0431] The server receives the output from the generative AI and operates hardware (e.g., IoT devices) to perform specific actions.

[0432] Specific examples

[0433] The user wakes up in the morning and puts on the EEG measurement device.

[0434] The terminal receives data from the EEG measurement device and begins analyzing it in real time.

[0435] The device detects the intention to "drink coffee" from brain wave data.

[0436] The device uses a facial recognition camera to confirm that the user is expressing "joy."

[0437] The device generates a prompt saying, "The user feels happy and wants to drink coffee. Please suggest an appropriate action."

[0438] The server sends a prompt to the generative AI, which then generates the output "instruct the coffee maker to make coffee."

[0439] The server controls the coffee maker through an IoT device and makes coffee.

[0440] Prompt Sentence Examples

[0441] User's EEG data: [DATA]

[0442] User's emotional state: joy

[0443] User Intent: I want a coffee.

[0444] Prompt for generative AI: The user has a feeling of joy and wants to "drink coffee." Suggest an appropriate action.

[0445] In this way, the system can analyze the user's brainwaves and emotional state, use generative AI to generate output based on the user's intentions, and execute specific actions.

[0446] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0447] Step 1:

[0448] The user wears a dedicated electroencephalogram (EEG) measuring device.

[0449] Input: User's brainwaves

[0450] Output: Real-time EEG data

[0451] How it works: The user wears the EEG device on their head, and the device begins to measure their brain waves. The device then transmits the EEG data to a terminal in real time via Bluetooth or USB connection.

[0452] Step 2:

[0453] The terminal acquires brain wave data in real time from the brain wave measurement device.

[0454] Input: Real-time EEG data

[0455] Output: Acquired EEG data

[0456] Specific operation: The terminal receives the EEG data sent from the EEG measurement device and stores it in a database.

[0457] Step 3:

[0458] The device analyzes the brain wave data it acquires and extracts the user's intentions.

[0459] Input: Acquired EEG data

[0460] Output: User intent

[0461] Specific operation: The device uses EEG analysis software (e.g., OpenBCI) to analyze EEG data. It detects specific EEG patterns and extracts the user's intention based on them. For example, it detects the intention of "I want to drink coffee."

[0462] Step 4:

[0463] The device analyzes the user's emotional state.

[0464] Input: User's facial image and voice data

[0465] Output: User's emotional state

[0466] Specific operation: The device uses the emotion engine to acquire emotion data through facial recognition cameras and voice analysis. As a result of the analysis, the user's emotional state, such as "joy" or "sadness," is identified.

[0467] Step 5:

[0468] Based on the analysis results, the terminal generates and inputs a prompt sentence to the generative AI.

[0469] Input: User intent and emotional state

[0470] Output: Prompt for the generative AI

[0471] Specific behavior: The device integrates the user's intention and emotional state and generates a prompt sentence such as, "The user is feeling happy and wants to drink coffee. Please suggest an appropriate action."

[0472] Step 6:

[0473] The server sends a prompt to the generative AI, which then generates an output.

[0474] Input: Prompt sentence for generative AI

[0475] Output: Output from generative AI

[0476] Specific operation: The server sends a prompt to a generative AI (e.g., OpenAI GPT-4), which then generates an output based on the prompt: "Instruct the coffee maker to make coffee."

[0477] Step 7:

[0478] The server receives the output from the generative AI and executes specific actions.

[0479] Input: Output from generative AI

[0480] Output: Actions taken

[0481] Specific operation: The server operates the coffee maker through the IoT device to make coffee. Specifically, it sends the command "Make coffee" to the coffee maker, and the coffee is actually made.

[0482] (Application example 1)

[0483] Next, a description will be given of Application Example 1 of Embodiment Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0484] Conventional brain-machine interfaces (BMIs) exist that analyze the user's brainwaves to understand their intentions, but their application range is limited, and they are not fully utilized, especially in supporting work in factories. Furthermore, there is a lack of systems that can understand the intentions of workers in real time and automatically perform appropriate tasks, which means that improvements in work efficiency and collaboration between humans and machines have not been fully realized.

[0485] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means. In this invention, the server includes: means for generating a minimally invasive, highly accurate brain-machine interface (BMI) that combines electroencephalograms and generative AI; means for providing a platform for popularizing the BMI throughout society; means for providing intuitive communication services using brain activity, a metaverse interface, health management services, and biometric authentication services provided through the platform; and means installed on robots performing work in factories, analyzing the electroencephalograms of workers in real time, understanding the workers' intentions, and automatically performing appropriate tasks. This makes it possible to understand the workers' intentions in real time and automatically perform appropriate tasks.

[0486] "Electroencephalograms" are electrical signals generated by neural activity in the brain.

[0487] "Generative AI" is an artificial intelligence technology that generates new data and output based on input data.

[0488] Brain-machine interface (BMI) is an interface technology that analyzes brain waves and controls machines and computers based on the results.

[0489] "Minimally invasive" means causing less strain and damage to the body.

[0490] "High accuracy" means very accurate.

[0491] A "platform" is a system or environment that serves as the foundation for providing specific services or functions.

[0492] An "intuitive communication service" is one that allows users to exchange information in a natural way.

[0493] "Metaverse interface" refers to the user interface within the virtual space.

[0494] A "health management service" is a service for monitoring and managing a user's health condition.

[0495] A "biometric authentication service" is a service that uses an individual's biometric information to perform authentication.

[0496] A "robot working in a factory" is a machine that automatically performs specific tasks in a factory.

[0497] "Worker" refers to a person who performs work in a factory.

[0498] "Real-time" means that processing and analysis occur almost simultaneously.

[0499] "Intent" refers to the purpose or idea of ​​what the user wants to do.

[0500] "Output" refers to the results or actions produced by a system or machine.

[0501] The system for implementing this invention uses a minimally invasive, highly accurate brain-machine interface (BMI) that combines EEG and generative AI. The server analyzes the user's EEG in real time and inputs the results into the generative AI, which then understands the user's intentions and generates output based on them. Furthermore, by combining it with an emotion engine, the system simultaneously analyzes the user's emotional state, and generates output based on an understanding of the user's intentions and emotional state.

[0502] This system is installed on robots working in factories and analyzes the worker's brain waves in real time. When a worker wants to perform a specific task, the robot understands that intention and automatically performs the appropriate task. For example, if a worker thinks, "I want to assemble a part," the robot will understand that intention and begin assembling the part.

[0503] The server uses an EEG sensor to acquire EEG data, which is analyzed in real time using a Python library. An emotion engine is used to analyze the emotional state. A generative AI model (e.g., OpenAI GPT-3 (registered trademark)) understands the user's intention based on the analyzed EEG data and emotional state, and generates appropriate output.

[0504] As a specific example, the prompt sentence when a worker thinks "I want to assemble parts" is shown below.

[0505] Example prompt sentence:

[0506] User's EEG data: [0.1, 0.2, 0.3, ...]

[0507] User's emotional state: joy

[0508] User Intent: Assemble the parts.

[0509] By inputting this prompt into a generative AI model, the system can understand the user's intent and generate an appropriate action, resulting in a specific output such as a robot starting to assemble parts.

[0510] This system makes it possible to understand the intentions of workers in real time and automatically carry out appropriate tasks, thereby improving work efficiency in factories and realizing collaboration between humans and machines.

[0511] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0512] Step 1:

[0513] The server acquires the user's brainwave data in real time using an EEG sensor. The input is the user's brainwave signal, and the output is the acquired brainwave data. This data is transmitted from the EEG sensor to the server.

[0514] Step 2:

[0515] The server uses a Python library to analyze the acquired EEG data in real time. The input is the acquired EEG data, and the output is the analyzed EEG data. This analysis extracts EEG features and patterns.

[0516] Step 3:

[0517] The server uses an emotion engine to analyze the user's emotional state from the analyzed EEG data. The input is the analyzed EEG data, and the output is the user's emotional state. This analysis determines whether the user is experiencing emotions such as joy, sadness, or anger.

[0518] Step 4:

[0519] The server uses a generative AI model (e.g., OpenAI GPT-3) to understand the user's intention based on the analyzed EEG data and emotional state. The input is the analyzed EEG data and emotional state, and the output is the user's intention. The generative AI model inputs these data as prompts and infers what the user wants to do.

[0520] Step 5:

[0521] The server generates appropriate output based on the user's intention. The input is the user's intention, and the output is a specific action. For example, if the user thinks, "I want to assemble parts," the server will instruct the robot to start assembling the parts.

[0522] Step 6:

[0523] The robot receives instructions from the server and performs specific tasks. The input is the instruction from the server, and the output is the executed task. For example, this includes a series of actions from when the robot starts assembling parts to when it completes the task.

[0524] Step 7:

[0525] The server monitors the progress of the work and provides feedback as needed. The input is work progress data from the robot, and the output is feedback information. This improves the accuracy and efficiency of the work.

[0526] Example 2

[0527] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0528] Conventional brain-machine interfaces (BMIs) lacked the ability to effectively manage users' EEG data and share it with third parties as needed. Furthermore, the accuracy and real-time nature of EEG data analysis made it difficult to accurately understand the user's intentions. Furthermore, the acquisition and management of EEG data was cumbersome, making it difficult for general users to use.

[0529] The specific processing by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes a means for a user to acquire EEG data using a dedicated EEG measuring device and upload the data to the platform via a terminal, a means for the server to receive the EEG data and store it in a database, and a means for the server to preprocess and analyze the EEG data. This allows users to effectively manage their own EEG data and share it with third parties as needed. In addition, the accuracy of EEG data analysis is improved, allowing for accurate understanding of the user's intentions in real time. Furthermore, the acquisition and management of EEG data is simplified, providing a system that is easy for general users to use.

[0530] Brain-machine interface (BMI) is a technology that uses brainwave data to understand the user's intentions and enable intuitive interaction with machines and computers.

[0531] "Generative AI" is an artificial intelligence technology that analyzes a user's brainwave data and generates output based on the results.

[0532] The "platform" is a system infrastructure that allows users to manage their EEG data and share it with third parties as needed.

[0533] An "EEG measuring device" is a device worn on the user's head to acquire brain wave data.

[0534] A "terminal" is a device that allows users to input EEG data and upload it to the platform.

[0535] A "server" is a computer system that receives EEG data, stores it in a database, and performs preprocessing and analysis.

[0536] The "database" is an information management system for storing received electroencephalogram data.

[0537] "Preprocessing" refers to the cleaning of data before analysis, such as removing noise from EEG data and filling in missing values.

[0538] "Analysis" is a process for classifying and understanding the user's emotional state and intentions based on pre-processed EEG data.

[0539] "Third parties" refer to those with whom users share their EEG data, such as medical institutions and research institutions.

[0540] MODE FOR CARRYING OUT THE INVENTION

[0541] This invention is a system in which users acquire EEG data using a dedicated EEG measuring device and upload it to a platform via their terminal. The server stores the received EEG data in a database and performs preprocessing and analysis. This allows users to effectively manage their own EEG data and share it with third parties as needed.

[0542] Hardware and software used

[0543] Hardware:

[0544] EEG measurement device: The user acquires EEG data using a dedicated EEG measurement device (e.g., a general-purpose EEG device), which is worn on the user's head and collects EEG data in real time.

[0545] software:

[0546] Terminal application: The terminal provides an interface for users to receive acquired EEG data and upload it to the platform. Users upload data through the terminal application.

[0547] Server: The server uses Python to preprocess and analyze the EEG data. Specifically, it cleans and filters the data using libraries such as NumPy and Pandas. It also builds and analyzes machine learning models using libraries such as TensorFlow and PyTorch.

[0548] Database: The server uses a relational database such as MySQL or PostgreSQL to store the EEG data.

[0549] Specific examples

[0550] As a concrete example, consider a scenario in which a user wants to share their EEG data when they are experiencing the emotion of joy with a medical institution. The user acquires the data using an EEG measurement device and uploads it to the platform via their terminal. The server receives the data, analyzes it, and then transmits it to the medical institution.

[0551] Example prompts to input to a generative AI model:

[0552] "Please explain the procedure for sharing EEG data from users experiencing pleasure with medical institutions."

[0553] Using this prompt, the generative AI model can provide detailed instructions on specific steps and required hardware and software.

[0554] In this way, users can effectively manage their own EEG data and share it with third parties as needed. Furthermore, the accuracy of EEG data analysis is improved, allowing users' intentions to be accurately understood in real time. Furthermore, the acquisition and management of EEG data is simplified, providing a system that is easy for general users to use.

[0555] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0556] Program processing flow

[0557] Step 1:

[0558] The user acquires EEG data

[0559] Input: User's brainwaves

[0560] Output: EEG data

[0561] How it works: The user wears a dedicated EEG device on their head to capture real-time EEG data. The device uses sensors to detect the user's brainwaves and stores them as digital data.

[0562] Step 2:

[0563] The device uploads the brainwave data to the platform.

[0564] Input: EEG data

[0565] Output: Data uploaded to the platform

[0566] Specific operation: The user opens the application on the device, selects the acquired EEG data, and presses the "Upload" button. The device then sends the data to the platform.

[0567] Step 3:

[0568] The server receives the EEG data and stores it in a database.

[0569] Input: Uploaded EEG data

[0570] Output: Data stored in the database

[0571] How it works: The server receives the EEG data sent from the device and stores it in a relational database such as MySQL or PostgreSQL, along with metadata such as user IDs and timestamps.

[0572] Step 4:

[0573] The server preprocesses the EEG data

[0574] Input: EEG data stored in a database

[0575] Output: Preprocessed data

[0576] What it does: The server runs Python scripts and uses libraries like NumPy and Pandas to clean and filter the data, specifically removing noise and imputing missing values.

[0577] Step 5:

[0578] The server analyzes the brainwave data

[0579] Input: Preprocessed data

[0580] Output: Analysis results (e.g., classification of emotional state)

[0581] How it works: The server uses libraries such as TensorFlow and PyTorch to run machine learning models to classify emotional states from EEG data. The analysis results are output as data indicating the user's emotional state and intentions.

[0582] Step 6:

[0583] Users manage their EEG data

[0584] Input: Analysis results and original EEG data

[0585] Output: Managed data (e.g. filtered data display)

[0586] What it does: Users access the platform's dashboard to view analytics results, and can filter and view data for specific time periods.

[0587] Step 7:

[0588] Users share their EEG data with third parties

[0589] Input: Managed Data

[0590] Output: Shared data

[0591] How it works: The user presses the "Share" button on the dashboard, enters the contact information of a third party (e.g., a medical institution) and gives permission to share. The server encrypts the data and sends it to the third party using HTTPS.

[0592] In this way, by performing specific actions at each step, users can effectively manage their own brainwave data and share it with third parties as needed.

[0593] (Application example 2)

[0594] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0595] Conventional brain-machine interfaces (BMIs) can collect and analyze a user's brainwave data, but lack a means to safely and efficiently share that data with third parties. Furthermore, there was no system that could analyze a user's emotional state in real time and take appropriate action if an abnormality was detected. This made it difficult to share data with medical and research institutions, hindering the improvement of user health management and biometric authentication accuracy.

[0596] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0597] In this invention, the server includes: a means for generating a minimally invasive, highly accurate brain-machine interface (BMI) that combines EEG and generative AI; a means for providing a platform for disseminating the BMI to society; a means for providing intuitive communication services, metaverse interfaces, health management services, and biometric authentication services that utilize brain activity, all of which are provided through the platform; a means for collecting a user's EEG data in real time, encrypting it, and sharing it with third parties; and a means for analyzing the user's emotional state and issuing an alert if an abnormality is detected. This enables the safe and efficient sharing of user EEG data and strengthens collaboration with medical and research institutions. Furthermore, by monitoring a user's emotional state in real time and responding quickly if an abnormality is detected, the accuracy of health management and biometric authentication can be improved.

[0598] "Electroencephalograms" are electrical signals generated by neural activity in the brain.

[0599] "Generative AI" is a type of artificial intelligence, a system that has the ability to generate new data and information based on input data.

[0600] "Minimally invasive" refers to minimal strain or damage to the body.

[0601] A "brain-machine interface (BMI)" is an interface that uses brain waves to operate machines and computers.

[0602] A "platform" is a system or environment that serves as the foundation for providing specific functions or services.

[0603] An "intuitive communication service" is one that allows users to exchange information in a natural way.

[0604] A "metaverse interface" is an interface that allows users to operate and experience things within a virtual space.

[0605] A "health management service" is a service that monitors the user's health status and provides appropriate advice and support.

[0606] A "biometric authentication service" is a service that verifies a user's identity using their biometric information.

[0607] "Encryption" is a technology that converts data using a specific algorithm to make it difficult for third parties to decipher.

[0608] An "emotional state" is the emotional state that a user is feeling at a particular moment.

[0609] An "alert" is a notification or warning to notify of an abnormality or emergency.

[0610] The following system configuration will be described as an embodiment of the present invention.

[0611] System Configuration

[0612] The system includes hardware and software for collecting and analyzing the user's brainwave data using generative AI. Specifically, it includes the following elements:

[0613] 1. EEG sensor: A device that collects the user's brainwaves in real time. The EEG sensor is worn on the user's head and measures the electrical activity of the brain.

[0614] 2. Smartphone: A device that receives, encrypts, and analyzes data from the EEG sensor. A dedicated application is installed on the smartphone.

[0615] 3. Server: Provides a platform for receiving encrypted EEG data and sharing it with third parties (medical institutions and research institutions) as needed. The server includes software for managing and sharing data.

[0616] Program processing

[0617] The server performs the following process.

[0618] 1. Collecting EEG data: The EEG sensor collects the user's EEG data in real time and transmits it to a smartphone.

[0619] 2. Data encryption: The smartphone encrypts the received EEG data using the Fernet cryptography library.

[0620] 3. Data sharing: The encrypted data is sent from the smartphone to a server, which then shares the received data with designated third parties (medical institutions or research institutions).

[0621] 4. Emotional state analysis: The smartphone analyzes the EEG data to identify the user's emotional state. If an abnormality is detected, the smartphone will send an alert.

[0622] Hardware and software used

[0623] Brainwave sensor: A device that collects the user's brainwaves.

[0624] Smartphone: A device that receives, encrypts, analyzes, and shares data.

[0625] Server: A platform for managing and sharing data.

[0626] cryptography library: Software used to encrypt data.

[0627] requests library: Software used to send data.

[0628] Specific examples

[0629] For example, consider the case where brain wave data is collected when a user feels stressed and sent to a medical institution. An brain wave sensor collects the user's brain waves and sends them to a smartphone. The smartphone encrypts the received data and sends it to a server. The server shares the encrypted data with the medical institution. The medical institution analyzes the received data and evaluates the user's stress level.

[0630] Prompt Sentence Examples

[0631] "Write a Python program that collects a user's EEG data, encrypts it, and sends it to a designated medical institution. The EEG data can be virtual. Use the Fernet cryptography library to encrypt it, and the requests library to send the data."

[0632] The above is an embodiment of the present invention.

[0633] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0634] Step 1:

[0635] The user wears an EEG sensor, which collects the user's EEG data in real time. The input is the user's EEG signal, and the output is the raw data collected by the EEG sensor.

[0636] Step 2:

[0637] The terminal (smartphone) receives EEG data from the EEG sensor. The terminal acquires the data using wireless communication such as Bluetooth or Wi-Fi. The input is the raw data sent from the EEG sensor, and the output is the EEG data stored in the terminal.

[0638] Step 3:

[0639] The EEG data received by the device is encrypted. The encryption is performed using the Fernet cryptography library. The input is the EEG data stored on the device, and the output is the encrypted EEG data. Specifically, an encryption key is generated and the EEG data is encrypted with that key.

[0640] Step 4:

[0641] The device sends encrypted EEG data to the server. The requests library is used for sending. The input is the encrypted EEG data, and the output is the data sent to the server. Specifically, the data is sent using an HTTP POST request.

[0642] Step 5:

[0643] The server stores the received encrypted EEG data. The server manages the data using a database. The input is the encrypted data sent from the device, and the output is the encrypted data stored in the database.

[0644] Step 6:

[0645] The server shares the encrypted EEG data with third parties (medical institutions or research institutions) as needed. A secure communication protocol is used for sharing. The input is the encrypted data stored in the database, and the output is the data sent to the third party. Specifically, the server authenticates the third party and sends the data if authentication is successful.

[0646] Step 7:

[0647] The device analyzes the EEG data to identify the user's emotional state. A generative AI model is used for the analysis. The input is the EEG data stored on the device, and the output is the emotional state as a result of the analysis. Specifically, the EEG data is input into the generative AI model, which estimates the emotional state.

[0648] Step 8:

[0649] The device monitors the emotional state and issues an alert if an abnormality is detected. The input is the emotional state as an analysis result, and the output is an alert when an abnormality is detected. Specifically, if the emotional state exceeds a preset threshold, a notification is sent to the user or a designated third party.

[0650] Example 3

[0651] Next, a description will be given of a third embodiment of the third embodiment. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0652] Conventional brain-machine interfaces (BMIs) require high accuracy when analyzing the user's brain waves, but it has been difficult to achieve high-precision analysis in a minimally invasive manner. Furthermore, there has been a lack of a system for providing real-time feedback to the user with the analysis results, making it impossible to immediately reflect the user's intentions or emotional state. Furthermore, there has also been a lack of functionality for users to manage their own brain wave data and share it with third parties as needed.

[0653] The identification process by the identification processing unit 290 of the data processing device 12 in Example 3 is realized by the following means. In this invention, the server includes means for measuring the user's electroencephalogram data and transmitting it to the server in real time, means for the server to analyze the electroencephalogram data and identify the user's emotional state and intention, and means for storing the analysis results in a database and transmitting them to the terminal. This enables minimally invasive and highly accurate electroencephalogram analysis, which can reflect the user's intention and emotional state in real time. It also enables the user to manage their own electroencephalogram data and share it with third parties as necessary.

[0654] Brain-machine interface (BMI) is a technology that uses brain waves to control computers and other devices.

[0655] "Generative AI" is a type of artificial intelligence that has the ability to generate new data and information based on input data.

[0656] A "platform" is a system or environment that serves as the foundation for providing specific services or functions.

[0657] "EEG data" is data obtained by electrically measuring a user's brain activity, and reflects their emotional state and intentions.

[0658] "Real-time" means that data is measured and processed immediately.

[0659] "Analysis" is the process of processing measured data and extracting specific information or patterns.

[0660] An "emotional state" is a state that indicates the user's current feelings or mood.

[0661] "Intent" describes what a user is trying to do or what they want.

[0662] A "database" is a system that stores data in an organized manner and allows it to be searched and updated as needed.

[0663] "Feedback" is the response or information that a system provides to a user.

[0664] "Minimally invasive" means that it causes little physical strain or discomfort to the user.

[0665] "High precision" means that the data measurements and analysis are extremely accurate.

[0666] A "virtual reality interface" is a means by which a user interacts with a virtual reality environment.

[0667] A "health management service" is a service that monitors a user's health status and provides appropriate advice and support.

[0668] "Biometric authentication service" refers to a service that authenticates individuals using their biometric information.

[0669] This invention is a system that provides a minimally invasive, highly accurate brain-machine interface (BMI) that combines EEG and generative AI. This system measures and analyzes the user's EEG data in real time to identify the user's emotional state and intentions, and provides feedback based on that information.

[0670] Hardware and software used

[0671] Hardware

[0672] EEG measurement device: A device for measuring the user's brain waves (e.g., EEG headset).

[0673] Terminal: A device (e.g., smartphone, tablet, PC) that connects to the EEG measurement device and transmits data to the server.

[0674] Server: A computer system for receiving and analyzing EEG data.

[0675] software

[0676] Data transmission program: A program installed on the device to transmit EEG data to the server.

[0677] Data analysis program: A program installed on the server to analyze EEG data (e.g., FFT analysis using Python)

[0678] Database management system: A system for storing analysis results and retrieving and updating them as needed.

[0679] Feedback program: A program installed on the device to display analysis results to the user.

[0680] Specific operation of the system

[0681] User Actions

[0682] The user wears an EEG measurement device (e.g., EEG headset) and connects it to a terminal. The user generates EEG data by performing specific actions. For example, if the user is relaxed, the EEG data will have specific frequency components.

[0683] Device behavior

[0684] The device measures the user's brain waves in real time and transmits them to the server using a data transmission program. The device waits for a response from the server to confirm that the data transmission was successful.

[0685] Server Operation

[0686] The server receives the EEG data sent from the device. The received data is analyzed using a data analysis program. Specifically, the EEG data is broken down into frequency components using FFT (Fast Fourier Transform) and specific patterns are extracted. For example, the intensity of alpha and beta waves is calculated to identify the user's emotional state.

[0687] Saving and sending analysis results

[0688] The server saves the analysis results in a database. The saved data includes the user ID, measurement date and time, and analysis results (emotional state and intention). This makes it possible to refer to the data later. The server then sends the analysis results to the device. A secure communication protocol (e.g., HTTPS) is used for transmission.

[0689] Device feedback

[0690] The device displays the analysis results received from the server to the user using a graphical user interface (GUI). For example, if the user is relaxed, the device displays "Relaxed state" and suggests an appropriate action (e.g., playing relaxing music).

[0691] Examples of specific examples and prompts

[0692] Specific examples

[0693] For example, when a user uses an intuitive communication service, the following procedure is performed.

[0694] 1. The user wears an EEG headset and connects it to the device.

[0695] 2. The device measures EEG data in real time and sends it to the server.

[0696] 3. The server receives the EEG data and analyzes it using FFT. For example, if the intensity of alpha waves is high, it determines that the user is relaxed.

[0697] 4. The analysis results are saved in the database. For example, data such as "User ID: 12345, Date and Time: 2023-10-01 10:00, State: Relaxed" is saved.

[0698] 5. The server sends the analysis results to the device, which displays "Relaxed state" and suggests playing relaxing music.

[0699] 6. The user plays relaxing music as suggested by the device and relaxes.

[0700] Prompt Sentence Examples

[0701] "Your goal is to analyze the user's brainwave data to identify their emotional state and generate a program that will enable intuitive communication. The hardware used is an EEG headset, and the software is Python. Specifically, you will analyze the brainwave data using FFT and calculate the strength of alpha and beta waves to identify their emotional state."

[0702] In this way, by specifically explaining the operation of the entire system, it is possible to clearly show the mode for carrying out the invention. The flow of the specific processing in the third embodiment will be explained with reference to FIG.

[0703] Step 1: Measuring the user's brainwave data

[0704] The user wears an electroencephalogram (EEG) measurement device (e.g., an EEG headset). The device measures the user's brain waves in real time and converts them into digital signals. The input is the user's brain waves, and the output is digitized brain wave data. Specifically, the device contacts the user's scalp and detects electrical signals.

[0705] Step 2: Sending EEG data

[0706] The device transmits the measured EEG data to a server via Bluetooth or Wi-Fi. The input is digitized EEG data, and the output is the data transmitted to the server. Specifically, the device executes a data transmission program and transmits the data to the server's IP address.

[0707] Step 3: Receiving and analyzing EEG data by the server

[0708] The server receives the EEG data sent from the device. The input is the EEG data sent from the device, and the output is the analysis results. The server uses Python to perform an FFT (Fast Fourier Transform) and break down the EEG data into frequency components. Specifically, the server runs a data analysis program and calculates the intensity of alpha and beta waves.

[0709] Step 4: Save the analysis results

[0710] The server saves the analysis results in a database. The input is the analysis results, and the output is the data saved in the database. Specifically, the server uses a database management system to save the user ID, measurement date and time, and analysis results.

[0711] Step 5: Submitting the analysis results

[0712] The server sends the analysis results to the terminal. The input is the analysis results stored in the database, and the output is the analysis results sent to the terminal. Specifically, the server sends data using a secure communication protocol (e.g., HTTPS).

[0713] Step 6: Displaying the analysis results on your device

[0714] The terminal displays the analysis results received from the server to the user. The input is the analysis results sent from the server, and the output is the information displayed to the user. In concrete terms, the terminal runs a feedback program and displays the analysis results using a graphical user interface (GUI).

[0715] Step 7: User feedback

[0716] The user confirms their emotional state and intentions based on the feedback provided by the device. The input is the feedback information from the device, and the output is the user's next action. Specifically, the user takes an action such as playing relaxing music according to the device's suggestion.

[0717] (Application example 3)

[0718] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0719] Conventional brain-machine interfaces (BMIs) can understand a user's intentions by analyzing their brain waves, but there are issues with accuracy and real-time performance. Furthermore, intuitive operation and emotion-based action control within the metaverse are difficult, creating a need for improved user experience. Furthermore, there has been a lack of user-friendly systems for managing brain wave data and sharing it with third parties.

[0720] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[0721] In this invention, the server includes: a means for generating a minimally invasive, highly accurate brain-machine interface (BMI) that combines EEG and generative AI; a means for providing a platform for disseminating the BMI to society; a means for providing intuitive communication services using brain activity, a metaverse interface, health management services, and biometric authentication services provided through the platform; a means for analyzing a user's EEG in real time, grasping their emotions and intentions, and controlling their behavior in the metaverse; and a means for controlling actions in the metaverse based on the user's emotional state. This allows users to intuitively operate the metaverse using EEG, realizing emotion-based action control. Furthermore, EEG data can be easily managed and shared with third parties.

[0722] "Electroencephalograms" are electrical signals generated by neural activity in the brain.

[0723] "Generative AI" is an artificial intelligence technology that generates new information and content based on data.

[0724] A "brain-machine interface (BMI)" is an interface that analyzes brain waves to control machines and computers.

[0725] A "platform" is a system or environment that serves as the foundation for providing specific services or functions.

[0726] An "intuitive communication service" is a service that analyzes the user's brain waves to understand their emotions and intentions, and then communicates based on that.

[0727] A "metaverse interface" is an interface for controlling actions in the world of virtual reality (VR) and augmented reality (AR).

[0728] The "health management service" analyzes the user's brain waves to determine their stress level, sleep state, etc., and then makes health management suggestions based on that information.

[0729] "Biometric authentication service" is a service that analyzes a user's brain wave patterns to authenticate them.

[0730] "Real-time" refers to processing or reaction occurring immediately, without delay.

[0731] "Emotional state" refers to the emotional state of the user as analyzed from their brain waves.

[0732] "Action control" refers to controlling the behavior of a system or avatar based on the user's intentions and emotions.

[0733] A system for implementing this invention includes means for generating a minimally invasive, highly accurate brain-machine interface (BMI) that combines brain waves and generative AI, means for providing a platform for popularizing the BMI throughout society, means for providing intuitive communication services, metaverse interfaces, health management services, and biometric authentication services that utilize brain activity provided through the platform, means for analyzing a user's brain waves in real time to understand their emotions and intentions and control their behavior within the metaverse, and means for controlling actions within the metaverse based on the user's emotional state.

[0734] System Configuration

[0735] The server comprises a system consisting of an EEG sensor, a head-mounted display (HMD), and generative AI. The EEG sensor captures the user's brainwaves in real time, and the HMD provides the user with visual information within the metaverse. The generative AI analyzes the captured EEG data to understand the user's emotions and intentions.

[0736] Program processing

[0737] The server uses the BrainFlow library to acquire EEG data, filter the data to remove noise, and then analyzes the data using generative AI to detect the user's emotions (e.g., joy). Based on the detected emotions, the server controls actions within the metaverse. Specifically, if the user is feeling joyful, the server controls the avatar to perform a joyful dance.

[0738] Hardware and software used

[0739] Hardware: EEG sensor (e.g., BrainFlow-compatible device), head-mounted display (HMD)

[0740] Software: Python, BrainFlow library

[0741] Specific examples

[0742] For example, consider a scenario in which a user wears an HMD and uses an EEG sensor to control their actions in the Metaverse. If the user feels joy, the avatar will dance in joy. This allows the user to intuitively operate the Metaverse, realizing emotion-based action control.

[0743] Prompt Sentence Examples

[0744] Example prompts to input to a generative AI model:

[0745] Write a Python program that analyzes a user's EEG data to detect emotions and control actions in the Metaverse. If the user has the emotion of joy, make the avatar do a joyful dance.

[0746] In this way, an intuitive metaverse interface using brain waves can be realized.

[0747] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[0748] Step 1:

[0749] The server uses an EEG sensor to acquire the user's EEG data in real time.

[0750] Input: User's brainwave signal

[0751] Output: Raw EEG data

[0752] How it works: The server uses the BrainFlow library to collect data from the EEG sensor and store it in memory.

[0753] Step 2:

[0754] The server filters the acquired EEG data to remove noise.

[0755] Input: Raw EEG data

[0756] Output: filtered EEG data

[0757] Specific operation: The server uses the filtering function of the BrainFlow library to remove noise in specific frequency bands.

[0758] Step 3:

[0759] The server inputs the filtered brainwave data into a generative AI system to analyze the user's emotions.

[0760] Input: filtered EEG data

[0761] Output: User's emotional state (e.g., happy, sad)

[0762] How it works: The server inputs data into a generative AI model and runs an emotion analysis algorithm to identify the user's emotions.

[0763] Step 4:

[0764] The server determines actions within the metaverse based on the analyzed emotional state.

[0765] Input: User's emotional state

[0766] Output: Instructions for action within the metaverse (e.g., dance of joy)

[0767] Specific operation: The server determines the action to be taken according to the emotional state and sends the instruction to the metaverse system.

[0768] Step 5:

[0769] The device (HMD) controls an avatar in the metaverse based on action instructions received from the server.

[0770] Input: Action instructions within the metaverse

[0771] Output: Avatar behavior (e.g., dance of joy)

[0772] Specific actions: The device provides visual information to the user through the HMD and controls the avatar to perform the specified actions.

[0773] Step 6:

[0774] The user visually confirms actions within the metaverse through the HMD.

[0775] Input: Avatar movement

[0776] Output: Visual information (e.g., a scene of an avatar dancing for joy)

[0777] Specific operation: The user wears the HMD and visually confirms the avatar's movements within the metaverse in real time.

[0778] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0779] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0780] Another example of generative AI is Gemini (registered trademark) (Internet search engine). <url: https: gemini.google.com ?hl="ja">) are mentioned.

[0781] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0782] [Second embodiment]

[0783] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0784] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0785] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0786] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0787] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0788] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0789] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0790] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0791] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0792] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0793] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0794] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0795] "Example 1"

[0796] One embodiment of the present invention is a system that generates a brain-machine interface (BMI) that combines electroencephalograms and generative AI. This system analyzes a user's electroencephalograms in real time and inputs the results into the generative AI. The generative AI understands the user's intentions based on this electroencephalogram data and generates output based on that. For example, when a user thinks, "I want to drink coffee," the system analyzes their electroencephalograms, and the generative AI understands that intention and can take action such as "making coffee."

[0797] "Example 2"

[0798] One embodiment of the present invention is a platform for popularizing BMI in society. This platform has a function that allows users to manage their own EEG data and share it with third parties as needed. For example, users can manage their own EEG data on the platform and share it with medical institutions and research institutions, thereby enabling more accurate medical services and research results.

[0799] "Example 3"

[0800] Furthermore, one embodiment of the present invention provides new services through the platform. These services include an intuitive communication service that utilizes brain activity, a metaverse interface, a health management service, and a biometric authentication service. For example, the intuitive communication service analyzes a user's brain waves to understand their emotions and intentions and communicate based on that information. The metaverse interface analyzes a user's brain waves to control their behavior in the VR or AR world. The health management service analyzes a user's brain waves to understand their stress level, sleep state, and other factors and proposes health management based on that information. The biometric authentication service analyzes a user's brain wave patterns to perform personal authentication.

[0801] The processing flow of each embodiment will be described below.

[0802] "Example 1"

[0803] Step 1: The user has a specific thought or intention.

[0804] Step 2: The system captures and analyzes the user's brainwaves in real time.

[0805] Step 3: Input the analysis results into the generative AI.

[0806] Step 4: The generative AI understands the user's intention based on the EEG data and generates an output based on that. For example, it takes an action such as "make coffee."

[0807] "Example 2"

[0808] Step 1: The user accesses the platform and uploads their EEG data.

[0809] Step 2: The platform manages the user's EEG data and shares it with third parties as necessary. For example, sharing it with medical institutions or research institutions will enable more accurate medical services and research results.

[0810] "Example 3"

[0811] Step 1: The user selects a specific service through the platform, such as an intuitive communication service, a metaverse interface, a health management service, or a biometric authentication service.

[0812] Step 2: The selected service analyzes the user's brainwaves and provides services based on them. For example, an intuitive communication service analyzes the user's brainwaves to understand their emotions and intentions, and communicates based on that.

[0813] Example 1

[0814] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0815] Conventional brain-machine interfaces (BMIs) have low accuracy in analyzing the user's brain waves, making it difficult to understand the user's intentions in real time. Furthermore, these systems are highly invasive and place a heavy burden on the user. Furthermore, it is not easy to manage and share brain wave data, which has hindered their widespread adoption in society.

[0816] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0817] In this invention, the server includes: means for collecting a user's brain waves in real time; means for analyzing the collected brain wave data; means for inputting the analyzed brain wave data into a generative AI model; means for the generative AI model to understand the user's intention and generate output based on that understanding; means for feeding back the generated output to the user; means for providing a platform for popularizing the brain-machine interface (BMI) in society; and means for providing intuitive communication services, virtual space interfaces, health management services, and biometric authentication services that utilize brain activity through the platform. This enables highly accurate and minimally invasive analysis of a user's brain waves and understanding of intentions in real time. Furthermore, it facilitates the management and sharing of brain wave data, promoting its widespread use in society.

[0818] "User" refers to an individual who uses a Brain-Machine Interface (BMI).

[0819] "Eeg" refers to the electrical signals generated by the user's brain.

[0820] "Real-time" refers to data being processed and analyzed as it is generated.

[0821] "Means for collection" refers to devices and methods for detecting the user's brain waves and acquiring them as data.

[0822] "Means for analyzing" refers to a method or device for processing collected EEG data and inferring the user's intentions.

[0823] A "generative AI model" refers to an artificial intelligence model that understands the user's intent based on input data and generates output based on that.

[0824] "Means of input" refers to the method or device for providing analyzed EEG data to a generative AI model.

[0825] "Output" refers to the results or suggestions that a generative AI model generates by understanding the user's intent.

[0826] "Feedback means" refers to the method or device used to communicate the generated output to the user.

[0827] "Platform" refers to the systems and services that form the foundation for popularizing brain-machine interfaces (BMIs) in society.

[0828] "Communication service" refers to a service that uses brain waves to enable users to intuitively exchange information with others.

[0829] A "virtual space interface" refers to an interface that allows users to operate and move within a virtual space using their brain waves.

[0830] "Health management service" refers to a service that uses a user's brainwave data to monitor and manage their health condition.

[0831] "Biometric authentication service" refers to a service that uses a user's brainwave data to authenticate an individual.

[0832] This invention is a brain-machine interface (BMI) system that analyzes a user's brain waves in real time and inputs the results into a generative AI model to understand the user's intentions and generate output based on those intentions. This system is implemented using the following hardware and software.

[0833] Hardware and Software Configuration

[0834] Brainwave sensor

[0835] The user wears an EEG sensor on their head, which detects the user's brain waves in real time and collects the data as electrical signals.

[0836] server

[0837] The server receives the EEG data sent from the EEG sensor, which is then sent to the device for analysis.

[0838] Terminal

[0839] The device analyzes the EEG data sent from the server using a Python-based EEG analysis library. The device then sends the analysis results to the server.

[0840] Generative AI Model

[0841] The server inputs the analyzed EEG data into a generative AI model (such as OpenAI's GPT-4), which understands the user's intent based on the input data and generates output based on that understanding.

[0842] feedback

[0843] The generated output is sent to the device via the server, and the device provides it as feedback to the user, who can then take the suggested action.

[0844] Specific examples

[0845] For example, consider the case where a user wants to "listen to music." The EEG sensor collects the user's EEG data, and the device analyzes the data. If the analysis result indicates the user's intent to "listen to music," the following prompt sentence is input to the generative AI model:

[0846] The user's EEG data has detected the intent "I want to listen to music." Please suggest an action to play appropriate music.

[0847] Based on this prompt, the generative AI model generates the output "play music" and sends it back to the user via their device. The user can then play the music according to the suggested action.

[0848] In this way, the system can analyze the user's brain waves with high accuracy and minimal invasiveness, enabling them to understand their intentions in real time. It also makes it easier to manage and share brain wave data, promoting its widespread adoption in society.

[0849] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0850] Step 1:

[0851] The user wears an EEG sensor on their head. The EEG sensor detects the user's brain waves in real time and collects data as electrical signals. The collected EEG data is then sent to a server.

[0852] Input: User's brainwaves

[0853] Output: EEG data (electrical signal)

[0854] Step 2:

[0855] The server receives the EEG data transmitted from the EEG sensor, which is then sent to the device for analysis.

[0856] Input: EEG data (electrical signal)

[0857] Output: EEG data (transferred from server to device)

[0858] Step 3:

[0859] The device receives the EEG data sent from the server. The device analyzes the EEG data using a Python-based EEG analysis library. During the analysis, specific EEG patterns are detected and the user's intentions are inferred. The analysis results are then sent to the server.

[0860] Input: EEG data (transferred from the server)

[0861] Output: Analysis results (user intent)

[0862] Step 4:

[0863] The server receives the analysis results sent from the device, generates a prompt sentence based on the analysis results, and inputs it into the generative AI model.

[0864] Input: Analysis result (user intent)

[0865] Output: prompt statement

[0866] Step 5:

[0867] The generative AI model receives the prompt sent from the server. The generative AI model understands the user's intent based on the prompt and generates output based on that. The generated output is then sent to the server.

[0868] Input: prompt statement

[0869] Output: Output

[0870] Step 6:

[0871] The server receives the output sent from the generative AI model and sends the output to the device.

[0872] Input: Output (transferred from generative AI model)

[0873] Output: Output (transfer from server to device)

[0874] Step 7:

[0875] The terminal receives the output sent from the server, and feeds the output back to the user, who can then take the suggested action.

[0876] Input: Output (transfer from server)

[0877] Output: Feedback to the user

[0878] (Application example 1)

[0879] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0880] Conventional factory robot control methods mainly involve manual operation or program control, which poses the problem of difficulty in intuitive operation. Furthermore, in order to improve work efficiency, a more intuitive and rapid control method is required. Furthermore, brain-machine interfaces (BMIs) that use brain waves are being increasingly applied in the fields of medicine and entertainment, but have not yet been fully utilized in factory automation or robot control. To solve these issues, a system that can intuitively control factory robots using brain waves is needed.

[0881] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0882] In this invention, the server includes: means for generating a minimally invasive, highly accurate brain-machine interface (BMI) that combines brainwaves and generative AI; means for providing a platform for popularizing the BMI throughout society; means for providing intuitive communication services, metaverse interfaces, health management services, and biometric authentication services that utilize brain activity, all of which are provided through the platform; means for factory workers to wear head-mounted displays and control factory robots using brainwaves; means for analyzing the user's brainwave data in real time and inputting the results into generative AI to understand the user's intentions and generate output based on those intentions; and means for controlling factory robots based on the user's intentions, thereby enabling factory workers to intuitively control factory robots using brainwaves.

[0883] "Electroencephalograms" are electrical signals generated by neural activity in the brain.

[0884] "Generative AI" is a type of artificial intelligence that generates new data and output based on input data.

[0885] A brain-machine interface (BMI) is an interface that uses brain waves to control machines and computers.

[0886] "Minimally invasive" refers to minimal strain or damage to the body.

[0887] "High precision" means very accurate.

[0888] A "platform" is a system or environment that serves as the foundation for providing specific services or functions.

[0889] An "intuitive communication service" is a service that allows users to operate and communicate intuitively.

[0890] A "metaverse interface" is an interface for performing activities and operations in a virtual space.

[0891] "Health management service" refers to a service for managing and improving the user's health condition.

[0892] "Biometric authentication service" refers to a service that authenticates individuals using biometric information such as fingerprints or facial recognition.

[0893] A "factory worker" is a worker engaged in production activities in a factory.

[0894] A "head-mounted display" is a display device that is worn on the head.

[0895] A "factory robot" is a robot used to automate production work in a factory.

[0896] "Real-time" refers to processing or reaction occurring immediately, without delay.

[0897] "User intent" refers to what the user wants to do or what they are thinking.

[0898] "Output" is the result or action that is produced by a system or device.

[0899] To implement the present invention, the following system configuration and program are required.

[0900] System Configuration

[0901] 1. Hardware

[0902] Head-mounted display (HMD): Equipped with a built-in brainwave sensor, it collects brainwave data in real time.

[0903] Factory robot: A robot that performs actions based on the user's intentions.

[0904] Server: Analyzes EEG data and runs generative AI models.

[0905] 2. Software

[0906] BrainwaveReader: A module for collecting EEG data.

[0907] Generative AI Model: A generative AI model that analyzes brainwave data and understands user intent.

[0908] RobotController: A module for controlling a robot based on the user's intentions.

[0909] Program processing explanation

[0910] The server first collects brainwave data from a head-mounted display (HMD). The BrainwaveReader module analyzes this data in real time and inputs it into a generative AI model (GenerativeAIModel). The generative AI model interprets the user's intention based on the brainwave data and passes that intention to the RobotController. The RobotController controls the factory robot based on the interpreted intention and performs the corresponding action.

[0911] Specific examples

[0912] For example, if a factory worker thinks, "I'm going to assemble parts," the following prompt sentence is input into the generative AI model:

[0913] Example prompt sentence:

[0914] Analysis of the user's brainwave data reveals that the user is thinking "assemble parts." Based on this intent, instruct the robot to assemble the parts.

[0915] Based on this prompt, the generative AI model interprets the user's intention and instructs the RobotController to "assemble the parts," which causes the factory robot to automatically begin assembling the parts.

[0916] This system allows factory workers to intuitively control factory robots using their brain waves, greatly improving work efficiency.

[0917] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0918] Step 1:

[0919] The user wears a head-mounted display (HMD). The HMD has a built-in brainwave sensor that collects the user's brainwave data in real time. The input is the user's brainwaves, and the output is the collected brainwave data.

[0920] Step 2:

[0921] The server uses the BrainwaveReader module to receive EEG data transmitted from the HMD. BrainwaveReader analyzes this data in real time, removing noise and extracting features. The input is EEG data from the HMD, and the output is the analyzed EEG data.

[0922] Step 3:

[0923] The server inputs the analyzed EEG data into the GenerativeAIModel, which is a generative AI model for interpreting the user's intentions based on EEG data. The input is the analyzed EEG data, and the output is data that indicates the user's intentions.

[0924] Step 4:

[0925] The server generates a prompt based on the results of its interpretation of the user's intent. This prompt specifically describes the user's intent. The input is data that indicates the user's intent, and the output is the prompt.

[0926] Step 5:

[0927] The server passes the generated prompt sentence to the RobotController, which generates specific instructions for the factory robot based on this prompt sentence. The input is the prompt sentence, and the output is the instruction for the robot.

[0928] Step 6:

[0929] The RobotController sends instructions to the factory robots, which then perform the corresponding actions based on these instructions. The input is the instruction to the robot, and the output is the robot's action.

[0930] Step 7:

[0931] The user checks the operation of a factory robot through an HMD. The HMD displays the robot's operating status in real time, allowing the user to check whether the robot is operating as intended. The input is the robot's operating status, and the output is the image displayed on the HMD.

[0932] Example 2

[0933] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0934] Conventional brain-machine interfaces (BMIs) lacked a platform for effectively managing users' EEG data and safely sharing it with third parties. Furthermore, real-time analysis of EEG data and integration with generative AI were insufficient, making it difficult to accurately understand the user's intentions and generate appropriate output. Furthermore, data security was not ensured due to insufficient encryption of EEG data and management of access rights.

[0935] The specific processing by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes a means for uploading EEG data acquired by a user using a dedicated measuring device from a terminal to the server, a means for the server to store the EEG data received in a database, a means for the user to access the platform and manage the data, a means for the user to set up data sharing with a third party, and a means for the server to encrypt the data and grant access rights to the third party. This allows the user to effectively manage their own EEG data and safely share it with a third party. Furthermore, by linking with generative AI, the system can accurately understand the user's intentions and generate appropriate output.

[0936] A "brain-machine interface (BMI)" is an interface that uses brain waves to communicate the user's intentions to a machine.

[0937] "Generative AI" is artificial intelligence that analyzes a user's brainwave data and generates appropriate output based on the results.

[0938] The "platform" is a system that allows users to manage their EEG data and share it with third parties as needed.

[0939] "Electroencephalogram data" refers to data obtained by measuring the user's brain activity, and is obtained using an electroencephalogram measuring device.

[0940] A "measuring device" is a device for measuring a user's brain waves in real time.

[0941] A "terminal" is a device such as a computer or smartphone that allows a user to upload EEG data acquired using a measurement device to a server.

[0942] A "server" is a computer system for storing and managing brain wave data received from a user.

[0943] The "database" is a system for efficiently storing and accessing the EEG data received by the server.

[0944] "Encryption" is a technology that converts data into a format that cannot be deciphered by third parties.

[0945] "Access authority" is a function that sets the rights that allow specific users or third parties to access data.

[0946] "Output" refers to the output generated by the generative AI based on the results of analyzing the user's brainwave data.

[0947] MODE FOR CARRYING OUT THE INVENTION

[0948] This invention provides a system that provides a platform for users to manage EEG data acquired using a dedicated measuring device and share it with third parties as needed. Specific embodiments of this system are described below.

[0949] Hardware and Software Configuration

[0950] The user acquires EEG data using a dedicated EEG measuring device, which measures the user's EEG in real time and transmits the data to a device (such as a PC or smartphone).

[0951] The device uploads the acquired EEG data to a server via a dedicated application, which converts the data into an appropriate format (e.g., CSV or JSON) and transmits it securely to the server using the HTTPS protocol.

[0952] The server stores the received EEG data in a database (e.g., MySQL or PostgreSQL). The database organizes the data for each user and sets up indexes for efficient access.

[0953] Users access the platform through a web browser or mobile application. The platform authenticates users (e.g., via OAuth or JWT) and provides an interface where users can view, edit, and delete their own EEG data.

[0954] Data Sharing and Encryption

[0955] If a user wants to share data with a third party, they select the third party (e.g., a medical institution or research institution) they want to share with on the platform, set the scope and period of data to be shared, and submit a sharing request.

[0956] The server receives the user's sharing request, encrypts the data to be shared, and grants access privileges to the designated third party, generating a secure access link to the shared data.

[0957] Specific examples

[0958] For example, if a user wants to share their EEG data with a medical institution, they follow the steps below.

[0959] 1. The user wears the EEG sensor and launches the dedicated application.

[0960] 2. The device converts the acquired EEG data into CSV format and uploads it to the server using HTTPS.

[0961] 3. The server stores the received data in a MySQL database.

[0962] 4. The user logs into the platform using a web browser and checks their own EEG data.

[0963] 5. The user selects a medical institution and sets the scope and period of data to be shared.

[0964] 6. The server encrypts the data with AES-256 and grants access privileges to the medical institution.

[0965] Example prompts to input to a generative AI model:

[0966] "Please explain the steps a user takes to use the EEG sensor to obtain EEG data and share it with a medical institution."

[0967] By inputting this prompt into the generative AI model, the model outputs specific steps for the user to obtain EEG data and share it with a medical institution.

[0968] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0969] Step 1:

[0970] The user acquires EEG data using a dedicated EEG measurement device. The user wears the measurement device and launches a dedicated application. The measurement device measures the user's EEG in real time and transmits the data to a terminal. The input is the user's EEG, and the output is the EEG data transmitted to the terminal.

[0971] Step 2:

[0972] The device uploads the EEG data to the server. The device converts the acquired EEG data into CSV format using a dedicated application and sends it to the server using the HTTPS protocol. The input is the EEG data sent from the measurement device, and the output is the CSV-formatted EEG data uploaded to the server.

[0973] Step 3:

[0974] The server stores the received EEG data in a database. The server imports the received CSV format EEG data into the database and organizes the data for each user. The input is the CSV format EEG data sent from the device, and the output is the EEG data stored in the database.

[0975] Step 4:

[0976] Users access the platform and manage their data. They log in to the platform through a web browser or mobile application to view, edit, and delete their own EEG data. The input is the user's login information and the EEG data stored in the database, and the output is the EEG data that the user viewed, edited, or deleted.

[0977] Step 5:

[0978] The user sets up data sharing with third parties. The user selects the third parties with whom they want to share data on the platform and sets the scope and period of data to be shared. The input is the user's sharing setting information, and the output is a sharing request sent to the server.

[0979] Step 6:

[0980] The server encrypts the data and grants access permissions to the third party. The server receives the user's sharing request and encrypts the data to be shared with AES-256. The server grants access permissions to the specified third party and generates a secure access link to the shared data. The input is the user's sharing request and the EEG data stored in the database, and the output is the encrypted data and the access permissions granted to the third party.

[0981] (Application example 2)

[0982] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0983] Conventional brain-machine interfaces (BMIs) lacked a platform for effectively managing users' EEG data and sharing it with third parties as needed. Furthermore, there was insufficient management of EEG data security and sharing history, making it difficult to smoothly share data with medical and research institutions. Furthermore, there was a lack of functionality to collect users' EEG data in real time, encrypt it, and upload it to the cloud, which meant that data security and convenience were not ensured.

[0984] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for generating a minimally invasive, highly accurate brain-machine interface (BMI) that combines electroencephalograms and generative AI; means for providing a platform for popularizing the BMI throughout society; means for providing intuitive communication services using brain activity, a metaverse interface, health management services, and biometric authentication services provided through the platform; means for collecting a user's electroencephalogram data in real time, encrypting it, and uploading it to the cloud; and means for sharing the data with medical institutions and research institutions and managing the data sharing history. This makes it possible to safely and efficiently manage a user's electroencephalogram data and share it with third parties as needed.

[0985] "Electroencephalograms" are electrical signals generated by neural activity in the brain.

[0986] "Generative AI" is a type of artificial intelligence, a system that has the ability to generate new data and information based on input data.

[0987] "Minimally invasive" means causing less strain and damage to the body.

[0988] "High accuracy" means very accurate.

[0989] A "brain-machine interface (BMI)" is an interface that uses brain waves to operate machines and computers.

[0990] A "platform" is a system or environment that serves as the foundation for providing specific functions or services.

[0991] An "intuitive communication service" is one that allows users to exchange information in a natural way.

[0992] A "metaverse interface" is an interface that allows users to operate and experience things within a virtual space.

[0993] A "health management service" is a service for monitoring and managing a user's health condition.

[0994] A "biometric authentication service" is a service that verifies a user's identity using their biometric information.

[0995] "Real-time" means that data or information is processed immediately.

[0996] "Encryption" means transforming data using a specific algorithm into a form that is not easily understandable to third parties.

[0997] The "cloud" is a collection of computer resources and services provided over the Internet.

[0998] "Sharing history" is a record of who shared data, when, and how.

[0999] A "medical institution" is a facility that provides medical services, such as a hospital or clinic.

[1000] A "research institution" is an organization or facility for conducting scientific research.

[1001] The following system configuration will be described as an embodiment of the present invention.

[1002] System Configuration

[1003] The system collects users' brainwave data in real time, encrypts it, and uploads it to the cloud. It also has the ability to share the data with medical and research institutions and manage the data sharing history.

[1004] Hardware used

[1005] Smartphone: A device that allows users to manage EEG data and upload it to the cloud.

[1006] Brainwave sensor: A device used to collect a user's brainwave data (e.g., Muse, Emotiv).

[1007] Software used

[1008] Python: A programming language used to perform the main operations of a program.

[1009] NeuroKit2: A library for simulating and analyzing EEG data.

[1010] Requests: A library for sending HTTP requests.

[1011] Cryptography: A library for performing data encryption.

[1012] Processing flow

[1013] 1. EEG data collection: EEG sensors are used to collect users' EEG data in real time, which is then stored on the smartphone.

[1014] 2. Data encryption: The collected EEG data is encrypted using the Cryptography library, which ensures the security of the data.

[1015] 3. Data Upload: The encrypted data is uploaded to the cloud using the Requests library, where it is securely stored.

[1016] 4. Data sharing: Users can share data with medical institutions or research institutes as needed. Sharing history is stored locally, allowing users to see who accessed what data and when.

[1017] Specific examples

[1018] The user wears the EEG sensor and launches the NeuroGuard app. The app collects EEG data, encrypts it, and uploads it to the cloud. Medical institutions can then retrieve the data from the cloud with the user's permission and use it for diagnosis.

[1019] Prompt Sentence Examples

[1020] The user wears the EEG sensor and launches the NeuroGuard app. The app collects EEG data, encrypts it, and uploads it to the cloud. With the user's permission, medical institutions can retrieve the data from the cloud and use it for diagnosis.

[1021] This system allows users' brainwave data to be managed safely and efficiently, and to be shared with third parties if necessary.

[1022] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1023] Step 1:

[1024] The user wears the EEG sensor and launches the NeuroGuard app on their smartphone. The EEG sensor collects the user's brainwave data in real time and transmits it to the smartphone. The input is the brainwave data from the EEG sensor, and the output is the raw brainwave data stored on the smartphone.

[1025] Step 2:

[1026] The device (smartphone) analyzes the collected EEG data using the NeuroKit2 library. This analysis extracts features from the EEG data. The input is raw EEG data, and the output is the features of the analyzed EEG data. Specifically, it calls NeuroKit2 functions to analyze the EEG data and extract features.

[1027] Step 3:

[1028] The device encrypts the analyzed EEG data using the Cryptography library. The input is the characteristics of the analyzed EEG data, and the output is the encrypted EEG data. Specifically, the device generates an encryption key and uses the key to encrypt the EEG data.

[1029] Step 4:

[1030] The device uses the Requests library to upload encrypted EEG data to the cloud. The input is encrypted EEG data, and the output is encrypted data stored on the cloud. Specifically, the device sends an HTTP POST request to upload the data to the cloud server.

[1031] Step 5:

[1032] The server manages the encrypted data stored on the cloud and shares it with medical institutions and research institutions as needed. The input is the encrypted data on the cloud, and the output is the shared data. Specifically, with the user's permission, the server decrypts the data and provides it to a designated third party.

[1033] Step 6:

[1034] The server manages the data sharing history and records who accessed which data and when. The input is the data access request information, and the output is a log of the sharing history. Specifically, it generates an access log and stores it locally or on the cloud.

[1035] In this way, the user's brain wave data can be managed safely and efficiently and shared with third parties as needed.

[1036] Example 3

[1037] Next, a description will be given of Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1038] Conventional brain-machine interfaces (BMIs) have had difficulty effectively analyzing a user's brainwave data and identifying emotions and intentions in real time. Furthermore, there has been a lack of a platform for providing services based on this data, such as intuitive communication, health management, and biometric authentication. Furthermore, there is a lack of functionality for users to manage their own brainwave data and share it with third parties as needed. To solve these issues, a system that can analyze brainwave data with high precision and provide a variety of services based on that data is needed.

[1039] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[1040] In this invention, the server includes: means for a user to wear an EEG measuring device and connect it to a terminal; means for the terminal to collect EEG data from the EEG measuring device and transmit it to the server; means for the server to receive and preprocess the EEG data; means for the server to analyze the preprocessed data and identify emotions and intentions; means for the server to transmit the analysis results to the terminal; means for the terminal to display the analysis results and provide feedback to the user; means for providing a platform for popularizing the system in society; and means for providing intuitive communication services, virtual reality interfaces, health management services, and biometric authentication services that utilize brain activity, which are provided through the platform. This makes it possible to analyze users' EEG data with high accuracy and provide a variety of services based on the analysis results in real time.

[1041] "User" refers to an individual who wears an EEG measurement device and uses the system.

[1042] "Electroencephalogram measuring device" refers to a device for measuring a user's electroencephalogram data.

[1043] "Terminal" refers to an electronic device that transmits data collected from an EEG measuring device to a server and displays the analysis results to the user.

[1044] "Server" refers to a computer system that receives EEG data, performs pre-processing and analysis, and transmits the results to a terminal.

[1045] "Preprocessing" refers to the cleansing of data before analysis, such as removing noise and filtering EEG data.

[1046] "Analysis" refers to data processing to identify the user's emotions and intentions based on preprocessed EEG data.

[1047] "Generative AI model" refers to a machine learning model used to analyze EEG data and identify a user's emotions and intent.

[1048] The "platform" refers to the infrastructure for disseminating the system to society, allowing users to manage their EEG data, and share it with third parties as needed.

[1049] An "intuitive communication service" is a service that understands a user's emotions and intentions based on their brainwave data and communicates based on that.

[1050] A "virtual reality interface" refers to an interface that controls behavior in virtual reality or augmented reality worlds based on the user's brainwave data.

[1051] "Health management service" refers to a service that uses a user's brainwave data to understand their stress level, sleep state, etc., and makes health management suggestions based on that information.

[1052] "Biometric authentication service" refers to a service that analyzes a user's brain wave patterns to authenticate them.

[1053] This invention begins with the user wearing an EEG device and connecting it to a terminal. The user wears the EEG device (e.g., EEG device) on their head and connects it to a terminal (e.g., smartphone or PC) using Bluetooth or a USB cable.

[1054] The device collects EEG data in real time from the EEG device using a dedicated application. The collected data is then sent to a server via the internet. The server receives the EEG data and performs preprocessing. This preprocessing includes noise removal and filtering, and data cleansing is performed using the Python MNE library.

[1055] The preprocessed data is then fed into a machine learning model (e.g., using TensorFlow or PyTorch) on the server to analyze emotions and intent. The analysis results are used to identify the user's emotions (e.g., joy, sadness, surprise, etc.) and intent (e.g., the desire to perform a specific action).

[1056] The analysis results are sent from the server to the device. The device then displays the analysis results received from the server to the user. For example, a chat app might display a message saying "The user is happy." The metaverse interface also controls behavior in the VR or AR world based on the user's intentions.

[1057] As a concrete example, consider an intuitive communication service. A user wears an EEG device and connects it to a smartphone. The smartphone collects brainwave data from the EEG device in real time and sends it to a server. The server preprocesses the received data and analyzes emotions using TensorFlow. If the analysis shows that the user is "happy," that information is reflected in the smartphone's chat app, and a message saying "the user is happy" is sent to the other party.

[1058] Example prompt sentence:

[1059] The user wears an EEG device and connects it to a smartphone. The smartphone collects brainwave data in real time and sends it to a server. The server preprocesses the data and analyzes emotions using TensorFlow. If the analysis shows that the user is "happy," the server reflects this information in a chat app and sends the other party a message saying "the user is happy."

[1060] In this way, a system is realized in which the server, the terminal, and the user work together to provide new services using electroencephalogram data. The flow of the identification process in the third embodiment will be described with reference to FIG.

[1061] Step 1:

[1062] The user wears the EEG measurement device and connects it to the terminal.

[1063] Specifically, the user wears an EEG device on their head and connects it to a device (e.g., a smartphone or PC) using Bluetooth or a USB cable. The input is the user's brainwaves, and the output is the EEG device connected to the device.

[1064] Step 2:

[1065] The terminal collects brain wave data from the EEG measuring device and transmits it to the server.

[1066] Specifically, the terminal uses a dedicated application to collect brainwave data from the EEG device in real time. The collected data is then sent to a server via the Internet. The input is the brainwave data from the EEG measurement device, and the output is the brainwave data sent to the server.

[1067] Step 3:

[1068] The server receives the EEG data and performs preprocessing.

[1069] Specifically, the server receives EEG data sent from the device. The received data undergoes preprocessing such as noise removal and filtering. The data is cleansed using the Python MNE library. The input is the EEG data sent from the device, and the output is the preprocessed EEG data.

[1070] Step 4:

[1071] The server analyzes the pre-processed data to identify emotions and intent.

[1072] Specifically, the server inputs the preprocessed data into a machine learning model (using, for example, TensorFlow or PyTorch). The model identifies the user's emotions and intentions from the EEG data. The input is the preprocessed EEG data, and the output is the analysis result, which is information on emotions and intentions.

[1073] Step 5:

[1074] The server sends the analysis results to the device.

[1075] Specifically, the server sends the analysis results to the device. The analysis results include the user's emotions (e.g., joy, sadness, surprise, etc.) and intentions (e.g., the intention to perform a specific action). The input is the analysis results, and the output is the analysis results sent to the device.

[1076] Step 6:

[1077] The device displays the analysis results and provides feedback to the user.

[1078] Specifically, the device displays the analysis results received from the server to the user. For example, in a chat app, it displays a message saying "The user is happy." In addition, the metaverse interface controls behavior in the VR or AR world based on the user's intentions. The input is the analysis results received from the server, and the output is the feedback displayed to the user.

[1079] (Application example 3)

[1080] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1081] Conventional brain-machine interfaces (BMIs) have limited technology for analyzing users' brain waves, making it difficult to perform high-precision analysis in real time. They also lacked intuitive means for navigation and interaction within the metaverse, making it difficult to execute actions that reflect the user's intentions and emotions. Furthermore, the functionality for managing and sharing brainwave data was inadequate, resulting in low user convenience.

[1082] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[1083] In this invention, the server includes: means for generating a minimally invasive, highly accurate brain-machine interface (BMI) that combines brain waves and generative AI; means for providing a platform for popularizing the BMI throughout society; means for providing intuitive communication services, metaverse interfaces, health management services, and biometric authentication services that utilize brain activity through the platform; means for analyzing a user's brain waves in real time and controlling navigation and interaction within the metaverse; and means for executing actions within the metaverse based on the user's intentions and emotions. This enables users to intuitively navigate and interact within the metaverse using their brain waves.

[1084] "Electroencephalograms" are records of the brain's electrical activity and are used to analyze a user's intentions and emotions.

[1085] "Generative AI" is a type of artificial intelligence, a technology that generates new information and output based on input data.

[1086] A "brain-machine interface (BMI)" is an interface that analyzes brain waves to control machines and computers.

[1087] A "platform" is a system that serves as the foundation for providing specific services and functions.

[1088] An "intuitive communication service" is a service that analyzes the user's brain waves to understand their emotions and intentions, and then communicates based on that.

[1089] A "metaverse interface" is an interface that allows users to interact in virtual reality and augmented reality worlds.

[1090] The "health management service" analyzes the user's brain waves to determine their stress level and sleep state, and then makes health management suggestions based on that information.

[1091] "Biometric authentication service" is a service that analyzes a user's brain wave patterns to authenticate them.

[1092] "Navigation" refers to giving instructions and performing operations to allow a user to move within a virtual space.

[1093] "Interaction" refers to a user interacting with other objects or characters in a virtual space.

[1094] "Intent" is the user's intention to perform a particular action or operation.

[1095] "Emotion" refers to the user's psychological state or feelings.

[1096] An "action" is a movement or reaction that is performed in a virtual space based on the user's intentions and emotions.

[1097] A system for implementing this invention includes means for generating a minimally invasive, highly accurate brain-machine interface (BMI) that combines brain waves and generative AI, means for providing a platform for popularizing the BMI throughout society, means for providing intuitive communication services, metaverse interfaces, health management services, and biometric authentication services that utilize brain activity provided through the platform, means for analyzing a user's brain waves in real time and controlling navigation and interaction within the metaverse, and means for executing actions within the metaverse based on the user's intentions and emotions.

[1098] System configuration

[1099] The system uses the following hardware and software:

[1100] Hardware: EEG sensor, head-mounted display

[1101] Software: Python, EEG sensor API, Metaverse controller API

[1102] Program processing explanation

[1103] The server analyzes the brainwave data acquired from the EEG sensor in real time and inputs it into the generative AI. The generative AI analyzes the user's intentions and emotions based on the input brainwave data and generates output based on that. Specifically, if the user wants to move forward, the brainwave sensor detects that intention and the avatar moves forward. Also, if the user feels happy, that emotion is detected and the avatar smiles.

[1104] Specific examples

[1105] For example, if a user wants to move forward in the Metaverse, the EEG sensor will detect that intention and the avatar will move forward. If the user feels happy, the sensor will detect that emotion and the avatar will smile.

[1106] Prompt Sentence Examples

[1107] An example of a prompt to input to a generative AI model is as follows:

[1108] If the user wants to move forward, the EEG sensor should detect that intention and generate a program that makes the avatar move forward. Also, if the user feels happy, the sensor should detect that emotion and make the avatar smile.

[1109] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[1110] Step 1:

[1111] The server acquires the brain wave data from the brain wave sensor.

[1112] Input: Raw data from EEG sensor

[1113] Output: Acquired EEG data

[1114] Specific operation: The EEG sensor measures the user's brain waves in real time and sends the data to the server.

[1115] Step 2:

[1116] The server preprocesses the acquired electroencephalogram data.

[1117] Input: Acquired EEG data

[1118] Output: Preprocessed EEG data

[1119] Specific operation: Perform preprocessing such as noise removal and filtering, and convert the data into a format suitable for analysis.

[1120] Step 3:

[1121] The server inputs the preprocessed EEG data into the generative AI.

[1122] Input: Preprocessed EEG data

[1123] Output: Data input to the generative AI

[1124] Specific operation: Preprocessed EEG data is input into a generative AI model to analyze the user's intentions and emotions.

[1125] Step 4:

[1126] Generative AI analyzes the user's intentions and emotions based on input brainwave data.

[1127] Input: Data entered into the generative AI

[1128] Output: Analyzed intent and emotion

[1129] How it works: A generative AI model analyzes EEG data to determine the user's intentions and emotions.

[1130] Step 5:

[1131] The server determines actions within the metaverse based on the analysis results.

[1132] Input: Parsed intent and emotion

[1133] Output: Actions taken within the metaverse

[1134] Specific actions: If the user's intention is "move forward," the avatar will move forward; if the emotion is "happy," the avatar will smile.

[1135] Step 6:

[1136] The server sends the determined action to the metaverse controller.

[1137] Input: Actions performed within the metaverse

[1138] Output: Action instructions sent to the metaverse controller

[1139] Specific action: The determined action is sent through the metaverse controller's API, instructing the avatar to actually perform the action.

[1140] Step 7:

[1141] The metaverse controller controls the avatar based on the received action instructions.

[1142] Input: Action instructions sent to the Metaverse Controller

[1143] Output: Avatar movement

[1144] Specific behavior: The Metaverse Controller controls the avatar and performs actions based on the user's intentions and emotions, such as making the avatar move forward or smile.

[1145] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1146] "Example 1"

[1147] In one embodiment of the present invention, a brain-machine interface (BMI) analyzes the user's brain waves in real time and inputs the results into a generative AI. Furthermore, by combining an emotion engine, the user's emotional state is also analyzed simultaneously. Based on the analysis results, the system understands the user's intentions and emotional state and generates output based on that. For example, if a user feels joy and thinks, "I'm going to make coffee," the system recognizes that joy and takes appropriate action to make coffee.

[1148] "Example 2"

[1149] In another embodiment of the present invention, the platform has a function that allows users to manage their own brain wave data and emotional state and share it with third parties as needed. For example, sharing brain wave data when a user is feeling joy with medical institutions or research institutions can lead to more accurate medical services and research results.

[1150] "Example 3"

[1151] In a further embodiment of the present invention, the services provided through the platform have a function of providing services that utilize the user's emotional state. For example, an intuitive communication service analyzes the user's brain waves to understand their emotions and intentions and communicates based on them. If the user is feeling happy, the service will recognize that feeling and provide communication tailored to that emotion.

[1152] The processing flow of each embodiment will be described below.

[1153] "Example 1"

[1154] Step 1: The user thinks about a specific action, for example, "make coffee."

[1155] Step 2: The brain-machine interface (BMI) analyzes the user's brain waves in real time.

[1156] Step 3: The emotion engine analyzes the user's emotional state. In this example, it analyzes that the user has the emotion of joy.

[1157] Step 4: Based on the analysis results, understand the user's intention and emotional state and generate output based on that. In this example, we will trigger the action of making coffee in line with the emotion of joy. Rub.

[1158] "Example 2"

[1159] Step 1: The user experiences a specific emotion, for example, joy.

[1160] Step 2: The platform manages the user's brainwave data and emotional state.

[1161] Step 3: The user optionally shares their EEG data and emotional state with third parties, in this example, medical or research institutions.

[1162] "Example 3"

[1163] Step 1: A user uses a service through the platform, for example, an intuitive communication service.

[1164] Step 2: The service analyzes the user's brainwaves to understand their emotions and intentions.

[1165] Step 3: The service uses the analysis results to provide communication tailored to the user's emotional state. In this example, if the user is feeling happy, the service provides communication tailored to that emotion.

[1166] Example 1

[1167] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1168] Conventional brain-machine interfaces (BMIs) only analyzed the user's brain waves, but were unable to take the user's emotional state into account. This made it difficult to accurately understand the user's intentions and generate appropriate output. Furthermore, there was a lack of a platform for managing the user's brainwave data and sharing it with third parties. This hindered the widespread adoption of BMIs in society, making it difficult to provide intuitive communication services, health management services, and biometric authentication services.

[1169] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1170] In this invention, the server includes means for acquiring a user's brain waves in real time, means for analyzing the acquired brain wave data and extracting the user's intention, means for analyzing the user's emotional state, means for generating and inputting prompt sentences to the generative AI based on the analysis results, and means for executing the output generated by the generative AI. This enables the system to accurately understand the user's intention and emotional state and generate appropriate output. Furthermore, a platform for disseminating this system throughout society can be provided, allowing for intuitive communication services, virtual space interfaces, health management services, and biometric authentication services that utilize brain activity.

[1171] "User" refers to an individual who uses the System.

[1172] "Electroencephalograms" refer to electrical signals generated by neural activity in the brain.

[1173] "Real-time" refers to processing data as it is generated.

[1174] "Means of acquisition" refers to the devices and methods used to collect data.

[1175] "Means for analyzing" refers to devices and methods for analyzing acquired data and extracting meaningful information.

[1176] "Intent" refers to information that indicates what a user wants to do or what they are thinking.

[1177] "Emotional state" refers to information that indicates a user's emotions or mood.

[1178] "Generative AI" refers to artificial intelligence that generates new information or output based on given data.

[1179] A "prompt sentence" refers to an input sentence used to give instructions or ask questions to a generative AI.

[1180] "Output" refers to the results or responses that a generative AI generates based on a prompt.

[1181] "Means of execution" refers to the devices and methods used to convert the output generated by generative AI into concrete actions or operations.

[1182] "Platform" refers to the infrastructure for operating the system and providing services to users.

[1183] An "intuitive communication service" is a service that allows users to exchange information with others in a natural way.

[1184] "Virtual space interface" refers to an interface that allows users to operate and experience things within a virtual environment.

[1185] "Health management service" refers to a service for monitoring and managing a user's health status.

[1186] "Biometric authentication service" refers to a service that verifies a user's identity using their biometric information.

[1187] This invention is a system that analyzes a user's brain waves in real time and inputs the results into a generative AI to understand the user's intentions and emotional state and generate output based on that. This system includes an EEG measurement device for acquiring the user's brain waves, software for analyzing the EEG data, an emotion engine for analyzing the emotional state, the generative AI, and hardware for executing the generated output.

[1188] Hardware and software used

[1189] 1. EEG measurement device:

[1190] The user wears a specially designed electroencephalogram (EEG) measuring device, which captures the user's brainwaves in real time.

[1191] 2. EEG analysis software:

[1192] The device uses EEG analysis software (e.g., OpenBCI) to analyze the EEG data acquired from the EEG measurement device. This software analyzes the EEG data and extracts the user's intentions.

[1193] 3. Emotion Engine:

[1194] The device uses an emotion engine to analyze the user's emotional state, which obtains emotional data through facial recognition cameras and voice analysis to understand the user's emotional state.

[1195] 4. Generative AI:

[1196] Based on the analysis results, the server generates and inputs a prompt to the generative AI (e.g., OpenAI GPT-4), which then generates an appropriate output based on the prompt.

[1197] 5. Output Execution Hardware:

[1198] The server receives the output from the generative AI and operates hardware (e.g., IoT devices) to perform specific actions.

[1199] Specific examples

[1200] The user wakes up in the morning and puts on the EEG measurement device.

[1201] The terminal receives data from the EEG measurement device and begins analyzing it in real time.

[1202] The device detects the intention to "drink coffee" from brain wave data.

[1203] The device uses a facial recognition camera to confirm that the user is expressing "joy."

[1204] The device generates a prompt saying, "The user feels happy and wants to drink coffee. Please suggest an appropriate action."

[1205] The server sends a prompt to the generative AI, which then generates the output "instruct the coffee maker to make coffee."

[1206] The server controls the coffee maker through an IoT device and makes coffee.

[1207] Prompt Sentence Examples

[1208] User's EEG data: [DATA]

[1209] User's emotional state: joy

[1210] User Intent: I want a coffee.

[1211] Prompt for generative AI: The user has a feeling of joy and wants to "drink coffee." Suggest an appropriate action.

[1212] In this way, the system can analyze the user's brainwaves and emotional state, use generative AI to generate output based on the user's intentions, and execute specific actions.

[1213] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1214] Step 1:

[1215] The user wears a dedicated electroencephalogram (EEG) measuring device.

[1216] Input: User's brainwaves

[1217] Output: Real-time EEG data

[1218] How it works: The user wears the EEG device on their head, and the device begins to measure their brain waves. The device then transmits the EEG data to a terminal in real time via Bluetooth or USB connection.

[1219] Step 2:

[1220] The terminal acquires brain wave data in real time from the brain wave measurement device.

[1221] Input: Real-time EEG data

[1222] Output: Acquired EEG data

[1223] Specific operation: The terminal receives the EEG data sent from the EEG measurement device and stores it in a database.

[1224] Step 3:

[1225] The device analyzes the brain wave data it acquires and extracts the user's intentions.

[1226] Input: Acquired EEG data

[1227] Output: User intent

[1228] Specific operation: The device uses EEG analysis software (e.g., OpenBCI) to analyze EEG data. It detects specific EEG patterns and extracts the user's intention based on them. For example, it detects the intention of "I want to drink coffee."

[1229] Step 4:

[1230] The device analyzes the user's emotional state.

[1231] Input: User's facial image and voice data

[1232] Output: User's emotional state

[1233] Specific operation: The device uses the emotion engine to acquire emotion data through facial recognition cameras and voice analysis. As a result of the analysis, the user's emotional state, such as "joy" or "sadness," is identified.

[1234] Step 5:

[1235] Based on the analysis results, the terminal generates and inputs a prompt sentence to the generative AI.

[1236] Input: User intent and emotional state

[1237] Output: Prompt for the generative AI

[1238] Specific behavior: The device integrates the user's intention and emotional state and generates a prompt sentence such as, "The user is feeling happy and wants to drink coffee. Please suggest an appropriate action."

[1239] Step 6:

[1240] The server sends a prompt to the generative AI, which then generates an output.

[1241] Input: Prompt sentence for generative AI

[1242] Output: Output from generative AI

[1243] Specific operation: The server sends a prompt to a generative AI (e.g., OpenAI GPT-4), which then generates an output based on the prompt: "Instruct the coffee maker to make coffee."

[1244] Step 7:

[1245] The server receives the output from the generative AI and executes specific actions.

[1246] Input: Output from generative AI

[1247] Output: Actions taken

[1248] Specific operation: The server operates the coffee maker through the IoT device to make coffee. Specifically, it sends the command "Make coffee" to the coffee maker, and the coffee is actually made.

[1249] (Application example 1)

[1250] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1251] Conventional brain-machine interfaces (BMIs) exist that analyze the user's brainwaves to understand their intentions, but their application range is limited, and they are not fully utilized, especially in supporting work in factories. Furthermore, there is a lack of systems that can understand the intentions of workers in real time and automatically perform appropriate tasks, which means that improvements in work efficiency and collaboration between humans and machines have not been fully realized.

[1252] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means. In this invention, the server includes: means for generating a minimally invasive, highly accurate brain-machine interface (BMI) that combines electroencephalograms and generative AI; means for providing a platform for popularizing the BMI throughout society; means for providing intuitive communication services using brain activity, a metaverse interface, health management services, and biometric authentication services provided through the platform; and means installed on robots performing work in factories, analyzing the electroencephalograms of workers in real time, understanding the workers' intentions, and automatically performing appropriate tasks. This makes it possible to understand the workers' intentions in real time and automatically perform appropriate tasks.

[1253] "Electroencephalograms" are electrical signals generated by neural activity in the brain.

[1254] "Generative AI" is an artificial intelligence technology that generates new data and output based on input data.

[1255] Brain-machine interface (BMI) is an interface technology that analyzes brain waves and controls machines and computers based on the results.

[1256] "Minimally invasive" means causing less strain and damage to the body.

[1257] "High accuracy" means very accurate.

[1258] A "platform" is a system or environment that serves as the foundation for providing specific services or functions.

[1259] An "intuitive communication service" is one that allows users to exchange information in a natural way.

[1260] "Metaverse interface" refers to the user interface within the virtual space.

[1261] A "health management service" is a service for monitoring and managing a user's health condition.

[1262] A "biometric authentication service" is a service that uses an individual's biometric information to perform authentication.

[1263] A "robot working in a factory" is a machine that automatically performs specific tasks in a factory.

[1264] "Worker" refers to a person who performs work in a factory.

[1265] "Real-time" means that processing and analysis occur almost simultaneously.

[1266] "Intent" refers to the purpose or idea of ​​what the user wants to do.

[1267] "Output" refers to the results or actions produced by a system or machine.

[1268] The system for implementing this invention uses a minimally invasive, highly accurate brain-machine interface (BMI) that combines EEG and generative AI. The server analyzes the user's EEG in real time and inputs the results into the generative AI, which then understands the user's intentions and generates output based on them. Furthermore, by combining it with an emotion engine, the system simultaneously analyzes the user's emotional state, and generates output based on an understanding of the user's intentions and emotional state.

[1269] This system is installed on robots working in factories and analyzes the worker's brain waves in real time. When a worker wants to perform a specific task, the robot understands that intention and automatically performs the appropriate task. For example, if a worker thinks, "I want to assemble a part," the robot will understand that intention and begin assembling the part.

[1270] The server uses an EEG sensor to acquire EEG data, which is analyzed in real time using a Python library. An emotion engine is used to analyze the emotional state. A generative AI model (e.g., OpenAI GPT-3) understands the user's intention based on the analyzed EEG data and emotional state, and generates appropriate output.

[1271] As a specific example, the prompt sentence when a worker thinks "I want to assemble parts" is shown below.

[1272] Example prompt sentence:

[1273] User's EEG data: [0.1, 0.2, 0.3, ...]

[1274] User's emotional state: joy

[1275] User Intent: Assemble the parts.

[1276] By inputting this prompt into a generative AI model, the system can understand the user's intent and generate an appropriate action, resulting in a specific output such as a robot starting to assemble parts.

[1277] This system makes it possible to understand the intentions of workers in real time and automatically carry out appropriate tasks, thereby improving work efficiency in factories and realizing collaboration between humans and machines.

[1278] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1279] Step 1:

[1280] The server acquires the user's brainwave data in real time using an EEG sensor. The input is the user's brainwave signal, and the output is the acquired brainwave data. This data is transmitted from the EEG sensor to the server.

[1281] Step 2:

[1282] The server uses a Python library to analyze the acquired EEG data in real time. The input is the acquired EEG data, and the output is the analyzed EEG data. This analysis extracts EEG features and patterns.

[1283] Step 3:

[1284] The server uses an emotion engine to analyze the user's emotional state from the analyzed EEG data. The input is the analyzed EEG data, and the output is the user's emotional state. This analysis determines whether the user is experiencing emotions such as joy, sadness, or anger.

[1285] Step 4:

[1286] The server uses a generative AI model (e.g., OpenAI GPT-3) to understand the user's intention based on the analyzed EEG data and emotional state. The input is the analyzed EEG data and emotional state, and the output is the user's intention. The generative AI model inputs these data as prompts and infers what the user wants to do.

[1287] Step 5:

[1288] The server generates appropriate output based on the user's intention. The input is the user's intention, and the output is a specific action. For example, if the user thinks, "I want to assemble parts," the server will instruct the robot to start assembling the parts.

[1289] Step 6:

[1290] The robot receives instructions from the server and performs specific tasks. The input is the instruction from the server, and the output is the executed task. For example, this includes a series of actions from when the robot starts assembling parts to when it completes the task.

[1291] Step 7:

[1292] The server monitors the progress of the work and provides feedback as needed. The input is work progress data from the robot, and the output is feedback information. This improves the accuracy and efficiency of the work.

[1293] Example 2

[1294] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1295] Conventional brain-machine interfaces (BMIs) lacked the ability to effectively manage users' EEG data and share it with third parties as needed. Furthermore, the accuracy and real-time nature of EEG data analysis made it difficult to accurately understand the user's intentions. Furthermore, the acquisition and management of EEG data was cumbersome, making it difficult for general users to use.

[1296] The specific processing by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes a means for a user to acquire EEG data using a dedicated EEG measuring device and upload the data to the platform via a terminal, a means for the server to receive the EEG data and store it in a database, and a means for the server to preprocess and analyze the EEG data. This allows users to effectively manage their own EEG data and share it with third parties as needed. In addition, the accuracy of EEG data analysis is improved, allowing for accurate understanding of the user's intentions in real time. Furthermore, the acquisition and management of EEG data is simplified, providing a system that is easy for general users to use.

[1297] Brain-machine interface (BMI) is a technology that uses brainwave data to understand the user's intentions and enable intuitive interaction with machines and computers.

[1298] "Generative AI" is an artificial intelligence technology that analyzes a user's brainwave data and generates output based on the results.

[1299] The "platform" is a system infrastructure that allows users to manage their EEG data and share it with third parties as needed.

[1300] An "EEG measuring device" is a device worn on the user's head to acquire brain wave data.

[1301] A "terminal" is a device that allows users to input EEG data and upload it to the platform.

[1302] A "server" is a computer system that receives EEG data, stores it in a database, and performs preprocessing and analysis.

[1303] The "database" is an information management system for storing received electroencephalogram data.

[1304] "Preprocessing" refers to the cleaning of data before analysis, such as removing noise from EEG data and filling in missing values.

[1305] "Analysis" is a process for classifying and understanding the user's emotional state and intentions based on pre-processed EEG data.

[1306] "Third parties" refer to those with whom users share their EEG data, such as medical institutions and research institutions.

[1307] MODE FOR CARRYING OUT THE INVENTION

[1308] This invention is a system in which users acquire EEG data using a dedicated EEG measuring device and upload it to a platform via their terminal. The server stores the received EEG data in a database and performs preprocessing and analysis. This allows users to effectively manage their own EEG data and share it with third parties as needed.

[1309] Hardware and software used

[1310] Hardware:

[1311] EEG measurement device: The user acquires EEG data using a dedicated EEG measurement device (e.g., a general-purpose EEG device), which is worn on the user's head and collects EEG data in real time.

[1312] software:

[1313] Terminal application: The terminal provides an interface for users to receive acquired EEG data and upload it to the platform. Users upload data through the terminal application.

[1314] Server: The server uses Python to preprocess and analyze the EEG data. Specifically, it cleans and filters the data using libraries such as NumPy and Pandas. It also builds and analyzes machine learning models using libraries such as TensorFlow and PyTorch.

[1315] Database: The server uses a relational database such as MySQL or PostgreSQL to store the EEG data.

[1316] Specific examples

[1317] As a concrete example, consider a scenario in which a user wants to share their EEG data when they are experiencing the emotion of joy with a medical institution. The user acquires the data using an EEG measurement device and uploads it to the platform via their terminal. The server receives the data, analyzes it, and then transmits it to the medical institution.

[1318] Example prompts to input to a generative AI model:

[1319] "Please explain the procedure for sharing EEG data from users experiencing pleasure with medical institutions."

[1320] Using this prompt, the generative AI model can provide detailed instructions on specific steps and required hardware and software.

[1321] In this way, users can effectively manage their own EEG data and share it with third parties as needed. Furthermore, the accuracy of EEG data analysis is improved, allowing users' intentions to be accurately understood in real time. Furthermore, the acquisition and management of EEG data is simplified, providing a system that is easy for general users to use.

[1322] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1323] Program processing flow

[1324] Step 1:

[1325] The user acquires EEG data

[1326] Input: User's brainwaves

[1327] Output: EEG data

[1328] How it works: The user wears a dedicated EEG device on their head to capture real-time EEG data. The device uses sensors to detect the user's brainwaves and stores them as digital data.

[1329] Step 2:

[1330] The device uploads the brainwave data to the platform.

[1331] Input: EEG data

[1332] Output: Data uploaded to the platform

[1333] Specific operation: The user opens the application on the device, selects the acquired EEG data, and presses the "Upload" button. The device then sends the data to the platform.

[1334] Step 3:

[1335] The server receives the EEG data and stores it in a database.

[1336] Input: Uploaded EEG data

[1337] Output: Data stored in the database

[1338] How it works: The server receives the EEG data sent from the device and stores it in a relational database such as MySQL or PostgreSQL, along with metadata such as user IDs and timestamps.

[1339] Step 4:

[1340] The server preprocesses the EEG data

[1341] Input: EEG data stored in a database

[1342] Output: Preprocessed data

[1343] What it does: The server runs Python scripts and uses libraries like NumPy and Pandas to clean and filter the data, specifically removing noise and imputing missing values.

[1344] Step 5:

[1345] The server analyzes the brainwave data

[1346] Input: Preprocessed data

[1347] Output: Analysis results (e.g., classification of emotional state)

[1348] How it works: The server uses libraries such as TensorFlow and PyTorch to run machine learning models to classify emotional states from EEG data. The analysis results are output as data indicating the user's emotional state and intentions.

[1349] Step 6:

[1350] Users manage their EEG data

[1351] Input: Analysis results and original EEG data

[1352] Output: Managed data (e.g. filtered data display)

[1353] What it does: Users access the platform's dashboard to view analytics results, and can filter and view data for specific time periods.

[1354] Step 7:

[1355] Users share their EEG data with third parties

[1356] Input: Managed Data

[1357] Output: Shared data

[1358] How it works: The user presses the "Share" button on the dashboard, enters the contact information of a third party (e.g., a medical institution) and gives permission to share. The server encrypts the data and sends it to the third party using HTTPS.

[1359] In this way, by performing specific actions at each step, users can effectively manage their own brainwave data and share it with third parties as needed.

[1360] (Application example 2)

[1361] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1362] Conventional brain-machine interfaces (BMIs) can collect and analyze a user's brainwave data, but lack a means to safely and efficiently share that data with third parties. Furthermore, there was no system that could analyze a user's emotional state in real time and take appropriate action if an abnormality was detected. This made it difficult to share data with medical and research institutions, hindering the improvement of user health management and biometric authentication accuracy.

[1363] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1364] In this invention, the server includes: a means for generating a minimally invasive, highly accurate brain-machine interface (BMI) that combines EEG and generative AI; a means for providing a platform for disseminating the BMI to society; a means for providing intuitive communication services, metaverse interfaces, health management services, and biometric authentication services that utilize brain activity, all of which are provided through the platform; a means for collecting a user's EEG data in real time, encrypting it, and sharing it with third parties; and a means for analyzing the user's emotional state and issuing an alert if an abnormality is detected. This enables the safe and efficient sharing of user EEG data and strengthens collaboration with medical and research institutions. Furthermore, by monitoring a user's emotional state in real time and responding quickly if an abnormality is detected, the accuracy of health management and biometric authentication can be improved.

[1365] "Electroencephalograms" are electrical signals generated by neural activity in the brain.

[1366] "Generative AI" is a type of artificial intelligence, a system that has the ability to generate new data and information based on input data.

[1367] "Minimally invasive" refers to minimal strain or damage to the body.

[1368] A "brain-machine interface (BMI)" is an interface that uses brain waves to operate machines and computers.

[1369] A "platform" is a system or environment that serves as the foundation for providing specific functions or services.

[1370] An "intuitive communication service" is one that allows users to exchange information in a natural way.

[1371] A "metaverse interface" is an interface that allows users to operate and experience things within a virtual space.

[1372] A "health management service" is a service that monitors the user's health status and provides appropriate advice and support.

[1373] A "biometric authentication service" is a service that verifies a user's identity using their biometric information.

[1374] "Encryption" is a technology that converts data using a specific algorithm to make it difficult for third parties to decipher.

[1375] An "emotional state" is the emotional state that a user is feeling at a particular moment.

[1376] An "alert" is a notification or warning to notify of an abnormality or emergency.

[1377] The following system configuration will be described as an embodiment of the present invention.

[1378] System Configuration

[1379] The system includes hardware and software for collecting and analyzing the user's brainwave data using generative AI. Specifically, it includes the following elements:

[1380] 1. EEG sensor: A device that collects the user's brainwaves in real time. The EEG sensor is worn on the user's head and measures the electrical activity of the brain.

[1381] 2. Smartphone: A device that receives, encrypts, and analyzes data from the EEG sensor. A dedicated application is installed on the smartphone.

[1382] 3. Server: Provides a platform for receiving encrypted EEG data and sharing it with third parties (medical institutions and research institutions) as needed. The server includes software for managing and sharing data.

[1383] Program processing

[1384] The server performs the following process.

[1385] 1. Collecting EEG data: The EEG sensor collects the user's EEG data in real time and transmits it to a smartphone.

[1386] 2. Data encryption: The smartphone encrypts the received EEG data using the Fernet cryptography library.

[1387] 3. Data sharing: The encrypted data is sent from the smartphone to a server, which then shares the received data with designated third parties (medical institutions or research institutions).

[1388] 4. Emotional state analysis: The smartphone analyzes the EEG data to identify the user's emotional state. If an abnormality is detected, the smartphone will send an alert.

[1389] Hardware and software used

[1390] Brainwave sensor: A device that collects the user's brainwaves.

[1391] Smartphone: A device that receives, encrypts, analyzes, and shares data.

[1392] Server: A platform for managing and sharing data.

[1393] cryptography library: Software used to encrypt data.

[1394] requests library: Software used to send data.

[1395] Specific examples

[1396] For example, consider the case where brain wave data is collected when a user feels stressed and sent to a medical institution. An brain wave sensor collects the user's brain waves and sends them to a smartphone. The smartphone encrypts the received data and sends it to a server. The server shares the encrypted data with the medical institution. The medical institution analyzes the received data and evaluates the user's stress level.

[1397] Prompt Sentence Examples

[1398] "Write a Python program that collects a user's EEG data, encrypts it, and sends it to a designated medical institution. The EEG data can be virtual. Use the Fernet cryptography library to encrypt it, and the requests library to send the data."

[1399] The above is an embodiment of the present invention.

[1400] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1401] Step 1:

[1402] The user wears an EEG sensor, which collects the user's EEG data in real time. The input is the user's EEG signal, and the output is the raw data collected by the EEG sensor.

[1403] Step 2:

[1404] The terminal (smartphone) receives EEG data from the EEG sensor. The terminal acquires the data using wireless communication such as Bluetooth or Wi-Fi. The input is the raw data sent from the EEG sensor, and the output is the EEG data stored in the terminal.

[1405] Step 3:

[1406] The EEG data received by the device is encrypted. The encryption is performed using the Fernet cryptography library. The input is the EEG data stored on the device, and the output is the encrypted EEG data. Specifically, an encryption key is generated and the EEG data is encrypted with that key.

[1407] Step 4:

[1408] The device sends encrypted EEG data to the server. The requests library is used for sending. The input is the encrypted EEG data, and the output is the data sent to the server. Specifically, the data is sent using an HTTP POST request.

[1409] Step 5:

[1410] The server stores the received encrypted EEG data. The server manages the data using a database. The input is the encrypted data sent from the device, and the output is the encrypted data stored in the database.

[1411] Step 6:

[1412] The server shares the encrypted EEG data with third parties (medical institutions or research institutions) as needed. A secure communication protocol is used for sharing. The input is the encrypted data stored in the database, and the output is the data sent to the third party. Specifically, the server authenticates the third party and sends the data if authentication is successful.

[1413] Step 7:

[1414] The device analyzes the EEG data to identify the user's emotional state. A generative AI model is used for the analysis. The input is the EEG data stored on the device, and the output is the emotional state as a result of the analysis. Specifically, the EEG data is input into the generative AI model, which estimates the emotional state.

[1415] Step 8:

[1416] The device monitors the emotional state and issues an alert if an abnormality is detected. The input is the emotional state as an analysis result, and the output is an alert when an abnormality is detected. Specifically, if the emotional state exceeds a preset threshold, a notification is sent to the user or a designated third party.

[1417] Example 3

[1418] Next, a description will be given of Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1419] Conventional brain-machine interfaces (BMIs) require high accuracy when analyzing the user's brain waves, but it has been difficult to achieve high-precision analysis in a minimally invasive manner. Furthermore, there has been a lack of a system for providing real-time feedback to the user with the analysis results, making it impossible to immediately reflect the user's intentions or emotional state. Furthermore, there has also been a lack of functionality for users to manage their own brain wave data and share it with third parties as needed.

[1420] The identification process by the identification processing unit 290 of the data processing device 12 in Example 3 is realized by the following means. In this invention, the server includes means for measuring the user's electroencephalogram data and transmitting it to the server in real time, means for the server to analyze the electroencephalogram data and identify the user's emotional state and intention, and means for storing the analysis results in a database and transmitting them to the terminal. This enables minimally invasive and highly accurate electroencephalogram analysis, which can reflect the user's intention and emotional state in real time. It also enables the user to manage their own electroencephalogram data and share it with third parties as necessary.

[1421] Brain-machine interface (BMI) is a technology that uses brain waves to control computers and other devices.

[1422] "Generative AI" is a type of artificial intelligence that has the ability to generate new data and information based on input data.

[1423] A "platform" is a system or environment that serves as the foundation for providing specific services or functions.

[1424] "EEG data" is data obtained by electrically measuring a user's brain activity, and reflects their emotional state and intentions.

[1425] "Real-time" means that data is measured and processed immediately.

[1426] "Analysis" is the process of processing measured data and extracting specific information or patterns.

[1427] An "emotional state" is a state that indicates the user's current feelings or mood.

[1428] "Intent" describes what a user is trying to do or what they want.

[1429] A "database" is a system that stores data in an organized manner and allows it to be searched and updated as needed.

[1430] "Feedback" is the response or information that a system provides to a user.

[1431] "Minimally invasive" means that it causes little physical strain or discomfort to the user.

[1432] "High precision" means that the data measurements and analysis are extremely accurate.

[1433] A "virtual reality interface" is a means by which a user interacts with a virtual reality environment.

[1434] A "health management service" is a service that monitors a user's health status and provides appropriate advice and support.

[1435] "Biometric authentication service" refers to a service that authenticates individuals using their biometric information.

[1436] This invention is a system that provides a minimally invasive, highly accurate brain-machine interface (BMI) that combines EEG and generative AI. This system measures and analyzes the user's EEG data in real time to identify the user's emotional state and intentions, and provides feedback based on that information.

[1437] Hardware and software used

[1438] Hardware

[1439] EEG measurement device: A device for measuring the user's brain waves (e.g., EEG headset).

[1440] Terminal: A device (e.g., smartphone, tablet, PC) that connects to the EEG measurement device and transmits data to the server.

[1441] Server: A computer system for receiving and analyzing EEG data.

[1442] software

[1443] Data transmission program: A program installed on the device to transmit EEG data to the server.

[1444] Data analysis program: A program installed on the server to analyze EEG data (e.g., FFT analysis using Python)

[1445] Database management system: A system for storing analysis results and retrieving and updating them as needed.

[1446] Feedback program: A program installed on the device to display analysis results to the user.

[1447] Specific operation of the system

[1448] User Actions

[1449] The user wears an EEG measurement device (e.g., EEG headset) and connects it to a terminal. The user generates EEG data by performing specific actions. For example, if the user is relaxed, the EEG data will have specific frequency components.

[1450] Device behavior

[1451] The device measures the user's brain waves in real time and transmits them to the server using a data transmission program. The device waits for a response from the server to confirm that the data transmission was successful.

[1452] Server Operation

[1453] The server receives the EEG data sent from the device. The received data is analyzed using a data analysis program. Specifically, the EEG data is broken down into frequency components using FFT (Fast Fourier Transform) and specific patterns are extracted. For example, the intensity of alpha and beta waves is calculated to identify the user's emotional state.

[1454] Saving and sending analysis results

[1455] The server saves the analysis results in a database. The saved data includes the user ID, measurement date and time, and analysis results (emotional state and intention). This makes it possible to refer to the data later. The server then sends the analysis results to the device. A secure communication protocol (e.g., HTTPS) is used for transmission.

[1456] Device feedback

[1457] The device displays the analysis results received from the server to the user using a graphical user interface (GUI). For example, if the user is relaxed, the device displays "Relaxed state" and suggests an appropriate action (e.g., playing relaxing music).

[1458] Examples of specific examples and prompts

[1459] Specific examples

[1460] For example, when a user uses an intuitive communication service, the following procedure is performed.

[1461] 1. The user wears an EEG headset and connects it to the device.

[1462] 2. The device measures EEG data in real time and sends it to the server.

[1463] 3. The server receives the EEG data and analyzes it using FFT. For example, if the intensity of alpha waves is high, it determines that the user is relaxed.

[1464] 4. The analysis results are saved in the database. For example, data such as "User ID: 12345, Date and Time: 2023-10-01 10:00, State: Relaxed" is saved.

[1465] 5. The server sends the analysis results to the device, which displays "Relaxed state" and suggests playing relaxing music.

[1466] 6. The user plays relaxing music as suggested by the device and relaxes.

[1467] Prompt Sentence Examples

[1468] "Your goal is to analyze the user's brainwave data to identify their emotional state and generate a program that will enable intuitive communication. The hardware used is an EEG headset, and the software is Python. Specifically, you will analyze the brainwave data using FFT and calculate the strength of alpha and beta waves to identify their emotional state."

[1469] In this way, by specifically explaining the operation of the entire system, it is possible to clearly show the mode for carrying out the invention. The flow of the specific processing in the third embodiment will be explained with reference to FIG.

[1470] Step 1: Measuring the user's brainwave data

[1471] The user wears an electroencephalogram (EEG) measurement device (e.g., an EEG headset). The device measures the user's brain waves in real time and converts them into digital signals. The input is the user's brain waves, and the output is digitized brain wave data. Specifically, the device contacts the user's scalp and detects electrical signals.

[1472] Step 2: Sending EEG data

[1473] The device transmits the measured EEG data to a server via Bluetooth or Wi-Fi. The input is digitized EEG data, and the output is the data transmitted to the server. Specifically, the device executes a data transmission program and transmits the data to the server's IP address.

[1474] Step 3: Receiving and analyzing EEG data by the server

[1475] The server receives the EEG data sent from the device. The input is the EEG data sent from the device, and the output is the analysis results. The server uses Python to perform an FFT (Fast Fourier Transform) and break down the EEG data into frequency components. Specifically, the server runs a data analysis program and calculates the intensity of alpha and beta waves.

[1476] Step 4: Save the analysis results

[1477] The server saves the analysis results in a database. The input is the analysis results, and the output is the data saved in the database. Specifically, the server uses a database management system to save the user ID, measurement date and time, and analysis results.

[1478] Step 5: Submitting the analysis results

[1479] The server sends the analysis results to the terminal. The input is the analysis results stored in the database, and the output is the analysis results sent to the terminal. Specifically, the server sends data using a secure communication protocol (e.g., HTTPS).

[1480] Step 6: Displaying the analysis results on your device

[1481] The terminal displays the analysis results received from the server to the user. The input is the analysis results sent from the server, and the output is the information displayed to the user. In concrete terms, the terminal runs a feedback program and displays the analysis results using a graphical user interface (GUI).

[1482] Step 7: User feedback

[1483] The user confirms their emotional state and intentions based on the feedback provided by the device. The input is the feedback information from the device, and the output is the user's next action. Specifically, the user takes an action such as playing relaxing music according to the device's suggestion.

[1484] (Application example 3)

[1485] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1486] Conventional brain-machine interfaces (BMIs) can understand a user's intentions by analyzing their brain waves, but there are issues with accuracy and real-time performance. Furthermore, intuitive operation and emotion-based action control within the metaverse are difficult, creating a need for improved user experience. Furthermore, there has been a lack of user-friendly systems for managing brain wave data and sharing it with third parties.

[1487] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[1488] In this invention, the server includes: a means for generating a minimally invasive, highly accurate brain-machine interface (BMI) that combines EEG and generative AI; a means for providing a platform for disseminating the BMI to society; a means for providing intuitive communication services using brain activity, a metaverse interface, health management services, and biometric authentication services provided through the platform; a means for analyzing a user's EEG in real time, grasping their emotions and intentions, and controlling their behavior in the metaverse; and a means for controlling actions in the metaverse based on the user's emotional state. This allows users to intuitively operate the metaverse using EEG, realizing emotion-based action control. Furthermore, EEG data can be easily managed and shared with third parties.

[1489] "Electroencephalograms" are electrical signals generated by neural activity in the brain.

[1490] "Generative AI" is an artificial intelligence technology that generates new information and content based on data.

[1491] A "brain-machine interface (BMI)" is an interface that analyzes brain waves to control machines and computers.

[1492] A "platform" is a system or environment that serves as the foundation for providing specific services or functions.

[1493] An "intuitive communication service" is a service that analyzes the user's brain waves to understand their emotions and intentions, and then communicates based on that.

[1494] A "metaverse interface" is an interface for controlling actions in the world of virtual reality (VR) and augmented reality (AR).

[1495] The "health management service" analyzes the user's brain waves to determine their stress level, sleep state, etc., and then makes health management suggestions based on that information.

[1496] "Biometric authentication service" is a service that analyzes a user's brain wave patterns to authenticate them.

[1497] "Real-time" refers to processing or reaction occurring immediately, without delay.

[1498] "Emotional state" refers to the emotional state of the user as analyzed from their brain waves.

[1499] "Action control" refers to controlling the behavior of a system or avatar based on the user's intentions and emotions.

[1500] A system for implementing this invention includes means for generating a minimally invasive, highly accurate brain-machine interface (BMI) that combines brain waves and generative AI, means for providing a platform for popularizing the BMI throughout society, means for providing intuitive communication services, metaverse interfaces, health management services, and biometric authentication services that utilize brain activity provided through the platform, means for analyzing a user's brain waves in real time to understand their emotions and intentions and control their behavior within the metaverse, and means for controlling actions within the metaverse based on the user's emotional state.

[1501] System Configuration

[1502] The server comprises a system consisting of an EEG sensor, a head-mounted display (HMD), and generative AI. The EEG sensor captures the user's brainwaves in real time, and the HMD provides the user with visual information within the metaverse. The generative AI analyzes the captured EEG data to understand the user's emotions and intentions.

[1503] Program processing

[1504] The server uses the BrainFlow library to acquire EEG data, filter the data to remove noise, and then analyzes the data using generative AI to detect the user's emotions (e.g., joy). Based on the detected emotions, the server controls actions within the metaverse. Specifically, if the user is feeling joyful, the server controls the avatar to perform a joyful dance.

[1505] Hardware and software used

[1506] Hardware: EEG sensor (e.g., BrainFlow-compatible device), head-mounted display (HMD)

[1507] Software: Python, BrainFlow library

[1508] Specific examples

[1509] For example, consider a scenario in which a user wears an HMD and uses an EEG sensor to control their actions in the Metaverse. If the user feels joy, the avatar will dance in joy. This allows the user to intuitively operate the Metaverse, realizing emotion-based action control.

[1510] Prompt Sentence Examples

[1511] Example prompts to input to a generative AI model:

[1512] Write a Python program that analyzes a user's EEG data to detect emotions and control actions in the Metaverse. If the user has the emotion of joy, make the avatar do a joyful dance.

[1513] In this way, an intuitive metaverse interface using brain waves can be realized.

[1514] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[1515] Step 1:

[1516] The server uses an EEG sensor to acquire the user's EEG data in real time.

[1517] Input: User's brainwave signal

[1518] Output: Raw EEG data

[1519] How it works: The server uses the BrainFlow library to collect data from the EEG sensor and store it in memory.

[1520] Step 2:

[1521] The server filters the acquired EEG data to remove noise.

[1522] Input: Raw EEG data

[1523] Output: filtered EEG data

[1524] Specific operation: The server uses the filtering function of the BrainFlow library to remove noise in specific frequency bands.

[1525] Step 3:

[1526] The server inputs the filtered brainwave data into a generative AI system to analyze the user's emotions.

[1527] Input: filtered EEG data

[1528] Output: User's emotional state (e.g., happy, sad)

[1529] How it works: The server inputs data into a generative AI model and runs an emotion analysis algorithm to identify the user's emotions.

[1530] Step 4:

[1531] The server determines actions within the metaverse based on the analyzed emotional state.

[1532] Input: User's emotional state

[1533] Output: Instructions for action within the metaverse (e.g., dance of joy)

[1534] Specific operation: The server determines the action to be taken according to the emotional state and sends the instruction to the metaverse system.

[1535] Step 5:

[1536] The device (HMD) controls an avatar in the metaverse based on action instructions received from the server.

[1537] Input: Action instructions within the metaverse

[1538] Output: Avatar behavior (e.g., dance of joy)

[1539] Specific actions: The device provides visual information to the user through the HMD and controls the avatar to perform the specified actions.

[1540] Step 6:

[1541] The user visually confirms actions within the metaverse through the HMD.

[1542] Input: Avatar movement

[1543] Output: Visual information (e.g., a scene of an avatar dancing for joy)

[1544] Specific operation: The user wears the HMD and visually confirms the avatar's movements within the metaverse in real time.

[1545] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1546] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1547] Another example of generative AI is Gemini (internet search engine). <url: https: gemini.google.com ?hl="ja">) are mentioned.

[1548] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1549] [Third embodiment]

[1550] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1551] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1552] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1553] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1554] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1555] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1556] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1557] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1558] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1559] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1560] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1561] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[1562] "Example 1"

[1563] As one embodiment of the present invention, we are developing a brain-machine interface that combines brain waves and generative AI.

[1564] There is a system that generates a BMI (Body Machine Interface). This system analyzes the user's brain waves in real time and inputs the results into a generative AI. The generative AI understands the user's intentions based on this brain wave data and generates output based on that. For example, when a user thinks, "I want to drink coffee," the generative AI analyzes their brain waves, understands their intention, and can take action such as "making coffee."

[1565] "Example 2"

[1566] One embodiment of the present invention is a platform for popularizing BMI in society. This platform has a function that allows users to manage their own EEG data and share it with third parties as needed. For example, users can manage their own EEG data on the platform and share it with medical institutions and research institutions, thereby enabling more accurate medical services and research results.

[1567] "Example 3"

[1568] Furthermore, one embodiment of the present invention provides new services through the platform. These services include an intuitive communication service that utilizes brain activity, a metaverse interface, a health management service, and a biometric authentication service. For example, the intuitive communication service analyzes a user's brain waves to understand their emotions and intentions and communicate based on that information. The metaverse interface analyzes a user's brain waves to control their behavior in the VR or AR world. The health management service analyzes a user's brain waves to understand their stress level, sleep state, and other factors and proposes health management based on that information. The biometric authentication service analyzes a user's brain wave patterns to perform personal authentication.

[1569] The processing flow of each embodiment will be described below.

[1570] "Example 1"

[1571] Step 1: The user has a specific thought or intention.

[1572] Step 2: The system captures and analyzes the user's brainwaves in real time.

[1573] Step 3: Input the analysis results into the generative AI.

[1574] Step 4: The generative AI understands the user's intention based on the EEG data and generates an output based on that. For example, it takes an action such as "make coffee."

[1575] "Example 2"

[1576] Step 1: The user accesses the platform and uploads their EEG data.

[1577] Step 2: The platform manages the user's EEG data and shares it with third parties as necessary. For example, sharing it with medical institutions or research institutions will enable more accurate medical services and research results.

[1578] "Example 3"

[1579] Step 1: The user selects a specific service through the platform, such as an intuitive communication service, a metaverse interface, a health management service, or a biometric authentication service.

[1580] Step 2: The selected service analyzes the user's brainwaves and provides services based on them. For example, an intuitive communication service analyzes the user's brainwaves to understand their emotions and intentions, and communicates based on that.

[1581] Example 1

[1582] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1583] Conventional brain-machine interfaces (BMIs) have low accuracy in analyzing the user's brain waves, making it difficult to understand the user's intentions in real time. Furthermore, these systems are highly invasive and place a heavy burden on the user. Furthermore, it is not easy to manage and share brain wave data, which has hindered their widespread adoption in society.

[1584] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1585] In this invention, the server includes: means for collecting a user's brain waves in real time; means for analyzing the collected brain wave data; means for inputting the analyzed brain wave data into a generative AI model; means for the generative AI model to understand the user's intention and generate output based on that understanding; means for feeding back the generated output to the user; means for providing a platform for popularizing the brain-machine interface (BMI) in society; and means for providing intuitive communication services, virtual space interfaces, health management services, and biometric authentication services that utilize brain activity through the platform. This enables highly accurate and minimally invasive analysis of a user's brain waves and understanding of intentions in real time. Furthermore, it facilitates the management and sharing of brain wave data, promoting its widespread use in society.

[1586] "User" refers to an individual who uses a Brain-Machine Interface (BMI).

[1587] "Eeg" refers to the electrical signals generated by the user's brain.

[1588] "Real-time" refers to data being processed and analyzed as it is generated.

[1589] "Means for collection" refers to devices and methods for detecting the user's brain waves and acquiring them as data.

[1590] "Means for analyzing" refers to a method or device for processing collected EEG data and inferring the user's intentions.

[1591] A "generative AI model" refers to an artificial intelligence model that understands the user's intent based on input data and generates output based on that.

[1592] "Means of input" refers to the method or device for providing analyzed EEG data to a generative AI model.

[1593] "Output" refers to the results or suggestions that a generative AI model generates by understanding the user's intent.

[1594] "Feedback means" refers to the method or device used to communicate the generated output to the user.

[1595] "Platform" refers to the systems and services that form the foundation for popularizing brain-machine interfaces (BMIs) in society.

[1596] "Communication service" refers to a service that uses brain waves to enable users to intuitively exchange information with others.

[1597] A "virtual space interface" refers to an interface that allows users to operate and move within a virtual space using their brain waves.

[1598] "Health management service" refers to a service that uses a user's brainwave data to monitor and manage their health condition.

[1599] "Biometric authentication service" refers to a service that uses a user's brainwave data to authenticate an individual.

[1600] This invention is a brain-machine interface (BMI) system that analyzes a user's brain waves in real time and inputs the results into a generative AI model to understand the user's intentions and generate output based on those intentions. This system is implemented using the following hardware and software.

[1601] Hardware and Software Configuration

[1602] Brainwave sensor

[1603] The user wears an EEG sensor on their head, which detects the user's brain waves in real time and collects the data as electrical signals.

[1604] server

[1605] The server receives the EEG data sent from the EEG sensor, which is then sent to the device for analysis.

[1606] Terminal

[1607] The device analyzes the EEG data sent from the server using a Python-based EEG analysis library. The device then sends the analysis results to the server.

[1608] Generative AI Model

[1609] The server inputs the analyzed EEG data into a generative AI model (such as OpenAI's GPT-4), which understands the user's intent based on the input data and generates output based on that understanding.

[1610] feedback

[1611] The generated output is sent to the device via the server, and the device provides it as feedback to the user, who can then take the suggested action.

[1612] Specific examples

[1613] For example, consider the case where a user wants to "listen to music." The EEG sensor collects the user's EEG data, and the device analyzes the data. If the analysis result indicates the user's intent to "listen to music," the following prompt sentence is input to the generative AI model:

[1614] The user's EEG data has detected the intent "I want to listen to music." Please suggest an action to play appropriate music.

[1615] Based on this prompt, the generative AI model generates the output "play music" and sends it back to the user via their device. The user can then play the music according to the suggested action.

[1616] In this way, the system can analyze the user's brain waves with high accuracy and minimal invasiveness, enabling them to understand their intentions in real time. It also makes it easier to manage and share brain wave data, promoting its widespread adoption in society.

[1617] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1618] Step 1:

[1619] The user wears an EEG sensor on their head. The EEG sensor detects the user's brain waves in real time and collects data as electrical signals. The collected EEG data is then sent to a server.

[1620] Input: User's brainwaves

[1621] Output: EEG data (electrical signal)

[1622] Step 2:

[1623] The server receives the EEG data transmitted from the EEG sensor, which is then sent to the device for analysis.

[1624] Input: EEG data (electrical signal)

[1625] Output: EEG data (transferred from server to device)

[1626] Step 3:

[1627] The device receives the EEG data sent from the server. The device analyzes the EEG data using a Python-based EEG analysis library. During the analysis, specific EEG patterns are detected and the user's intentions are inferred. The analysis results are then sent to the server.

[1628] Input: EEG data (transferred from the server)

[1629] Output: Analysis results (user intent)

[1630] Step 4:

[1631] The server receives the analysis results sent from the device, generates a prompt sentence based on the analysis results, and inputs it into the generative AI model.

[1632] Input: Analysis result (user intent)

[1633] Output: prompt statement

[1634] Step 5:

[1635] The generative AI model receives the prompt sent from the server. The generative AI model understands the user's intent based on the prompt and generates output based on that. The generated output is then sent to the server.

[1636] Input: prompt statement

[1637] Output: Output

[1638] Step 6:

[1639] The server receives the output sent from the generative AI model and sends the output to the device.

[1640] Input: Output (transferred from generative AI model)

[1641] Output: Output (transfer from server to device)

[1642] Step 7:

[1643] The terminal receives the output sent from the server, and feeds the output back to the user, who can then take the suggested action.

[1644] Input: Output (transfer from server)

[1645] Output: Feedback to the user

[1646] (Application example 1)

[1647] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1648] Conventional factory robot control methods mainly involve manual operation or program control, which poses the problem of difficulty in intuitive operation. Furthermore, in order to improve work efficiency, a more intuitive and rapid control method is required. Furthermore, brain-machine interfaces (BMIs) that use brain waves are being increasingly applied in the fields of medicine and entertainment, but have not yet been fully utilized in factory automation or robot control. To solve these issues, a system that can intuitively control factory robots using brain waves is needed.

[1649] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1650] In this invention, the server includes: means for generating a minimally invasive, highly accurate brain-machine interface (BMI) that combines brainwaves and generative AI; means for providing a platform for popularizing the BMI throughout society; means for providing intuitive communication services, metaverse interfaces, health management services, and biometric authentication services that utilize brain activity, all of which are provided through the platform; means for factory workers to wear head-mounted displays and control factory robots using brainwaves; means for analyzing the user's brainwave data in real time and inputting the results into generative AI to understand the user's intentions and generate output based on those intentions; and means for controlling factory robots based on the user's intentions, thereby enabling factory workers to intuitively control factory robots using brainwaves.

[1651] "Electroencephalograms" are electrical signals generated by neural activity in the brain.

[1652] "Generative AI" is a type of artificial intelligence that generates new data and output based on input data.

[1653] A brain-machine interface (BMI) is an interface that uses brain waves to control machines and computers.

[1654] "Minimally invasive" refers to minimal strain or damage to the body.

[1655] "High precision" means very accurate.

[1656] A "platform" is a system or environment that serves as the foundation for providing specific services or functions.

[1657] An "intuitive communication service" is a service that allows users to operate and communicate intuitively.

[1658] A "metaverse interface" is an interface for performing activities and operations in a virtual space.

[1659] "Health management service" refers to a service for managing and improving the user's health condition.

[1660] "Biometric authentication service" refers to a service that authenticates individuals using biometric information such as fingerprints or facial recognition.

[1661] A "factory worker" is a worker engaged in production activities in a factory.

[1662] A "head-mounted display" is a display device that is worn on the head.

[1663] A "factory robot" is a robot used to automate production work in a factory.

[1664] "Real-time" refers to processing or reaction occurring immediately, without delay.

[1665] "User intent" refers to what the user wants to do or what they are thinking.

[1666] "Output" is the result or action that is produced by a system or device.

[1667] To implement the present invention, the following system configuration and program are required.

[1668] System Configuration

[1669] 1. Hardware

[1670] Head-mounted display (HMD): Equipped with a built-in brainwave sensor, it collects brainwave data in real time.

[1671] Factory robot: A robot that performs actions based on the user's intentions.

[1672] Server: Analyzes EEG data and runs generative AI models.

[1673] 2. Software

[1674] BrainwaveReader: A module for collecting EEG data.

[1675] Generative AI Model: A generative AI model that analyzes brainwave data and understands user intent.

[1676] RobotController: A module for controlling a robot based on the user's intentions.

[1677] Program processing explanation

[1678] The server first collects brainwave data from a head-mounted display (HMD). The BrainwaveReader module analyzes this data in real time and inputs it into a generative AI model (GenerativeAIModel). The generative AI model interprets the user's intention based on the brainwave data and passes that intention to the RobotController. The RobotController controls the factory robot based on the interpreted intention and performs the corresponding action.

[1679] Specific examples

[1680] For example, if a factory worker thinks, "I'm going to assemble parts," the following prompt sentence is input into the generative AI model:

[1681] Example prompt sentence:

[1682] Analysis of the user's brainwave data reveals that the user is thinking "assemble parts." Based on this intent, instruct the robot to assemble the parts.

[1683] Based on this prompt, the generative AI model interprets the user's intention and instructs the RobotController to "assemble the parts," which causes the factory robot to automatically begin assembling the parts.

[1684] This system allows factory workers to intuitively control factory robots using their brain waves, greatly improving work efficiency.

[1685] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1686] Step 1:

[1687] The user wears a head-mounted display (HMD). The HMD has a built-in brainwave sensor that collects the user's brainwave data in real time. The input is the user's brainwaves, and the output is the collected brainwave data.

[1688] Step 2:

[1689] The server uses the BrainwaveReader module to receive EEG data transmitted from the HMD. BrainwaveReader analyzes this data in real time, removing noise and extracting features. The input is EEG data from the HMD, and the output is the analyzed EEG data.

[1690] Step 3:

[1691] The server inputs the analyzed EEG data into the GenerativeAIModel, which is a generative AI model for interpreting the user's intentions based on EEG data. The input is the analyzed EEG data, and the output is data that indicates the user's intentions.

[1692] Step 4:

[1693] The server generates a prompt based on the results of its interpretation of the user's intent. This prompt specifically describes the user's intent. The input is data that indicates the user's intent, and the output is the prompt.

[1694] Step 5:

[1695] The server passes the generated prompt sentence to the RobotController, which generates specific instructions for the factory robot based on this prompt sentence. The input is the prompt sentence, and the output is the instruction for the robot.

[1696] Step 6:

[1697] The RobotController sends instructions to the factory robots, which then perform the corresponding actions based on these instructions. The input is the instruction to the robot, and the output is the robot's action.

[1698] Step 7:

[1699] The user checks the operation of a factory robot through an HMD. The HMD displays the robot's operating status in real time, allowing the user to check whether the robot is operating as intended. The input is the robot's operating status, and the output is the image displayed on the HMD.

[1700] Example 2

[1701] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1702] Conventional brain-machine interfaces (BMIs) lacked a platform for effectively managing users' EEG data and safely sharing it with third parties. Furthermore, real-time analysis of EEG data and integration with generative AI were insufficient, making it difficult to accurately understand the user's intentions and generate appropriate output. Furthermore, data security was not ensured due to insufficient encryption of EEG data and management of access rights.

[1703] The specific processing by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes a means for uploading EEG data acquired by a user using a dedicated measuring device from a terminal to the server, a means for the server to store the EEG data received in a database, a means for the user to access the platform and manage the data, a means for the user to set up data sharing with a third party, and a means for the server to encrypt the data and grant access rights to the third party. This allows the user to effectively manage their own EEG data and safely share it with a third party. Furthermore, by linking with generative AI, the system can accurately understand the user's intentions and generate appropriate output.

[1704] A "brain-machine interface (BMI)" is an interface that uses brain waves to communicate the user's intentions to a machine.

[1705] "Generative AI" is artificial intelligence that analyzes a user's brainwave data and generates appropriate output based on the results.

[1706] The "platform" is a system that allows users to manage their EEG data and share it with third parties as needed.

[1707] "Electroencephalogram data" refers to data obtained by measuring the user's brain activity, and is obtained using an electroencephalogram measuring device.

[1708] A "measuring device" is a device for measuring a user's brain waves in real time.

[1709] A "terminal" is a device such as a computer or smartphone that allows a user to upload EEG data acquired using a measurement device to a server.

[1710] A "server" is a computer system for storing and managing brain wave data received from a user.

[1711] The "database" is a system for efficiently storing and accessing the EEG data received by the server.

[1712] "Encryption" is a technology that converts data into a format that cannot be deciphered by third parties.

[1713] "Access authority" is a function that sets the rights that allow specific users or third parties to access data.

[1714] "Output" refers to the output generated by the generative AI based on the results of analyzing the user's brainwave data.

[1715] MODE FOR CARRYING OUT THE INVENTION

[1716] This invention provides a system that provides a platform for users to manage EEG data acquired using a dedicated measuring device and share it with third parties as needed. Specific embodiments of this system are described below.

[1717] Hardware and Software Configuration

[1718] The user acquires EEG data using a dedicated EEG measuring device, which measures the user's EEG in real time and transmits the data to a device (such as a PC or smartphone).

[1719] The device uploads the acquired EEG data to a server via a dedicated application, which converts the data into an appropriate format (e.g., CSV or JSON) and transmits it securely to the server using the HTTPS protocol.

[1720] The server stores the received EEG data in a database (e.g., MySQL or PostgreSQL). The database organizes the data for each user and sets up indexes for efficient access.

[1721] Users access the platform through a web browser or mobile application. The platform authenticates users (e.g., via OAuth or JWT) and provides an interface where users can view, edit, and delete their own EEG data.

[1722] Data Sharing and Encryption

[1723] If a user wants to share data with a third party, they select the third party (e.g., a medical institution or research institution) they want to share with on the platform, set the scope and period of data to be shared, and submit a sharing request.

[1724] The server receives the user's sharing request, encrypts the data to be shared, and grants access privileges to the designated third party, generating a secure access link to the shared data.

[1725] Specific examples

[1726] For example, if a user wants to share their EEG data with a medical institution, they follow the steps below.

[1727] 1. The user wears the EEG sensor and launches the dedicated application.

[1728] 2. The device converts the acquired EEG data into CSV format and uploads it to the server using HTTPS.

[1729] 3. The server stores the received data in a MySQL database.

[1730] 4. The user logs into the platform using a web browser and checks their own EEG data.

[1731] 5. The user selects a medical institution and sets the scope and period of data to be shared.

[1732] 6. The server encrypts the data with AES-256 and grants access privileges to the medical institution.

[1733] Example prompts to input to a generative AI model:

[1734] "Please explain the steps a user takes to use the EEG sensor to obtain EEG data and share it with a medical institution."

[1735] By inputting this prompt into the generative AI model, the model outputs specific steps for the user to obtain EEG data and share it with a medical institution.

[1736] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1737] Step 1:

[1738] The user acquires EEG data using a dedicated EEG measurement device. The user wears the measurement device and launches a dedicated application. The measurement device measures the user's EEG in real time and transmits the data to a terminal. The input is the user's EEG, and the output is the EEG data transmitted to the terminal.

[1739] Step 2:

[1740] The device uploads the EEG data to the server. The device converts the acquired EEG data into CSV format using a dedicated application and sends it to the server using the HTTPS protocol. The input is the EEG data sent from the measurement device, and the output is the CSV-formatted EEG data uploaded to the server.

[1741] Step 3:

[1742] The server stores the received EEG data in a database. The server imports the received CSV format EEG data into the database and organizes the data for each user. The input is the CSV format EEG data sent from the device, and the output is the EEG data stored in the database.

[1743] Step 4:

[1744] Users access the platform and manage their data. They log in to the platform through a web browser or mobile application to view, edit, and delete their own EEG data. The input is the user's login information and the EEG data stored in the database, and the output is the EEG data that the user viewed, edited, or deleted.

[1745] Step 5:

[1746] The user sets up data sharing with third parties. The user selects the third parties with whom they want to share data on the platform and sets the scope and period of data to be shared. The input is the user's sharing setting information, and the output is a sharing request sent to the server.

[1747] Step 6:

[1748] The server encrypts the data and grants access permissions to the third party. The server receives the user's sharing request and encrypts the data to be shared with AES-256. The server grants access permissions to the specified third party and generates a secure access link to the shared data. The input is the user's sharing request and the EEG data stored in the database, and the output is the encrypted data and the access permissions granted to the third party.

[1749] (Application example 2)

[1750] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1751] Conventional brain-machine interfaces (BMIs) lacked a platform for effectively managing users' EEG data and sharing it with third parties as needed. Furthermore, there was insufficient management of EEG data security and sharing history, making it difficult to smoothly share data with medical and research institutions. Furthermore, there was a lack of functionality to collect users' EEG data in real time, encrypt it, and upload it to the cloud, which meant that data security and convenience were not ensured.

[1752] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for generating a minimally invasive, highly accurate brain-machine interface (BMI) that combines electroencephalograms and generative AI; means for providing a platform for popularizing the BMI throughout society; means for providing intuitive communication services using brain activity, a metaverse interface, health management services, and biometric authentication services provided through the platform; means for collecting a user's electroencephalogram data in real time, encrypting it, and uploading it to the cloud; and means for sharing the data with medical institutions and research institutions and managing the data sharing history. This makes it possible to safely and efficiently manage a user's electroencephalogram data and share it with third parties as needed.

[1753] "Electroencephalograms" are electrical signals generated by neural activity in the brain.

[1754] "Generative AI" is a type of artificial intelligence, a system that has the ability to generate new data and information based on input data.

[1755] "Minimally invasive" means causing less strain and damage to the body.

[1756] "High accuracy" means very accurate.

[1757] A "brain-machine interface (BMI)" is an interface that uses brain waves to operate machines and computers.

[1758] A "platform" is a system or environment that serves as the foundation for providing specific functions or services.

[1759] An "intuitive communication service" is one that allows users to exchange information in a natural way.

[1760] A "metaverse interface" is an interface that allows users to operate and experience things within a virtual space.

[1761] A "health management service" is a service for monitoring and managing a user's health condition.

[1762] A "biometric authentication service" is a service that verifies a user's identity using their biometric information.

[1763] "Real-time" means that data or information is processed immediately.

[1764] "Encryption" means transforming data using a specific algorithm into a form that is not easily understandable to third parties.

[1765] The "cloud" is a collection of computer resources and services provided over the Internet.

[1766] "Sharing history" is a record of who shared data, when, and how.

[1767] A "medical institution" is a facility that provides medical services, such as a hospital or clinic.

[1768] A "research institution" is an organization or facility for conducting scientific research.

[1769] The following system configuration will be described as an embodiment of the present invention.

[1770] System Configuration

[1771] The system collects users' brainwave data in real time, encrypts it, and uploads it to the cloud. It also has the ability to share the data with medical and research institutions and manage the data sharing history.

[1772] Hardware used

[1773] Smartphone: A device that allows users to manage EEG data and upload it to the cloud.

[1774] Brainwave sensor: A device used to collect a user's brainwave data (e.g., Muse, Emotiv).

[1775] Software used

[1776] Python: A programming language used to perform the main operations of a program.

[1777] NeuroKit2: A library for simulating and analyzing EEG data.

[1778] Requests: A library for sending HTTP requests.

[1779] Cryptography: A library for performing data encryption.

[1780] Processing flow

[1781] 1. EEG data collection: EEG sensors are used to collect users' EEG data in real time, which is then stored on the smartphone.

[1782] 2. Data encryption: The collected EEG data is encrypted using the Cryptography library, which ensures the security of the data.

[1783] 3. Data Upload: The encrypted data is uploaded to the cloud using the Requests library, where it is securely stored.

[1784] 4. Data sharing: Users can share data with medical institutions or research institutes as needed. Sharing history is stored locally, allowing users to see who accessed what data and when.

[1785] Specific examples

[1786] The user wears the EEG sensor and launches the NeuroGuard app. The app collects EEG data, encrypts it, and uploads it to the cloud. Medical institutions can then retrieve the data from the cloud with the user's permission and use it for diagnosis.

[1787] Prompt Sentence Examples

[1788] The user wears the EEG sensor and launches the NeuroGuard app. The app collects EEG data, encrypts it, and uploads it to the cloud. With the user's permission, medical institutions can retrieve the data from the cloud and use it for diagnosis.

[1789] This system allows users' brainwave data to be managed safely and efficiently, and to be shared with third parties if necessary.

[1790] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1791] Step 1:

[1792] The user wears the EEG sensor and launches the NeuroGuard app on their smartphone. The EEG sensor collects the user's brainwave data in real time and transmits it to the smartphone. The input is the brainwave data from the EEG sensor, and the output is the raw brainwave data stored on the smartphone.

[1793] Step 2:

[1794] The device (smartphone) analyzes the collected EEG data using the NeuroKit2 library. This analysis extracts features from the EEG data. The input is raw EEG data, and the output is the features of the analyzed EEG data. Specifically, it calls NeuroKit2 functions to analyze the EEG data and extract features.

[1795] Step 3:

[1796] The device encrypts the analyzed EEG data using the Cryptography library. The input is the characteristics of the analyzed EEG data, and the output is the encrypted EEG data. Specifically, the device generates an encryption key and uses the key to encrypt the EEG data.

[1797] Step 4:

[1798] The device uses the Requests library to upload encrypted EEG data to the cloud. The input is encrypted EEG data, and the output is encrypted data stored on the cloud. Specifically, the device sends an HTTP POST request to upload the data to the cloud server.

[1799] Step 5:

[1800] The server manages the encrypted data stored on the cloud and shares it with medical institutions and research institutions as needed. The input is the encrypted data on the cloud, and the output is the shared data. Specifically, with the user's permission, the server decrypts the data and provides it to a designated third party.

[1801] Step 6:

[1802] The server manages the data sharing history and records who accessed which data and when. The input is the data access request information, and the output is a log of the sharing history. Specifically, it generates an access log and stores it locally or on the cloud.

[1803] In this way, the user's brain wave data can be managed safely and efficiently and shared with third parties as needed.

[1804] Example 3

[1805] Next, a third embodiment of the third embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1806] Conventional brain-machine interfaces (BMIs) have had difficulty effectively analyzing a user's brainwave data and identifying emotions and intentions in real time. Furthermore, there has been a lack of a platform for providing services based on this data, such as intuitive communication, health management, and biometric authentication. Furthermore, there is a lack of functionality for users to manage their own brainwave data and share it with third parties as needed. To solve these issues, a system that can analyze brainwave data with high precision and provide a variety of services based on that data is needed.

[1807] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[1808] In this invention, the server includes: means for a user to wear an EEG measuring device and connect it to a terminal; means for the terminal to collect EEG data from the EEG measuring device and transmit it to the server; means for the server to receive and preprocess the EEG data; means for the server to analyze the preprocessed data and identify emotions and intentions; means for the server to transmit the analysis results to the terminal; means for the terminal to display the analysis results and provide feedback to the user; means for providing a platform for popularizing the system in society; and means for providing intuitive communication services, virtual reality interfaces, health management services, and biometric authentication services that utilize brain activity, which are provided through the platform. This makes it possible to analyze users' EEG data with high accuracy and provide a variety of services based on the analysis results in real time.

[1809] "User" refers to an individual who wears an EEG measurement device and uses the system.

[1810] "Electroencephalogram measuring device" refers to a device for measuring a user's electroencephalogram data.

[1811] "Terminal" refers to an electronic device that transmits data collected from an EEG measuring device to a server and displays the analysis results to the user.

[1812] "Server" refers to a computer system that receives EEG data, performs pre-processing and analysis, and transmits the results to a terminal.

[1813] "Preprocessing" refers to the cleansing of data before analysis, such as removing noise and filtering EEG data.

[1814] "Analysis" refers to data processing to identify the user's emotions and intentions based on preprocessed EEG data.

[1815] "Generative AI model" refers to a machine learning model used to analyze EEG data and identify a user's emotions and intent.

[1816] The "platform" refers to the infrastructure for disseminating the system to society, allowing users to manage their EEG data, and share it with third parties as needed.

[1817] An "intuitive communication service" is a service that understands a user's emotions and intentions based on their brainwave data and communicates based on that.

[1818] A "virtual reality interface" refers to an interface that controls behavior in virtual reality or augmented reality worlds based on the user's brainwave data.

[1819] "Health management service" refers to a service that uses a user's brainwave data to understand their stress level, sleep state, etc., and makes health management suggestions based on that information.

[1820] "Biometric authentication service" refers to a service that analyzes a user's brain wave patterns to authenticate them.

[1821] This invention begins with the user wearing an EEG device and connecting it to a terminal. The user wears the EEG device (e.g., EEG device) on their head and connects it to a terminal (e.g., smartphone or PC) using Bluetooth or a USB cable.

[1822] The device collects EEG data in real time from the EEG device using a dedicated application. The collected data is then sent to a server via the internet. The server receives the EEG data and performs preprocessing. This preprocessing includes noise removal and filtering, and data cleansing is performed using the Python MNE library.

[1823] The preprocessed data is then fed into a machine learning model (e.g., using TensorFlow or PyTorch) on the server to analyze emotions and intent. The analysis results are used to identify the user's emotions (e.g., joy, sadness, surprise, etc.) and intent (e.g., the desire to perform a specific action).

[1824] The analysis results are sent from the server to the device. The device then displays the analysis results received from the server to the user. For example, a chat app might display a message saying "The user is happy." The metaverse interface also controls behavior in the VR or AR world based on the user's intentions.

[1825] As a concrete example, consider an intuitive communication service. A user wears an EEG device and connects it to a smartphone. The smartphone collects brainwave data from the EEG device in real time and sends it to a server. The server preprocesses the received data and analyzes emotions using TensorFlow. If the analysis shows that the user is "happy," that information is reflected in the smartphone's chat app, and a message saying "the user is happy" is sent to the other party.

[1826] Example prompt sentence:

[1827] The user wears an EEG device and connects it to a smartphone. The smartphone collects brainwave data in real time and sends it to a server. The server preprocesses the data and analyzes emotions using TensorFlow. If the analysis shows that the user is "happy," the server reflects this information in a chat app and sends the other party a message saying "the user is happy."

[1828] In this way, a system is realized in which the server, the terminal, and the user work together to provide new services using electroencephalogram data. The flow of the identification process in the third embodiment will be described with reference to FIG.

[1829] Step 1:

[1830] The user wears the EEG measurement device and connects it to the terminal.

[1831] Specifically, the user wears an EEG device on their head and connects it to a device (e.g., a smartphone or PC) using Bluetooth or a USB cable. The input is the user's brainwaves, and the output is the EEG device connected to the device.

[1832] Step 2:

[1833] The terminal collects brain wave data from the EEG measuring device and transmits it to the server.

[1834] Specifically, the terminal uses a dedicated application to collect brainwave data from the EEG device in real time. The collected data is then sent to a server via the Internet. The input is the brainwave data from the EEG measurement device, and the output is the brainwave data sent to the server.

[1835] Step 3:

[1836] The server receives the EEG data and performs preprocessing.

[1837] Specifically, the server receives EEG data sent from the device. The received data undergoes preprocessing such as noise removal and filtering. The data is cleansed using the Python MNE library. The input is the EEG data sent from the device, and the output is the preprocessed EEG data.

[1838] Step 4:

[1839] The server analyzes the pre-processed data to identify emotions and intent.

[1840] Specifically, the server inputs the preprocessed data into a machine learning model (using, for example, TensorFlow or PyTorch). The model identifies the user's emotions and intentions from the EEG data. The input is the preprocessed EEG data, and the output is the analysis result, which is information on emotions and intentions.

[1841] Step 5:

[1842] The server sends the analysis results to the device.

[1843] Specifically, the server sends the analysis results to the device. The analysis results include the user's emotions (e.g., joy, sadness, surprise, etc.) and intentions (e.g., the intention to perform a specific action). The input is the analysis results, and the output is the analysis results sent to the device.

[1844] Step 6:

[1845] The device displays the analysis results and provides feedback to the user.

[1846] Specifically, the device displays the analysis results received from the server to the user. For example, in a chat app, it displays a message saying "The user is happy." In addition, the metaverse interface controls behavior in the VR or AR world based on the user's intentions. The input is the analysis results received from the server, and the output is the feedback displayed to the user.

[1847] (Application example 3)

[1848] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1849] Conventional brain-machine interfaces (BMIs) have limited technology for analyzing users' brain waves, making it difficult to perform high-precision analysis in real time. They also lacked intuitive means for navigation and interaction within the metaverse, making it difficult to execute actions that reflect the user's intentions and emotions. Furthermore, the functionality for managing and sharing brainwave data was inadequate, resulting in low user convenience.

[1850] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[1851] In this invention, the server includes: means for generating a minimally invasive, highly accurate brain-machine interface (BMI) that combines brain waves and generative AI; means for providing a platform for popularizing the BMI throughout society; means for providing intuitive communication services, metaverse interfaces, health management services, and biometric authentication services that utilize brain activity through the platform; means for analyzing a user's brain waves in real time and controlling navigation and interaction within the metaverse; and means for executing actions within the metaverse based on the user's intentions and emotions. This enables users to intuitively navigate and interact within the metaverse using their brain waves.

[1852] "Electroencephalograms" are records of the brain's electrical activity and are used to analyze a user's intentions and emotions.

[1853] "Generative AI" is a type of artificial intelligence, a technology that generates new information and output based on input data.

[1854] A "brain-machine interface (BMI)" is an interface that analyzes brain waves to control machines and computers.

[1855] A "platform" is a system that serves as the foundation for providing specific services and functions.

[1856] An "intuitive communication service" is a service that analyzes the user's brain waves to understand their emotions and intentions, and then communicates based on that.

[1857] A "metaverse interface" is an interface that allows users to interact in virtual reality and augmented reality worlds.

[1858] The "health management service" analyzes the user's brain waves to determine their stress level and sleep state, and then makes health management suggestions based on that information.

[1859] "Biometric authentication service" is a service that analyzes a user's brain wave patterns to authenticate them.

[1860] "Navigation" refers to giving instructions and performing operations to allow a user to move within a virtual space.

[1861] "Interaction" refers to a user interacting with other objects or characters in a virtual space.

[1862] "Intent" is the user's intention to perform a particular action or operation.

[1863] "Emotion" refers to the user's psychological state or feelings.

[1864] An "action" is a movement or reaction that is performed in a virtual space based on the user's intentions and emotions.

[1865] A system for implementing this invention includes means for generating a minimally invasive, highly accurate brain-machine interface (BMI) that combines brain waves and generative AI, means for providing a platform for popularizing the BMI throughout society, means for providing intuitive communication services, metaverse interfaces, health management services, and biometric authentication services that utilize brain activity provided through the platform, means for analyzing a user's brain waves in real time and controlling navigation and interaction within the metaverse, and means for executing actions within the metaverse based on the user's intentions and emotions.

[1866] System configuration

[1867] The system uses the following hardware and software:

[1868] Hardware: EEG sensor, head-mounted display

[1869] Software: Python, EEG sensor API, Metaverse controller API

[1870] Program processing explanation

[1871] The server analyzes the brainwave data acquired from the EEG sensor in real time and inputs it into the generative AI. The generative AI analyzes the user's intentions and emotions based on the input brainwave data and generates output based on that. Specifically, if the user wants to move forward, the brainwave sensor detects that intention and the avatar moves forward. Also, if the user feels happy, that emotion is detected and the avatar smiles.

[1872] Specific examples

[1873] For example, if a user wants to move forward in the Metaverse, the EEG sensor will detect that intention and the avatar will move forward. If the user feels happy, the sensor will detect that emotion and the avatar will smile.

[1874] Prompt Sentence Examples

[1875] An example of a prompt to input to a generative AI model is as follows:

[1876] If the user wants to move forward, the EEG sensor should detect that intention and generate a program that makes the avatar move forward. Also, if the user feels happy, the sensor should detect that emotion and make the avatar smile.

[1877] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[1878] Step 1:

[1879] The server acquires the brain wave data from the brain wave sensor.

[1880] Input: Raw data from EEG sensor

[1881] Output: Acquired EEG data

[1882] Specific operation: The EEG sensor measures the user's brain waves in real time and sends the data to the server.

[1883] Step 2:

[1884] The server preprocesses the acquired electroencephalogram data.

[1885] Input: Acquired EEG data

[1886] Output: Preprocessed EEG data

[1887] Specific operation: Perform preprocessing such as noise removal and filtering, and convert the data into a format suitable for analysis.

[1888] Step 3:

[1889] The server inputs the preprocessed EEG data into the generative AI.

[1890] Input: Preprocessed EEG data

[1891] Output: Data input to the generative AI

[1892] Specific operation: Preprocessed EEG data is input into a generative AI model to analyze the user's intentions and emotions.

[1893] Step 4:

[1894] Generative AI analyzes the user's intentions and emotions based on input brainwave data.

[1895] Input: Data entered into the generative AI

[1896] Output: Analyzed intent and emotion

[1897] How it works: A generative AI model analyzes EEG data to determine the user's intentions and emotions.

[1898] Step 5:

[1899] The server determines actions within the metaverse based on the analysis results.

[1900] Input: Parsed intent and emotion

[1901] Output: Actions taken within the metaverse

[1902] Specific actions: If the user's intention is "move forward," the avatar will move forward; if the emotion is "happy," the avatar will smile.

[1903] Step 6:

[1904] The server sends the determined action to the metaverse controller.

[1905] Input: Actions performed within the metaverse

[1906] Output: Action instructions sent to the metaverse controller

[1907] Specific action: The determined action is sent through the metaverse controller's API, instructing the avatar to actually perform the action.

[1908] Step 7:

[1909] The metaverse controller controls the avatar based on the received action instructions.

[1910] Input: Action instructions sent to the Metaverse Controller

[1911] Output: Avatar movement

[1912] Specific behavior: The Metaverse Controller controls the avatar and performs actions based on the user's intentions and emotions, such as making the avatar move forward or smile.

[1913] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1914] "Example 1"

[1915] In one embodiment of the present invention, a brain-machine interface (BMI) analyzes the user's brain waves in real time and inputs the results into a generative AI. Furthermore, by combining an emotion engine, the user's emotional state is also analyzed simultaneously. Based on the analysis results, the system understands the user's intentions and emotional state and generates output based on that. For example, if a user feels joy and thinks, "I'm going to make coffee," the system recognizes that joy and takes appropriate action to make coffee.

[1916] "Example 2"

[1917] In another embodiment of the present invention, the platform has a function that allows users to manage their own brain wave data and emotional state and share it with third parties as needed. For example, sharing brain wave data when a user is feeling joy with medical institutions or research institutions can lead to more accurate medical services and research results.

[1918] "Example 3"

[1919] In a further embodiment of the present invention, the services provided through the platform have a function of providing services that utilize the user's emotional state. For example, an intuitive communication service analyzes the user's brain waves to understand their emotions and intentions and communicates based on them. If the user is feeling happy, the service will recognize that feeling and provide communication tailored to that emotion.

[1920] The processing flow of each embodiment will be described below.

[1921] "Example 1"

[1922] Step 1: The user thinks about a specific action, for example, "make coffee."

[1923] Step 2: The brain-machine interface (BMI) analyzes the user's brain waves in real time.

[1924] Step 3: The emotion engine analyzes the user's emotional state. In this example, it analyzes that the user has the emotion of joy.

[1925] Step 4: Based on the analysis results, understand the user's intention and emotional state and generate an output based on that. In this example, it triggers an action to make coffee that matches the emotion of joy.

[1926] "Example 2"

[1927] Step 1: The user experiences a specific emotion, for example, joy.

[1928] Step 2: The platform manages the user's brainwave data and emotional state.

[1929] Step 3: The user optionally shares their EEG data and emotional state with third parties, in this example, medical or research institutions.

[1930] "Example 3"

[1931] Step 1: A user uses a service through the platform, for example, an intuitive communication service.

[1932] Step 2: The service analyzes the user's brainwaves to understand their emotions and intentions.

[1933] Step 3: The service uses the analysis results to provide communication tailored to the user's emotional state. In this example, if the user is feeling happy, the service provides communication tailored to that emotion.

[1934] Example 1

[1935] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1936] Conventional brain-machine interfaces (BMIs) only analyzed the user's brain waves, but were unable to take the user's emotional state into account. This made it difficult to accurately understand the user's intentions and generate appropriate output. Furthermore, there was a lack of a platform for managing the user's brainwave data and sharing it with third parties. This hindered the widespread adoption of BMIs in society, making it difficult to provide intuitive communication services, health management services, and biometric authentication services.

[1937] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1938] In this invention, the server includes means for acquiring a user's brain waves in real time, means for analyzing the acquired brain wave data and extracting the user's intention, means for analyzing the user's emotional state, means for generating and inputting prompt sentences to the generative AI based on the analysis results, and means for executing the output generated by the generative AI. This enables the system to accurately understand the user's intention and emotional state and generate appropriate output. Furthermore, a platform for disseminating this system throughout society can be provided, allowing for intuitive communication services, virtual space interfaces, health management services, and biometric authentication services that utilize brain activity.

[1939] "User" refers to an individual who uses the System.

[1940] "Electroencephalograms" refer to electrical signals generated by neural activity in the brain.

[1941] "Real-time" refers to processing data as it is generated.

[1942] "Means of acquisition" refers to the devices and methods used to collect data.

[1943] "Means for analyzing" refers to devices and methods for analyzing acquired data and extracting meaningful information.

[1944] "Intent" refers to information that indicates what a user wants to do or what they are thinking.

[1945] "Emotional state" refers to information that indicates a user's emotions or mood.

[1946] "Generative AI" refers to artificial intelligence that generates new information or output based on given data.

[1947] A "prompt sentence" refers to an input sentence used to give instructions or ask questions to a generative AI.

[1948] "Output" refers to the results or responses that a generative AI generates based on a prompt.

[1949] "Means of execution" refers t...

Claims

1. A means for acquiring a user's brainwave data in real time; A means for analyzing the acquired electroencephalogram data and extracting the user's intention; means for analyzing an emotional state of the user; A means for executing an output using a generative AI based on a prompt sentence corresponding to the extracted intention of the user and the analyzed emotional state of the user; A system including:

2. A means for providing a platform that allows the user to manage his / her own brainwave data and emotion data and share it with a third party as necessary; and a means for providing at least one of an intuitive communication service, a virtual space interface, a health management service, and a biometric authentication service using the brainwave data, which are provided through the platform. The system of claim 1 .

3. A means for analyzing the electroencephalogram data of the user in real time and controlling navigation and interaction within the virtual space; and a means for executing an action of an avatar corresponding to the user in the virtual space based on at least one of an intention and an emotion of the user. The system of claim 1 .

Images