System

Abstract

To provide a system.SOLUTION: A system includes: means for converting a brain wave signal of a thinking pattern of a user, who will operate an information processing device, into a digital signal; means which generates a prompt sentence to be inputted to a generative AI model on the basis of the converted digital signal; means which inputs the generated prompt sentence to the generative AI model and acquires an operation command for operating the information processing device from the generative AI model; and means for transmitting the acquired operation command to the information processing device.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional smartphone operation requires physical manipulation, limiting multitasking for users with physical limitations and in interfaces such as VR, AR, and vehicles. [Means for solving the problem]

[0005] This invention provides a system that converts EEG signals into digital signals and uses them to operate a smartphone. Furthermore, by utilizing generative AI to understand the user's intentions and interpret EEG signals, it enables efficient operation by directly linking the user's thoughts to smartphone operations. This enables smooth operation even for users with physical limitations, or when multitasking with interfaces such as VR, AR, and vehicles. [Brief explanation of the drawings]

[0006] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 2 is a sequence diagram showing a flow of processing in the data processing system according to the first embodiment of the first form example. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Embodiment 1. [Figure 13] FIG. 10 is a sequence diagram showing a processing flow of a data processing system in a second embodiment of the second form example. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 of Embodiment Example 2. [Figure 15] FIG. 10 is a sequence diagram showing the flow of processing in a data processing system according to a third embodiment of the third embodiment. [Figure 16] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 3 of Embodiment 3. [Figure 17] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the first embodiment of the first form example when an emotion engine is combined. [Figure 18] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Embodiment 1 when an emotion engine is combined. [Figure 19] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the second embodiment of the second form example when an emotion engine is combined. [Figure 20] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 of Form Example 2 when an emotion engine is combined. [Figure 21] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the third embodiment of the third form example when an emotion engine is combined. [Figure 22] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 3 of Form Example 3 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0007] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0008] First, the terms used in the following description will be explained.

[0009] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, the processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (TENSOR PROCESSING UNIT (registered trademark)).

[0010] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0011] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0012] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0013] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0014] [First embodiment]

[0015] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0016] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0017] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0018] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0019] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0020] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0021] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0022] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0023] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0024] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0025] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0026] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0027] "Example 1"

[0028] One embodiment of the present invention is a system that includes a device that converts EEG signals into digital signals, a device that uses the digital signals to operate a smartphone, and a device that uses generative AI to understand a user's intentions and interpret the EEG signals. Specifically, when a user engages in a specific thought, an EEG signal is generated, and a device that converts this into a digital signal captures the EEG signal. Next, a device that uses the digital signal to operate the smartphone analyzes the digital signal and executes the corresponding smartphone operation. Furthermore, the device that uses generative AI to understand a user's intentions and interprets the EEG signals learns the user's thought patterns, enabling more accurate operation.

[0029] "Example 2"

[0030] As a concrete example, consider a user with physical limitations operating a smartphone. When the user thinks about tapping the smartphone screen, an EEG signal is generated and converted into a digital signal. A device that uses this digital signal to operate the smartphone analyzes the digital signal and performs the tap operation on the smartphone screen. Furthermore, the device uses generative AI to understand the user's intentions and interprets the EEG signal, learning the user's thought patterns and allowing them to perform more accurate screen tap operations with the same thought the next time.

[0031] "Example 3"

[0032] Similar operations are also possible when multitasking with interfaces such as VR, AR, and vehicles. For example, when operating a navigation system while driving a car, if the user thinks about a destination, an EEG signal is generated and converted into a digital signal. A device that uses this digital signal to operate the navigation system analyzes the digital signal and executes the destination setting operation on the navigation system. Furthermore, the device, which uses generative AI to understand the user's intentions and interprets the EEG signal, can learn the user's thought patterns and execute a more accurate destination setting operation with the same thought next time.

[0033] The processing flow of each embodiment will be described below.

[0034] "Example 1"

[0035] Step 1: The user engages in a specific thought.

[0036] Step 2: EEG signals are generated and captured by a device that converts the EEG signals into digital signals.

[0037] Step 3: The device that uses the digital signals to operate the smartphone analyzes the digital signals.

[0038] Step 4: Based on the analysis results, perform the corresponding smartphone operation.

[0039] Step 5: Using generative AI to understand user intent and interpret EEG signals, the device learns the user's thought patterns.

[0040] "Example 2"

[0041] Step 1: A user with physical limitations thinks about tapping on a smartphone screen.

[0042] Step 2: An EEG signal is generated and converted into a digital signal.

[0043] Step 3: The device that uses the digital signals to operate the smartphone analyzes the digital signals.

[0044] Step 4: Based on the analysis results, perform a tap operation on the smartphone screen.

[0045] Step 5: Using generative AI to understand user intent and interpret EEG signals, the device learns the user's thought patterns.

[0046] "Example 3"

[0047] Step 1: To operate the navigation system while driving, the user thinks about a destination.

[0048] Step 2: An EEG signal is generated and converted into a digital signal.

[0049] Step 3: A device that utilizes the digital signal to operate the navigation system analyzes the digital signal.

[0050] Step 4: Based on the analysis results, the navigation system sets the destination. Step 5: Using generative AI to understand the user's intentions and interpret brainwave signals, the device learns the user's thought patterns.

[0051] Example 1

[0052] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0053] Conventional methods for operating information processing devices require users to operate them manually, making operation particularly difficult for users with physical limitations. Efficient operation is also required for multitasking in interfaces such as virtual reality, augmented reality, and vehicles. Furthermore, it has been difficult to accurately understand the user's intentions and quickly reflect them.

[0054] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means. In this invention, the server includes means for converting an EEG signal into a digital signal, means for operating the information processing device using the digital signal, means for understanding the user's intention and interpreting the EEG signal by making full use of generative artificial intelligence, means for transmitting the digital signal using wireless communication technology, and means for analyzing the digital signal. This allows the user's thoughts to be directly linked to the operation of the information processing device, enabling efficient multitasking for users with physical limitations and in interfaces such as virtual reality, augmented reality, and vehicles.

[0055] "Electroencephalogram signals" are electrical signals generated by neural activity in the brain.

[0056] A "digital signal" is a signal that has been converted from an analog signal into a digital form that can be processed by a computer or electronic device.

[0057] An "information processing device" is a device that inputs, processes, and outputs data, and includes computers, smartphones, etc.

[0058] "Generative AI" is an AI technology that understands the user's intentions and performs appropriate processing based on the generated data.

[0059] "Wireless communication technology" refers to technology that transmits and receives data using radio waves without using cables, and includes Wi-Fi and Bluetooth.

[0060] "Analysis" is the process of examining data in detail to understand its structure and meaning.

[0061] A "thought" is a mental activity consciously performed by the user, with a specific purpose or intention.

[0062] "Virtual reality" is a technology that immerses users in a virtual environment generated using computer technology.

[0063] "Augmented reality" is a technology that displays computer-generated information overlaid on the real environment.

[0064] "Vehicle" means a vehicle for travelling on land, sea or air, including automobiles, ships, aircraft, etc.

[0065] "Multitasking" is the ability or process of performing multiple tasks simultaneously.

[0066] The present invention relates to a system for operating an information processing device using a user's electroencephalogram signal. Specific embodiments of this system will be described below.

[0067] Hardware and Software Configuration

[0068] Brain wave capture device

[0069] The user wears an EEG capture device, which has the function of capturing EEG signals and converting them into digital signals. Specifically, a typical EEG capture device is a headset with a built-in EEG sensor.

[0070] Terminal

[0071] The terminal receives digital signals from the EEG capture device and transmits them to the server. The terminal is equipped with wireless communication technology (such as Wi-Fi or Bluetooth).

[0072] server

[0073] The server receives and analyzes the digital signals sent from the device. The analysis is performed using Python libraries such as "MNE-Python" and "SciPy." Furthermore, generative artificial intelligence (e.g., a generative AI model) is used to understand the user's intentions and interpret the EEG signals. This AI model uses machine learning frameworks such as TENSORFLOW (registered trademark) and PyTorch to learn the user's thought patterns and enable more accurate operation.

[0074] Specific examples

[0075] Example 1: Sending a message

[0076] The user thinks, "I want to send a message." The EEG capture device captures the EEG signal corresponding to this thought and converts it into a digital signal. The device then sends this digital signal to a server via Wi-Fi. The server uses MNE-Python to analyze the digital signal and recognizes the intention to "send a message." The server then uses ADB (ANDROID (registered trademark) Debug Bridge) to launch the Android smartphone's messaging app and send a message to the specified contacts.

[0077] Example 2: Launching an app

[0078] The user thinks, "I want to launch the camera app." The EEG capture device captures the EEG signal corresponding to this thought and converts it into a digital signal. The device then sends this digital signal to a server via Bluetooth. The server uses SciPy to analyze the digital signal and recognizes the intention to "launch the camera app." The server then uses XCUITest to launch the camera app on the iOS smartphone.

[0079] Prompt Sentence Examples

[0080] "Please describe a system that captures the EEG signals generated when a user engages in specific thoughts, converts them into digital signals, and operates an information processing device. Please also specify the names of any specific hardware and software."

[0081] This system provides a more intuitive and efficient interface by directly reflecting the user's thoughts in the operation of the information processing device.

[0082] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0083] Step 1:

[0084] When a user has a specific thought, the EEG capture device captures the EEG signal. The input is the user's thought, and the output is the captured EEG signal. In concrete terms, when a user thinks, "I want to send a message," the EEG capture device detects the EEG signal.

[0085] Step 2:

[0086] The terminal converts the captured EEG signal into a digital signal. The input is the captured EEG signal, and the output is a digital signal. Specifically, the built-in software of the EEG capture device converts the analog signal into a digital signal.

[0087] Step 3:

[0088] The terminal sends the converted digital signal to the server. The input is a digital signal, and the output is the digital signal sent to the server. Specifically, the terminal sends the digital signal to the server using Wi-Fi or Bluetooth.

[0089] Step 4:

[0090] The server analyzes the received digital signal. The input is the digital signal sent to the server, and the output is the analysis result. Specifically, the server uses the Python libraries "MNE-Python" and "SciPy" to extract the characteristics of the digital signal and determine whether it is intended to "send a message."

[0091] Step 5:

[0092] The server executes operations on the information processing device based on the analysis results. The input is the analysis results, and the output is the executed operations. Specifically, the server uses ADB (Android Debug Bridge) to launch the Android smartphone's messaging app and send a message to the specified contacts.

[0093] Step 6:

[0094] The server uses generative AI to understand the user's intentions and interpret the EEG signals. The input is the digital signal and past thought patterns, and the output is the interpretation of the user's intentions. Specifically, the server uses a generative AI model to learn the user's thought patterns and respond quickly to similar thoughts in the future.

[0095] In this way, a system is realized in which the user's thoughts are directly linked to the operation of an information processing device.

[0096] (Application example 1)

[0097] Next, a description will be given of Application Example 1 of Embodiment Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0098] Conventional methods for operating autonomous vehicles rely on physical interfaces, making them difficult for users with physical limitations. Efficient operation is also difficult in multitasking environments where multiple interfaces must be operated simultaneously. Furthermore, the lack of technology to accurately reflect user intentions can lead to reduced operation accuracy.

[0099] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0100] In this invention, the server includes means for converting EEG signals into digital signals, means for operating an information processing device using the digital signals, means for understanding a user's intentions by making full use of generative AI and interpreting the EEG signals, and means for operating an autonomous vehicle using the user's thoughts. This makes it possible to operate an autonomous vehicle using the user's thoughts directly, enabling efficient and accurate operation even for users with physical limitations or in a multitasking environment.

[0101] An "EEG signal" is an electrical signal that reflects the electrical activity of the brain.

[0102] A "digital signal" is a signal that has been converted from an analog signal into a digital format that can be processed by a computer or other electronic device.

[0103] An "information processing device" is a device that has the function of inputting, processing, and outputting data, and specifically includes computers and smartphones.

[0104] "Generative AI" is a type of artificial intelligence that understands the user's intentions and performs appropriate responses and operations based on generated data.

[0105] "User intent" refers to the thoughts or goals that a user has when they want to perform a particular operation or action.

[0106] An "autonomous vehicle" is a vehicle that uses artificial intelligence and sensor technology to drive autonomously.

[0107] "Thinking" is an information processing activity that takes place in the brain and includes conscious judgments and decisions.

[0108] "Physical constraints" refers to physical disabilities or limitations that a user has that make it difficult to perform certain actions or operations.

[0109] "Virtual reality" refers to a computer-generated virtual environment or experience in which users can interact.

[0110] "Augmented reality" is a technology that overlays digital information onto a real environment, allowing users to experience the real world and digital information simultaneously.

[0111] "Multitasking" refers to performing multiple tasks or operations simultaneously, with the aim of completing tasks efficiently.

[0112] A system for implementing this invention includes a device that converts EEG signals into digital signals, a device that uses the digital signals to operate an information processing device, a device that uses generative AI to understand a user's intentions and interpret EEG signals, and a device that uses the user's thoughts to operate an autonomous vehicle.

[0113] First, the user wears a device to capture EEG signals. This device captures EEG signals in real time and converts them into digital signals. The converted digital signals are sent to an information processing device. The information processing device analyzes the digital signals using a generative AI model to understand the user's intentions. The generative AI model is trained in advance using training data and can interpret the user's thought patterns with high accuracy.

[0114] The autonomous vehicle is then controlled based on the interpreted user intent. For example, if the user thinks, "Turn right," their brainwave signal is captured and converted into a digital signal. This digital signal is analyzed and an instruction to steer the vehicle to the right is issued. This allows the user to control the autonomous vehicle with just their thoughts, without having to perform any physical operation.

[0115] The following hardware and software are used to realize this system: An EEG sensor is used as the device for capturing EEG signals. A Fourier transform algorithm is used as the software for converting them into digital signals. An MLP Classifier (Multilayer Perceptron Classifier), for example, is used as the generative AI model. A computer or smartphone is used as the information processing device. A vehicle control system is used to operate the autonomous vehicle.

[0116] As a concrete example, the following prompt sentence is shown.

[0117] "When a driver thinks 'turn right,' their brainwave signals are captured and converted into digital signals. This digital signal is then analyzed and an instruction is issued to steer the vehicle to the right."

[0118] In this way, it becomes possible to directly utilize the user's thoughts to operate an autonomous vehicle, enabling efficient and precise operation even for users with physical limitations or in multitasking environments.

[0119] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0120] Step 1:

[0121] The user wears an EEG sensor.

[0122] Input: User's EEG signal

[0123] Output: Captured EEG signal

[0124] How it works: The user wears an EEG sensor on their head to capture EEG signals in real time. The sensor detects the brain's electrical activity and outputs it as an analog signal.

[0125] Step 2:

[0126] The server converts the EEG signal into a digital signal.

[0127] Input: Captured EEG signal (analog signal)

[0128] Output: Digital signal

[0129] What it does: The server uses a Fourier transform algorithm to convert the analog signal into a digital signal, putting the EEG signal in a format that can be processed by a computer.

[0130] Step 3:

[0131] The server uses a generative AI model to analyze the digital signals and understand the user's intent.

[0132] Input: Digital signal

[0133] Output: User intent (e.g., "Turn right")

[0134] How it works: The server analyzes digital signals using a pre-trained MLPClassifier model. The model learns the user's thought patterns and can accurately predict the user's intentions from the input digital signals.

[0135] Step 4:

[0136] The server generates operating instructions for the autonomous vehicle based on the user's intentions.

[0137] Input: User intent (e.g., "turn right")

[0138] Output: Vehicle operation instructions (e.g., steer right)

[0139] Specific operation: The server receives the user's intention and generates a corresponding vehicle operation instruction. For example, if the intention is "turn right," an instruction to turn the steering wheel to the right is generated.

[0140] Step 5:

[0141] The terminal transmits vehicle operation instructions to the autonomous vehicle.

[0142] Input: Vehicle operation instructions

[0143] Output: Vehicle movement (e.g. steering wheel turns right)

[0144] Specific operation: The terminal sends the generated vehicle operation instructions to the autonomous vehicle control system, which then performs actual vehicle operation based on the received instructions.

[0145] Step 6:

[0146] Autonomous vehicles operate based on the user's intentions.

[0147] Input: Vehicle operation instructions

[0148] Output: Vehicle action (e.g. turn right)

[0149] Specific operation: The autonomous vehicle will follow the received operation instructions and perform operations such as turning the steering wheel to the right. This allows the user to operate the vehicle using only their thoughts.

[0150] Example 2

[0151] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0152] Users with physical limitations often have difficulty intuitively and efficiently operating information processing devices. Conventional interfaces also have the problem of making it difficult to accurately understand the user's intentions and execute appropriate operations. Furthermore, there is a lack of mechanisms for learning the user's thought patterns and improving the accuracy of subsequent operations.

[0153] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0154] In this invention, the server includes means for converting EEG signals into digital signals, means for operating an information processing device using the digital signals, means for understanding a user's intentions by utilizing a generative AI model and interpreting the EEG signals, means for analyzing the digital signals and generating operation commands, means for transmitting the operation commands to the information processing device, means for the information processing device to perform operations in accordance with the operation commands, and means for learning the user's thought patterns, thereby enabling users with physical limitations to intuitively and efficiently operate the information processing device using EEG signals.

[0155] "EEG signals" are signals that reflect the electrical activity of the user's brain.

[0156] A "digital signal" is a signal that is an analog signal converted into digital form.

[0157] An "information processing device" is a device that receives and analyzes digital signals and performs operations based on the user's intentions.

[0158] A "generative AI model" is a model that uses artificial intelligence technology to understand user intent and interpret brainwave signals.

[0159] An "operation command" is an instruction to cause an information processing device to perform a specific operation.

[0160] A "thought pattern" is a characteristic pattern of brain wave signals that occurs when a user performs a specific operation.

[0161] "Wireless communication" is a means of sending and receiving data using wireless technologies such as Wi-Fi and Bluetooth.

[0162] "Analysis" is the process of interpreting the user's intent based on the received digital signal.

[0163] "Learning" is the process of recording the user's past brainwave signals and corresponding operations to improve the accuracy of future operations.

[0164] MODE FOR CARRYING OUT THE INVENTION

[0165] This invention is a system that enables users with physical limitations to intuitively and efficiently operate information processing devices. The system includes a series of processes that convert EEG signals into digital signals, understand the user's intentions using a generative AI model, and operate the information processing device.

[0166] Hardware and software used

[0167] Hardware:

[0168] 1. EEG measurement device: A device for acquiring the user's brainwave signals. A specific example is the EEG measurement device "Epoc+."

[0169] 2. Information processing device: A device that receives and analyzes digital signals and performs actions based on the user's intentions. A good example is a smartphone.

[0170] 3. Server: A device for analyzing digital signals and generating operational commands.

[0171] software:

[0172] 1. Generative AI model: An artificial intelligence model that understands user intent and interprets EEG signals. A specific example is the generative AI model "GPT-4 (registered trademark)."

[0173] Data processing and calculation

[0174] Acquisition and transformation of EEG signals:

[0175] When a user wears an EEG measuring device and thinks about a specific operation, the device acquires an EEG signal, which is then converted into a digital signal by a signal processing unit within the device.

[0176] Digital signal transmission and analysis:

[0177] The device (EEG measurement device) transmits the converted digital signal to a server via wireless communication (Wi-Fi or Bluetooth), which then analyzes the digital signal using a generative AI model to interpret the user's intent.

[0178] Generate and send operation commands:

[0179] The server generates an operation command for the information processing device based on the analysis result, and transmits the generated operation command to the information processing device via wireless communication.

[0180] Execution of operations by information processing device:

[0181] The information processing device executes an operation in accordance with the received operation command. For example, when a smartphone receives a command to tap the "home" button, the smartphone taps the "home" button.

[0182] Learning Thought Patterns:

[0183] The server records the user's EEG signals and the corresponding actions, and uses a generative AI model to learn the user's thought patterns, allowing for more accurate actions the next time the same thought is used.

[0184] Specific examples

[0185] As a concrete example, consider the case where a user thinks about tapping the "Home" button on a smartphone. When the user thinks about tapping the "Home" button, an EEG measurement device captures the EEG signal and converts it into a digital signal. This digital signal is sent to a server, which uses a generative AI model to analyze the user's intention. Based on the analysis results, the server generates a command to tap the "Home" button and sends it to the smartphone. The smartphone then taps the "Home" button according to the command.

[0186] Example prompt sentence:

[0187] "When a user thinks about tapping the home button on their smartphone, please generate a program that analyzes the brainwave signal and sends a command to the smartphone to tap the home button."

[0188] In this way, a physically limited user can intuitively and efficiently operate an information processing device using an electroencephalogram signal.

[0189] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0190] Step 1:

[0191] A user wears an electroencephalogram (EEG) measuring device and thinks about a specific operation, and the EEG measuring device acquires an electroencephalogram signal from the user.

[0192] Input: User's EEG signal

[0193] Output: Captured EEG signal

[0194] Step 2:

[0195] The terminal (EEG measurement device) converts the acquired EEG signal into a digital signal. This conversion process is performed by a signal processing unit within the EEG measurement device.

[0196] Input: Acquired EEG signal

[0197] Output: Digital signal

[0198] Step 3:

[0199] The terminal transmits the converted digital signal to the server via wireless communication (Wi-Fi or Bluetooth).

[0200] Input: Digital signal

[0201] Output: Digital signal sent to the server

[0202] Step 4:

[0203] The server uses a generative AI model to analyze the received digital signals and interpret the user's intentions from their brainwave signals.

[0204] Input: Digital signal sent to the server

[0205] Output: Analysis of user intent

[0206] Step 5:

[0207] The server generates an operation command for the information processing device based on the analysis result. For example, if the user thinks about tapping the "Home" button, the server generates a command to tap the "Home" button.

[0208] Input: Analysis results regarding user intent

[0209] Output: Operation command

[0210] Step 6:

[0211] The server transmits the generated operation command to the information processing device via wireless communication.

[0212] Input: Operation command

[0213] Output: Operation commands sent to the information processing device

[0214] Step 7:

[0215] The information processing device executes an operation in accordance with the received operation command. For example, when a command to tap the "home" button is received, the information processing device taps the "home" button.

[0216] Input: Operation command sent to the information processing device

[0217] Output: Actions taken

[0218] Step 8:

[0219] The server records the user's EEG signals and the corresponding actions, and uses a generative AI model to learn the user's thought patterns, allowing them to perform more accurate actions with the same thoughts next time.

[0220] Input: Records of the user's EEG signals and corresponding actions

[0221] Output: Learned thought patterns

[0222] (Application example 2)

[0223] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0224] Conventional factory robots have been difficult for workers with physical limitations to operate, resulting in inefficient work. Furthermore, the complexity of operation has been an issue even when performing multitasking using multiple interfaces. There is a need to provide a system that can solve these problems and enable workers with physical limitations to operate factory robots efficiently.

[0225] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0226] In this invention, the server includes means for converting EEG signals into digital signals, means for operating a smartphone using the digital signals, means for understanding a user's intentions using generative AI and interpreting the EEG signals, and means for operating a factory robot using the interpreted EEG signals, thereby enabling even physically limited workers to efficiently operate factory robots using EEG signals.

[0227] "EEG signals" are electrical signals generated based on the user's brain activity.

[0228] A "digital signal" is a signal that has been converted from an analog signal into a digital format that can be processed by a computer or other electronic device.

[0229] A "smartphone" is a multi-functional mobile device that can connect to the Internet and run applications in addition to the functions of a mobile phone.

[0230] "Generative AI" is a type of artificial intelligence technology that understands user intent and generates models to perform specific tasks.

[0231] "User intent" refers to the thoughts or goals that a user has when they want to perform a particular operation or action.

[0232] "Means for interpretation" refers to a method or device for analyzing acquired signals or data and understanding their meaning or intent.

[0233] A "factory robot" is a mechanical device used to automate work within a factory, and can manage production lines and perform maintenance work.

[0234] "Means for operating" refers to a method or device for controlling a specific device or system to perform a desired operation.

[0235] "Physical limitations" refers to a condition in which a user has difficulty performing a particular action or operation due to physical reasons.

[0236] "Virtual reality" refers to a virtual environment or experience generated using computer technology in which users can interact.

[0237] "Augmented reality" is a technology that overlays digital information onto a real environment, allowing users to experience the real world and digital information simultaneously.

[0238] A "vehicle" is a means of transportation for transporting people or goods, and includes automobiles, trains, etc.

[0239] "Multitasking" refers to the simultaneous execution of multiple tasks, and is a state in which a user can perform multiple operations or tasks in parallel.

[0240] The system for implementing this invention converts EEG signals into digital signals, interprets the user's intentions using generative AI, and operates a factory robot. A specific embodiment of this system is described below.

[0241] First, the user wears an EEG sensor, which captures the user's brainwave signals in real time and converts them into digital signals, which are then sent to a server.

[0242] The server performs data preprocessing to standardize the acquired digital signals. Specifically, it standardizes the signals using libraries such as StandardScaler. Next, it uses a generative AI model to analyze the standardized signals and interpret the user's intentions. This generative AI model is pre-trained and can generate specific operation commands from the user's EEG signals.

[0243] The interpreted operation command is sent to the factory robot via serial communication. The factory robot performs the specified action based on the received command. For example, if a user thinks, "Lift a part," the generative AI model interprets this intention and generates the command "Lift." The factory robot receives this command and actually performs the action of lifting the part.

[0244] As a concrete example of this system, the following prompt sentence can be input into the generative AI model:

[0245] "When the user thinks 'I want to lift a part,' the brainwave signal is analyzed and a 'lift' command is sent to the robot arm."

[0246] In this way, even physically challenged workers can efficiently operate factory robots using EEG signals. It can also be applied to multitasking using interfaces such as virtual reality, augmented reality, and vehicles, so it is expected to be used in a wide range of applications.

[0247] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0248] Step 1:

[0249] The user wears an EEG sensor, which captures the user's EEG signals in real time. The input is the user's EEG signal, and the output is a digital signal. The EEG sensor converts the analog signal into a digital signal.

[0250] Step 2:

[0251] The terminal transmits the acquired digital signal to the server. The input is a digital signal, and the output is data transmission to the server. The terminal transmits the digital signal to the server using wireless or wired communication.

[0252] Step 3:

[0253] The server normalizes the received digital signal. The input is a digital signal and the output is a normalized signal. The server normalizes the signal using a library such as StandardScaler.

[0254] Step 4:

[0255] The server uses a generative AI model to analyze the standardized signals and interpret the user's intentions. The input is the standardized signal, and the output is the operation command. The server uses a pre-trained generative AI model to analyze the signal and generate the operation command.

[0256] Step 5:

[0257] The server sends the generated operation command to the factory robot via serial communication. The input is the operation command and the output is the command sent to the factory robot. The server sends the operation command to the factory robot using serial communication.

[0258] Step 6:

[0259] The factory robot executes the specified operation based on the received operation command. The input is the operation command and the output is the actual operation. The factory robot performs an operation such as lifting a part according to the received command.

[0260] Example 3

[0261] Next, a description will be given of a third embodiment of the third embodiment. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0262] Conventional methods for operating information processing devices require users to use their hands or voice, making operation particularly difficult for users who are driving or have physical limitations. Efficient operation is also required in multitasking environments where multiple tasks must be performed simultaneously. To solve these issues, a system is needed that directly reflects the user's thoughts in the operation of the information processing device.

[0263] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[0264] In this invention, the server includes means for converting EEG signals into digital signals, means for operating an information processing device using the digital signals, means for understanding a user's intentions by utilizing generative AI and interpreting the EEG signals, and means for learning the user's thought patterns and performing subsequent operations more accurately. This enables a user to operate an information processing device using only their thoughts, without using their hands or voice, enabling efficient operation while driving, for users with physical limitations, and even in a multitasking environment.

[0265] "Electroencephalogram signals" are signals indicative of the electrical activity of a user's brain.

[0266] A "digital signal" is a signal that is an analog signal converted into digital form.

[0267] An "information processing device" is an electronic device for processing and manipulating data.

[0268] "Generative AI" is an artificial intelligence technology that understands user intent and interprets brainwave signals.

[0269] "User intent" is the goal or desire expressed by the user through their thoughts.

[0270] A "thought pattern" is a characteristic pattern of brain wave signals when a user performs a specific purpose or operation.

[0271] "Virtual reality" is a virtual environment created using computer technology.

[0272] "Augmented reality" is a technology that overlays digital information onto the real environment.

[0273] "Multitasking" refers to performing multiple tasks simultaneously.

[0274] MODE FOR CARRYING OUT THE INVENTION

[0275] This invention relates to a system that allows a user to operate an information processing device simply by thinking. Specifically, the system converts electroencephalogram signals into digital signals, analyzes the digital signals to understand the user's intentions, and then operates the information processing device.

[0276] Hardware and software used

[0277] Hardware: EEG sensors, information processing devices (e.g., navigation systems)

[0278] Software: EEG signal analysis software, generative AI model

[0279] System Overview

[0280] 1. EEG signal acquisition:

[0281] A user wants to operate an information processing device.

[0282] The terminal (brainwave sensor) acquires the user's brainwave signals in real time.

[0283] 2. Conversion to digital signal:

[0284] The brainwave signal acquired by the terminal (brainwave sensor) is converted into a digital signal.

[0285] 3. Digital signal analysis:

[0286] The server receives the digital signal and analyzes it using EEG signal analysis software.

[0287] The server inputs the analysis results into a generative AI model to understand the user's intentions.

[0288] 4. Operation of information processing devices:

[0289] The server transmits the analysis results to the information processing device.

[0290] The terminal (information processing device) performs an operation based on the received information.

[0291] 5. Generative AI Learning:

[0292] The server uses a generative AI model to learn the user's thought patterns.

[0293] The server updates the model for future operations.

[0294] Specific examples

[0295] For example, if a user thinks, "I want to go to Tokyo Tower" while driving a car, the EEG sensor picks up the EEG signal and converts it into a digital signal. The server analyzes the digital signal and identifies the destination, "Tokyo Tower." The navigation system receives this information and sets the destination to "Tokyo Tower." The generative AI learns this series of operations, and from then on, simply by thinking, "I want to go to Tokyo Tower," the destination will be set more quickly and accurately.

[0296] Prompt Sentence Examples

[0297] "Describe a system that captures brainwave signals when a user thinks about a destination while driving a car, converts them into digital signals, and sends them to a navigation system."

[0298] This system allows the user to operate the information processing device without using their hands while driving, thereby improving safety and convenience. The flow of the identification process in the third embodiment will be described with reference to FIG.

[0299] Step 1:

[0300] A user wants to operate an information processing device.

[0301] Specific operation: The user thinks, "I want to go to Tokyo Tower."

[0302] Input: User thoughts

[0303] Output: EEG signal

[0304] Step 2:

[0305] The terminal (brainwave sensor) acquires the user's brainwave signals in real time.

[0306] What it does: The EEG sensor captures the user's brainwave signals.

[0307] Input: User's EEG signal

[0308] Output: Analog EEG signal

[0309] Step 3:

[0310] The analog brainwave signal acquired by the terminal (brainwave sensor) is converted into a digital signal.

[0311] Specific operation: The EEG sensor converts analog signals into digital signals and outputs them as data.

[0312] Input: Analog EEG signal

[0313] Output: Digital EEG signal

[0314] Step 4:

[0315] The server receives the digital EEG signals and analyzes them using EEG signal analysis software.

[0316] What it does: The server analyzes the digital signals to determine the user's intent.

[0317] Input: Digital EEG signal

[0318] Output: Analysis result (user intent)

[0319] Step 5:

[0320] The server inputs the analysis results into a generative AI model to understand the user's intentions.

[0321] Specific operation: The generative AI model interprets the user's intention based on the analysis results.

[0322] Input: Analysis result (user intent)

[0323] Output: Interpreted user intent

[0324] Step 6:

[0325] The server transmits the interpreted user's intention to the information processing device.

[0326] Specific operation: The server transmits the interpreted intention to the information processing device.

[0327] Input: Interpreted user intent

[0328] Output: Instructions to the information processing device

[0329] Step 7:

[0330] The terminal (information processing device) executes an operation based on the received instruction.

[0331] Specific operation: The information processing device performs operations such as setting a destination.

[0332] Input: Instructions to the information processing device

[0333] Output: The action taken (e.g., set destination)

[0334] Step 8:

[0335] The server uses a generative AI model to learn the user's thought patterns.

[0336] Specific operation: The generative AI model learns the user's thought patterns and performs operations more accurately from the next time onwards.

[0337] Input: The operations performed and the user's thought patterns

[0338] Output: An updated generative AI model

[0339] (Application example 3)

[0340] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0341] Conventional navigation systems require users to manually input their destination, making operation difficult, especially for those driving or those with physical limitations. Furthermore, when performing multiple tasks simultaneously, the system becomes increasingly complicated and places a strain on the user, limiting the use of navigation systems.

[0342] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[0343] In this invention, the server includes a means for converting EEG signals into digital signals, a means for using the digital signals to operate the navigation system of the autonomous vehicle, and a means for understanding the user's intentions and interpreting the EEG signals using generative AI. This allows the user to set a destination in the navigation system of the autonomous vehicle simply by thinking, making it easy to operate even while driving or for users with physical limitations. Furthermore, even when performing multiple tasks simultaneously, the complexity of operation is reduced, significantly reducing the burden on the user.

[0344] "EEG signals" are electrical signals generated in conjunction with brain activity, and reflect the user's thoughts and emotions.

[0345] A "digital signal" is a digitalized analog signal that can be processed by computers and other electronic devices.

[0346] An "autonomous vehicle" is a vehicle that uses artificial intelligence and sensor technology to drive autonomously, without the need for human operation.

[0347] A "navigation system" is a system that guides you to your destination and provides the optimal route based on map information and current location information.

[0348] "Generative AI" is artificial intelligence that understands the user's intentions and thoughts and generates appropriate responses and operations, using machine learning and natural language processing technologies.

[0349] "User intent" is the idea or goal of what the user wants or is trying to do.

[0350] "Interpreting" means analyzing input information and understanding its meaning and intent.

[0351] The following system configuration will be described as an embodiment of the present invention.

[0352] System Configuration

[0353] This system includes a means for converting EEG signals into digital signals, a means for using the digital signals to operate the navigation system of an autonomous vehicle, and a means for using generative AI to understand the user's intentions and interpret the EEG signals.

[0354] Hardware

[0355] EEG sensor: A device that captures the user's brainwave signals.

[0356] Navigation systems for autonomous vehicles: Navigation systems installed in autonomous vehicles.

[0357] software

[0358] Generative AI model: Software that converts EEG signals into digital signals and interprets user intent.

[0359] Processing flow

[0360] 1. Capturing EEG signals: The EEG sensor captures the user's EEG signals.

[0361] 2. Conversion to digital signals: The generative AI model converts the captured EEG signals into digital signals.

[0362] 3. Digital signal analysis: Generative AI models analyze digital signals to interpret user intent and identify destinations.

[0363] 4. Destination setting: The autonomous vehicle's navigation system sets the specified destination.

[0364] Specific examples

[0365] For example, if a user thinks, "I want to go to Tokyo Tower," the EEG sensor captures the EEG signal, and the generative AI model analyzes the signal to identify the destination, "Tokyo Tower." The navigation system of the autonomous vehicle receives this information and sets "Tokyo Tower" as the destination in the vehicle's navigation system.

[0366] Prompt Sentence Examples

[0367] Analyze the EEG signals when the user thinks, "I want to go to Tokyo Tower," and generate a prompt that identifies the destination.

[0368] In this way, the brainwave navigation system can read the user's thoughts and set a destination, making it easier to operate while driving or for users with physical limitations, and reducing the complexity of operation when performing multiple tasks at the same time.

[0369] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[0370] Step 1:

[0371] Capturing EEG signals

[0372] The user wears the EEG sensor and thinks. The EEG sensor captures the user's EEG signals in real time. The input is the user's EEG signals, and the output is the captured raw EEG data. This data is in analog form and is converted into a digital signal in the next step.

[0373] Step 2:

[0374] Conversion to digital signals

[0375] The server receives the captured raw EEG data and converts it into a digital signal using a generative AI model. The input is raw EEG data, and the output is a digital signal. This conversion includes data processing such as filtering and noise removal of the EEG data. Specifically, the EEG data is sampled and quantized.

[0376] Step 3:

[0377] Digital Signal Analysis

[0378] The server uses a generative AI model to analyze digital signals and interpret the user's intent. The input is the digital signal, and the output is the user's intent (e.g., the name of a destination). This analysis involves pattern recognition using machine learning algorithms. Specific operations include feature extraction and classification of the digital signal.

[0379] Step 4:

[0380] Destination setting

[0381] The server sends the analyzed user intention to the navigation system of the autonomous vehicle. The input is the user intention (the name of the destination), and the output is the destination set in the navigation system. Specifically, the server calls the navigation system's API to set the destination.

[0382] Step 5:

[0383] Start navigation

[0384] The navigation system of an autonomous vehicle calculates the optimal route based on the set destination and starts navigation. The input is the set destination and the output is the navigation route. Specific operations include searching map data and calculating the route.

[0385] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0386] "Example 1"

[0387] In one embodiment of the present invention, a system is provided that includes a means for converting EEG signals into digital signals, a means for operating a smartphone using the digital signals, a means for understanding a user's intentions and interpreting the EEG signals using generative AI, and an emotion engine that recognizes the user's emotions. Specifically, when the user is feeling joy, the system can operate the smartphone in accordance with the user's emotions, such as by brightening the color tone of the smartphone screen.

[0388] "Example 2"

[0389] In another embodiment of the present invention, the emotion engine provides a means for executing smartphone operations according to the user's emotions. Specifically, when the user feels angry, the smartphone can be operated according to the user's emotions, such as temporarily stopping smartphone notifications.

[0390] "Example 3"

[0391] In a further embodiment of the present invention, the emotion engine provides a means for adjusting the interface in response to the user's emotion, for example, by lowering the volume of the smartphone when the user is feeling sad.

[0392] The processing flow of each embodiment will be described below.

[0393] "Example 1"

[0394] Step 1: Acquire the user's EEG signal.

[0395] Step 2: Convert the acquired EEG signal into a digital signal.

[0396] Step 3: Use generative AI to interpret user intent from digital signals.

[0397] Step 4: Use the emotion engine to recognize the user's emotions.

[0398] Step 5: Operate the smartphone based on the recognized emotion. For example, if the user is feeling happy, the smartphone screen will brighten up.

[0399] "Example 2"

[0400] Step 1: Acquire the user's EEG signal.

[0401] Step 2: Convert the acquired EEG signal into a digital signal.

[0402] Step 3: Use generative AI to interpret user intent from digital signals.

[0403] Step 4: Use the emotion engine to recognize the user's emotions.

[0404] Step 5: Implement smartphone actions based on the recognized emotion. For example, temporarily suspend smartphone notifications if the user is feeling angry.

[0405] "Example 3"

[0406] Step 1: Acquire the user's EEG signal.

[0407] Step 2: Convert the acquired EEG signal into a digital signal.

[0408] Step 3: Use generative AI to interpret user intent from digital signals.

[0409] Step 4: Use the emotion engine to recognize the user's emotions.

[0410] Step 5: Adjust the smartphone interface based on the recognized emotion, for example, lowering the volume of the smartphone when the user feels sad.

[0411] Example 1

[0412] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0413] Conventional methods for operating information processing devices require users to use their hands or voice, which makes operation difficult for users with physical limitations or in multitasking environments with interfaces such as virtual reality, augmented reality, or in vehicles. Furthermore, the inability to accurately understand the user's intentions and emotions and operate accordingly poses a problem of not improving the user experience.

[0414] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0415] In this invention, the server includes means for converting EEG signals into digital signals, means for operating an information processing device using the digital signals, means for understanding a user's intentions and interpreting EEG signals by making full use of generative artificial intelligence, and means for recognizing the user's emotions. This makes it possible to operate the information processing device based on the user's thoughts and emotions, enabling intuitive and efficient operation even for users with physical limitations or in a multitasking environment.

[0416] "Electroencephalogram signals" are weak electrical signals generated by electrical activity in the brain.

[0417] A "digital signal" is a signal that has been converted from an analog signal into a digital form that can be processed by a computer or electronic device.

[0418] An "information processing device" is an electronic device that has the function of inputting, processing, and outputting data, and specifically refers to smartphones, computers, etc.

[0419] "Generative AI" is an artificial intelligence technology that understands the user's intentions and thoughts and generates appropriate responses and operations.

[0420] "User intent" refers to the thoughts or goals that a user has when they want to perform a particular operation or action.

[0421] "User emotion" refers to the psychological state a user feels in response to a particular situation or stimulus, including joy, sadness, anger, etc.

[0422] "Means for recognizing emotions" refers to technologies and devices for detecting and identifying a user's emotions.

[0423] "Thought patterns" are specific patterns of brain waves that are generated when a user has a specific thought or intention.

[0424] "Analysis" is the process of processing a digital signal and extracting useful information from it.

[0425] An "operation instruction" is a signal or command that instructs an information processing device to perform a specific operation.

[0426] This invention is a system that converts a user's electroencephalogram (EEG) signals into digital signals and uses these digital signals to operate an information processing device. Furthermore, it utilizes generative artificial intelligence to understand the user's intentions and interpret the EEG signals. It can also recognize the user's emotions and perform operations accordingly.

[0427] Hardware and software used

[0428] Hardware

[0429] 1. EEG headset: A device that captures the user's brainwave signals and converts them into digital signals. Specifically, it is a device with sensors that is worn on the scalp.

[0430] 2. Information processing device: A device such as a smartphone or computer that receives digital signals and performs operations based on the analysis results.

[0431] 3. Server: A device that receives digital signals and interprets their intent using analytical and generative artificial intelligence.

[0432] software

[0433] 1. Generative AI model: A machine learning model to understand user intent and interpret EEG signals. It uses techniques such as neural networks.

[0434] 2. Signal analysis algorithm: An algorithm that analyzes digital signals and infers the user's intentions. Specifically, it uses FFT (Fast Fourier Transform) and machine learning algorithms.

[0435] 3. Emotion recognition engine: Software that recognizes the user's emotions and determines the appropriate operation.

[0436] Specific examples

[0437] Example 1: Sending email

[0438] When a user thinks about "sending an email," the following process takes place:

[0439] 1. The user thinks, "I want to send an email."

[0440] 2. The EEG headset captures the brainwave signals and converts them into digital signals.

[0441] 3. The device sends a digital signal to the server.

[0442] 4. The server analyzes the digital signals and uses a generative artificial intelligence model to interpret the user's intent as "send an email."

[0443] 5. The server sends an instruction to the information processing device (smartphone) to launch the email application and display a new email creation screen.

[0444] 6. Follow the instructions received on your smartphone to launch the email app and display the new email composition screen.

[0445] Example 2: Emotional manipulation

[0446] If the user is happy, the following process is performed:

[0447] 1. Users feel delighted.

[0448] 2. The EEG headset captures the brainwave signals and converts them into digital signals.

[0449] 3. The device sends a digital signal to the server.

[0450] 4. The server analyzes the digital signal and uses an emotion recognition engine to identify the user's emotion as "joy."

[0451] 5. The server sends an instruction to the information processing device (smartphone) to brighten the screen color tone.

[0452] 6. Follow the instructions received by your smartphone to brighten the screen color tone.

[0453] Prompt Sentence Examples

[0454] "Design a system that, when a user has a specific thought, executes an operation on an information processing device that corresponds to that thought. For example, if a user thinks, 'I want to send an email,' the system should recognize that intention and send an instruction to the information processing device to launch the email application. Also, add a function that recognizes the user's emotions and performs an operation according to those emotions. For example, if the user is feeling happy, make the color tone of the information processing device screen brighter."

[0455] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0456] Step 1:

[0457] The user performs a specific thought.

[0458] Input: User thoughts (e.g., "send email")

[0459] Output: EEG signal

[0460] Specific operation: When a user thinks about "sending an email," electrical activity occurs in the brain, generating an EEG signal.

[0461] Step 2:

[0462] The terminal (EEG headset) captures the brainwave signal and converts it into a digital signal.

[0463] Input: EEG signal

[0464] Output: Digital signal

[0465] How it works: The EEG headset's sensors contact the user's scalp and measure brain waves in real time. The measured analog signal is converted to a digital signal using an ADC (analog-to-digital converter).

[0466] Step 3:

[0467] The terminal transmits a digital signal to the server.

[0468] Input: Digital signal

[0469] Output: Send data to the server

[0470] What it does: The device uses Wi-Fi or Bluetooth to send a digital signal to the server.

[0471] Step 4:

[0472] The server analyzes the received digital signal.

[0473] Input: Digital signal

[0474] Output: Analysis results (frequency components, etc.)

[0475] Specific operation: The server analyzes the frequency components of the digital signal using FFT (Fast Fourier Transform), and extracts the signal features as a result of the analysis.

[0476] Step 5:

[0477] The server uses a generative artificial intelligence model to interpret the user's intent.

[0478] Input: Analysis results

[0479] Output: User intent (e.g. "send email")

[0480] Specific operation: The server uses a machine learning model (e.g., a neural network) to infer the user's intent from the analysis results. The inferred intent is "send an email."

[0481] Step 6:

[0482] The server determines an operation to be performed on the information processing device based on the estimated user's intention.

[0483] Input: User intent

[0484] Output: Operation instructions (e.g., launch the email app)

[0485] Specific operation: The server determines the corresponding operation based on the user's intention. For example, if the intention is to "send an email," it determines the operation to launch the email app.

[0486] Step 7:

[0487] The server transmits an operation instruction to the information processing device.

[0488] Input: Operation instructions

[0489] Output: Data transmission to information processing device

[0490] Specific operation: The server uses the API of the information processing device (smartphone) to send operation instructions.

[0491] Step 8:

[0492] The information processing device executes the operation according to the received instruction.

[0493] Input: Operation instructions

[0494] Output: Operation result (e.g. launching email app)

[0495] Specific operation: According to the operation instructions received by the smartphone, the email app is launched and a new email creation screen is displayed.

[0496] Step 9:

[0497] The device recognizes the user's emotions.

[0498] Input: EEG signal

[0499] Output: Emotional information (e.g., joy)

[0500] How it works: The EEG headset captures brainwave signals, and the emotion recognition engine analyzes the signals to recognize the user's emotions.

[0501] Step 10:

[0502] The device transmits the recognized emotion information to the server.

[0503] Input: Emotion information

[0504] Output: Send data to the server

[0505] Specific operation: The device uses Wi-Fi or Bluetooth to send emotional information to the server.

[0506] Step 11:

[0507] The server determines the operation of the information processing device based on the emotion information.

[0508] Input: Emotion information

[0509] Output: Operation instructions (e.g., brighten the screen color)

[0510] Specific operation: The server determines the corresponding operation based on the emotion information. For example, if the user feels happy, it determines the operation to brighten the color tone of the screen.

[0511] Step 12:

[0512] The server transmits an operation instruction according to the emotion to the information processing device.

[0513] Input: Operation instructions

[0514] Output: Data transmission to information processing device

[0515] Specific operation: The server uses the API of the information processing device (smartphone) to send operation instructions.

[0516] Step 13:

[0517] The information processing device executes the operation according to the received instruction.

[0518] Input: Operation instructions

[0519] Output: Operation result (e.g., change in screen color)

[0520] Specific operation: The smartphone brightens the screen color according to the operation instructions received.

[0521] (Application example 1)

[0522] Next, a description will be given of Application Example 1 of Embodiment Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0523] Conventional methods for operating information processing devices rely on manual operation or voice recognition, making them difficult for users with physical limitations or in certain environments. From a security perspective, conventional authentication methods pose a high risk of unauthorized access. Furthermore, the inability to take the user's emotional state into account in operations hinders the user experience.

[0524] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0525] In this invention, the server includes means for converting EEG signals into digital signals, means for operating an information processing device using the digital signals, means for understanding a user's intentions using generative artificial intelligence and interpreting the EEG signals, means including an emotion engine for recognizing the user's emotions, means for performing authentication using the user's EEG signals, and means for determining whether to authenticate the user based on the user's emotional state. This allows even users with physical limitations to easily operate the information processing device, improving security and the user experience.

[0526] An "electroencephalogram signal" is a signal that reflects the electrical activity of the brain and is measured using a device such as an electroencephalograph.

[0527] A "digital signal" is a signal that has been converted from an analog signal into a digital format that can be processed by a computer or information processing device.

[0528] An "information processing device" is a device that inputs, processes, and outputs data, and includes computers, smartphones, etc.

[0529] "Generative AI" is AI that uses machine learning and deep learning to learn patterns from data and understand user intentions and emotions.

[0530] An "emotion engine" is software or hardware that recognizes a user's emotional state and responds or takes appropriate action based on that.

[0531] "Authentication" is the process of verifying that a user is a legitimate user, and is carried out to ensure security.

[0532] "Electroencephalogram authentication" is a method of authentication that utilizes the user's brainwave signals, ensuring security by identifying each individual user's brainwave patterns.

[0533] "Emotional state" refers to the user's current emotional state, including emotions such as joy, sadness, anger, and fear.

[0534] An "operation" is an act of giving an instruction to an information processing device to perform a specific action.

[0535] A system for implementing this invention includes means for converting an EEG signal into a digital signal, means for operating an information processing device using the digital signal, means for understanding a user's intentions by making full use of generative artificial intelligence and interpreting the EEG signal, means including an emotion engine for recognizing the user's emotions, means for performing authentication using the user's EEG signal, and means for determining whether or not to authenticate the user based on the user's emotional state.

[0536] Hardware and Software Configuration

[0537] The server captures the user's brainwave signals using an EEG capture device (e.g., Emotiv Epoc+). The captured brainwave signals are converted into digital signals and sent to an information processing device (e.g., a smartphone or computer). A generative artificial intelligence model (e.g., a deep learning model using Keras) analyzes the digital signals and understands the user's intentions. The emotion engine recognizes the user's emotional state and determines whether or not to authenticate them.

[0538] Data processing and calculation

[0539] The EEG capture device captures the user's EEG signals in real time and converts them into digital signals. The digital signals undergo a standardization process and are then input into a generative AI model. The generative AI model analyzes the user's EEG patterns and predicts their intentions. The emotion engine recognizes the user's emotional state from the digital signals and provides data to determine whether or not to authenticate them.

[0540] Specific examples

[0541] For example, when a user tries to unlock their smartphone, an EEG capture device captures the user's brainwave signals and converts them into digital signals. A generative AI model analyzes the digital signals and recognizes that the user's intention is to "unlock." If the emotion engine determines that the user's emotional state is "calm," the smartphone will be unlocked.

[0542] Prompt Sentence Examples

[0543] "Create a program that uses an EEG capture device to capture brainwave signals when a user tries to unlock their smartphone, uses a generative artificial intelligence model to interpret the user's intent, and uses an emotion engine to recognize the user's emotions."

[0544] In this way, a system is realized that utilizes the user's electroencephalogram signal and emotional state to operate and authenticate an information processing device.

[0545] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0546] Step 1:

[0547] The user wears an EEG capture device and captures EEG signals.

[0548] Input: User's EEG signal

[0549] Output: Analog EEG signal

[0550] Specific operation: The user wears an EEG capture device on their head, and the device captures EEG signals in real time.

[0551] Step 2:

[0552] The server converts the captured analog EEG signals into digital signals.

[0553] Input: Analog EEG signal

[0554] Output: Digital EEG signal

[0555] Specific operation: The server receives the analog EEG signal and performs a standardization process to convert it into a digital signal.

[0556] Step 3:

[0557] The server inputs the digital EEG signals into a generative artificial intelligence model to analyze the user's intentions.

[0558] Input: Digital EEG signal

[0559] Output: User intent (e.g. unlock)

[0560] Specific operation: The server inputs digital EEG signals into a generative artificial intelligence model, which then analyzes the signals and predicts the user's intentions.

[0561] Step 4:

[0562] The server inputs the digital EEG signal into an emotion engine to recognize the user's emotional state.

[0563] Input: Digital EEG signal

[0564] Output: User's emotional state (e.g., calm)

[0565] Specific operation: The server inputs digital EEG signals into the emotion engine, which then analyzes the signals to recognize the user's emotional state.

[0566] Step 5:

[0567] The server determines whether to authenticate the user based on their intentions and emotional state.

[0568] Input: User intent, user emotional state

[0569] Output: Authentication result (e.g., authentication successful)

[0570] Specific operation: The server confirms that the user's intention is "unlock" and that the emotional state is "calm," and determines that the authentication is successful.

[0571] Step 6:

[0572] The server executes the operation of the information processing device based on the authentication result.

[0573] Input: Authentication result

[0574] Output: Operation results of information processing device (e.g. unlocking a smartphone)

[0575] Specific operation: The server sends the authentication result to the information processing device and performs the operation to unlock the smartphone.

[0576] Example 2

[0577] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0578] Users with physical limitations and in multitasking environments such as virtual reality, augmented reality, and in-vehicle interfaces face challenges in intuitive and efficient device operation. In particular, when operating a device using EEG signals, it is necessary to accurately understand the user's intentions and execute appropriate operations.

[0579] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0580] In this invention, the server includes means for converting EEG signals into digital signals, means for analyzing the digital signals, means for understanding the user's intention using a generative AI model, means for sending operation instructions to the smartphone based on the intention, and means for receiving the operation instructions and executing smartphone operations. This allows the user's thoughts to be directly linked to smartphone operations, enabling intuitive and efficient device operation even for users with physical limitations or in a multitasking environment.

[0581] "EEG signals" are signals that reflect the electrical activity of the brain and are used to detect a user's thoughts and emotions.

[0582] A "digital signal" is an analog signal converted into a digital format that can be processed by a computer or electronic device.

[0583] A "generative AI model" is a model that uses artificial intelligence technology to analyze data and learn user intentions and patterns.

[0584] "User intent" refers to the user's intention or purpose to perform a specific operation or action.

[0585] An "operation instruction" is a specific operation command for a device that is generated based on the user's intention.

[0586] A "smartphone" is a multi-functional mobile device that can connect to the Internet and run applications in addition to the functions of a mobile phone.

[0587] "Analysis" is a data processing method that processes digital signals to understand user intent and patterns.

[0588] "Interface" refers to the means or methods that enable interaction between a user and a device.

[0589] "Multitasking" refers to the state or ability to perform multiple tasks or operations simultaneously.

[0590] MODE FOR CARRYING OUT THE INVENTION

[0591] The present invention relates to a system for operating a smartphone using a user's electroencephalogram (EEG) signals. Specific embodiments of this system will be described below.

[0592] System configuration

[0593] The system consists of the following main components:

[0594] 1. EEG measurement device: A device for acquiring the user's EEG signals. A specific example is an EEG measurement device such as Emotiv Epoc+.

[0595] 2. Digital signal converter: A device that converts acquired EEG signals into digital signals. It may be built into the EEG measurement device.

[0596] 3. Server: A computer system that analyzes digital signals and understands user intent using a generative AI model. Signal processing algorithms (e.g., FFT, filtering) and generative AI models (e.g., GPT-4 by OpenAI®) are used for the analysis.

[0597] 4. Smartphone: A multi-functional mobile device that receives operation instructions from the server and actually executes the operations.

[0598] System Operation

[0599] When a user thinks about operating a smartphone, an electroencephalogram (EEG) signal is generated. The EEG measurement device acquires this EEG signal in real time and converts it into a digital signal. The converted digital signal is then sent to a server via the Internet.

[0600] The server analyzes the received digital signals and uses a generative AI model to understand the user's intentions. The generative AI model learns the user's thought patterns based on past data and infers their intentions. Based on the analysis results, the server generates operational instructions for the smartphone and sends them to the device. The smartphone performs specific operations (e.g., tap the home button) according to the received operational instructions.

[0601] Specific examples

[0602] As a concrete example, consider the case where a user thinks, "Tap the home button." The user's brainwave signals are acquired by an EEG measuring device and converted into digital signals. The server analyzes these digital signals and uses a generative AI model to understand the user's intention. Based on the analysis results, the server sends the instruction to the smartphone to "Tap the home button," and the smartphone carries out this instruction.

[0603] Prompt Sentence Examples

[0604] Analyze the EEG signals when the user thinks "tap the home button," and generate a program that will execute the action of tapping the home button on a smartphone.

[0605] This system allows the user's thoughts to be directly linked to the operation of the smartphone, enabling intuitive and efficient device operation even for users with physical limitations or in multitasking environments.

[0606] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0607] Step 1:

[0608] The user thinks about how to operate a smartphone.

[0609] A user thinks about tapping the home button on their smartphone. This thought generates an EEG signal. The input is the user's thought, and the output is the EEG signal.

[0610] Step 2:

[0611] The device acquires the brainwave signal.

[0612] The terminal (EEG measuring device) acquires the user's EEG signals in real time. The EEG measuring device is worn on the user's head and electrically detects the EEG signals. The input is the user's EEG signals, and the output is the acquired EEG signals.

[0613] Step 3:

[0614] The terminal converts the brainwave signal into a digital signal.

[0615] The terminal converts the acquired EEG signal into a digital signal, which makes the EEG signal in a format that can be processed by a computer. The input is the acquired EEG signal, and the output is a digital signal.

[0616] Step 4:

[0617] The terminal transmits a digital signal to the server.

[0618] The terminal transmits the converted digital signal to a server via the Internet using a communication method such as Wi-Fi or Bluetooth. The input is a digital signal, and the output is the digital signal transmitted to the server.

[0619] Step 5:

[0620] The server analyzes the digital signal.

[0621] The server analyzes the received digital signal. Signal processing algorithms (e.g. FFT, filtering) are used for the analysis. The input is the digital signal sent to the server, and the output is the analyzed digital signal.

[0622] Step 6:

[0623] The server uses a generative AI model to understand the user's intent.

[0624] The server inputs the analyzed digital signals into a generative AI model (e.g., GPT-4) to understand the user's intention. The generative AI model learns the user's thought patterns based on past data and infers their intention. The input is the analyzed digital signals, and the output is the user's intention.

[0625] Step 7:

[0626] The server sends operation instructions to the smartphone.

[0627] The server generates operational instructions for the smartphone based on the user's intention and sends them to the device. The operational instructions include specific actions (e.g., tapping the home button). The input is the user's intention, and the output is the operational instructions.

[0628] Step 8:

[0629] The device receives the operation instruction and executes the smartphone operation.

[0630] The terminal transmits the operation instruction received from the server to the smartphone, and the smartphone executes the operation of tapping the home button. The input is the operation instruction, and the output is the executed smartphone operation.

[0631] (Application example 2)

[0632] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0633] Conventional factory robot operation systems make it difficult for workers with physical limitations to operate robots efficiently. Furthermore, they are unable to operate robots while taking into account the emotional state of the workers, which leads to problems with work efficiency and safety. Furthermore, operation in a multitasking environment using multiple interfaces is also difficult.

[0634] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0635] In this invention, the server includes a means for converting EEG signals into digital signals, a means for operating a smartphone using the digital signals, a means for understanding a user's intentions and interpreting the EEG signals using generative AI, and a means for detecting the user's emotional state and adjusting the operation according to the emotional state. This enables even physically challenged workers to efficiently operate factory robots, realizing safe and efficient robot operation according to the worker's emotional state. Furthermore, smooth operation is possible even in a multitasking environment using multiple interfaces.

[0636] An "EEG signal" is a signal that reflects the electrical activity of the brain and indicates the state of the brain and thoughts.

[0637] A "digital signal" is a signal that has been converted from an analog signal into a digital format that can be processed by a computer or other electronic device.

[0638] "Generative AI" is a type of artificial intelligence that is a technology that understands the user's intentions and thoughts and generates appropriate responses and operations.

[0639] "User intent" describes the action the user wants to take or the goal they want to achieve.

[0640] "Emotional state" refers to a user's current feelings or mood, including states such as stress, joy, and anger.

[0641] "Means for adjusting operation" refers to techniques for changing the system's behavior and response depending on the user's emotional state.

[0642] "Physical Constraints" means any physical limitation or disability a User has, including any limitation or disability that makes normal operation difficult.

[0643] "Virtual reality" refers to virtual environments and experiences generated using computer technology.

[0644] "Augmented reality" is a technology that displays digital information overlaid on the real environment.

[0645] A "multitasking environment" is an environment in which multiple tasks or operations are required to be performed simultaneously.

[0646] A system for implementing this invention converts EEG signals into digital signals and uses the digital signals to operate a factory robot. The system uses generative AI to understand the user's intentions and interpret the EEG signals. It also includes a function to detect the user's emotional state and adjust its operation accordingly.

[0647] Hardware Configuration

[0648] The system uses the following hardware:

[0649] EEG capture device: A device that captures the user's brainwave signals.

[0650] Head-mounted display (HMD): A device worn by the user that provides visual information.

[0651] Factory robot: A robot that operates based on user instructions.

[0652] Software Configuration

[0653] The system uses the following software:

[0654] Python: A programming language used to implement programs.

[0655] scikit-learn: A library used to implement generative AI models.

[0656] EmotionEngine: Software for detecting the user's emotional state.

[0657] RobotController: Software for operating factory robots.

[0658] Data processing and calculation

[0659] 1. Capturing EEG signals: The EEG capture device captures the user's EEG signals.

[0660] 2. Conversion to digital signals: The captured EEG signals are converted into digital signals.

[0661] 3. Interpreting intent using a generative AI model: Digital signals are input into a generative AI model to interpret the user's intent.

[0662] 4. Emotional state detection: EmotionEngine detects the user's emotional state.

[0663] 5. Robot Control: The RobotController controls factory robots based on the user's intentions and emotional state.

[0664] Specific examples

[0665] As a concrete example, consider the case where a factory worker thinks, "I want to lift a part." This thought is captured as an EEG signal and converted into a digital signal. The generative AI model interprets the intention to "lift a part," and the robot carries out that command. If the worker is feeling stressed, the emotion engine detects this state and takes appropriate action, such as adjusting the robot's speed.

[0666] Prompt Sentence Examples

[0667] When the user thinks about "lifting a part," the brainwave signal is converted into a digital signal, and the robot performs the action of lifting the part.

[0668] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0669] Step 1:

[0670] The user wears an EEG capture device and thinks a specific thought. The EEG capture device captures the user's EEG signals. The captured EEG signals become input data.

[0671] Step 2:

[0672] The server receives the captured EEG signals and converts them into digital signals. Specifically, it processes the data to convert the analog EEG signals into digital format. This converted digital signal becomes the output data.

[0673] Step 3:

[0674] The server uses a generative AI model to interpret the user's intention from the digital signal. The input data is a digital signal, which the generative AI model analyzes to predict the user's intention. The output data is information that indicates the user's intention.

[0675] Step 4:

[0676] The server uses the Emotion Engine to detect the user's emotional state. The input data is the user's biosignals and behavioral data, which the Emotion Engine analyzes to identify the user's emotional state. The output data is information indicating the user's emotional state.

[0677] Step 5:

[0678] The server uses RobotController to operate factory robots based on the user's intentions and emotional state. The input data is information indicating the user's intentions and emotional state, and RobotController determines the robot's behavior based on this. The output data is specific operating instructions for the robot.

[0679] Step 6:

[0680] The factory robot receives operation instructions from the server and actually performs the operation. Specific operations include lifting, moving, and assembling parts, thereby realizing the user's intentions.

[0681] Example 3

[0682] Next, a description will be given of a third embodiment of the third embodiment. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0683] Conventional information processing systems require users to operate them manually, which is particularly difficult for users who are driving or have physical limitations. Furthermore, the interface does not adjust according to the user's emotions, resulting in a poor user experience. To address these issues, there is a need for a system that uses the user's brainwave signals to operate information processing devices and further adjusts the interface according to the user's emotions.

[0684] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[0685] In this invention, the server includes means for converting EEG signals into digital signals, means for operating an information processing device using the digital signals, means for understanding a user's intentions and interpreting EEG signals using generative artificial intelligence, and means for adjusting an interface according to the user's emotions using an emotion engine. This allows the user to operate the information processing device using EEG signals without manual operation, and further, the interface is adjusted according to the user's emotions, improving the user experience.

[0686] "Electroencephalogram signals" are signals obtained by detecting electrical activity in the user's brain.

[0687] A "digital signal" is a signal that is an analog signal converted into digital form.

[0688] An "information processing device" is a device that receives digital signals and performs specific operations or processes.

[0689] "Generative AI" is an AI technology used to understand user intent and interpret brainwave signals.

[0690] An "emotion engine" is a technology that detects a user's emotions and adjusts the interface accordingly.

[0691] An "interface" is a means for exchanging information between a user and an information processing device.

[0692] "User intent" is what a user wants to do or do.

[0693] A "brainwave sensor" is a device for detecting a user's brainwave signals.

[0694] "Real-time" refers to processing or operations being performed immediately without delay.

[0695] A "thought pattern" is a characteristic pattern of brain wave signals when a user performs a specific operation or action.

[0696] MODE FOR CARRYING OUT THE INVENTION

[0697] The present invention is a system that utilizes a user's electroencephalogram signals to operate an information processing device and further adjusts the interface in accordance with the user's emotions. A specific embodiment of this system will be described below.

[0698] Hardware and software used

[0699] Hardware: EEG sensors, information processing devices (e.g. smartphones, navigation systems)

[0700] Software: EEG signal analysis software, generative artificial intelligence, emotion engine

[0701] System Overview

[0702] When a user wears an EEG sensor and thinks a specific thought, the EEG sensor picks up the EEG signal. This EEG signal is converted into a digital signal and sent to a server. The server uses generative artificial intelligence to analyze the digital signal and understand the user's intention. The information processing device is operated based on the analysis results. An emotion engine is also used to detect the user's emotions and adjust the interface.

[0703] Specific examples

[0704] Example 1: A user is driving a car and thinks, "I want to go home."

[0705] The user wears a brainwave sensor and thinks, "I want to go home."

[0706] The EEG sensor picks up the EEG signal and converts it into a digital signal.

[0707] The server analyzes the digital signal and sets "home" as the destination in the navigation system.

[0708] The navigation system sets the destination to "home" and begins route guidance.

[0709] Example 2: When a user feels sad

[0710] The server detects the user's emotions using an emotion engine.

[0711] The smartphone receives instructions from the server and automatically lowers the volume.

[0712] Prompt Sentence Examples

[0713] "If a user thinks about a destination while driving a car, explain how you can capture their brainwave signals and operate the navigation system."

[0714] "Describe the behavior of a system that adjusts its interface based on the user's emotions."

[0715] This system enables the user to operate the information processing device using EEG signals without manual operation, and furthermore, the interface is adjusted according to the user's emotions, improving the user experience. The flow of the identification process in Example 3 will be described with reference to Fig. 21.

[0716] Step 1:

[0717] The user wears a brainwave sensor and thinks specific thoughts.

[0718] Input: User thoughts

[0719] Output: EEG signal

[0720] Specific action: The user wears a brainwave sensor on their head and thinks a specific thought, such as "I want to go home."

[0721] Step 2:

[0722] The terminal (brainwave sensor) acquires the user's brainwave signals and converts them into digital signals.

[0723] Input: EEG signal

[0724] Output: Digital signal

[0725] How it works: The signal processing unit inside the EEG sensor converts the analog EEG signal into a digital signal.

[0726] Step 3:

[0727] The server receives the digital signal and analyzes it using a generative AI model.

[0728] Input: Digital signal

[0729] Output: User intent (e.g., destination)

[0730] What it does: The server analyzes the digital signals and runs a generative AI model to understand the user's intent.

[0731] Step 4:

[0732] The server generates an operation instruction for the information processing device based on the analysis result.

[0733] Input: User intent

[0734] Output: Operation instructions

[0735] Specific operation: The server generates an instruction to the navigation system to set "home" as the destination.

[0736] Step 5:

[0737] The terminal (navigation system) receives instructions from the server and sets the destination.

[0738] Input: Operation instructions

[0739] Output: Destination setting

[0740] Specific operation: The navigation system sets the destination to "Home" and begins route guidance.

[0741] Step 6:

[0742] The server uses a generative AI model to learn the user's thought patterns.

[0743] Input: Historical data

[0744] Output: Training results

[0745] Specific operation: The server accumulates past data and improves the accuracy of analysis from the next time onwards.

[0746] Step 7:

[0747] The server detects the user's emotions using an emotion engine.

[0748] Input: User emotion data

[0749] Output: Emotion detection results

[0750] Specific operation: The server analyzes the user's emotions and executes the emotion engine.

[0751] Step 8:

[0752] The device (smartphone) receives instructions from the server and adjusts the interface.

[0753] Input: Emotion detection result

[0754] Output: Interface adjustments

[0755] Specific behavior: The smartphone automatically adjusts the interface, such as automatically lowering the volume.

[0756] (Application example 3)

[0757] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0758] Conventional navigation systems and smart devices require manual input from the user, making them difficult to operate, especially for those driving or with physical limitations. Furthermore, the interface does not adjust according to the user's emotional state, resulting in a lack of improvement in the user experience. To address these issues, there is a need for a system that uses EEG signals to understand the user's intentions and emotions and automates device operation and interface adjustments.

[0759] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[0760] In this invention, the server includes means for converting EEG signals into digital signals, means for operating a smart device using the digital signals, means for understanding a user's intentions and interpreting EEG signals using generative AI, and means for detecting the user's emotional state and adjusting the interface. This enables the user to set a destination in a navigation system or operate a smart device simply by thinking, and also enables the interface to be adjusted according to the user's emotional state.

[0761] "Electroencephalogram signals" are electrical signals generated in association with brain activity.

[0762] A "digital signal" is a signal that is generated by converting an analog signal into a numerical value and expressing it in digital form.

[0763] "Smart device" is a general term for electronic devices that have internet connectivity and advanced computing capabilities.

[0764] "Generative AI" is an artificial intelligence technology that understands a user's intentions and emotions and generates appropriate responses.

[0765] An "interface" is a point of contact or means by which a user and a system exchange information with each other.

[0766] "Emotional state" refers to the emotional state a user is feeling at a particular moment.

[0767] A "navigation system" is a system that provides guidance on routes to a destination.

[0768] "Thoughts" refer to what the user thinks in their head.

[0769] "Physical limitations" refers to a condition in which a user has difficulty performing a particular action or operation due to physical reasons.

[0770] "Virtual reality" is a virtual environment created using computer technology.

[0771] "Augmented reality" is a technology that displays digital information overlaid on the real environment.

[0772] "Multitasking" refers to performing multiple tasks simultaneously.

[0773] A system for carrying out this invention converts EEG signals into digital signals, understands the user's intentions and emotions, and operates a smart device. Specific embodiments are described below.

[0774] System Configuration

[0775] The system consists of the following main components:

[0776] 1. EEG signal receiver: Smart glasses with built-in sensors that capture the user's brainwave signals.

[0777] 2. Digital signal converter: A device that converts EEG signals into digital signals.

[0778] 3. Generative AI model: A machine learning model for understanding user intent and emotions and interpreting EEG signals.

[0779] 4. Navigation system: A system for setting destinations.

[0780] 5. Audio system: A system that plays music depending on the user's emotional state.

[0781] Program processing

[0782] The server receives EEG signals from the EEG receiver and converts them into digital signals using a digital signal converter. It then analyzes the user's intentions and emotions using a generative AI model. Based on the analysis results, it operates the navigation system and audio system.

[0783] Hardware and software used

[0784] Brainwave signal receiving device: smart glasses

[0785] Digital signal converter: Dedicated converter

[0786] Generative AI model: Machine learning model (e.g., MLPClassifier)

[0787] Navigation system: Navigation system installed in the vehicle

[0788] Audio system: The audio system installed in the vehicle

[0789] Specific examples

[0790] When a user wears the smart glasses and thinks, "I want to go home," the EEG signal receiver captures the EEG signal. The digital signal converter converts this EEG signal into a digital signal, and the generative AI model analyzes the user's intention. Based on the analysis results, the navigation system automatically sets home as the destination. If the user is feeling stressed, the audio system will play relaxing music.

[0791] Prompt Sentence Examples

[0792] "All a user has to do is put on the smart glasses and think about their destination, and the navigation system will automatically set it. It can also detect the user's emotional state and play relaxing music if they are feeling stressed."

[0793] The above is a specific embodiment for carrying out the invention.

[0794] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[0795] Step 1:

[0796] The user puts on the smart glasses and thinks about a destination.

[0797] Input: User's EEG signal

[0798] Output: EEG signal data

[0799] How it works: Sensors built into the smart glasses capture the user's brainwave signals.

[0800] Step 2:

[0801] The electroencephalogram signal receiving device acquires the electroencephalogram signal.

[0802] Input: EEG signal data

[0803] Output: Raw EEG signal data

[0804] Specific operation: The EEG signal receiving device transmits the captured EEG signal to the digital signal conversion device.

[0805] Step 3:

[0806] A digital signal conversion device converts the electroencephalogram signal into a digital signal.

[0807] Input: Raw EEG signal data

[0808] Output: Digital signal data

[0809] Specific operation: The digital signal converter analyzes the EEG signal and converts it into digital form.

[0810] Step 4:

[0811] The server uses a generative AI model to analyze the digital signals and understand the user's intent.

[0812] Input: Digital signal data

[0813] Output: User intent (destination information)

[0814] Specific operation: The server inputs digital signal data into a generative AI model to predict the user's intentions.

[0815] Step 5:

[0816] The server transmits the destination information to the navigation system.

[0817] Input: User intent (destination information)

[0818] Output: Destination setting for navigation system

[0819] Specific operation: The server sends the analysis results to the navigation system, which automatically sets the destination.

[0820] Step 6:

[0821] The server uses a generative AI model to analyze the user's emotional state.

[0822] Input: Digital signal data

[0823] Output: User's emotional state

[0824] How it works: The server inputs digital signal data into a generative AI model to predict the user's emotional state.

[0825] Step 7:

[0826] The server sends instructions to the audio system according to the emotional state.

[0827] Input: User's emotional state

[0828] Output: Audio system operating instructions

[0829] Specific behavior: The server instructs the audio system to play relaxing music based on the user's emotional state.

[0830] The above are the specific processing steps of the system that realizes the application example.

[0831] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires a voice indicating a user input for the result of the specific processing. The control unit 46A converts the voice data indicating the user input acquired by the microphone 38B into

[0832] The audio data is transmitted to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0833] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0834] Another example of generative AI is Gemini (registered trademark) (Internet search engine). <url: https: gemini.google.com ?hl="ja">) are listed.

[0835] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0836] [Second embodiment]

[0837] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0838] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0839] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0840] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0841] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0842] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0843] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0844] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0845] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0846] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0847] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0848] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0849] "Example 1"

[0850] One embodiment of the present invention is a system that includes a device that converts EEG signals into digital signals, a device that uses the digital signals to operate a smartphone, and a device that uses generative AI to understand a user's intentions and interpret the EEG signals. Specifically, when a user engages in a specific thought, an EEG signal is generated, and a device that converts this into a digital signal captures the EEG signal. Next, a device that uses the digital signal to operate the smartphone analyzes the digital signal and executes the corresponding smartphone operation. Furthermore, the device that uses generative AI to understand a user's intentions and interprets the EEG signals learns the user's thought patterns, enabling more accurate operation.

[0851] "Example 2"

[0852] As a concrete example, consider a user with physical limitations operating a smartphone. When the user thinks about tapping the smartphone screen, an EEG signal is generated and converted into a digital signal. A device that uses this digital signal to operate the smartphone analyzes the digital signal and performs the tap operation on the smartphone screen. Furthermore, the device uses generative AI to understand the user's intentions and interprets the EEG signal, learning the user's thought patterns and allowing them to perform more accurate screen tap operations with the same thought the next time.

[0853] "Example 3"

[0854] Similar operations are also possible when multitasking with interfaces such as VR, AR, and vehicles. For example, when operating a navigation system while driving a car, if the user thinks about a destination, an EEG signal is generated and converted into a digital signal. A device that uses this digital signal to operate the navigation system analyzes the digital signal and executes the destination setting operation on the navigation system. Furthermore, the device, which uses generative AI to understand the user's intentions and interprets the EEG signal, can learn the user's thought patterns and execute a more accurate destination setting operation with the same thought next time.

[0855] The processing flow of each embodiment will be described below.

[0856] "Example 1"

[0857] Step 1: The user engages in a specific thought.

[0858] Step 2: EEG signals are generated and captured by a device that converts the EEG signals into digital signals.

[0859] Step 3: The device that uses the digital signals to operate the smartphone analyzes the digital signals.

[0860] Step 4: Based on the analysis results, perform the corresponding smartphone operation.

[0861] Step 5: Using generative AI to understand user intent and interpret EEG signals, the device learns the user's thought patterns.

[0862] "Example 2"

[0863] Step 1: A user with physical limitations thinks about tapping on a smartphone screen.

[0864] Step 2: An EEG signal is generated and converted into a digital signal.

[0865] Step 3: The device that uses the digital signals to operate the smartphone analyzes the digital signals.

[0866] Step 4: Based on the analysis results, perform a tap operation on the smartphone screen.

[0867] Step 5: Using generative AI to understand user intent and interpret EEG signals, the device learns the user's thought patterns.

[0868] "Example 3"

[0869] Step 1: To operate the navigation system while driving, the user thinks about a destination.

[0870] Step 2: An EEG signal is generated and converted into a digital signal.

[0871] Step 3: A device that utilizes the digital signal to operate the navigation system analyzes the digital signal.

[0872] Step 4: Based on the analysis results, the navigation system sets the destination. Step 5: Using generative AI to understand the user's intentions and interpret brainwave signals, the device learns the user's thought patterns.

[0873] Example 1

[0874] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0875] Conventional methods for operating information processing devices require users to operate them manually, making operation particularly difficult for users with physical limitations. Efficient operation is also required for multitasking in interfaces such as virtual reality, augmented reality, and vehicles. Furthermore, it has been difficult to accurately understand the user's intentions and quickly reflect them.

[0876] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means. In this invention, the server includes means for converting an EEG signal into a digital signal, means for operating the information processing device using the digital signal, means for understanding the user's intention and interpreting the EEG signal by making full use of generative artificial intelligence, means for transmitting the digital signal using wireless communication technology, and means for analyzing the digital signal. This allows the user's thoughts to be directly linked to the operation of the information processing device, enabling efficient multitasking for users with physical limitations and in interfaces such as virtual reality, augmented reality, and vehicles.

[0877] "Electroencephalogram signals" are electrical signals generated by neural activity in the brain.

[0878] A "digital signal" is a signal that has been converted from an analog signal into a digital form that can be processed by a computer or electronic device.

[0879] An "information processing device" is a device that inputs, processes, and outputs data, and includes computers, smartphones, etc.

[0880] "Generative AI" is an AI technology that understands the user's intentions and performs appropriate processing based on the generated data.

[0881] "Wireless communication technology" refers to technology that transmits and receives data using radio waves without using cables, and includes Wi-Fi and Bluetooth.

[0882] "Analysis" is the process of examining data in detail to understand its structure and meaning.

[0883] A "thought" is a mental activity consciously performed by the user, with a specific purpose or intention.

[0884] "Virtual reality" is a technology that immerses users in a virtual environment generated using computer technology.

[0885] "Augmented reality" is a technology that displays computer-generated information overlaid on the real environment.

[0886] "Vehicle" means a vehicle for travelling on land, sea or air, including automobiles, ships, aircraft, etc.

[0887] "Multitasking" is the ability or process of performing multiple tasks simultaneously.

[0888] The present invention relates to a system for operating an information processing device using a user's electroencephalogram signal. Specific embodiments of this system will be described below.

[0889] Hardware and Software Configuration

[0890] Brain wave capture device

[0891] The user wears an EEG capture device, which has the function of capturing EEG signals and converting them into digital signals. Specifically, a typical EEG capture device is a headset with a built-in EEG sensor.

[0892] Terminal

[0893] The terminal receives digital signals from the EEG capture device and transmits them to the server. The terminal is equipped with wireless communication technology (such as Wi-Fi or Bluetooth).

[0894] server

[0895] The server receives and analyzes the digital signals sent from the device. It uses the Python libraries "MNE-Python" and "SciPy" for analysis. It also uses generative artificial intelligence (e.g., a generative AI model) to understand the user's intentions and interpret the EEG signals. This AI model uses machine learning frameworks such as TensorFlow and PyTorch to learn the user's thought patterns and enable more accurate operation.

[0896] Specific examples

[0897] Example 1: Sending a message

[0898] The user thinks, "I want to send a message." The EEG capture device captures the EEG signal corresponding to this thought and converts it into a digital signal. The device then sends this digital signal to a server via Wi-Fi. The server uses MNE-Python to analyze the digital signal and recognizes the intention to "send a message." The server then uses ADB (Android Debug Bridge) to launch the messaging app on the Android smartphone and send the message to the specified contacts.

[0899] Example 2: Launching an app

[0900] The user thinks, "I want to launch the camera app." The EEG capture device captures the EEG signal corresponding to this thought and converts it into a digital signal. The device then sends this digital signal to a server via Bluetooth. The server uses SciPy to analyze the digital signal and recognizes the intention to "launch the camera app." The server then uses XCUITest to launch the camera app on the iOS smartphone.

[0901] Prompt Sentence Examples

[0902] "Please describe a system that captures the EEG signals generated when a user engages in specific thoughts, converts them into digital signals, and operates an information processing device. Please also specify the names of any specific hardware and software."

[0903] This system provides a more intuitive and efficient interface by directly reflecting the user's thoughts in the operation of the information processing device.

[0904] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0905] Step 1:

[0906] When a user has a specific thought, the EEG capture device captures the EEG signal. The input is the user's thought, and the output is the captured EEG signal. In concrete terms, when a user thinks, "I want to send a message," the EEG capture device detects the EEG signal.

[0907] Step 2:

[0908] The terminal converts the captured EEG signal into a digital signal. The input is the captured EEG signal, and the output is a digital signal. Specifically, the built-in software of the EEG capture device converts the analog signal into a digital signal.

[0909] Step 3:

[0910] The terminal sends the converted digital signal to the server. The input is a digital signal, and the output is the digital signal sent to the server. Specifically, the terminal sends the digital signal to the server using Wi-Fi or Bluetooth.

[0911] Step 4:

[0912] The server analyzes the received digital signal. The input is the digital signal sent to the server, and the output is the analysis result. Specifically, the server uses the Python libraries "MNE-Python" and "SciPy" to extract the characteristics of the digital signal and determine whether it is intended to "send a message."

[0913] Step 5:

[0914] The server executes operations on the information processing device based on the analysis results. The input is the analysis results, and the output is the executed operations. Specifically, the server uses ADB (Android Debug Bridge) to launch the Android smartphone's messaging app and send a message to the specified contacts.

[0915] Step 6:

[0916] The server uses generative AI to understand the user's intentions and interpret the EEG signals. The input is the digital signal and past thought patterns, and the output is the interpretation of the user's intentions. Specifically, the server uses a generative AI model to learn the user's thought patterns and respond quickly to similar thoughts in the future.

[0917] In this way, a system is realized in which the user's thoughts are directly linked to the operation of an information processing device.

[0918] (Application example 1)

[0919] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0920] Conventional methods for operating autonomous vehicles rely on physical interfaces, making them difficult for users with physical limitations. Efficient operation is also difficult in multitasking environments where multiple interfaces must be operated simultaneously. Furthermore, the lack of technology to accurately reflect user intentions can lead to reduced operation accuracy.

[0921] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0922] In this invention, the server includes means for converting EEG signals into digital signals, means for operating an information processing device using the digital signals, means for understanding a user's intentions by making full use of generative AI and interpreting the EEG signals, and means for operating an autonomous vehicle using the user's thoughts. This makes it possible to operate an autonomous vehicle using the user's thoughts directly, enabling efficient and accurate operation even for users with physical limitations or in a multitasking environment.

[0923] An "EEG signal" is an electrical signal that reflects the electrical activity of the brain.

[0924] A "digital signal" is a signal that has been converted from an analog signal into a digital format that can be processed by a computer or other electronic device.

[0925] An "information processing device" is a device that has the function of inputting, processing, and outputting data, and specifically includes computers and smartphones.

[0926] "Generative AI" is a type of artificial intelligence that understands the user's intentions and performs appropriate responses and operations based on generated data.

[0927] "User intent" refers to the thoughts or goals that a user has when they want to perform a particular operation or action.

[0928] An "autonomous vehicle" is a vehicle that uses artificial intelligence and sensor technology to drive autonomously.

[0929] "Thinking" is an information processing activity that takes place in the brain and includes conscious judgments and decisions.

[0930] "Physical constraints" refers to physical disabilities or limitations that a user has that make it difficult to perform certain actions or operations.

[0931] "Virtual reality" refers to a computer-generated virtual environment or experience in which users can interact.

[0932] "Augmented reality" is a technology that overlays digital information onto a real environment, allowing users to experience the real world and digital information simultaneously.

[0933] "Multitasking" refers to performing multiple tasks or operations simultaneously, with the aim of completing tasks efficiently.

[0934] A system for implementing this invention includes a device that converts EEG signals into digital signals, a device that uses the digital signals to operate an information processing device, a device that uses generative AI to understand a user's intentions and interpret EEG signals, and a device that uses the user's thoughts to operate an autonomous vehicle.

[0935] First, the user wears a device to capture EEG signals. This device captures EEG signals in real time and converts them into digital signals. The converted digital signals are sent to an information processing device. The information processing device analyzes the digital signals using a generative AI model to understand the user's intentions. The generative AI model is trained in advance using training data and can interpret the user's thought patterns with high accuracy.

[0936] The autonomous vehicle is then controlled based on the interpreted user intent. For example, if the user thinks, "Turn right," their brainwave signal is captured and converted into a digital signal. This digital signal is analyzed and an instruction to steer the vehicle to the right is issued. This allows the user to control the autonomous vehicle with just their thoughts, without having to perform any physical operation.

[0937] The following hardware and software are used to realize this system: An EEG sensor is used as the device for capturing EEG signals. A Fourier transform algorithm is used as the software for converting them into digital signals. An MLP Classifier (Multilayer Perceptron Classifier), for example, is used as the generative AI model. A computer or smartphone is used as the information processing device. A vehicle control system is used to operate the autonomous vehicle.

[0938] As a concrete example, the following prompt sentence is shown.

[0939] "When a driver thinks 'turn right,' their brainwave signals are captured and converted into digital signals. This digital signal is then analyzed and an instruction is issued to steer the vehicle to the right."

[0940] In this way, it becomes possible to directly utilize the user's thoughts to operate an autonomous vehicle, enabling efficient and precise operation even for users with physical limitations or in multitasking environments.

[0941] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0942] Step 1:

[0943] The user wears an EEG sensor.

[0944] Input: User's EEG signal

[0945] Output: Captured EEG signal

[0946] How it works: The user wears an EEG sensor on their head to capture EEG signals in real time. The sensor detects the brain's electrical activity and outputs it as an analog signal.

[0947] Step 2:

[0948] The server converts the EEG signal into a digital signal.

[0949] Input: Captured EEG signal (analog signal)

[0950] Output: Digital signal

[0951] What it does: The server uses a Fourier transform algorithm to convert the analog signal into a digital signal, putting the EEG signal in a format that can be processed by a computer.

[0952] Step 3:

[0953] The server uses a generative AI model to analyze the digital signals and understand the user's intent.

[0954] Input: Digital signal

[0955] Output: User intent (e.g., "Turn right")

[0956] How it works: The server analyzes digital signals using a pre-trained MLPClassifier model. The model learns the user's thought patterns and can accurately predict the user's intentions from the input digital signals.

[0957] Step 4:

[0958] The server generates operating instructions for the autonomous vehicle based on the user's intentions.

[0959] Input: User intent (e.g., "turn right")

[0960] Output: Vehicle operation instructions (e.g., steer right)

[0961] Specific operation: The server receives the user's intention and generates a corresponding vehicle operation instruction. For example, if the intention is "turn right," an instruction to turn the steering wheel to the right is generated.

[0962] Step 5:

[0963] The terminal transmits vehicle operation instructions to the autonomous vehicle.

[0964] Input: Vehicle operation instructions

[0965] Output: Vehicle movement (e.g. steering wheel turns right)

[0966] Specific operation: The terminal sends the generated vehicle operation instructions to the autonomous vehicle control system, which then performs actual vehicle operation based on the received instructions.

[0967] Step 6:

[0968] Autonomous vehicles operate based on the user's intentions.

[0969] Input: Vehicle operation instructions

[0970] Output: Vehicle action (e.g. turn right)

[0971] Specific operation: The autonomous vehicle will follow the received operation instructions and perform operations such as turning the steering wheel to the right. This allows the user to operate the vehicle using only their thoughts.

[0972] Example 2

[0973] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0974] Users with physical limitations often have difficulty intuitively and efficiently operating information processing devices. Conventional interfaces also have the problem of making it difficult to accurately understand the user's intentions and execute appropriate operations. Furthermore, there is a lack of mechanisms for learning the user's thought patterns and improving the accuracy of subsequent operations.

[0975] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0976] In this invention, the server includes means for converting EEG signals into digital signals, means for operating an information processing device using the digital signals, means for understanding a user's intentions by utilizing a generative AI model and interpreting the EEG signals, means for analyzing the digital signals and generating operation commands, means for transmitting the operation commands to the information processing device, means for the information processing device to perform operations in accordance with the operation commands, and means for learning the user's thought patterns, thereby enabling users with physical limitations to intuitively and efficiently operate the information processing device using EEG signals.

[0977] "EEG signals" are signals that reflect the electrical activity of the user's brain.

[0978] A "digital signal" is a signal that is an analog signal converted into digital form.

[0979] An "information processing device" is a device that receives and analyzes digital signals and performs operations based on the user's intentions.

[0980] A "generative AI model" is a model that uses artificial intelligence technology to understand user intent and interpret brainwave signals.

[0981] An "operation command" is an instruction to cause an information processing device to perform a specific operation.

[0982] A "thought pattern" is a characteristic pattern of brain wave signals that occurs when a user performs a specific operation.

[0983] "Wireless communication" is a means of sending and receiving data using wireless technologies such as Wi-Fi and Bluetooth.

[0984] "Analysis" is the process of interpreting the user's intent based on the received digital signal.

[0985] "Learning" is the process of recording the user's past brainwave signals and corresponding operations to improve the accuracy of future operations.

[0986] MODE FOR CARRYING OUT THE INVENTION

[0987] This invention is a system that enables users with physical limitations to intuitively and efficiently operate information processing devices. The system includes a series of processes that convert EEG signals into digital signals, understand the user's intentions using a generative AI model, and operate the information processing device.

[0988] Hardware and software used

[0989] Hardware:

[0990] 1. EEG measurement device: A device for acquiring the user's brainwave signals. A specific example is the EEG measurement device "Epoc+."

[0991] 2. Information processing device: A device that receives and analyzes digital signals and performs actions based on the user's intentions. A good example is a smartphone.

[0992] 3. Server: A device for analyzing digital signals and generating operational commands.

[0993] software:

[0994] 1. Generative AI model: An artificial intelligence model that understands user intent and interprets EEG signals. An example of a generative AI model is the GPT-4.

[0995] Data processing and calculation

[0996] Acquisition and transformation of EEG signals:

[0997] When a user wears an EEG measuring device and thinks about a specific operation, the device acquires an EEG signal, which is then converted into a digital signal by a signal processing unit within the device.

[0998] Digital signal transmission and analysis:

[0999] The device (EEG measurement device) transmits the converted digital signal to a server via wireless communication (Wi-Fi or Bluetooth), which then analyzes the digital signal using a generative AI model to interpret the user's intent.

[1000] Generate and send operation commands:

[1001] The server generates an operation command for the information processing device based on the analysis result, and transmits the generated operation command to the information processing device via wireless communication.

[1002] Execution of operations by information processing device:

[1003] The information processing device executes an operation in accordance with the received operation command. For example, when a smartphone receives a command to tap the "home" button, the smartphone taps the "home" button.

[1004] Learning Thought Patterns:

[1005] The server records the user's EEG signals and the corresponding actions, and uses a generative AI model to learn the user's thought patterns, allowing for more accurate actions the next time the same thought is used.

[1006] Specific examples

[1007] As a concrete example, consider the case where a user thinks about tapping the "Home" button on a smartphone. When the user thinks about tapping the "Home" button, an EEG measurement device captures the EEG signal and converts it into a digital signal. This digital signal is sent to a server, which uses a generative AI model to analyze the user's intention. Based on the analysis results, the server generates a command to tap the "Home" button and sends it to the smartphone. The smartphone then taps the "Home" button according to the command.

[1008] Example prompt sentence:

[1009] "When a user thinks about tapping the home button on their smartphone, please generate a program that analyzes the brainwave signal and sends a command to the smartphone to tap the home button."

[1010] In this way, a physically limited user can intuitively and efficiently operate an information processing device using an electroencephalogram signal.

[1011] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1012] Step 1:

[1013] A user wears an electroencephalogram (EEG) measuring device and thinks about a specific operation, and the EEG measuring device acquires an electroencephalogram signal from the user.

[1014] Input: User's EEG signal

[1015] Output: Captured EEG signal

[1016] Step 2:

[1017] The terminal (EEG measurement device) converts the acquired EEG signal into a digital signal. This conversion process is performed by a signal processing unit within the EEG measurement device.

[1018] Input: Acquired EEG signal

[1019] Output: Digital signal

[1020] Step 3:

[1021] The terminal transmits the converted digital signal to the server via wireless communication (Wi-Fi or Bluetooth).

[1022] Input: Digital signal

[1023] Output: Digital signal sent to the server

[1024] Step 4:

[1025] The server uses a generative AI model to analyze the received digital signals and interpret the user's intentions from their brainwave signals.

[1026] Input: Digital signal sent to the server

[1027] Output: Analysis of user intent

[1028] Step 5:

[1029] The server generates an operation command for the information processing device based on the analysis result. For example, if the user thinks about tapping the "Home" button, the server generates a command to tap the "Home" button.

[1030] Input: Analysis results regarding user intent

[1031] Output: Operation command

[1032] Step 6:

[1033] The server transmits the generated operation command to the information processing device via wireless communication.

[1034] Input: Operation command

[1035] Output: Operation commands sent to the information processing device

[1036] Step 7:

[1037] The information processing device executes an operation in accordance with the received operation command. For example, when a command to tap the "home" button is received, the information processing device taps the "home" button.

[1038] Input: Operation command sent to the information processing device

[1039] Output: Actions taken

[1040] Step 8:

[1041] The server records the user's EEG signals and the corresponding actions, and uses a generative AI model to learn the user's thought patterns, allowing them to perform more accurate actions with the same thoughts next time.

[1042] Input: Records of the user's EEG signals and corresponding actions

[1043] Output: Learned thought patterns

[1044] (Application example 2)

[1045] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1046] Conventional factory robots have been difficult for workers with physical limitations to operate, resulting in inefficient work. Furthermore, the complexity of operation has been an issue even when performing multitasking using multiple interfaces. There is a need to provide a system that can solve these problems and enable workers with physical limitations to operate factory robots efficiently.

[1047] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1048] In this invention, the server includes means for converting EEG signals into digital signals, means for operating a smartphone using the digital signals, means for understanding a user's intentions using generative AI and interpreting the EEG signals, and means for operating a factory robot using the interpreted EEG signals, thereby enabling even physically limited workers to efficiently operate factory robots using EEG signals.

[1049] "EEG signals" are electrical signals generated based on the user's brain activity.

[1050] A "digital signal" is a signal that has been converted from an analog signal into a digital format that can be processed by a computer or other electronic device.

[1051] A "smartphone" is a multi-functional mobile device that can connect to the Internet and run applications in addition to the functions of a mobile phone.

[1052] "Generative AI" is a type of artificial intelligence technology that understands user intent and generates models to perform specific tasks.

[1053] "User intent" refers to the thoughts or goals that a user has when they want to perform a particular operation or action.

[1054] "Means for interpretation" refers to a method or device for analyzing acquired signals or data and understanding their meaning or intent.

[1055] A "factory robot" is a mechanical device used to automate work within a factory, and can manage production lines and perform maintenance work.

[1056] "Means for operating" refers to a method or device for controlling a specific device or system to perform a desired operation.

[1057] "Physical limitations" refers to a condition in which a user has difficulty performing a particular action or operation due to physical reasons.

[1058] "Virtual reality" refers to a virtual environment or experience generated using computer technology in which users can interact.

[1059] "Augmented reality" is a technology that overlays digital information onto a real environment, allowing users to experience the real world and digital information simultaneously.

[1060] A "vehicle" is a means of transportation for transporting people or goods, and includes automobiles, trains, etc.

[1061] "Multitasking" refers to the simultaneous execution of multiple tasks, and is a state in which a user can perform multiple operations or tasks in parallel.

[1062] The system for implementing this invention converts EEG signals into digital signals, interprets the user's intentions using generative AI, and operates a factory robot. A specific embodiment of this system is described below.

[1063] First, the user wears an EEG sensor, which captures the user's brainwave signals in real time and converts them into digital signals, which are then sent to a server.

[1064] The server performs data preprocessing to standardize the acquired digital signals. Specifically, it standardizes the signals using libraries such as StandardScaler. Next, it uses a generative AI model to analyze the standardized signals and interpret the user's intentions. This generative AI model is pre-trained and can generate specific operation commands from the user's EEG signals.

[1065] The interpreted operation command is sent to the factory robot via serial communication. The factory robot performs the specified action based on the received command. For example, if a user thinks, "Lift a part," the generative AI model interprets this intention and generates the command "Lift." The factory robot receives this command and actually performs the action of lifting the part.

[1066] As a concrete example of this system, the following prompt sentence can be input into the generative AI model:

[1067] "When the user thinks 'I want to lift a part,' the brainwave signal is analyzed and a 'lift' command is sent to the robot arm."

[1068] In this way, even physically challenged workers can efficiently operate factory robots using EEG signals. It can also be applied to multitasking using interfaces such as virtual reality, augmented reality, and vehicles, so it is expected to be used in a wide range of applications.

[1069] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1070] Step 1:

[1071] The user wears an EEG sensor, which captures the user's EEG signals in real time. The input is the user's EEG signal, and the output is a digital signal. The EEG sensor converts the analog signal into a digital signal.

[1072] Step 2:

[1073] The terminal transmits the acquired digital signal to the server. The input is a digital signal, and the output is data transmission to the server. The terminal transmits the digital signal to the server using wireless or wired communication.

[1074] Step 3:

[1075] The server normalizes the received digital signal. The input is a digital signal and the output is a normalized signal. The server normalizes the signal using a library such as StandardScaler.

[1076] Step 4:

[1077] The server uses a generative AI model to analyze the standardized signals and interpret the user's intentions. The input is the standardized signal, and the output is the operation command. The server uses a pre-trained generative AI model to analyze the signal and generate the operation command.

[1078] Step 5:

[1079] The server sends the generated operation command to the factory robot via serial communication. The input is the operation command and the output is the command sent to the factory robot. The server sends the operation command to the factory robot using serial communication.

[1080] Step 6:

[1081] The factory robot executes the specified operation based on the received operation command. The input is the operation command and the output is the actual operation. The factory robot performs an operation such as lifting a part according to the received command.

[1082] Example 3

[1083] Next, a description will be given of Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1084] Conventional methods for operating information processing devices require users to use their hands or voice, making operation particularly difficult for users who are driving or have physical limitations. Efficient operation is also required in multitasking environments where multiple tasks must be performed simultaneously. To solve these issues, a system is needed that directly reflects the user's thoughts in the operation of the information processing device.

[1085] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[1086] In this invention, the server includes means for converting EEG signals into digital signals, means for operating an information processing device using the digital signals, means for understanding a user's intentions by utilizing generative AI and interpreting the EEG signals, and means for learning the user's thought patterns and performing subsequent operations more accurately. This enables a user to operate an information processing device using only their thoughts, without using their hands or voice, enabling efficient operation while driving, for users with physical limitations, and even in a multitasking environment.

[1087] "Electroencephalogram signals" are signals indicative of the electrical activity of a user's brain.

[1088] A "digital signal" is a signal that is an analog signal converted into digital form.

[1089] An "information processing device" is an electronic device for processing and manipulating data.

[1090] "Generative AI" is an artificial intelligence technology that understands user intent and interprets brainwave signals.

[1091] "User intent" is the goal or desire expressed by the user through their thoughts.

[1092] A "thought pattern" is a characteristic pattern of brain wave signals when a user performs a specific purpose or operation.

[1093] "Virtual reality" is a virtual environment created using computer technology.

[1094] "Augmented reality" is a technology that overlays digital information onto the real environment.

[1095] "Multitasking" refers to performing multiple tasks simultaneously.

[1096] MODE FOR CARRYING OUT THE INVENTION

[1097] This invention relates to a system that allows a user to operate an information processing device simply by thinking. Specifically, the system converts electroencephalogram signals into digital signals, analyzes the digital signals to understand the user's intentions, and then operates the information processing device.

[1098] Hardware and software used

[1099] Hardware: EEG sensors, information processing devices (e.g., navigation systems)

[1100] Software: EEG signal analysis software, generative AI model

[1101] System Overview

[1102] 1. EEG signal acquisition:

[1103] A user wants to operate an information processing device.

[1104] The terminal (brainwave sensor) acquires the user's brainwave signals in real time.

[1105] 2. Conversion to digital signal:

[1106] The brainwave signal acquired by the terminal (brainwave sensor) is converted into a digital signal.

[1107] 3. Digital signal analysis:

[1108] The server receives the digital signal and analyzes it using EEG signal analysis software.

[1109] The server inputs the analysis results into a generative AI model to understand the user's intentions.

[1110] 4. Operation of information processing devices:

[1111] The server transmits the analysis results to the information processing device.

[1112] The terminal (information processing device) performs an operation based on the received information.

[1113] 5. Generative AI Learning:

[1114] The server uses a generative AI model to learn the user's thought patterns.

[1115] The server updates the model for future operations.

[1116] Specific examples

[1117] For example, if a user thinks, "I want to go to Tokyo Tower" while driving a car, the EEG sensor picks up the EEG signal and converts it into a digital signal. The server analyzes the digital signal and identifies the destination, "Tokyo Tower." The navigation system receives this information and sets the destination to "Tokyo Tower." The generative AI learns this series of operations, and from then on, simply by thinking, "I want to go to Tokyo Tower," the destination will be set more quickly and accurately.

[1118] Prompt Sentence Examples

[1119] "Describe a system that captures brainwave signals when a user thinks about a destination while driving a car, converts them into digital signals, and sends them to a navigation system."

[1120] This system allows the user to operate the information processing device without using their hands while driving, thereby improving safety and convenience. The flow of the identification process in the third embodiment will be described with reference to FIG.

[1121] Step 1:

[1122] A user wants to operate an information processing device.

[1123] Specific operation: The user thinks, "I want to go to Tokyo Tower."

[1124] Input: User thoughts

[1125] Output: EEG signal

[1126] Step 2:

[1127] The terminal (brainwave sensor) acquires the user's brainwave signals in real time.

[1128] What it does: The EEG sensor captures the user's brainwave signals.

[1129] Input: User's EEG signal

[1130] Output: Analog EEG signal

[1131] Step 3:

[1132] The analog brainwave signal acquired by the terminal (brainwave sensor) is converted into a digital signal.

[1133] Specific operation: The EEG sensor converts analog signals into digital signals and outputs them as data.

[1134] Input: Analog EEG signal

[1135] Output: Digital EEG signal

[1136] Step 4:

[1137] The server receives the digital EEG signals and analyzes them using EEG signal analysis software.

[1138] What it does: The server analyzes the digital signals to determine the user's intent.

[1139] Input: Digital EEG signal

[1140] Output: Analysis result (user intent)

[1141] Step 5:

[1142] The server inputs the analysis results into a generative AI model to understand the user's intentions.

[1143] Specific operation: The generative AI model interprets the user's intention based on the analysis results.

[1144] Input: Analysis result (user intent)

[1145] Output: Interpreted user intent

[1146] Step 6:

[1147] The server transmits the interpreted user's intention to the information processing device.

[1148] Specific operation: The server transmits the interpreted intention to the information processing device.

[1149] Input: Interpreted user intent

[1150] Output: Instructions to the information processing device

[1151] Step 7:

[1152] The terminal (information processing device) executes an operation based on the received instruction.

[1153] Specific operation: The information processing device performs operations such as setting a destination.

[1154] Input: Instructions to the information processing device

[1155] Output: The action taken (e.g., set destination)

[1156] Step 8:

[1157] The server uses a generative AI model to learn the user's thought patterns.

[1158] Specific operation: The generative AI model learns the user's thought patterns and performs operations more accurately from the next time onwards.

[1159] Input: The operations performed and the user's thought patterns

[1160] Output: An updated generative AI model

[1161] (Application example 3)

[1162] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1163] Conventional navigation systems require users to manually input their destination, making operation difficult, especially for those driving or those with physical limitations. Furthermore, when performing multiple tasks simultaneously, the system becomes increasingly complicated and places a strain on the user, limiting the use of navigation systems.

[1164] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[1165] In this invention, the server includes a means for converting EEG signals into digital signals, a means for using the digital signals to operate the navigation system of the autonomous vehicle, and a means for understanding the user's intentions and interpreting the EEG signals using generative AI. This allows the user to set a destination in the navigation system of the autonomous vehicle simply by thinking, making it easy to operate even while driving or for users with physical limitations. Furthermore, even when performing multiple tasks simultaneously, the complexity of operation is reduced, significantly reducing the burden on the user.

[1166] "EEG signals" are electrical signals generated in conjunction with brain activity, and reflect the user's thoughts and emotions.

[1167] A "digital signal" is a digitalized analog signal that can be processed by computers and other electronic devices.

[1168] An "autonomous vehicle" is a vehicle that uses artificial intelligence and sensor technology to drive autonomously, without the need for human operation.

[1169] A "navigation system" is a system that guides you to your destination and provides the optimal route based on map information and current location information.

[1170] "Generative AI" is artificial intelligence that understands the user's intentions and thoughts and generates appropriate responses and operations, using machine learning and natural language processing technologies.

[1171] "User intent" is the idea or goal of what the user wants or is trying to do.

[1172] "Interpreting" means analyzing input information and understanding its meaning and intent.

[1173] The following system configuration will be described as an embodiment of the present invention.

[1174] System Configuration

[1175] This system includes a means for converting EEG signals into digital signals, a means for using the digital signals to operate the navigation system of an autonomous vehicle, and a means for using generative AI to understand the user's intentions and interpret the EEG signals.

[1176] Hardware

[1177] EEG sensor: A device that captures the user's brainwave signals.

[1178] Navigation systems for autonomous vehicles: Navigation systems installed in autonomous vehicles.

[1179] software

[1180] Generative AI model: Software that converts EEG signals into digital signals and interprets user intent.

[1181] Processing flow

[1182] 1. Capturing EEG signals: The EEG sensor captures the user's EEG signals.

[1183] 2. Conversion to digital signals: The generative AI model converts the captured EEG signals into digital signals.

[1184] 3. Digital signal analysis: Generative AI models analyze digital signals to interpret user intent and identify destinations.

[1185] 4. Destination setting: The autonomous vehicle's navigation system sets the specified destination.

[1186] Specific examples

[1187] For example, if a user thinks, "I want to go to Tokyo Tower," the EEG sensor captures the EEG signal, and the generative AI model analyzes the signal to identify the destination, "Tokyo Tower." The navigation system of the autonomous vehicle receives this information and sets "Tokyo Tower" as the destination in the vehicle's navigation system.

[1188] Prompt Sentence Examples

[1189] Analyze the EEG signals when the user thinks, "I want to go to Tokyo Tower," and generate a prompt that identifies the destination.

[1190] In this way, the brainwave navigation system can read the user's thoughts and set a destination, making it easier to operate while driving or for users with physical limitations, and reducing the complexity of operation when performing multiple tasks at the same time.

[1191] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[1192] Step 1:

[1193] Capturing EEG signals

[1194] The user wears the EEG sensor and thinks. The EEG sensor captures the user's EEG signals in real time. The input is the user's EEG signals, and the output is the captured raw EEG data. This data is in analog form and is converted into a digital signal in the next step.

[1195] Step 2:

[1196] Conversion to digital signals

[1197] The server receives the captured raw EEG data and converts it into a digital signal using a generative AI model. The input is raw EEG data, and the output is a digital signal. This conversion includes data processing such as filtering and noise removal of the EEG data. Specifically, the EEG data is sampled and quantized.

[1198] Step 3:

[1199] Digital Signal Analysis

[1200] The server uses a generative AI model to analyze digital signals and interpret the user's intent. The input is the digital signal, and the output is the user's intent (e.g., the name of a destination). This analysis involves pattern recognition using machine learning algorithms. Specific operations include feature extraction and classification of the digital signal.

[1201] Step 4:

[1202] Destination setting

[1203] The server sends the analyzed user intention to the navigation system of the autonomous vehicle. The input is the user intention (the name of the destination), and the output is the destination set in the navigation system. Specifically, the server calls the navigation system's API to set the destination.

[1204] Step 5:

[1205] Start navigation

[1206] The navigation system of an autonomous vehicle calculates the optimal route based on the set destination and starts navigation. The input is the set destination and the output is the navigation route. Specific operations include searching map data and calculating the route.

[1207] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1208] "Example 1"

[1209] In one embodiment of the present invention, a system is provided that includes a means for converting EEG signals into digital signals, a means for operating a smartphone using the digital signals, a means for understanding a user's intentions and interpreting the EEG signals using generative AI, and an emotion engine that recognizes the user's emotions. Specifically, when the user is feeling joy, the system can operate the smartphone in accordance with the user's emotions, such as by brightening the color tone of the smartphone screen.

[1210] "Example 2"

[1211] In another embodiment of the present invention, the emotion engine provides a means for executing smartphone operations according to the user's emotions. Specifically, when the user feels angry, the smartphone can be operated according to the user's emotions, such as temporarily stopping smartphone notifications.

[1212] "Example 3"

[1213] In a further embodiment of the present invention, the emotion engine provides a means for adjusting the interface in response to the user's emotion, for example, by lowering the volume of the smartphone when the user is feeling sad.

[1214] The processing flow of each embodiment will be described below.

[1215] "Example 1"

[1216] Step 1: Acquire the user's EEG signal.

[1217] Step 2: Convert the acquired EEG signal into a digital signal.

[1218] Step 3: Use generative AI to interpret user intent from digital signals.

[1219] Step 4: Use the emotion engine to recognize the user's emotions.

[1220] Step 5: Operate the smartphone based on the recognized emotion. For example, if the user is feeling happy, the smartphone screen will brighten up.

[1221] "Example 2"

[1222] Step 1: Acquire the user's EEG signal.

[1223] Step 2: Convert the acquired EEG signal into a digital signal.

[1224] Step 3: Use generative AI to interpret user intent from digital signals.

[1225] Step 4: Use the emotion engine to recognize the user's emotions.

[1226] Step 5: Implement smartphone actions based on the recognized emotion. For example, temporarily suspend smartphone notifications if the user is feeling angry.

[1227] "Example 3"

[1228] Step 1: Acquire the user's EEG signal.

[1229] Step 2: Convert the acquired EEG signal into a digital signal.

[1230] Step 3: Use generative AI to interpret user intent from digital signals.

[1231] Step 4: Use the emotion engine to recognize the user's emotions.

[1232] Step 5: Adjust the smartphone interface based on the recognized emotion, for example, lowering the volume of the smartphone when the user feels sad.

[1233] Example 1

[1234] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1235] Conventional methods for operating information processing devices require users to use their hands or voice, which makes operation difficult for users with physical limitations or in multitasking environments with interfaces such as virtual reality, augmented reality, or in vehicles. Furthermore, the inability to accurately understand the user's intentions and emotions and operate accordingly poses a problem of not improving the user experience.

[1236] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1237] In this invention, the server includes means for converting EEG signals into digital signals, means for operating an information processing device using the digital signals, means for understanding a user's intentions and interpreting EEG signals by making full use of generative artificial intelligence, and means for recognizing the user's emotions. This makes it possible to operate the information processing device based on the user's thoughts and emotions, enabling intuitive and efficient operation even for users with physical limitations or in a multitasking environment.

[1238] "Electroencephalogram signals" are weak electrical signals generated by electrical activity in the brain.

[1239] A "digital signal" is a signal that has been converted from an analog signal into a digital form that can be processed by a computer or electronic device.

[1240] An "information processing device" is an electronic device that has the function of inputting, processing, and outputting data, and specifically refers to smartphones, computers, etc.

[1241] "Generative AI" is an artificial intelligence technology that understands the user's intentions and thoughts and generates appropriate responses and operations.

[1242] "User intent" refers to the thoughts or goals that a user has when they want to perform a particular operation or action.

[1243] "User emotion" refers to the psychological state a user feels in response to a particular situation or stimulus, including joy, sadness, anger, etc.

[1244] "Means for recognizing emotions" refers to technologies and devices for detecting and identifying a user's emotions.

[1245] "Thought patterns" are specific patterns of brain waves that are generated when a user has a specific thought or intention.

[1246] "Analysis" is the process of processing a digital signal and extracting useful information from it.

[1247] An "operation instruction" is a signal or command that instructs an information processing device to perform a specific operation.

[1248] This invention is a system that converts a user's electroencephalogram (EEG) signals into digital signals and uses these digital signals to operate an information processing device. Furthermore, it utilizes generative artificial intelligence to understand the user's intentions and interpret the EEG signals. It can also recognize the user's emotions and perform operations accordingly.

[1249] Hardware and software used

[1250] Hardware

[1251] 1. EEG headset: A device that captures the user's brainwave signals and converts them into digital signals. Specifically, it is a device with sensors that is worn on the scalp.

[1252] 2. Information processing device: A device such as a smartphone or computer that receives digital signals and performs operations based on the analysis results.

[1253] 3. Server: A device that receives digital signals and interprets their intent using analytical and generative artificial intelligence.

[1254] software

[1255] 1. Generative AI model: A machine learning model to understand user intent and interpret EEG signals. It uses techniques such as neural networks.

[1256] 2. Signal analysis algorithm: An algorithm that analyzes digital signals and infers the user's intentions. Specifically, it uses FFT (Fast Fourier Transform) and machine learning algorithms.

[1257] 3. Emotion recognition engine: Software that recognizes the user's emotions and determines the appropriate operation.

[1258] Specific examples

[1259] Example 1: Sending email

[1260] When a user thinks about "sending an email," the following process takes place:

[1261] 1. The user thinks, "I want to send an email."

[1262] 2. The EEG headset captures the brainwave signals and converts them into digital signals.

[1263] 3. The device sends a digital signal to the server.

[1264] 4. The server analyzes the digital signals and uses a generative artificial intelligence model to interpret the user's intent as "send an email."

[1265] 5. The server sends an instruction to the information processing device (smartphone) to launch the email application and display a new email creation screen.

[1266] 6. Follow the instructions received on your smartphone to launch the email app and display the new email composition screen.

[1267] Example 2: Emotional manipulation

[1268] If the user is happy, the following process is performed:

[1269] 1. Users feel delighted.

[1270] 2. The EEG headset captures the brainwave signals and converts them into digital signals.

[1271] 3. The device sends a digital signal to the server.

[1272] 4. The server analyzes the digital signal and uses an emotion recognition engine to identify the user's emotion as "joy."

[1273] 5. The server sends an instruction to the information processing device (smartphone) to brighten the screen color tone.

[1274] 6. Follow the instructions received by your smartphone to brighten the screen color tone.

[1275] Prompt Sentence Examples

[1276] "Design a system that, when a user has a specific thought, executes an operation on an information processing device that corresponds to that thought. For example, if a user thinks, 'I want to send an email,' the system should recognize that intention and send an instruction to the information processing device to launch the email application. Also, add a function that recognizes the user's emotions and performs an operation according to those emotions. For example, if the user is feeling happy, make the color tone of the information processing device screen brighter."

[1277] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1278] Step 1:

[1279] The user performs a specific thought.

[1280] Input: User thoughts (e.g., "send email")

[1281] Output: EEG signal

[1282] Specific operation: When a user thinks about "sending an email," electrical activity occurs in the brain, generating an EEG signal.

[1283] Step 2:

[1284] The terminal (EEG headset) captures the brainwave signal and converts it into a digital signal.

[1285] Input: EEG signal

[1286] Output: Digital signal

[1287] How it works: The EEG headset's sensors contact the user's scalp and measure brain waves in real time. The measured analog signal is converted to a digital signal using an ADC (analog-to-digital converter).

[1288] Step 3:

[1289] The terminal transmits a digital signal to the server.

[1290] Input: Digital signal

[1291] Output: Send data to the server

[1292] What it does: The device uses Wi-Fi or Bluetooth to send a digital signal to the server.

[1293] Step 4:

[1294] The server analyzes the received digital signal.

[1295] Input: Digital signal

[1296] Output: Analysis results (frequency components, etc.)

[1297] Specific operation: The server analyzes the frequency components of the digital signal using FFT (Fast Fourier Transform), and extracts the signal features as a result of the analysis.

[1298] Step 5:

[1299] The server uses a generative artificial intelligence model to interpret the user's intent.

[1300] Input: Analysis results

[1301] Output: User intent (e.g. "send email")

[1302] Specific operation: The server uses a machine learning model (e.g., a neural network) to infer the user's intent from the analysis results. The inferred intent is "send an email."

[1303] Step 6:

[1304] The server determines an operation to be performed on the information processing device based on the estimated user's intention.

[1305] Input: User intent

[1306] Output: Operation instructions (e.g., launch the email app)

[1307] Specific operation: The server determines the corresponding operation based on the user's intention. For example, if the intention is to "send an email," it determines the operation to launch the email app.

[1308] Step 7:

[1309] The server transmits an operation instruction to the information processing device.

[1310] Input: Operation instructions

[1311] Output: Data transmission to information processing device

[1312] Specific operation: The server uses the API of the information processing device (smartphone) to send operation instructions.

[1313] Step 8:

[1314] The information processing device executes the operation according to the received instruction.

[1315] Input: Operation instructions

[1316] Output: Operation result (e.g. launching email app)

[1317] Specific operation: According to the operation instructions received by the smartphone, the email app is launched and a new email creation screen is displayed.

[1318] Step 9:

[1319] The device recognizes the user's emotions.

[1320] Input: EEG signal

[1321] Output: Emotional information (e.g., joy)

[1322] How it works: The EEG headset captures brainwave signals, and the emotion recognition engine analyzes the signals to recognize the user's emotions.

[1323] Step 10:

[1324] The device transmits the recognized emotion information to the server.

[1325] Input: Emotion information

[1326] Output: Send data to the server

[1327] Specific operation: The device uses Wi-Fi or Bluetooth to send emotional information to the server.

[1328] Step 11:

[1329] The server determines the operation of the information processing device based on the emotion information.

[1330] Input: Emotion information

[1331] Output: Operation instructions (e.g., brighten the screen color)

[1332] Specific operation: The server determines the corresponding operation based on the emotion information. For example, if the user feels happy, it determines the operation to brighten the color tone of the screen.

[1333] Step 12:

[1334] The server transmits an operation instruction according to the emotion to the information processing device.

[1335] Input: Operation instructions

[1336] Output: Data transmission to information processing device

[1337] Specific operation: The server uses the API of the information processing device (smartphone) to send operation instructions.

[1338] Step 13:

[1339] The information processing device executes the operation according to the received instruction.

[1340] Input: Operation instructions

[1341] Output: Operation result (e.g., change in screen color)

[1342] Specific operation: The smartphone brightens the screen color according to the operation instructions received.

[1343] (Application example 1)

[1344] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1345] Conventional methods for operating information processing devices rely on manual operation or voice recognition, making them difficult for users with physical limitations or in certain environments. From a security perspective, conventional authentication methods pose a high risk of unauthorized access. Furthermore, the inability to take the user's emotional state into account in operations hinders the user experience.

[1346] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1347] In this invention, the server includes means for converting EEG signals into digital signals, means for operating an information processing device using the digital signals, means for understanding a user's intentions using generative artificial intelligence and interpreting the EEG signals, means including an emotion engine for recognizing the user's emotions, means for performing authentication using the user's EEG signals, and means for determining whether to authenticate the user based on the user's emotional state. This allows even users with physical limitations to easily operate the information processing device, improving security and the user experience.

[1348] An "electroencephalogram signal" is a signal that reflects the electrical activity of the brain and is measured using a device such as an electroencephalograph.

[1349] A "digital signal" is a signal that has been converted from an analog signal into a digital format that can be processed by a computer or information processing device.

[1350] An "information processing device" is a device that inputs, processes, and outputs data, and includes computers, smartphones, etc.

[1351] "Generative AI" is AI that uses machine learning and deep learning to learn patterns from data and understand user intentions and emotions.

[1352] An "emotion engine" is software or hardware that recognizes a user's emotional state and responds or takes appropriate action based on that.

[1353] "Authentication" is the process of verifying that a user is a legitimate user, and is carried out to ensure security.

[1354] "Electroencephalogram authentication" is a method of authentication that utilizes the user's brainwave signals, ensuring security by identifying each individual user's brainwave patterns.

[1355] "Emotional state" refers to the user's current emotional state, including emotions such as joy, sadness, anger, and fear.

[1356] An "operation" is an act of giving an instruction to an information processing device to perform a specific action.

[1357] A system for implementing this invention includes means for converting an EEG signal into a digital signal, means for operating an information processing device using the digital signal, means for understanding a user's intentions by making full use of generative artificial intelligence and interpreting the EEG signal, means including an emotion engine for recognizing the user's emotions, means for performing authentication using the user's EEG signal, and means for determining whether or not to authenticate the user based on the user's emotional state.

[1358] Hardware and Software Configuration

[1359] The server captures the user's brainwave signals using an EEG capture device (e.g., Emotiv Epoc+). The captured brainwave signals are converted into digital signals and sent to an information processing device (e.g., a smartphone or computer). A generative artificial intelligence model (e.g., a deep learning model using Keras) analyzes the digital signals and understands the user's intentions. The emotion engine recognizes the user's emotional state and determines whether or not to authenticate them.

[1360] Data processing and calculation

[1361] The EEG capture device captures the user's EEG signals in real time and converts them into digital signals. The digital signals undergo a standardization process and are then input into a generative AI model. The generative AI model analyzes the user's EEG patterns and predicts their intentions. The emotion engine recognizes the user's emotional state from the digital signals and provides data to determine whether or not to authenticate them.

[1362] Specific examples

[1363] For example, when a user tries to unlock their smartphone, an EEG capture device captures the user's brainwave signals and converts them into digital signals. A generative AI model analyzes the digital signals and recognizes that the user's intention is to "unlock." If the emotion engine determines that the user's emotional state is "calm," the smartphone will be unlocked.

[1364] Prompt Sentence Examples

[1365] "Create a program that uses an EEG capture device to capture brainwave signals when a user tries to unlock their smartphone, uses a generative artificial intelligence model to interpret the user's intent, and uses an emotion engine to recognize the user's emotions."

[1366] In this way, a system is realized that utilizes the user's electroencephalogram signal and emotional state to operate and authenticate an information processing device.

[1367] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1368] Step 1:

[1369] The user wears an EEG capture device and captures EEG signals.

[1370] Input: User's EEG signal

[1371] Output: Analog EEG signal

[1372] Specific operation: The user wears an EEG capture device on their head, and the device captures EEG signals in real time.

[1373] Step 2:

[1374] The server converts the captured analog EEG signals into digital signals.

[1375] Input: Analog EEG signal

[1376] Output: Digital EEG signal

[1377] Specific operation: The server receives the analog EEG signal and performs a standardization process to convert it into a digital signal.

[1378] Step 3:

[1379] The server inputs the digital EEG signals into a generative artificial intelligence model to analyze the user's intentions.

[1380] Input: Digital EEG signal

[1381] Output: User intent (e.g. unlock)

[1382] Specific operation: The server inputs digital EEG signals into a generative artificial intelligence model, which then analyzes the signals and predicts the user's intentions.

[1383] Step 4:

[1384] The server inputs the digital EEG signal into an emotion engine to recognize the user's emotional state.

[1385] Input: Digital EEG signal

[1386] Output: User's emotional state (e.g., calm)

[1387] Specific operation: The server inputs digital EEG signals into the emotion engine, which then analyzes the signals to recognize the user's emotional state.

[1388] Step 5:

[1389] The server determines whether to authenticate the user based on their intentions and emotional state.

[1390] Input: User intent, user emotional state

[1391] Output: Authentication result (e.g., authentication successful)

[1392] Specific operation: The server confirms that the user's intention is "unlock" and that the emotional state is "calm," and determines that the authentication is successful.

[1393] Step 6:

[1394] The server executes the operation of the information processing device based on the authentication result.

[1395] Input: Authentication result

[1396] Output: Operation results of information processing device (e.g. unlocking a smartphone)

[1397] Specific operation: The server sends the authentication result to the information processing device and performs the operation to unlock the smartphone.

[1398] Example 2

[1399] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1400] Users with physical limitations and in multitasking environments such as virtual reality, augmented reality, and in-vehicle interfaces face challenges in intuitive and efficient device operation. In particular, when operating a device using EEG signals, it is necessary to accurately understand the user's intentions and execute appropriate operations.

[1401] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1402] In this invention, the server includes means for converting EEG signals into digital signals, means for analyzing the digital signals, means for understanding the user's intention using a generative AI model, means for sending operation instructions to the smartphone based on the intention, and means for receiving the operation instructions and executing smartphone operations. This allows the user's thoughts to be directly linked to smartphone operations, enabling intuitive and efficient device operation even for users with physical limitations or in a multitasking environment.

[1403] "EEG signals" are signals that reflect the electrical activity of the brain and are used to detect a user's thoughts and emotions.

[1404] A "digital signal" is an analog signal converted into a digital format that can be processed by a computer or electronic device.

[1405] A "generative AI model" is a model that uses artificial intelligence technology to analyze data and learn user intentions and patterns.

[1406] "User intent" refers to the user's intention or purpose to perform a specific operation or action.

[1407] An "operation instruction" is a specific operation command for a device that is generated based on the user's intention.

[1408] A "smartphone" is a multi-functional mobile device that can connect to the Internet and run applications in addition to the functions of a mobile phone.

[1409] "Analysis" is a data processing method that processes digital signals to understand user intent and patterns.

[1410] "Interface" refers to the means or methods that enable interaction between a user and a device.

[1411] "Multitasking" refers to the state or ability to perform multiple tasks or operations simultaneously.

[1412] MODE FOR CARRYING OUT THE INVENTION

[1413] The present invention relates to a system for operating a smartphone using a user's electroencephalogram (EEG) signals. Specific embodiments of this system will be described below.

[1414] System configuration

[1415] The system consists of the following main components:

[1416] 1. EEG measurement device: A device for acquiring the user's EEG signals. A specific example is an EEG measurement device such as Emotiv Epoc+.

[1417] 2. Digital signal converter: A device that converts acquired EEG signals into digital signals. It may be built into the EEG measurement device.

[1418] 3. Server: A computer system that analyzes digital signals and uses generative AI models to understand user intent. Signal processing algorithms (e.g., FFT, filtering) and generative AI models (e.g., OpenAI's GPT-4) are used for analysis.

[1419] 4. Smartphone: A multi-functional mobile device that receives operation instructions from the server and actually executes the operations.

[1420] System Operation

[1421] When a user thinks about operating a smartphone, an electroencephalogram (EEG) signal is generated. The EEG measurement device acquires this EEG signal in real time and converts it into a digital signal. The converted digital signal is then sent to a server via the Internet.

[1422] The server analyzes the received digital signals and uses a generative AI model to understand the user's intentions. The generative AI model learns the user's thought patterns based on past data and infers their intentions. Based on the analysis results, the server generates operational instructions for the smartphone and sends them to the device. The smartphone performs specific operations (e.g., tap the home button) according to the received operational instructions.

[1423] Specific examples

[1424] As a concrete example, consider the case where a user thinks, "Tap the home button." The user's brainwave signals are acquired by an EEG measuring device and converted into digital signals. The server analyzes these digital signals and uses a generative AI model to understand the user's intention. Based on the analysis results, the server sends the instruction to the smartphone to "Tap the home button," and the smartphone carries out this instruction.

[1425] Prompt Sentence Examples

[1426] Analyze the EEG signals when the user thinks "tap the home button," and generate a program that will execute the action of tapping the home button on a smartphone.

[1427] This system allows the user's thoughts to be directly linked to the operation of the smartphone, enabling intuitive and efficient device operation even for users with physical limitations or in multitasking environments.

[1428] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1429] Step 1:

[1430] The user thinks about how to operate a smartphone.

[1431] A user thinks about tapping the home button on their smartphone. This thought generates an EEG signal. The input is the user's thought, and the output is the EEG signal.

[1432] Step 2:

[1433] The device acquires the brainwave signal.

[1434] The terminal (EEG measuring device) acquires the user's EEG signals in real time. The EEG measuring device is worn on the user's head and electrically detects the EEG signals. The input is the user's EEG signals, and the output is the acquired EEG signals.

[1435] Step 3:

[1436] The terminal converts the brainwave signal into a digital signal.

[1437] The terminal converts the acquired EEG signal into a digital signal, which makes the EEG signal in a format that can be processed by a computer. The input is the acquired EEG signal, and the output is a digital signal.

[1438] Step 4:

[1439] The terminal transmits a digital signal to the server.

[1440] The terminal transmits the converted digital signal to a server via the Internet using a communication method such as Wi-Fi or Bluetooth. The input is a digital signal, and the output is the digital signal transmitted to the server.

[1441] Step 5:

[1442] The server analyzes the digital signal.

[1443] The server analyzes the received digital signal. Signal processing algorithms (e.g. FFT, filtering) are used for the analysis. The input is the digital signal sent to the server, and the output is the analyzed digital signal.

[1444] Step 6:

[1445] The server uses a generative AI model to understand the user's intent.

[1446] The server inputs the analyzed digital signals into a generative AI model (e.g., GPT-4) to understand the user's intention. The generative AI model learns the user's thought patterns based on past data and infers their intention. The input is the analyzed digital signals, and the output is the user's intention.

[1447] Step 7:

[1448] The server sends operation instructions to the smartphone.

[1449] The server generates operational instructions for the smartphone based on the user's intention and sends them to the device. The operational instructions include specific actions (e.g., tapping the home button). The input is the user's intention, and the output is the operational instructions.

[1450] Step 8:

[1451] The device receives the operation instruction and executes the smartphone operation.

[1452] The terminal transmits the operation instruction received from the server to the smartphone, and the smartphone executes the operation of tapping the home button. The input is the operation instruction, and the output is the executed smartphone operation.

[1453] (Application example 2)

[1454] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1455] Conventional factory robot operation systems make it difficult for workers with physical limitations to operate robots efficiently. Furthermore, they are unable to operate robots while taking into account the emotional state of the workers, which leads to problems with work efficiency and safety. Furthermore, operation in a multitasking environment using multiple interfaces is also difficult.

[1456] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1457] In this invention, the server includes a means for converting EEG signals into digital signals, a means for operating a smartphone using the digital signals, a means for understanding a user's intentions and interpreting the EEG signals using generative AI, and a means for detecting the user's emotional state and adjusting the operation according to the emotional state. This enables even physically challenged workers to efficiently operate factory robots, realizing safe and efficient robot operation according to the worker's emotional state. Furthermore, smooth operation is possible even in a multitasking environment using multiple interfaces.

[1458] An "EEG signal" is a signal that reflects the electrical activity of the brain and indicates the state of the brain and thoughts.

[1459] A "digital signal" is a signal that has been converted from an analog signal into a digital format that can be processed by a computer or other electronic device.

[1460] "Generative AI" is a type of artificial intelligence that is a technology that understands the user's intentions and thoughts and generates appropriate responses and operations.

[1461] "User intent" describes the action the user wants to take or the goal they want to achieve.

[1462] "Emotional state" refers to a user's current feelings or mood, including states such as stress, joy, and anger.

[1463] "Means for adjusting operation" refers to techniques for changing the system's behavior and response depending on the user's emotional state.

[1464] "Physical Constraints" means any physical limitation or disability a User has, including any limitation or disability that makes normal operation difficult.

[1465] "Virtual reality" refers to virtual environments and experiences generated using computer technology.

[1466] "Augmented reality" is a technology that displays digital information overlaid on the real environment.

[1467] A "multitasking environment" is an environment in which multiple tasks or operations are required to be performed simultaneously.

[1468] A system for implementing this invention converts EEG signals into digital signals and uses the digital signals to operate a factory robot. The system uses generative AI to understand the user's intentions and interpret the EEG signals. It also includes a function to detect the user's emotional state and adjust its operation accordingly.

[1469] Hardware Configuration

[1470] The system uses the following hardware:

[1471] EEG capture device: A device that captures the user's brainwave signals.

[1472] Head-mounted display (HMD): A device worn by the user that provides visual information.

[1473] Factory robot: A robot that operates based on user instructions.

[1474] Software Configuration

[1475] The system uses the following software:

[1476] Python: A programming language used to implement programs.

[1477] scikit-learn: A library used to implement generative AI models.

[1478] EmotionEngine: Software for detecting the user's emotional state.

[1479] RobotController: Software for operating factory robots.

[1480] Data processing and calculation

[1481] 1. Capturing EEG signals: The EEG capture device captures the user's EEG signals.

[1482] 2. Conversion to digital signals: The captured EEG signals are converted into digital signals.

[1483] 3. Interpreting intent using a generative AI model: Digital signals are input into a generative AI model to interpret the user's intent.

[1484] 4. Emotional state detection: EmotionEngine detects the user's emotional state.

[1485] 5. Robot Control: The RobotController controls factory robots based on the user's intentions and emotional state.

[1486] Specific examples

[1487] As a concrete example, consider the case where a factory worker thinks, "I want to lift a part." This thought is captured as an EEG signal and converted into a digital signal. The generative AI model interprets the intention to "lift a part," and the robot carries out that command. If the worker is feeling stressed, the emotion engine detects this state and takes appropriate action, such as adjusting the robot's speed.

[1488] Prompt Sentence Examples

[1489] When the user thinks about "lifting a part," the brainwave signal is converted into a digital signal, and the robot performs the action of lifting the part.

[1490] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1491] Step 1:

[1492] The user wears an EEG capture device and thinks a specific thought. The EEG capture device captures the user's EEG signals. The captured EEG signals become input data.

[1493] Step 2:

[1494] The server receives the captured EEG signals and converts them into digital signals. Specifically, it processes the data to convert the analog EEG signals into digital format. This converted digital signal becomes the output data.

[1495] Step 3:

[1496] The server uses a generative AI model to interpret the user's intention from the digital signal. The input data is a digital signal, which the generative AI model analyzes to predict the user's intention. The output data is information that indicates the user's intention.

[1497] Step 4:

[1498] The server uses the Emotion Engine to detect the user's emotional state. The input data is the user's biosignals and behavioral data, which the Emotion Engine analyzes to identify the user's emotional state. The output data is information indicating the user's emotional state.

[1499] Step 5:

[1500] The server uses RobotController to operate factory robots based on the user's intentions and emotional state. The input data is information indicating the user's intentions and emotional state, and RobotController determines the robot's behavior based on this. The output data is specific operating instructions for the robot.

[1501] Step 6:

[1502] The factory robot receives operation instructions from the server and actually performs the operation. Specific operations include lifting, moving, and assembling parts, thereby realizing the user's intentions.

[1503] Example 3

[1504] Next, a description will be given of Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1505] Conventional information processing systems require users to operate them manually, which is particularly difficult for users who are driving or have physical limitations. Furthermore, the interface does not adjust according to the user's emotions, resulting in a poor user experience. To address these issues, there is a need for a system that uses the user's brainwave signals to operate information processing devices and further adjusts the interface according to the user's emotions.

[1506] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[1507] In this invention, the server includes means for converting EEG signals into digital signals, means for operating an information processing device using the digital signals, means for understanding a user's intentions and interpreting EEG signals using generative artificial intelligence, and means for adjusting an interface according to the user's emotions using an emotion engine. This allows the user to operate the information processing device using EEG signals without manual operation, and further, the interface is adjusted according to the user's emotions, improving the user experience.

[1508] "Electroencephalogram signals" are signals obtained by detecting electrical activity in the user's brain.

[1509] A "digital signal" is a signal that is an analog signal converted into digital form.

[1510] An "information processing device" is a device that receives digital signals and performs specific operations or processes.

[1511] "Generative AI" is an AI technology used to understand user intent and interpret brainwave signals.

[1512] An "emotion engine" is a technology that detects a user's emotions and adjusts the interface accordingly.

[1513] An "interface" is a means for exchanging information between a user and an information processing device.

[1514] "User intent" is what a user wants to do or do.

[1515] A "brainwave sensor" is a device for detecting a user's brainwave signals.

[1516] "Real-time" refers to processing or operations being performed immediately without delay.

[1517] A "thought pattern" is a characteristic pattern of brain wave signals when a user performs a specific operation or action.

[1518] MODE FOR CARRYING OUT THE INVENTION

[1519] The present invention is a system that utilizes a user's electroencephalogram signals to operate an information processing device and further adjusts the interface in accordance with the user's emotions. A specific embodiment of this system will be described below.

[1520] Hardware and software used

[1521] Hardware: EEG sensors, information processing devices (e.g. smartphones, navigation systems)

[1522] Software: EEG signal analysis software, generative artificial intelligence, emotion engine

[1523] System Overview

[1524] When a user wears an EEG sensor and thinks a specific thought, the EEG sensor picks up the EEG signal. This EEG signal is converted into a digital signal and sent to a server. The server uses generative artificial intelligence to analyze the digital signal and understand the user's intention. The information processing device is operated based on the analysis results. An emotion engine is also used to detect the user's emotions and adjust the interface.

[1525] Specific examples

[1526] Example 1: A user is driving a car and thinks, "I want to go home."

[1527] The user wears a brainwave sensor and thinks, "I want to go home."

[1528] The EEG sensor picks up the EEG signal and converts it into a digital signal.

[1529] The server analyzes the digital signal and sets "home" as the destination in the navigation system.

[1530] The navigation system sets the destination to "home" and begins route guidance.

[1531] Example 2: When a user feels sad

[1532] The server detects the user's emotions using an emotion engine.

[1533] The smartphone receives instructions from the server and automatically lowers the volume.

[1534] Prompt Sentence Examples

[1535] "If a user thinks about a destination while driving a car, explain how you can capture their brainwave signals and operate the navigation system."

[1536] "Describe the behavior of a system that adjusts its interface based on the user's emotions."

[1537] This system enables the user to operate the information processing device using EEG signals without manual operation, and furthermore, the interface is adjusted according to the user's emotions, improving the user experience. The flow of the identification process in Example 3 will be described with reference to Fig. 21.

[1538] Step 1:

[1539] The user wears a brainwave sensor and thinks specific thoughts.

[1540] Input: User thoughts

[1541] Output: EEG signal

[1542] Specific action: The user wears a brainwave sensor on their head and thinks a specific thought, such as "I want to go home."

[1543] Step 2:

[1544] The terminal (brainwave sensor) acquires the user's brainwave signals and converts them into digital signals.

[1545] Input: EEG signal

[1546] Output: Digital signal

[1547] How it works: The signal processing unit inside the EEG sensor converts the analog EEG signal into a digital signal.

[1548] Step 3:

[1549] The server receives the digital signal and analyzes it using a generative AI model.

[1550] Input: Digital signal

[1551] Output: User intent (e.g., destination)

[1552] What it does: The server analyzes the digital signals and runs a generative AI model to understand the user's intent.

[1553] Step 4:

[1554] The server generates an operation instruction for the information processing device based on the analysis result.

[1555] Input: User intent

[1556] Output: Operation instructions

[1557] Specific operation: The server generates an instruction to the navigation system to set "home" as the destination.

[1558] Step 5:

[1559] The terminal (navigation system) receives instructions from the server and sets the destination.

[1560] Input: Operation instructions

[1561] Output: Destination setting

[1562] Specific operation: The navigation system sets the destination to "Home" and begins route guidance.

[1563] Step 6:

[1564] The server uses a generative AI model to learn the user's thought patterns.

[1565] Input: Historical data

[1566] Output: Training results

[1567] Specific operation: The server accumulates past data and improves the accuracy of analysis from the next time onwards.

[1568] Step 7:

[1569] The server detects the user's emotions using an emotion engine.

[1570] Input: User emotion data

[1571] Output: Emotion detection results

[1572] Specific operation: The server analyzes the user's emotions and executes the emotion engine.

[1573] Step 8:

[1574] The device (smartphone) receives instructions from the server and adjusts the interface.

[1575] Input: Emotion detection result

[1576] Output: Interface adjustments

[1577] Specific behavior: The smartphone automatically adjusts the interface, such as automatically lowering the volume.

[1578] (Application example 3)

[1579] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1580] Conventional navigation systems and smart devices require manual input from the user, making them difficult to operate, especially for those driving or with physical limitations. Furthermore, the interface does not adjust according to the user's emotional state, resulting in a lack of improvement in the user experience. To address these issues, there is a need for a system that uses EEG signals to understand the user's intentions and emotions and automates device operation and interface adjustments.

[1581] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[1582] In this invention, the server includes means for converting EEG signals into digital signals, means for operating a smart device using the digital signals, means for understanding a user's intentions and interpreting EEG signals using generative AI, and means for detecting the user's emotional state and adjusting the interface. This enables the user to set a destination in a navigation system or operate a smart device simply by thinking, and also enables the interface to be adjusted according to the user's emotional state.

[1583] "Electroencephalogram signals" are electrical signals generated in association with brain activity.

[1584] A "digital signal" is a signal that is generated by converting an analog signal into a numerical value and expressing it in digital form.

[1585] "Smart device" is a general term for electronic devices that have internet connectivity and advanced computing capabilities.

[1586] "Generative AI" is an artificial intelligence technology that understands a user's intentions and emotions and generates appropriate responses.

[1587] An "interface" is a point of contact or means by which a user and a system exchange information with each other.

[1588] "Emotional state" refers to the emotional state a user is feeling at a particular moment.

[1589] A "navigation system" is a system that provides guidance on routes to a destination.

[1590] "Thoughts" refer to what the user thinks in their head.

[1591] "Physical limitations" refers to a condition in which a user has difficulty performing a particular action or operation due to physical reasons.

[1592] "Virtual reality" is a virtual environment created using computer technology.

[1593] "Augmented reality" is a technology that displays digital information overlaid on the real environment.

[1594] "Multitasking" refers to performing multiple tasks simultaneously.

[1595] A system for carrying out this invention converts EEG signals into digital signals, understands the user's intentions and emotions, and operates a smart device. Specific embodiments are described below.

[1596] System Configuration

[1597] The system consists of the following main components:

[1598] 1. EEG signal receiver: Smart glasses with built-in sensors that capture the user's brainwave signals.

[1599] 2. Digital signal converter: A device that converts EEG signals into digital signals.

[1600] 3. Generative AI model: A machine learning model for understanding user intent and emotions and interpreting EEG signals.

[1601] 4. Navigation system: A system for setting destinations.

[1602] 5. Audio system: A system that plays music depending on the user's emotional state.

[1603] Program processing

[1604] The server receives EEG signals from the EEG receiver and converts them into digital signals using a digital signal converter. It then analyzes the user's intentions and emotions using a generative AI model. Based on the analysis results, it operates the navigation system and audio system.

[1605] Hardware and software used

[1606] Brainwave signal receiving device: smart glasses

[1607] Digital signal converter: Dedicated converter

[1608] Generative AI model: Machine learning model (e.g., MLPClassifier)

[1609] Navigation system: Navigation system installed in the vehicle

[1610] Audio system: The audio system installed in the vehicle

[1611] Specific examples

[1612] When a user wears the smart glasses and thinks, "I want to go home," the EEG signal receiver captures the EEG signal. The digital signal converter converts this EEG signal into a digital signal, and the generative AI model analyzes the user's intention. Based on the analysis results, the navigation system automatically sets home as the destination. If the user is feeling stressed, the audio system will play relaxing music.

[1613] Prompt Sentence Examples

[1614] "All a user has to do is put on the smart glasses and think about their destination, and the navigation system will automatically set it. It can also detect the user's emotional state and play relaxing music if they are feeling stressed."

[1615] The above is a specific embodiment for carrying out the invention.

[1616] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[1617] Step 1:

[1618] The user puts on the smart glasses and thinks about a destination.

[1619] Input: User's EEG signal

[1620] Output: EEG signal data

[1621] How it works: Sensors built into the smart glasses capture the user's brainwave signals.

[1622] Step 2:

[1623] The electroencephalogram signal receiving device acquires the electroencephalogram signal.

[1624] Input: EEG signal data

[1625] Output: Raw EEG signal data

[1626] Specific operation: The EEG signal receiving device transmits the captured EEG signal to the digital signal conversion device.

[1627] Step 3:

[1628] A digital signal conversion device converts the electroencephalogram signal into a digital signal.

[1629] Input: Raw EEG signal data

[1630] Output: Digital signal data

[1631] Specific operation: The digital signal converter analyzes the EEG signal and converts it into digital form.

[1632] Step 4:

[1633] The server uses a generative AI model to analyze the digital signals and understand the user's intent.

[1634] Input: Digital signal data

[1635] Output: User intent (destination information)

[1636] Specific operation: The server inputs digital signal data into a generative AI model to predict the user's intentions.

[1637] Step 5:

[1638] The server transmits the destination information to the navigation system.

[1639] Input: User intent (destination information)

[1640] Output: Destination setting for navigation system

[1641] Specific operation: The server sends the analysis results to the navigation system, which automatically sets the destination.

[1642] Step 6:

[1643] The server uses a generative AI model to analyze the user's emotional state.

[1644] Input: Digital signal data

[1645] Output: User's emotional state

[1646] How it works: The server inputs digital signal data into a generative AI model to predict the user's emotional state.

[1647] Step 7:

[1648] The server sends instructions to the audio system according to the emotional state.

[1649] Input: User's emotional state

[1650] Output: Audio system operating instructions

[1651] Specific behavior: The server instructs the audio system to play relaxing music based on the user's emotional state.

[1652] The above are the specific processing steps of the system that realizes the application example.

[1653] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1654] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1655] Another example of generative AI is Gemini (internet search engine). <url: https: gemini.google.com ?hl="ja">) are listed.

[1656] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1657] [Third embodiment]

[1658] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1659] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1660] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1661] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1662] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1663] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1664] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1665] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1666] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1667] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1668] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1669] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[1670] "Example 1"

[1671] One embodiment of the present invention is a system that includes a device that converts EEG signals into digital signals, a device that uses the digital signals to operate a smartphone, and a device that uses generative AI to understand a user's intentions and interpret the EEG signals. Specifically, when a user engages in a specific thought, an EEG signal is generated, and a device that converts this into a digital signal captures the EEG signal. Next, a device that uses the digital signal to operate the smartphone analyzes the digital signal and executes the corresponding smartphone operation. Furthermore, the device that uses generative AI to understand a user's intentions and interprets the EEG signals learns the user's thought patterns, enabling more accurate operation.

[1672] "Example 2"

[1673] As a concrete example, consider a user with physical limitations operating a smartphone. When the user thinks about tapping the smartphone screen, an EEG signal is generated and converted into a digital signal. A device that uses this digital signal to operate the smartphone analyzes the digital signal and performs the tap operation on the smartphone screen. Furthermore, the device uses generative AI to understand the user's intentions and interprets the EEG signal, learning the user's thought patterns and allowing them to perform more accurate screen tap operations with the same thought the next time.

[1674] "Example 3"

[1675] Similar operations are also possible when multitasking with interfaces such as VR, AR, and vehicles. For example, when operating a navigation system while driving a car, if the user thinks about a destination, an EEG signal is generated and converted into a digital signal. A device that uses this digital signal to operate the navigation system analyzes the digital signal and executes the destination setting operation on the navigation system. Furthermore, the device, which uses generative AI to understand the user's intentions and interprets the EEG signal, can learn the user's thought patterns and execute a more accurate destination setting operation with the same thought next time.

[1676] The processing flow of each embodiment will be described below.

[1677] "Example 1"

[1678] Step 1: The user engages in a specific thought.

[1679] Step 2: EEG signals are generated and captured by a device that converts the EEG signals into digital signals.

[1680] Step 3: The device that uses the digital signals to operate the smartphone analyzes the digital signals.

[1681] Step 4: Based on the analysis results, perform the corresponding smartphone operation.

[1682] Step 5: Using generative AI to understand user intent and interpret EEG signals, the device learns the user's thought patterns.

[1683] "Example 2"

[1684] Step 1: A user with physical limitations thinks about tapping on a smartphone screen.

[1685] Step 2: An EEG signal is generated and converted into a digital signal.

[1686] Step 3: The device that uses the digital signals to operate the smartphone analyzes the digital signals.

[1687] Step 4: Based on the analysis results, perform a tap operation on the smartphone screen.

[1688] Step 5: Using generative AI to understand user intent and interpret EEG signals, the device learns the user's thought patterns.

[1689] "Example 3"

[1690] Step 1: To operate the navigation system while driving, the user thinks about a destination.

[1691] Step 2: An EEG signal is generated and converted into a digital signal.

[1692] Step 3: A device that utilizes the digital signal to operate the navigation system analyzes the digital signal.

[1693] Step 4: Based on the analysis results, the navigation system sets the destination. Step 5: Using generative AI to understand the user's intentions and interpret brainwave signals, the device learns the user's thought patterns.

[1694] Example 1

[1695] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1696] Conventional methods for operating information processing devices require users to operate them manually, making operation particularly difficult for users with physical limitations. Efficient operation is also required for multitasking in interfaces such as virtual reality, augmented reality, and vehicles. Furthermore, it has been difficult to accurately understand the user's intentions and quickly reflect them.

[1697] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means. In this invention, the server includes means for converting an EEG signal into a digital signal, means for operating the information processing device using the digital signal, means for understanding the user's intention and interpreting the EEG signal by making full use of generative artificial intelligence, means for transmitting the digital signal using wireless communication technology, and means for analyzing the digital signal. This allows the user's thoughts to be directly linked to the operation of the information processing device, enabling efficient multitasking for users with physical limitations and in interfaces such as virtual reality, augmented reality, and vehicles.

[1698] "Electroencephalogram signals" are electrical signals generated by neural activity in the brain.

[1699] A "digital signal" is a signal that has been converted from an analog signal into a digital form that can be processed by a computer or electronic device.

[1700] An "information processing device" is a device that inputs, processes, and outputs data, and includes computers, smartphones, etc.

[1701] "Generative AI" is an AI technology that understands the user's intentions and performs appropriate processing based on the generated data.

[1702] "Wireless communication technology" refers to technology that transmits and receives data using radio waves without using cables, and includes Wi-Fi and Bluetooth.

[1703] "Analysis" is the process of examining data in detail to understand its structure and meaning.

[1704] A "thought" is a mental activity consciously performed by the user, with a specific purpose or intention.

[1705] "Virtual reality" is a technology that immerses users in a virtual environment generated using computer technology.

[1706] "Augmented reality" is a technology that displays computer-generated information overlaid on the real environment.

[1707] "Vehicle" means a vehicle for travelling on land, sea or air, including automobiles, ships, aircraft, etc.

[1708] "Multitasking" is the ability or process of performing multiple tasks simultaneously.

[1709] The present invention relates to a system for operating an information processing device using a user's electroencephalogram signal. Specific embodiments of this system will be described below.

[1710] Hardware and Software Configuration

[1711] Brain wave capture device

[1712] The user wears an EEG capture device, which has the function of capturing EEG signals and converting them into digital signals. Specifically, a typical EEG capture device is a headset with a built-in EEG sensor.

[1713] Terminal

[1714] The terminal receives digital signals from the EEG capture device and transmits them to the server. The terminal is equipped with wireless communication technology (such as Wi-Fi or Bluetooth).

[1715] server

[1716] The server receives and analyzes the digital signals sent from the device. It uses the Python libraries "MNE-Python" and "SciPy" for analysis. It also uses generative artificial intelligence (e.g., a generative AI model) to understand the user's intentions and interpret the EEG signals. This AI model uses machine learning frameworks such as TensorFlow and PyTorch to learn the user's thought patterns and enable more accurate operation.

[1717] Specific examples

[1718] Example 1: Sending a message

[1719] The user thinks, "I want to send a message." The EEG capture device captures the EEG signal corresponding to this thought and converts it into a digital signal. The device then sends this digital signal to a server via Wi-Fi. The server uses MNE-Python to analyze the digital signal and recognizes the intention to "send a message." The server then uses ADB (Android Debug Bridge) to launch the messaging app on the Android smartphone and send the message to the specified contacts.

[1720] Example 2: Launching an app

[1721] The user thinks, "I want to launch the camera app." The EEG capture device captures the EEG signal corresponding to this thought and converts it into a digital signal. The device then sends this digital signal to a server via Bluetooth. The server uses SciPy to analyze the digital signal and recognizes the intention to "launch the camera app." The server then uses XCUITest to launch the camera app on the iOS smartphone.

[1722] Prompt Sentence Examples

[1723] "Please describe a system that captures the EEG signals generated when a user engages in specific thoughts, converts them into digital signals, and operates an information processing device. Please also specify the names of any specific hardware and software."

[1724] This system provides a more intuitive and efficient interface by directly reflecting the user's thoughts in the operation of the information processing device.

[1725] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1726] Step 1:

[1727] When a user has a specific thought, the EEG capture device captures the EEG signal. The input is the user's thought, and the output is the captured EEG signal. In concrete terms, when a user thinks, "I want to send a message," the EEG capture device detects the EEG signal.

[1728] Step 2:

[1729] The terminal converts the captured EEG signal into a digital signal. The input is the captured EEG signal, and the output is a digital signal. Specifically, the built-in software of the EEG capture device converts the analog signal into a digital signal.

[1730] Step 3:

[1731] The terminal sends the converted digital signal to the server. The input is a digital signal, and the output is the digital signal sent to the server. Specifically, the terminal sends the digital signal to the server using Wi-Fi or Bluetooth.

[1732] Step 4:

[1733] The server analyzes the received digital signal. The input is the digital signal sent to the server, and the output is the analysis result. Specifically, the server uses the Python libraries "MNE-Python" and "SciPy" to extract the characteristics of the digital signal and determine whether it is intended to "send a message."

[1734] Step 5:

[1735] The server executes operations on the information processing device based on the analysis results. The input is the analysis results, and the output is the executed operations. Specifically, the server uses ADB (Android Debug Bridge) to launch the Android smartphone's messaging app and send a message to the specified contacts.

[1736] Step 6:

[1737] The server uses generative AI to understand the user's intentions and interpret the EEG signals. The input is the digital signal and past thought patterns, and the output is the interpretation of the user's intentions. Specifically, the server uses a generative AI model to learn the user's thought patterns and respond quickly to similar thoughts in the future.

[1738] In this way, a system is realized in which the user's thoughts are directly linked to the operation of an information processing device.

[1739] (Application example 1)

[1740] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1741] Conventional methods for operating autonomous vehicles rely on physical interfaces, making them difficult for users with physical limitations. Efficient operation is also difficult in multitasking environments where multiple interfaces must be operated simultaneously. Furthermore, the lack of technology to accurately reflect user intentions can lead to reduced operation accuracy.

[1742] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1743] In this invention, the server includes means for converting EEG signals into digital signals, means for operating an information processing device using the digital signals, means for understanding a user's intentions by making full use of generative AI and interpreting the EEG signals, and means for operating an autonomous vehicle using the user's thoughts. This makes it possible to operate an autonomous vehicle using the user's thoughts directly, enabling efficient and accurate operation even for users with physical limitations or in a multitasking environment.

[1744] An "EEG signal" is an electrical signal that reflects the electrical activity of the brain.

[1745] A "digital signal" is a signal that has been converted from an analog signal into a digital format that can be processed by a computer or other electronic device.

[1746] An "information processing device" is a device that has the function of inputting, processing, and outputting data, and specifically includes computers and smartphones.

[1747] "Generative AI" is a type of artificial intelligence that understands the user's intentions and performs appropriate responses and operations based on generated data.

[1748] "User intent" refers to the thoughts or goals that a user has when they want to perform a particular operation or action.

[1749] An "autonomous vehicle" is a vehicle that uses artificial intelligence and sensor technology to drive autonomously.

[1750] "Thinking" is an information processing activity that takes place in the brain and includes conscious judgments and decisions.

[1751] "Physical constraints" refers to physical disabilities or limitations that a user has that make it difficult to perform certain actions or operations.

[1752] "Virtual reality" refers to a computer-generated virtual environment or experience in which users can interact.

[1753] "Augmented reality" is a technology that overlays digital information onto a real environment, allowing users to experience the real world and digital information simultaneously.

[1754] "Multitasking" refers to performing multiple tasks or operations simultaneously, with the aim of completing tasks efficiently.

[1755] A system for implementing this invention includes a device that converts EEG signals into digital signals, a device that uses the digital signals to operate an information processing device, a device that uses generative AI to understand a user's intentions and interpret EEG signals, and a device that uses the user's thoughts to operate an autonomous vehicle.

[1756] First, the user wears a device to capture EEG signals. This device captures EEG signals in real time and converts them into digital signals. The converted digital signals are sent to an information processing device. The information processing device analyzes the digital signals using a generative AI model to understand the user's intentions. The generative AI model is trained in advance using training data and can interpret the user's thought patterns with high accuracy.

[1757] The autonomous vehicle is then controlled based on the interpreted user intent. For example, if the user thinks, "Turn right," their brainwave signal is captured and converted into a digital signal. This digital signal is analyzed and an instruction to steer the vehicle to the right is issued. This allows the user to control the autonomous vehicle with just their thoughts, without having to perform any physical operation.

[1758] The following hardware and software are used to realize this system: An EEG sensor is used as the device for capturing EEG signals. A Fourier transform algorithm is used as the software for converting them into digital signals. An MLP Classifier (Multilayer Perceptron Classifier), for example, is used as the generative AI model. A computer or smartphone is used as the information processing device. A vehicle control system is used to operate the autonomous vehicle.

[1759] As a concrete example, the following prompt sentence is shown.

[1760] "When a driver thinks 'turn right,' their brainwave signals are captured and converted into digital signals. This digital signal is then analyzed and an instruction is issued to steer the vehicle to the right."

[1761] In this way, it becomes possible to directly utilize the user's thoughts to operate an autonomous vehicle, enabling efficient and precise operation even for users with physical limitations or in multitasking environments.

[1762] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1763] Step 1:

[1764] The user wears an EEG sensor.

[1765] Input: User's EEG signal

[1766] Output: Captured EEG signal

[1767] How it works: The user wears an EEG sensor on their head to capture EEG signals in real time. The sensor detects the brain's electrical activity and outputs it as an analog signal.

[1768] Step 2:

[1769] The server converts the EEG signal into a digital signal.

[1770] Input: Captured EEG signal (analog signal)

[1771] Output: Digital signal

[1772] What it does: The server uses a Fourier transform algorithm to convert the analog signal into a digital signal, putting the EEG signal in a format that can be processed by a computer.

[1773] Step 3:

[1774] The server uses a generative AI model to analyze the digital signals and understand the user's intent.

[1775] Input: Digital signal

[1776] Output: User intent (e.g., "Turn right")

[1777] How it works: The server analyzes digital signals using a pre-trained MLPClassifier model. The model learns the user's thought patterns and can accurately predict the user's intentions from the input digital signals.

[1778] Step 4:

[1779] The server generates operating instructions for the autonomous vehicle based on the user's intentions.

[1780] Input: User intent (e.g., "turn right")

[1781] Output: Vehicle operation instructions (e.g., steer right)

[1782] Specific operation: The server receives the user's intention and generates a corresponding vehicle operation instruction. For example, if the intention is "turn right," an instruction to turn the steering wheel to the right is generated.

[1783] Step 5:

[1784] The terminal transmits vehicle operation instructions to the autonomous vehicle.

[1785] Input: Vehicle operation instructions

[1786] Output: Vehicle movement (e.g. steering wheel turns right)

[1787] Specific operation: The terminal sends the generated vehicle operation instructions to the autonomous vehicle control system, which then performs actual vehicle operation based on the received instructions.

[1788] Step 6:

[1789] Autonomous vehicles operate based on the user's intentions.

[1790] Input: Vehicle operation instructions

[1791] Output: Vehicle action (e.g. turn right)

[1792] Specific operation: The autonomous vehicle will follow the received operation instructions and perform operations such as turning the steering wheel to the right. This allows the user to operate the vehicle using only their thoughts.

[1793] Example 2

[1794] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1795] Users with physical limitations often have difficulty intuitively and efficiently operating information processing devices. Conventional interfaces also have the problem of making it difficult to accurately understand the user's intentions and execute appropriate operations. Furthermore, there is a lack of mechanisms for learning the user's thought patterns and improving the accuracy of subsequent operations.

[1796] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1797] In this invention, the server includes means for converting EEG signals into digital signals, means for operating an information processing device using the digital signals, means for understanding a user's intentions by utilizing a generative AI model and interpreting the EEG signals, means for analyzing the digital signals and generating operation commands, means for transmitting the operation commands to the information processing device, means for the information processing device to perform operations in accordance with the operation commands, and means for learning the user's thought patterns, thereby enabling users with physical limitations to intuitively and efficiently operate the information processing device using EEG signals.

[1798] "EEG signals" are signals that reflect the electrical activity of the user's brain.

[1799] A "digital signal" is a signal that is an analog signal converted into digital form.

[1800] An "information processing device" is a device that receives and analyzes digital signals and performs operations based on the user's intentions.

[1801] A "generative AI model" is a model that uses artificial intelligence technology to understand user intent and interpret brainwave signals.

[1802] An "operation command" is an instruction to cause an information processing device to perform a specific operation.

[1803] A "thought pattern" is a characteristic pattern of brain wave signals that occurs when a user performs a specific operation.

[1804] "Wireless communication" is a means of sending and receiving data using wireless technologies such as Wi-Fi and Bluetooth.

[1805] "Analysis" is the process of interpreting the user's intent based on the received digital signal.

[1806] "Learning" is the process of recording the user's past brainwave signals and corresponding operations to improve the accuracy of future operations.

[1807] MODE FOR CARRYING OUT THE INVENTION

[1808] This invention is a system that enables users with physical limitations to intuitively and efficiently operate information processing devices. The system includes a series of processes that convert EEG signals into digital signals, understand the user's intentions using a generative AI model, and operate the information processing device.

[1809] Hardware and software used

[1810] Hardware:

[1811] 1. EEG measurement device: A device for acquiring the user's brainwave signals. A specific example is the EEG measurement device "Epoc+."

[1812] 2. Information processing device: A device that receives and analyzes digital signals and performs actions based on the user's intentions. A good example is a smartphone.

[1813] 3. Server: A device for analyzing digital signals and generating operational commands.

[1814] software:

[1815] 1. Generative AI model: An artificial intelligence model that understands user intent and interprets EEG signals. An example of a generative AI model is the GPT-4.

[1816] Data processing and calculation

[1817] Acquisition and transformation of EEG signals:

[1818] When a user wears an EEG measuring device and thinks about a specific operation, the device acquires an EEG signal, which is then converted into a digital signal by a signal processing unit within the device.

[1819] Digital signal transmission and analysis:

[1820] The device (EEG measurement device) transmits the converted digital signal to a server via wireless communication (Wi-Fi or Bluetooth), which then analyzes the digital signal using a generative AI model to interpret the user's intent.

[1821] Generate and send operation commands:

[1822] The server generates an operation command for the information processing device based on the analysis result, and transmits the generated operation command to the information processing device via wireless communication.

[1823] Execution of operations by information processing device:

[1824] The information processing device executes an operation in accordance with the received operation command. For example, when a smartphone receives a command to tap the "home" button, the smartphone taps the "home" button.

[1825] Learning Thought Patterns:

[1826] The server records the user's EEG signals and the corresponding actions, and uses a generative AI model to learn the user's thought patterns, allowing for more accurate actions the next time the same thought is used.

[1827] Specific examples

[1828] As a concrete example, consider the case where a user thinks about tapping the "Home" button on a smartphone. When the user thinks about tapping the "Home" button, an EEG measurement device captures the EEG signal and converts it into a digital signal. This digital signal is sent to a server, which uses a generative AI model to analyze the user's intention. Based on the analysis results, the server generates a command to tap the "Home" button and sends it to the smartphone. The smartphone then taps the "Home" button according to the command.

[1829] Example prompt sentence:

[1830] "When a user thinks about tapping the home button on their smartphone, please generate a program that analyzes the brainwave signal and sends a command to the smartphone to tap the home button."

[1831] In this way, a physically limited user can intuitively and efficiently operate an information processing device using an electroencephalogram signal.

[1832] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1833] Step 1:

[1834] A user wears an electroencephalogram (EEG) measuring device and thinks about a specific operation, and the EEG measuring device acquires an electroencephalogram signal from the user.

[1835] Input: User's EEG signal

[1836] Output: Captured EEG signal

[1837] Step 2:

[1838] The terminal (EEG measurement device) converts the acquired EEG signal into a digital signal. This conversion process is performed by a signal processing unit within the EEG measurement device.

[1839] Input: Acquired EEG signal

[1840] Output: Digital signal

[1841] Step 3:

[1842] The terminal transmits the converted digital signal to the server via wireless communication (Wi-Fi or Bluetooth).

[1843] Input: Digital signal

[1844] Output: Digital signal sent to the server

[1845] Step 4:

[1846] The server uses a generative AI model to analyze the received digital signals and interpret the user's intentions from their brainwave signals.

[1847] Input: Digital signal sent to the server

[1848] Output: Analysis of user intent

[1849] Step 5:

[1850] The server generates an operation command for the information processing device based on the analysis result. For example, if the user thinks about tapping the "Home" button, the server generates a command to tap the "Home" button.

[1851] Input: Analysis results regarding user intent

[1852] Output: Operation command

[1853] Step 6:

[1854] The server transmits the generated operation command to the information processing device via wireless communication.

[1855] Input: Operation command

[1856] Output: Operation commands sent to the information processing device

[1857] Step 7:

[1858] The information processing device executes an operation in accordance with the received operation command. For example, when a command to tap the "home" button is received, the information processing device taps the "home" button.

[1859] Input: Operation command sent to the information processing device

[1860] Output: Actions taken

[1861] Step 8:

[1862] The server records the user's EEG signals and the corresponding actions, and uses a generative AI model to learn the user's thought patterns, allowing them to perform more accurate actions with the same thoughts next time.

[1863] Input: Records of the user's EEG signals and corresponding actions

[1864] Output: Learned thought patterns

[1865] (Application example 2)

[1866] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1867] Conventional factory robots have been difficult for workers with physical limitations to operate, resulting in inefficient work. Furthermore, the complexity of operation has been an issue even when performing multitasking using multiple interfaces. There is a need to provide a system that can solve these problems and enable workers with physical limitations to operate factory robots efficiently.

[1868] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1869] In this invention, the server includes means for converting EEG signals into digital signals, means for operating a smartphone using the digital signals, means for understanding a user's intentions using generative AI and interpreting the EEG signals, and means for operating a factory robot using the interpreted EEG signals, thereby enabling even physically limited workers to efficiently operate factory robots using EEG signals.

[1870] "EEG signals" are electrical signals generated based on the user's brain activity.

[1871] A "digital signal" is a signal that has been converted from an analog signal into a digital format that can be processed by a computer or other electronic device.

[1872] A "smartphone" is a multi-functional mobile device that can connect to the Internet and run applications in addition to the functions of a mobile phone.

[1873] "Generative AI" is a type of artificial intelligence technology that understands user intent and generates models to perform specific tasks.

[1874] "User intent" refers to the thoughts or goals that a user has when they want to perform a particular operation or action.

[1875] "Means for interpretation" refers to a method or device for analyzing acquired signals or data and understanding their meaning or intent.

[1876] A "factory robot" is a mechanical device used to automate work within a factory, and can manage production lines and perform maintenance work.

[1877] "Means for operating" refers to a method or device for controlling a specific device or system to perform a desired operation.

[1878] "Physical limitations" refers to a condition in which a user has difficulty performing a particular action or operation due to physical reasons.

[1879] "Virtual reality" refers to a virtual environment or experience generated using computer technology in which users can interact.

[1880] "Augmented reality" is a technology that overlays digital information onto a real environment, allowing users to experience the real world and digital information simultaneously.

[1881] A "vehicle" is a means of transportation for transporting people or goods, and includes automobiles, trains, etc.

[1882] "Multitasking" refers to the simultaneous execution of multiple tasks, and is a state in which a user can perform multiple operations or tasks in parallel.

[1883] The system for implementing this invention converts EEG signals into digital signals, interprets the user's intentions using generative AI, and operates a factory robot. A specific embodiment of this system is described below.

[1884] First, the user wears an EEG sensor, which captures the user's brainwave signals in real time and converts them into digital signals, which are then sent to a server.

[1885] The server performs data preprocessing to standardize the acquired digital signals. Specifically, it standardizes the signals using libraries such as StandardScaler. Next, it uses a generative AI model to analyze the standardized signals and interpret the user's intentions. This generative AI model is pre-trained and can generate specific operation commands from the user's EEG signals.

[1886] The interpreted operation command is sent to the factory robot via serial communication. The factory robot performs the specified action based on the received command. For example, if a user thinks, "Lift a part," the generative AI model interprets this intention and generates the command "Lift." The factory robot receives this command and actually performs the action of lifting the part.

[1887] As a concrete example of this system, the following prompt sentence can be input into the generative AI model:

[1888] "When the user thinks 'I want to lift a part,' the brainwave signal is analyzed and a 'lift' command is sent to the robot arm."

[1889] In this way, even physically challenged workers can efficiently operate factory robots using EEG signals. It can also be applied to multitasking using interfaces such as virtual reality, augmented reality, and vehicles, so it is expected to be used in a wide range of applications.

[1890] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1891] Step 1:

[1892] The user wears an EEG sensor, which captures the user's EEG signals in real time. The input is the user's EEG signal, and the output is a digital signal. The EEG sensor converts the analog signal into a digital signal.

[1893] Step 2:

[1894] The terminal transmits the acquired digital signal to the server. The input is a digital signal, and the output is data transmission to the server. The terminal transmits the digital signal to the server using wireless or wired communication.

[1895] Step 3:

[1896] The server normalizes the received digital signal. The input is a digital signal and the output is a normalized signal. The server normalizes the signal using a library such as StandardScaler.

[1897] Step 4:

[1898] The server uses a generative AI model to analyze the standardized signals and interpret the user's intentions. The input is the standardized signal, and the output is the operation command. The server uses a pre-trained generative AI model to analyze the signal and generate the operation command.

[1899] Step 5:

[1900] The server sends the generated operation command to the factory robot via serial communication. The input is the operation command and the output is the command sent to the factory robot. The server sends the operation command to the factory robot using serial communication.

[1901] Step 6:

[1902] The factory robot executes the specified operation based on the received operation command. The input is the operation command and the output is the actual operation. The factory robot performs an operation such as lifting a part according to the received command.

[1903] Example 3

[1904] Next, a third embodiment of the third embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1905] Conventional methods for operating information processing devices require users to use their hands or voice, making operation particularly difficult for users who are driving or have physical limitations. Efficient operation is also required in multitasking environments where multiple tasks must be performed simultaneously. To solve these issues, a system is needed that directly reflects the user's thoughts in the operation of the information processing device.

[1906] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[1907] In this invention, the server includes means for converting EEG signals into digital signals, means for operating an information processing device using the digital signals, means for understanding a user's intentions by utilizing generative AI and interpreting the EEG signals, and means for learning the user's thought patterns and performing subsequent operations more accurately. This enables a user to operate an information processing device using only their thoughts, without using their hands or voice, enabling efficient operation while driving, for users with physical limitations, and even in a multitasking environment.

[1908] "Electroencephalogram signals" are signals indicative of the electrical activity of a user's brain.

[1909] A "digital signal" is a signal that is an analog signal converted into digital form.

[1910] An "information processing device" is an electronic device for processing and manipulating data.

[1911] "Generative AI" is an artificial intelligence technology that understands user intent and interprets brainwave signals.

[1912] "User intent" is the goal or desire expressed by the user through their thoughts.

[1913] A "thought pattern" is a characteristic pattern of brain wave signals when a user performs a specific purpose or operation.

[1914] "Virtual reality" is a virtual environment created using computer technology.

[1915] "Augmented reality" is a technology that overlays digital information onto the real environment.

[1916] "Multitasking" refers to performing multiple tasks simultaneously.

[1917] MODE FOR CARRYING OUT THE INVENTION

[1918] This invention relates to a system that allows a user to operate an information processing device simply by thinking. Specifically, the system converts electroencephalogram signals into digital signals, analyzes the digital signals to understand the user's intentions, and then operates the information processing device.

[1919] Hardware and software used

[1920] Hardware: EEG sensors, information processing devices (e.g., navigation systems)

[1921] Software: EEG signal analysis software, generative AI model

[1922] System Overview

[1923] 1. EEG signal acquisition:

[1924] A user wants to operate an information processing device.

[1925] The terminal (brainwave sensor) acquires the user's brainwave signals in real time.

[1926] 2. Conversion to digital signal:

[1927] The brainwave signal acquired by the terminal (brainwave sensor) is converted into a digital signal.

[1928] 3. Digital signal analysis:

[1929] The server receives the digital signal and analyzes it using EEG signal analysis software.

[1930] The server inputs the analysis results into a generative AI model to understand the user's intentions.

[1931] 4. Operation of information processing devices:

[1932] The server transmits the analysis results to the information processing device.

[1933] The terminal (information processing device) performs an operation based on the received information.

[1934] 5. Generative AI Learning:

[1935] The server uses a generative AI model to learn the user's thought patterns.

[1936] The server updates the model for future operations.

[1937] Specific examples

[1938] For example, if a user thinks, "I want to go to Tokyo Tower" while driving a car, the EEG sensor picks up the EEG signal and converts it into a digital signal. The server analyzes the digital signal and identifies the destination, "Tokyo Tower." The navigation system receives this information and sets the destination to "Tokyo Tower." The generative AI learns this series of operations, and from then on, simply by thinking, "I want to go to Tokyo Tower," the destination will be set more quickly and accurately.

[1939] Prompt Sentence Examples

[1940] "Describe a system that captures brainwave signals when a user thinks about a destination while driving a car, converts them into digital signals, and sends them to a navigation system."

[1941] This system allows the user to operate the information processing device without using their hands while driving, thereby improving safety and convenience. The flow of the identification process in the third embodiment will be described with reference to FIG.

[1942] Step 1:

[1943] A user wants to operate an information processing device.

[1944] Specific operation: The user thinks, "I want to go to Tokyo Tower."

[1945] Input: User thoughts

[1946] Output: EEG signal

[1947] Step 2:

[1948] The terminal (brainwave sensor) acquires the user's brainwave signals in real time.

[1949] What it does: The EEG sensor captures the user's brainwave signals.

[1950] Input: User's EEG signal

[1951] Output: Analog EEG signal

[1952] Step 3:

[1953] The analog brainwave signal acquired by the terminal (brainwave sensor) is converted into a digital signal.

[1954] Specific operation: The EEG sensor converts analog signals into digital signals and outputs them as data.

[1955] Input: Analog EEG signal

[1956] Output: Digital EEG signal

[1957] Step 4:

[1958] The server receives the digital EEG signals and analyzes them using EEG signal analysis software.

[1959] What it does: The server analyzes the digital signals to determine the user's intent.

[1960] Input: Digital EEG signal

[1961] Output: Analysis result (user intent)

[1962] Step 5:

[1963] The server inputs the analysis results into a generative AI model to understand the user's intentions.

[1964] Specific operation: The generative AI model interprets the user's intention based on the analysis results.

[1965] Input: Analysis result (user intent)

[1966] Output: Interpreted user intent

[1967] Step 6:

[1968] The server transmits the interpreted user's intention to the information processing device.

[1969] Specific operation: The server transmits the interpreted intention to the information processing device.

[1970] Input: Interpreted user intent

[1971] Output: Instructions to the information processing device

[1972] Step 7:

[1973] The terminal (information processing device) executes an operation based on the received instruction.

[1974] Specific operation: The information processing device performs operations such as setting a destination.

[1975] Input: Instructions to the information processing device

[1976] Output: The action taken (e.g., set destination)

[1977] Step 8:

[1978] The server uses a generative AI model to learn the user's thought patterns.

[1979] Specific operation: The generative AI model learns the user's thought patterns and performs operations more accurately from the next time onwards.

[1980] Input: The operations performed and the user's thought patterns

[1981] Output: An updated generative AI model

[1982] (Application example 3)

[1983] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1984] Conventional navigation systems require users to manually input their destination, making operation difficult, especially for those driving or those with physical limitations. Furthermore, when performing multiple tasks simultaneously, the system becomes increasingly complicated and places a strain on the user, limiting the use of navigation systems.

[1985] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[1986] In this invention, the server includes a means for converting EEG signals into digital signals, a means for using the digital signals to operate the navigation system of the autonomous vehicle, and a means for understanding the user's intentions and interpreting the EEG signals using generative AI. This allows the user to set a destination in the navigation system of the autonomous vehicle simply by thinking, making it easy to operate even while driving or for users with physical limitations. Furthermore, even when performing multiple tasks simultaneously, the complexity of operation is reduced, significantly reducing the burden on the user.

[1987] "EEG signals" are electrical signals generated in conjunction with brain activity, and reflect the user's thoughts and emotions.

[1988] A "digital signal" is a digitalized analog signal that can be processed by computers and other electronic devices.

[1989] An "autonomous vehicle" is a vehicle that uses artificial intelligence and sensor technology to drive autonomously, without the need for human operation.

[1990] A "navigation system" is a system that guides you to your destination and provides the optimal route based on map information and current location information.

[1991] "Generative AI" is artificial intelligence that understands the user's intentions and thoughts and generates appropriate responses and operations, using machine learning and natural language processing technologies.

[1992] "User intent" is the idea or goal of what the user wants or is trying to do.

[1993] "Interpreting" means analyzing input information and understanding its meaning and intent.

[1994] The following system configuration will be described as an embodiment of the present invention.

[1995] System Configuration

[1996] This system includes a means for converting EEG signals into digital signals, a means for using the digital signals to operate the navigation system of an autonomous vehicle, and a means for using generative AI to understand the user's intentions and interpret the EEG signals.

[1997] Hardware

[1998] EEG sensor: A device that captures the user's brainwave signals.

[1999] Navigation systems for autonomous vehicles: Navigation systems installed in autonomous vehicles.

[2000] software

[2001] Generative AI model: Software that converts EEG signals into digital signals and interprets user intent.

[2002] Processing flow

[2003] 1. Capturing EEG signals: The EEG sensor captures the user's EEG signals.

[2004] 2. Conversion to digital signals: The generative AI model converts the captured EEG signals into digital signals.

[2005] 3. Digital signal analysis: Generative AI models analyze digital signals to interpret user intent and identify destinations.

[2006] 4. Destination setting: The autonomous vehicle's navigation system sets the specified destination.

[2007] Specific examples

[2008] For example, if a user thinks, "I want to go to Tokyo Tower," the EEG sensor captures the EEG signal, and the generative AI model analyzes the signal to identify the destination, "Tokyo Tower." The navigation system of the autonomous vehicle receives this information and sets "Tokyo Tower" as the destination in the vehicle's navigation system.

[2009] Prompt Sentence Examples

[2010] Analyze the EEG signals when the user thinks, "I want to go to Tokyo Tower," and generate a prompt that identifies the destination.

[2011] In this way, the brainwave navigation system can read the user's thoughts and set a destination, making it easier to operate while driving or for users with physical limitations, and reducing the complexity of operation when performing multiple tasks at the same time.

[2012] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[2013] Step 1:

[2014] Capturing EEG signals

[2015] The user wears the EEG sensor and thinks. The EEG sensor captures the user's EEG signals in real time. The input is the user's EEG signals, and the output is the captured raw EEG data. This data is in analog form and is converted into a digital signal in the next step.

[2016] Step 2:

[2017] Conversion to digital signals

[2018] The server receives the captured raw EEG data and converts it into a digital signal using a generative AI model. The input is raw EEG data, and the output is a digital signal. This conversion includes data processing such as filtering and noise removal of the EEG data. Specifically, the EEG data is sampled and quantized.

[2019] Step 3:

[2020] Digital Signal Analysis

[2021] The server uses a generative AI model to analyze digital signals and interpret the user's intent. The input is the digital signal, and the output is the user's intent (e.g., the name of a destination). This analysis involves pattern recognition using machine learning algorithms. Specific operations include feature extraction and classification of the digital signal.

[2022] Step 4:

[2023] Destination setting

[2024] The server sends the analyzed user intention to the navigation system of the autonomous vehicle. The input is the user intention (the name of the destination), and the output is the destination set in the navigation system. Specifically, the server calls the navigation system's API to set the destination.

[2025] Step 5:

[2026] Start navigation

[2027] The navigation system of an autonomous vehicle calculates the optimal route based on the set destination and starts navigation. The input is the set destination and the output is the navigation route. Specific operations include searching map data and calculating the route.

[2028] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[2029] "Example 1"

[2030] In one embodiment of the present invention, a system is provided that includes a means for converting EEG signals into digital signals, a means for operating a smartphone using the digital signals, a means for understanding a user's intentions and interpreting the EEG signals using generative AI, and an emotion engine that recognizes the user's emotions. Specifically, when the user is feeling joy, the system can operate the smartphone in accordance with the user's emotions, such as by brightening the color tone of the smartphone screen.

[2031] "Example 2"

[2032] In another embodiment of the present invention, the emotion engine provides a means for executing smartphone operations according to the user's emotions. Specifically, when the user feels angry, the smartphone can be operated according to the user's emotions, such as temporarily stopping smartphone notifications.

[2033] "Example 3"

[2034] In a further embodiment of the present invention, the emotion engine provides a means for adjusting the interface in response to the user's emotion, for example, by lowering the volume of the smartphone when the user is feeling sad.

[2035] The processing flow of each embodiment will be described below.

[2036] "Example 1"

[2037] Step 1: Acquire the user's EEG signal.

[2038] Step 2: Convert the acquired EEG signal into a digital signal.

[2039] Step 3: Use generative AI to interpret user intent from digital signals.

[2040] Step 4: Use the emotion engine to recognize the user's emotions.

[2041] Step 5: Operate the smartphone based on the recognized emotion. For example, if the user is feeling happy, the smartphone screen will brighten up.

[2042] "Example 2"

[2043] Step 1: Acquire the user's EEG signal.

[2044] Step 2: Convert the acquired EEG signal into a digital signal.

[2045] Step 3: Use generative AI to interpret user intent from digital signals.

[2046] Step 4: Use the emotion engine to recognize the user's emotions.

[2047] Step 5: Implement smartphone actions based on the recognized emotion. For example, temporarily suspend smartphone notifications if the user is feeling angry.

[2048] "Example 3"

[2049] Step 1: Acquire the user's EEG signal.

[2050] Step 2: Convert the acquired EEG signal into a digital signal.

[2051] Step 3: Use generative AI to interpret user intent from digital signals.

[2052] Step 4: Use the emotion engine to recognize the user's emotions.

[2053] Step 5: Adjust the smartphone interface based on the recognized emotion, for example, lowering the volume of the smartphone when the user feels sad.

[2054] Example 1

[2055] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[2056] Conventional methods for operating information processing devices require users to use their hands or voice, which makes operation difficult for users with physical limitations or in multitasking environments with interfaces such as virtual reality, augmented reality, or in vehicles. Furthermore, the inability to accurately understand the user's intentions and emotions and operate accordingly poses a problem of not improving the user experience.

[2057] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[2058] In this invention, the server includes means for converting EEG signals into digital signals, means for operating an information processing device using the digital signals, means for understanding a user's intentions and interpreting EEG signals by making full use of generative artificial intelligence, and means for recognizing the user's emotions. This makes it possible to operate the information processing device based on the user's thoughts and emotions, enabling intuitive and efficient operation even for users with physical limitations or in a multitasking environment.

[2059] "Electroencephalogram signals" are weak electrical signals generated by electrical activity in the brain.

[2060] A "digital signal" is a signal that has been converted from an analog signal into a digital form that can be processed by a computer or electronic device.

[2061] An "information processing device" is an electronic device that has the function of inputting, processing, and outputting data, and specifically refers to smartphones, computers, etc.

[2062] "Generative AI" is an artificial intelligence technology that understands the user's intentions and thoughts and generates appropriate responses and operations.

[2063] "User intent" refers to the thoughts or goals that a user has when they want to perform a particular operation or action.

[2064] "User emotion" refers to the psychological state a user feels in response to a particular situation or stimulus, including joy, sadness, anger, etc.

[2065] "Means for recognizing emotions" refers to technologies and devices for detecting and identifying a user's emotions.

[2066] "Thought patterns" are specific patterns of brain waves that are generated when a user has a specific thought or intention.

[2067] "Analysis" is the process of processing a digital signal and extracting useful information from it.

[2068] An "operation instruction" is a signal or command that instructs an information processing device to perform a specific operation.

[2069] This invention is a system that converts a user's electroencephalogram (EEG) signals into digital signals and uses these digital signals to operate an information processing device. Furthermore, it utilizes generative artificial intelligence to understand the user's intentions and interpret the EEG signals. It can also recognize the user's emotions and perform operations accordingly.

[2070] Hardware and software used

[2071] Hardware

[2072] 1. EEG headset: A device that captures the user's brainwave signals and converts them into digital signals. Specifically, it is a device with sensors that is worn on the scalp.

[2073] 2. Information processing device: A device such as a smartphone or computer that receives digital signals and performs operations based on the analysis results.

[2074] 3. Server: A device that receives digital signals and interprets their intent using analytical and generative artificial intelligence.

[2075] software

[2076] 1. Generative AI model: A machine learning model to understand user intent and interpret EEG signals. It uses techniques such as neural networks.

[2077] 2. Signal analysis algorithm: An algorithm that analyzes digital signals and infers the user's intentions. Specifically, it uses FFT (Fast Fourier Transform) and machine learning algorithms.

[2078] 3. Emotion recognition engine: Software that recognizes the user's emotions and determines the appropriate operation.

[2079] Specific examples

[2080] Example 1: Sending email

[2081] When a user thinks about "sending an email," the following process takes place:

[2082] 1. The user thinks, "I want to send an email."

[2083] 2. The EEG headset captures the brainwave signals and converts them into digital signals.

[2084] 3. The device sends a digital signal to the server.

[2085] 4. The server analyzes the digital signals and uses a generative artificial intelligence model to interpret the user's intent as "send an email."

[2086] 5. The server sends an instruction to the information processing device (smartphone) to launch the email application and display a new email creation screen.

[2087] 6. Follow the instructions received on your smartphone to launch the email app and display the new email composition screen.

[2088] Example 2: Emotional manipulation

[2089] If the user is happy, the following process is performed:

[2090] 1. Users feel delighted.

[2091] 2. The EEG headset captures the brainwave signals and converts them into digital signals.

[2092] 3. The device sends a digital signal to the server.

[2093] 4. The server analyzes the digital signal and uses an emotion recognition engine to identify the user's emotion as "joy."

[2094] 5. The server sends an instruction to the information processing device (smartphone) to brighten the screen color tone.

[2095] 6. Follow the instructions received by your smartphone to brighten the screen color tone.

[2096] Prompt Sentence Examples

[2097] "Design a system that, when a user has a specific thought, executes an operation on an information processing device that corresponds to that thought. For example, if a user thinks, 'I want to send an email,' the system should recognize that intention and send an instruction to the information processing device to launch the email application. Also, add a function that recognizes the user's emotions and performs an operation according to those emotions. For example, if the user is feeling happy, make the color tone of the information processing device screen brighter."

[2098] The flow of the identification process in the first embodiment will be described with reference to FIG.

[2099] Step 1:

[2100] The user performs a specific thought.

[2101] Input: User thoughts (e.g., "send email")

[2102] Output: EEG signal

[2103] Specific operation: When a user thinks about "sending an email," electrical activity occurs in the brain, generating an EEG signal.

[2104] Step 2:

[2105] The terminal (EEG headset) captures the brainwave signal and converts it into a digital signal.

[2106] Input: EEG signal

[2107] Output: Digital signal

[2108] How it works: The EEG headset's sensors contact the user's scalp and measure brain waves in real time. The measured analog signal is converted to a digital signal using an ADC (analog-to-digital converter).

[2109] Step 3:

[2110] The terminal transmits a digital signal to the server.

[2111] Input: Digital signal

[2112] Output: Send data to the server

[2113] What it does: The device uses Wi-Fi or Bluetooth to send a digital signal to the server.

[2114] Step 4:

[2115] The server analyzes the received digital signal.

[2116] Input: Digital signal

[2117] Output: Analysis results (frequency components, etc.)

[2118] Specific operation: The server analyzes the frequency components of the digital signal using FFT (Fast Fourier Transform), and extracts the signal features as a result of the analysis.

[2119] Step 5:

[2120] The server uses a generative artificial intelligence model to interpret the user's intent.

[2121] Input: Analysis results

[2122] Output: User intent (e.g. "send email")

[2123] Specific operation: The server uses a machine learning model (e.g., a neural network) to infer the user's intent from the analysis results. The inferred intent is "send an email."

[2124] Step 6:

[2125] The server determines an operation to be performed on the information processing device based on the estimated user's intention.

[2126] Input: User intent

[2127] Output: Operation instructions (e.g., launch the email app)

[2128] Specific operation: The server determines the corresponding operation based on the user's intention. For example, if the intention is to "send an email," it determines the operation to launch the email app.

[2129] Step 7:

[2130] The server transmits an operation instruction to the information processing device.

[2131] Input: Operation instructions

[2132] Output: Data transmission to information processing device

[2133] Specific operation: The server uses the API of the information processing device (smartphone) to send operation instructions.

[2134] Step 8:

[2135] The information processing device executes the operation according to the received instruction.

[2136] Input: Operation instructions

[2137] Output: Operation result (e.g. launching email app)

[2138] Specific operation: According to the operation instructions received by the smartphone, the email app is launched and a new email creation screen is displayed.

[2139] Step 9:

[2140] The device recognizes the user's emotions.

[2141] Input: EEG signal

[2142] Output: Emotional information (e.g., joy)

[2143] How it works: The EEG headset captures brainwave signals, and the emotion recognition engine analyzes the signals to recognize the user's emotions.

[2144] Step 10:

[2145] The device transmits the recognized emotion information to the server.

[2146] Input: Emotion information

[2147] Output: Send data to the server

[2148] Specific operation: The device uses Wi-Fi or Bluetooth to send emotional information to the server.

[2149] Step 11:

[2150] The server determines the operation of the information processing device based on the emotion information.

[2151] Input: Emotion information

[2152] Output: Operation instructions (e.g., brighten the screen color)

[2153] Specific operation: The server determines the corresponding operation based on the emotion information. For example, if the user feels happy, it determines the operation to brighten the color tone of the screen.

[2154] Step 12:

[2155] The server transmits an operation instruction according to the emotion to the information processing device.

[2156] Input: Operation instructions

[2157] Output: Data transmission to information processing device

[2158] Specific operation: The server uses the API of the information processing device (smartphone) to send operation instructions.

[2159] Step 13:

[2160] The information processing device executes the operation according to the received instruction.

[2161] Input: Operation instructions

[2162] Output: Operation result (e.g., change in screen color)

[2163] Specific operation: The smartphone brightens the screen color according to the operation instructions received.

[2164] (Application example 1)

[2165] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[2166] Conventional methods for operating information ...

Claims

1. A means for converting an electroencephalogram signal representing a thought pattern of a user who is about to operate an information processing device into a digital signal; A means for generating a prompt sentence for input to the generative AI model based on the converted digital signal; a means for inputting the generated prompt sentence into the generative AI model and acquiring an operation command for operating the information processing device from the generative AI model; means for transmitting the acquired operation command to the information processing device; A system including:

2. 2. The system according to claim 1, further comprising: means for analyzing the emotion of the user using an emotion engine, and adjusting an operation command for operating the information processing device based on the analyzed emotion of the user.

3. The system according to claim 1 , wherein the generative AI model learns a user's thought patterns and operation contents of an information processing device as teaching data.

Images