System

Abstract

A system is provided.SOLUTION: A system comprising: means for collecting brainwave data of a user in real time; means for pre-processing the brainwave data and extracting specific features; means for transmitting the pre-processed brainwave data to a remote server; means for analyzing the pre-processed brainwave data and identifying a thought of the user in the remote server; means for generating a specific image based on the thought; means for transmitting the generated image to a terminal of the user; and means for displaying the transmitted image on a display in the terminal.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional technology provides very limited communication tools for patients who cannot speak or write, or for people who have difficulty communicating. In such cases, it is difficult for users to express the images and thoughts they have in their minds to the outside world, resulting in a lack of communication tools. This presents a challenge in the fields of medicine and nursing care, making it difficult for patients who have difficulty expressing their wishes to receive appropriate care and treatment. The present invention aims to solve these challenges and provide a new communication tool that allows users to share their thoughts and images with others in a tangible form. [Means for solving the problem]

[0005] The present invention solves the above-mentioned problems by including the following means: a system including means for collecting a user's electroencephalogram data in real time, means for preprocessing the electroencephalogram data to extract specific features, means for transmitting the preprocessed electroencephalogram data to a remote server, means for analyzing the preprocessed electroencephalogram data in the remote server and identifying the user's thoughts, means for generating a concrete image based on the thoughts, means for transmitting the generated image to the user's terminal, and means for displaying the transmitted image on a display in the terminal. This allows even users who have difficulty speaking or writing to express their thoughts in concrete visual form and share them with others.

[0006] "User" means an entity that uses the system and provides brainwave data.

[0007] "Electroencephalogram data" is data obtained by measuring the electrical activity generated by the user's brain.

[0008] "Real-time" means that the entire process of data collection, processing, and transmission occurs instantly.

[0009] "Means of collection" refers to physical devices or systems for acquiring EEG data, such as EEG caps or EEG measurement devices.

[0010] "Preprocessing" is the process of cleaning the acquired EEG data, removing noise, filtering, and extracting features.

[0011] "Features" are important data points identified from preprocessed data that indicate the user's state or thoughts.

[0012] A "remote server" is a central server that performs data analysis and data generation and is accessible from a remote location via the Internet or a dedicated network.

[0013] "Analysis" is the process of processing received brainwave data to identify the user's thoughts and emotions.

[0014] "Images" refer to specific images or visuals generated based on the user's thoughts and emotions.

[0015] "Means of generation" refers to artificial intelligence technology and image generation models that convert the user's thoughts into concrete images.

[0016] "Transmission means" refers to the communications protocol or technology used to transmit the processed data or generated images from the remote server to the user's terminal.

[0017] A "terminal" is a device or system that collects and preprocesses a user's EEG data, communicates with a remote server, and displays the generated images.

[0018] "Display" refers to a screen or monitor used to visually present the generated image.

[0019] "Artificial intelligence technology" refers to computer technologies for automating intellectual tasks, including machine learning and deep learning.

[0020] "Emotional state" is a feature that indicates the user's psychological and emotional state and is identified through analysis. [Brief explanation of the drawings]

[0021] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4]FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0022] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0023] First, the terms used in the following description will be explained.

[0024] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0025] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0026] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0027] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0028] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0029] [First embodiment]

[0030] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0031] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0032] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0033] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0034] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0035] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0036] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0037] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0038] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0039] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0040] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0041] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0042] The present invention relates to a system that acquires images such as a scene envisioned by a user from electroencephalogram data in real time and displays them as concrete images on a display. The configuration and operation of this system will be specifically described below.

[0043] System Configuration

[0044] 1. User device (terminal)

[0045] Users wearing an EEG cap or EEG measuring device provide brainwave data.

[0046] The device measures brain waves in real time and collects initial data.

[0047] 2. Intermediate processing device (terminal)

[0048] After initial collection of EEG data, preprocessing is performed to remove noise and filter it, and to extract features.

[0049] The pre-processed data is encrypted and sent to a remote server.

[0050] 3. Remote Analysis Server (Server)

[0051] The received brainwave data is analyzed to determine what the user was thinking.

[0052] Using AI (artificial intelligence) technology, concrete images are generated based on the content of thoughts.

[0053] 4. Generated image transmission device (server)

[0054] The generated image data is sent to the user's terminal.

[0055] 5. Display Device (Terminal)

[0056] The received image data is displayed on the user's terminal.

[0057] Program processing flow (natural language explanation)

[0058] 1. Data Collection and Preprocessing

[0059] The user wears an EEG cap and conjures up a specific image.

[0060] The device collects the user's brainwave data in real time and stores the initial data in memory.

[0061] The device performs noise removal and filtering to extract specific features.

[0062] 2. Data transmission

[0063] The device encrypts the pre-processed EEG data and transmits it securely to a remote server.

[0064] 3. Data Analysis

[0065] The server analyzes the received data and identifies what the user was thinking, using AI models such as neural networks.

[0066] The server generates specific images based on the identified thoughts. For example, if you think of a seascape, it generates a detailed image corresponding to it.

[0067] 4. Sending and displaying the generated image

[0068] The server transmits the generated image data to the user's terminal.

[0069] The image data received by the terminal is displayed on the display at an appropriate resolution.

[0070] Specific examples

[0071] Example 1: A user with physical limitations visualizes a scene

[0072] 1. User A, who has physical limitations, imagines a scene of a quiet lake.

[0073] 2. The device collects Person A's EEG data and performs noise removal and feature extraction.

[0074] 3. The device sends the preprocessed data to the remote server.

[0075] 4. The server analyzes the data and recognizes that Person A is thinking of a lake.

[0076] 5. The server uses image generation AI to convert the tranquil lake scene into a concrete image.

[0077] 6. The server sends the generated image to Mr. A's device.

[0078] 7. The device displays the received image on the screen, showing the lake scenery that Mr. A had in mind.

[0079] Example 2: Researchers visualize thoughts by analyzing subjects' brain waves

[0080] 1. User research subject B imagines a complex geometric pattern.

[0081] 2. The device collects Mr. B's EEG data and performs preprocessing.

[0082] 3. The device sends the preprocessed data to the remote server.

[0083] 4. The server analyzes the data and recognizes that Person B is thinking of a geometric pattern.

[0084] 5. The server generates a geometric pattern based on the analysis results.

[0085] 6. The server sends the generated image data to the researcher's computer.

[0086] 7. The device displays geometric patterns on a screen, allowing researchers to observe what the subject is thinking.

[0087] In this way, the present invention generates a concrete image of a user's thoughts and images from electroencephalogram data, providing a new means of communication.

[0088] The processing flow will be explained below.

[0089] Step 1:

[0090] The user wears an EEG cap and conjures up a specific image.

[0091] The EEG cap is worn properly and the user is made aware of the scenery or image they have in mind.

[0092] Step 2:

[0093] The device collects the user's brainwave data in real time.

[0094] An electroencephalogram (EEG) measuring device captures the user's brainwave activity and stores the data in memory.

[0095] Step 3:

[0096] The device preprocesses the collected EEG data.

[0097] Clean up the data through a noise removal filter.

[0098] Features are extracted using algorithms such as the fast Fourier transform (FFT).

[0099] Step 4:

[0100] The terminal encrypts the pre-processed data and transmits it to a remote server.

[0101] Data is encrypted using secure communication protocols (e.g. SSL / TLS).

[0102] Sends the encrypted data to the remote server and waits for an acknowledgment (ACK) from the server.

[0103] Step 5:

[0104] The server analyzes the received brain wave data.

[0105] Use AI models (e.g., deep learning neural networks) to identify what users are thinking.

[0106] Based on the analysis results, the image or scenery that the user has in mind is identified.

[0107] Step 6:

[0108] The server generates a concrete image based on the identified thought content.

[0109] Generative AI (e.g., Generative Adversarial Network, GAN) is used to create specific images based on specific content.

[0110] The image is subjected to high resolution and filtering processing as necessary.

[0111] Step 7:

[0112] The server sends the generated image to the user's terminal.

[0113] Convert the generated image data into an appropriate format (e.g., JPEG, PNG).

[0114] The data is compressed and encrypted before being sent to the user's terminal.

[0115] Step 8:

[0116] The device receives the transmitted image and displays it on the screen.

[0117] Decompress and decrypt the received data.

[0118] The image is adjusted to match the display resolution and settings and displayed in real time.

[0119] Through these steps, the scenery or image the user has in mind is reproduced on the display as a concrete image.

[0120] Example 1

[0121] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0122] With current technology, it is difficult to display a user's thoughts and images as concrete images in real time. Furthermore, there are security concerns regarding the processing and transmission of EEG data, and it is also difficult to customize the generated images according to the user's emotional state. Therefore, there is a need for the development of technology that can quickly and safely visualize a user's thoughts.

[0123] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0124] In this invention, the server includes means for encrypting the preprocessed EEG data and transmitting it to a remote server, means for analyzing the preprocessed EEG data using artificial intelligence technology in the remote server to identify the content of the user's thoughts, and means for generating concrete images based on the content of the thoughts, thereby enabling the images that the user has in mind to be analyzed quickly and securely and visually embodied.

[0125] "User" refers to an individual who uses the system and provides EEG data.

[0126] "EEG Data" refers to electronic signal data acquired by a user's EEG cap or EEG measuring device.

[0127] "Means of collecting data in real time" refers to technology that uses an EEG cap or measuring device to instantly obtain a user's brainwave data.

[0128] "Preprocessing" refers to the process of removing noise and filtering the acquired EEG data to extract specific features.

[0129] "Specific features" refer to important patterns and indicators in EEG data, which serve as basic data for analyzing the user's thoughts.

[0130] "Encryption means" refers to technology that converts data to make it difficult for third parties to decipher, in order to ensure the security of brainwave data.

[0131] "Remote Server" refers to a server located at a remote location that receives, stores, and analyzes data sent from a user's device.

[0132] "Artificial intelligence technology" refers to technology that uses machine learning, deep learning, etc. to analyze data and draw conclusions and predictions.

[0133] "Thought content" refers to what the user specifically imagines or thinks about in their head.

[0134] "Means for generating concrete images" refers to technology that uses an image generation AI model to generate visual images based on the user's thoughts.

[0135] A "prompt sentence" refers to a sentence used as an instruction or input to an image-generating AI model.

[0136] "Means for displaying on a display" refers to the technology used to visually display images generated on the terminal on a screen.

[0137] "Emotional state" refers to the user's psychological or emotional state derived from EEG data.

[0138] The present invention relates to a system that acquires an image that a user has in mind from electroencephalogram data in real time and displays it as a concrete image. The configuration and operation of this system will be specifically described below.

[0139] System Configuration

[0140] 1. User device (terminal)

[0141] The user provides the brainwave data by wearing an EEG cap or EEG measurement device, which typically connects to a terminal using Bluetooth or other wireless communication technology.

[0142] The device measures brain waves in real time and collects initial data.

[0143] 2. Intermediate processing device (terminal)

[0144] After initial collection of EEG data, preprocessing is performed to remove noise and filter it, and to extract features. Specifically, the FFT (Fast Fourier Transform) algorithm is used.

[0145] The preprocessed data is encrypted and sent to a remote server using AES (Advanced Encryption Standard).

[0146] 3. Remote Analysis Server (Server)

[0147] The received EEG data is analyzed to identify what the user was thinking, using deep learning models (e.g., TensorFlow or PyTorch).

[0148] Based on the identified thought content, an image generation AI model (e.g., Stable Diffusion or DALL-E) is used to generate concrete images.

[0149] 4. Generated image transmission device (server)

[0150] The generated image data is then re-encrypted and sent to the user's device using the HTTPS protocol.

[0151] 5. Display Device (Terminal)

[0152] The received image data is decoded on the user's device and displayed on the display at an appropriate resolution.

[0153] Program processing flow

[0154] 1. The user wears the EEG cap and imagines a specific image. For example, the user imagines a "tranquil lake scene."

[0155] 2. The device collects the user's brainwave data in real time and stores the initial data in memory.

[0156] 3. The device uses an FFT algorithm to remove noise and filter the data, and extract specific features.

[0157] 4. The device encrypts the preprocessed EEG data with AES and transmits it to a remote server using the HTTPS protocol.

[0158] 5. The server uses a deep learning model to analyze the received EEG data and determine that the user is thinking of a "lake scene."

[0159] 6. The server uses an image generation AI model to generate a detailed image corresponding to the "lake scene."

[0160] 7. The server re-encrypts the generated image data and sends it to the user's device using the HTTPS protocol.

[0161] 8. The device decrypts the encrypted data and displays it on the display in full HD resolution.

[0162] Specific examples

[0163] Example 1: A user with physical limitations visualizes a scene

[0164] 1. A user with a physical limitation imagines a scene of a tranquil lake.

[0165] 2. The device collects the user's brainwave data and performs preprocessing.

[0166] 3. The device encrypts the preprocessed data with AES and sends it to the remote server.

[0167] 4. The server uses a deep learning model to analyze the data and determine that the user is thinking of a lake.

[0168] 5. The server uses image generation AI to convert the tranquil lake scene into a concrete image.

[0169] 6. The server re-encrypts the generated image and sends it to the user's device.

[0170] 7. The device decodes the image and displays it on the screen, allowing the user to visually confirm the lakescape they had in mind.

[0171] Prompt Sentence Examples

[0172] "Show me the image generated as a result of imagining a tranquil lake scene."

[0173] "Generate an image based on the results of EEG data analysis when imagining a complex geometric pattern."

[0174] In this way, the present invention makes it possible to materialize the user's thoughts and images from electroencephalogram data into concrete images and display them on a display.

[0175] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0176] Step 1:

[0177] The user wears an EEG cap and conjures up a specific image.

[0178] Specific operation: The user wears an EEG cap, sits in a chair, and relaxes while imagining a "lake scene."

[0179] Input: An image imagined by the user in a relaxed state.

[0180] Output: EEG data collected in real time.

[0181] Step 2:

[0182] The device collects the user's brainwave data in real time and stores it in memory.

[0183] Specific operation: The signal from the EEG cap is transmitted to the terminal via wireless communication technology (e.g., Bluetooth), and initial data is stored in the memory.

[0184] Input: Radio signal from EEG cap.

[0185] Output: Initial EEG data stored in memory.

[0186] Step 3:

[0187] The device performs noise removal and filtering on the EEG data to extract specific features.

[0188] Specific operation: Noise removal and filtering are performed using an FFT (Fast Fourier Transform) algorithm, and specific features such as alpha waves and beta waves are extracted.

[0189] Input: Initial EEG data stored in memory.

[0190] Output: Preprocessed EEG data with extracted features.

[0191] Step 4:

[0192] The device encrypts the preprocessed EEG data and transmits it to a remote server.

[0193] What it does: Encrypts data using AES (Advanced Encryption Standard) and sends it to a remote server via the HTTPS protocol.

[0194] Input: Preprocessed EEG data with extracted features.

[0195] Output: Encrypted EEG data is sent to a remote server.

[0196] Step 5:

[0197] The server receives the encrypted data, decrypts it, and then analyzes it using a deep learning model to determine what the user is thinking.

[0198] Specific operation: The received data is decrypted using AES, and then the user's thoughts are analyzed using a deep learning model (e.g., TensorFlow or PyTorch).

[0199] Input: Encrypted EEG data.

[0200] Output: Decoded data and analyzed thought (e.g., "lakescape").

[0201] Step 6:

[0202] The server uses an image generation AI model to generate specific images based on the identified thought content.

[0203] Specific operation: Generate an image equivalent to a "lake scene" using an image generation AI model (e.g., Stable Diffusion or DALL-E).

[0204] Input: Parsed thoughts.

[0205] Output: The concrete image generated.

[0206] Step 7:

[0207] The server re-encrypts the generated image data and sends it to the user's terminal.

[0208] Specific operation: The generated image data is encrypted with AES and sent to the user's device via the HTTPS protocol.

[0209] Input: The concrete image to be generated.

[0210] Output: The encrypted image data is sent to the user's device.

[0211] Step 8:

[0212] The terminal decodes the received image data and displays it on the display at the appropriate resolution.

[0213] Specific operation: The received encrypted data is decrypted using AES and displayed on the display in full HD resolution.

[0214] Input: Encrypted image data.

[0215] Output: The concrete image (picture) shown on the display.

[0216] (Application example 1)

[0217] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0218] Traditional methods have made it difficult to efficiently and accurately share work and repair procedures with other engineers within a factory. This has led to communication errors, work delays, and reduced productivity. Furthermore, the lack of a way to directly visualize engineers' thinking has made it difficult for experienced engineers to share their knowledge effectively.

[0219] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0220] In this invention, the server includes means for preprocessing the user's electroencephalogram data and extracting specific features, means for analyzing the preprocessed electroencephalogram data and identifying the user's thoughts, and means for generating concrete images based on the thoughts, thereby enabling the engineer's thoughts to be embodied and shared with other engineers in real time.

[0221] "User" refers to the entity that provides the EEG data, and is primarily a technician who performs work and repairs within the factory.

[0222] "EEG data" refers to data collected by an EEG measuring device when a user has a particular thought or image in their mind.

[0223] "Preprocessing" refers to data processing means for removing noise from electroencephalogram data and extracting specific features.

[0224] "Remote server" refers to a server device located remotely for receiving and analyzing electroencephalogram data.

[0225] A "neural network" is a type of artificial intelligence technology and a deep learning model used to analyze EEG data and generate images.

[0226] "Concrete imagery" refers to visual images and figures generated based on analyzed EEG data.

[0227] "Terminal" refers to equipment, including devices such as computers and tablets used by users and technicians.

[0228] A "display" is a display device installed in a terminal, and is a device for visually displaying the generated specific image.

[0229] "Sharing" refers to the viewing and use of the generated concrete image in real time among multiple engineers.

[0230] The present invention is a system for efficiently and accurately sharing work and repair procedures with other engineers in a factory, and a specific embodiment thereof will be described.

[0231] System Configuration

[0232] Hardware

[0233] 1. EEG Cap

[0234] This device is worn by the user and collects brainwave data in real time. It is used by engineers working in factories.

[0235] 2. EEG measurement device

[0236] This device works in conjunction with the EEG cap and transmits collected EEG data to a terminal.

[0237] 3. Terminal

[0238] A general computing device such as a PC or tablet used by engineers, and is mainly used for data preprocessing and display.

[0239] 4. Remote Server

[0240] This is a server installed outside the factory that analyzes EEG data and generates images.

[0241] software

[0242] 1. Preprocessing software

[0243] It runs on the device and is responsible for removing noise from EEG data and extracting specific features.

[0244] 2. AI analysis software

[0245] It runs on a remote server and uses artificial intelligence techniques such as neural networks to analyze brainwave data and generate concrete images.

[0246] 3. Image display application

[0247] It is an application that runs on the terminal and displays the generated image to the user.

[0248] System Operation Description

[0249] Data Collection Phase

[0250] The user wears an EEG cap and thinks about a specific work procedure or repair method. The EEG measurement device collects brainwave data in real time and transmits it to a terminal. Pre-processing software on the terminal removes noise and filters the data, and extracts features.

[0251] Data transmission phase

[0252] After preprocessing, the EEG data is encrypted on the device and securely transmitted to a remote server, ensuring data security.

[0253] Data analysis phase

[0254] The remote server uses AI analysis software to analyze the received EEG data, using deep learning models such as neural networks to identify what the user is thinking and generate specific images based on that.

[0255] Image Generation and Display Phase

[0256] The generated images are then sent back to the terminal from the remote server and displayed on the terminal's image display application, allowing the user to visualize the work or repair procedures they envision as concrete images that can be shared with other technicians.

[0257] Specific examples

[0258] When performing machine maintenance

[0259] Engineer A puts on an EEG cap and thinks about the maintenance procedures for a specific machine. The terminal collects A's EEG data and performs preprocessing. The preprocessed data is sent to a remote server, where AI analysis software analyzes it and generates an image of the maintenance procedures. The generated image is sent to Engineer A's terminal and shared with other engineers. This process makes work more efficient.

[0260] Prompt Sentence Examples

[0261] "Analyze EEG data to generate a detailed image of a specific repair procedure for a machine."

[0262] As described above, the present invention provides a system that enables factory engineers to visualize and efficiently share their thoughts using electroencephalogram data.

[0263] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0264] Step 1:

[0265] The user wears the EEG cap and thinks about a specific work procedure or repair method. As input, the user's brainwave data is collected in real time. As output, the real-time brainwave signal is sent to the EEG measurement device. Specifically, the EEG cap detects the user's brainwaves and sends the signal to the EEG measurement device.

[0266] Step 2:

[0267] The terminal receives EEG data in real time from the EEG measurement device and performs preprocessing. The input is the collected EEG data. The output is preprocessed data with noise removed and features extracted. Specifically, the preprocessing software applies a noise removal filter and executes a feature extraction algorithm.

[0268] Step 3:

[0269] The device encrypts the preprocessed data and sends it to a remote server. The input is the preprocessed EEG data. The output is the encrypted data sent to the remote server. Specifically, encryption software on the device encrypts the preprocessed data and transfers the data to the server using a secure communication protocol.

[0270] Step 4:

[0271] The server receives and decrypts the encrypted data. The input is the encrypted EEG data. The output is the decrypted data. The specific operation is to decrypt the data using a secure key management system and prepare it for data analysis.

[0272] Step 5:

[0273] The server analyzes the decoded data using AI analysis software. The input is the decoded EEG data. The output is the user's thoughts. Specifically, the AI ​​analysis software analyzes the data using neural networks and generative AI models to identify the user's thoughts.

[0274] Step 6:

[0275] The server generates a specific image based on the identified thought content. The input is data that identifies the thought content. The output is the generated specific image. As a specific action, the generative AI model generates a detailed image based on the analysis results. An example of a prompt statement would be, "Analyze specific repair procedures for a certain machine from EEG data and generate a detailed image."

[0276] Step 7:

[0277] The server sends the generated image data back to the terminal. The generated image data is the input. The image data is sent to the terminal as the output. In concrete terms, the sending software on the server transfers the generated image data to the terminal.

[0278] Step 8:

[0279] The terminal receives the image data and displays it in the image display application. The input is the transmitted image data. The output is a specific image displayed on the display. The specific operation is that the image display application interprets the image data and displays it on the display device.

[0280] This allows engineers to share their thoughts as concrete images with other engineers, improving work efficiency.

[0281] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0282] This invention relates to a system that interprets the user's thoughts and emotional state based on their electroencephalogram data, generates specific images based on this, and displays them on a display. In particular, by combining it with an emotion engine, it becomes possible to generate images that take the user's emotional state into consideration. The configuration and operation of this system will be specifically described.

[0283] System Configuration

[0284] 1. User device (terminal)

[0285] Users wear an EEG cap or EEG measuring device and provide brainwave data.

[0286] The device measures brain waves in real time and collects initial data.

[0287] 2. Intermediate processing device (terminal)

[0288] After initial collection of EEG data, preprocessing such as noise removal is performed to extract specific features.

[0289] After preprocessing, the data is encrypted and sent to a remote server.

[0290] 3. Remote Analysis Server (Server)

[0291] The user's thoughts are analyzed based on the received brain wave data.

[0292] AI technology converts the thoughts into concrete images.

[0293] An emotion engine is used to recognize the user's emotional state.

[0294] 4. Generated image transmission device (server)

[0295] The generated image is sent to the user's device.

[0296] Customize images based on information from the emotion engine.

[0297] 5. Display Device (Terminal)

[0298] The image data received by the user's terminal is decoded and displayed on the screen.

[0299] Program processing flow (natural language explanation)

[0300] 1. Data Collection and Preprocessing

[0301] The user wears an EEG cap to sense their thoughts and emotions.

[0302] The device collects the user's brainwave data in real time and stores the initial data in memory.

[0303] The device performs noise removal and filtering to extract specific features.

[0304] 2. Data transmission

[0305] The device encrypts the pre-processed EEG data and transmits it to a remote server.

[0306] 3. Data Analysis

[0307] The server analyzes the received brainwave data and identifies what the user was thinking.

[0308] At the same time, the server uses an emotion engine to determine the user's emotional state.

[0309] 4. Emotion-based image generation

[0310] The server uses the analysis results of the emotion engine to generate specific images that correspond to the user's emotional state.

[0311] 5. Sending and displaying the generated image

[0312] When the server sends the generated image to the user's device, it adaptively customizes the image based on information from the emotion engine.

[0313] The terminal decompresses and decodes the received image data and displays it on the display.

[0314] Specific examples

[0315] Example 1: Landscape imagery for stress reduction

[0316] 1. The user visualizes a relaxing beach scene while feeling stressed at work.

[0317] 2. The device collects EEG data, removes noise, and extracts features.

[0318] 3. The device sends the preprocessed data to the remote server.

[0319] 4. The server analyzes the data and determines that the user is thinking of a relaxing beach scene.

[0320] 5. At the same time, the server uses an emotion engine to detect when the user is in a high stress state.

[0321] 6. The server uses AI to generate a beach scene with colors and composition that enhance the relaxing effect.

[0322] 7. The server sends the generated image to the user's device, and the adaptively customized beach scene is displayed on the screen.

[0323] Example 2: Imagery to improve concentration

[0324] 1. Imagine a forest scene that helps users improve their focus while studying for exams.

[0325] 2. The device collects and preprocesses EEG data.

[0326] 3. The device sends the preprocessed data to the remote server.

[0327] 4. The server analyzes the data and determines that the user is imagining a forest scene to improve concentration.

[0328] 5. At the same time, the server detects the user's concentration level using an emotion engine.

[0329] 6. The server uses AI to generate a forest landscape suitable for maintaining concentration.

[0330] 7. The server sends the generated image to the user's device, and a forest scene that encourages concentration is displayed on the screen.

[0331] In this way, by combining emotion engines, it becomes possible to generate images that take into account not only the user's thoughts but also their emotional state, providing a more personalized experience.

[0332] The processing flow will be explained below.

[0333] Step 1:

[0334] The user wears an EEG cap and conjures up a specific image.

[0335] The user wears an EEG cap on their head and is asked to consciously visualize a specific scene or image.

[0336] Step 2:

[0337] The device collects the user's brainwave data in real time.

[0338] The electroencephalogram measuring device captures the user's electroencephalogram signals and transmits the signal data to the terminal.

[0339] Step 3:

[0340] The device preprocesses the collected EEG data.

[0341] Data cleansing removes noise and improves data quality.

[0342] Features are extracted using algorithms such as the fast Fourier transform (FFT).

[0343] Step 4:

[0344] The terminal encrypts the pre-processed data and transmits it to a remote server.

[0345] The EEG data from which the features have been extracted is encrypted using the SSL / TLS protocol.

[0346] Securely transmit encrypted data over the Internet to a remote server.

[0347] Step 5:

[0348] The server analyzes the received brain wave data.

[0349] It uses AI models (such as deep learning neural networks) to analyze data and identify what users are thinking.

[0350] Step 6:

[0351] The server uses an emotion engine to recognize the user's emotional state.

[0352] The emotion engine uses the received data to identify the user's stress level, concentration level, and type of emotion.

[0353] Step 7:

[0354] The server generates concrete images based on the identified thoughts and emotional states.

[0355] Using image generation AI (e.g., GAN), specific images are generated based on the identified thought content and emotional state.

[0356] The image's color, brightness, and composition are adaptively adjusted to match the user's emotional state.

[0357] Step 8:

[0358] The server sends the generated image to the user's terminal.

[0359] The generated image data is compressed and converted into an appropriate format (e.g., JPEG, PNG).

[0360] The data is encrypted using the SSL / TLS protocol and sent to the user's device.

[0361] Step 9:

[0362] The device receives the transmitted image and displays it on the screen.

[0363] The received image data is decompressed and decoded.

[0364] The image is adjusted to fit the display resolution and displayed in real time.

[0365] This allows images based on the scenery and emotions imagined by the user to be embodied and displayed on the screen.

[0366] Example 2

[0367] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0368] Conventional EEG data analysis systems have difficulty accurately grasping a user's emotional state and thoughts, resulting in images that often do not match the user's expectations. Furthermore, insufficient real-time data collection and processing can result in a poor user experience. Furthermore, insufficient privacy protection for collected data raises concerns about the risk of information leaks. A system that can resolve these issues and generate images that are both accurate and optimized for the user is needed.

[0369] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0370] In this invention, the server includes means for analyzing the preprocessed electroencephalogram data to identify the user's thoughts, means for generating specific images based on the thoughts, and means for identifying the user's emotional state using an emotion recognition engine and customizing the images according to the emotional state, thereby enabling the generation of personalized images that accurately reflect the user's thoughts and emotional state.

[0371] "User" means a person who provides EEG data and uses the system.

[0372] "EEG data" refers to data that measures a user's brainwave activity in real time.

[0373] "Preprocessing" refers to the process of removing noise from collected EEG data and extracting specific features.

[0374] "Features" are important data points or patterns extracted from preprocessed EEG data.

[0375] "Encryption" is the process of transforming data using a specific algorithm in order to transmit it securely.

[0376] A "remote server" is a server that can receive and process data via the Internet.

[0377] "Analysis" refers to data processing to identify the user's thoughts and emotional state based on brainwave data.

[0378] A "concrete image" is an image or figure that visually expresses the user's thoughts.

[0379] An "emotion recognition engine" is software or algorithms that analyze a user's emotional state based on brainwave data.

[0380] "Customization" refers to the process of adjusting the color tone and composition of the generated image according to the user's particular emotional state.

[0381] "Decryption" is the process of restoring encrypted data to its original state.

[0382] "Display" means a display device that allows a user to visually view the generated image.

[0383] This invention relates to a system that interprets the user's thoughts and emotional state based on their electroencephalogram data, generates specific images based on this, and displays them on a display. In particular, by combining it with an emotion engine, it is possible to generate images that take the user's emotional state into consideration. The configuration and operation of this system will be specifically explained.

[0384] System Configuration

[0385] 1. User device (terminal)

[0386] The user wears an EEG cap or EEG measurement device (e.g., Emotiv Epoc+) and provides brainwave data.

[0387] The device is used to measure brain waves in real time and collect initial data, which is collected using software such as Emotiv Pro and MATLAB.

[0388] 2. Intermediate processing device (terminal)

[0389] After initial collection of EEG data, preprocessing such as noise removal is performed to extract specific features (e.g., alpha waves, beta waves).

[0390] For preprocessing, we use Python libraries such as NumPy and SciPy.

[0391] 3. Remote Analysis Server (Server)

[0392] The user's thoughts are analyzed based on the received brain wave data.

[0393] Use an AI analysis engine (e.g., TensorFlow, PyTorch) to convert thought content into concrete images.

[0394] Recognize the user's emotional state using an emotion engine (e.g., Affectiva, IBM Watson).

[0395] 4. Generated image transmission device (server)

[0396] The generated image is sent to the user's device.

[0397] Adaptively customize images based on sentiment analysis results.

[0398] 5. Display Device (Terminal)

[0399] The image data received by the user's device is decoded and displayed on a monitor or VR headset.

[0400] Specific examples

[0401] Example 1: Landscape imagery for stress reduction

[0402] 1. The user visualizes a relaxing beach scene while feeling stressed at work.

[0403] 2. The device collects EEG data, removes noise, and extracts features.

[0404] 3. The device sends the preprocessed data to the remote server.

[0405] 4. The server analyzes the data and determines that the user is thinking of a relaxing beach scene.

[0406] 5. At the same time, the server uses an emotion engine to detect when the user is in a high stress state.

[0407] 6. The server uses AI to generate a relaxing beach scene.

[0408] 7. The server sends the generated image to the user's terminal and displays it on the display.

[0409] Prompt Sentence Examples

[0410] "Suppose a user is under high stress at work, but imagines a relaxing beach scene. In this state, a customized beach scene that maximizes the relaxation effect is generated and displayed on the display."

[0411] Example 2: Imagery to improve concentration

[0412] 1. Imagine a forest scene that helps users improve their focus while studying for exams.

[0413] 2. The device collects EEG data, removes noise, and extracts features.

[0414] 3. The device sends the preprocessed data to the remote server.

[0415] 4. The server analyzes the data and determines that the user is imagining a forest scene to improve concentration.

[0416] 5. At the same time, the server detects the user's concentration level using an emotion engine.

[0417] 6. The server uses AI to generate a forest landscape to help you maintain concentration.

[0418] 7. The server sends the generated image to the user's terminal and displays it on the display.

[0419] Prompt Sentence Examples

[0420] "Imagine a tranquil forest scene that helps users concentrate better while studying for an exam. In this state, we generate a customized forest scene that maximizes concentration and display it on the display."

[0421] Thus, a system for generating and displaying personalized images based on a user's thought content and emotional state has been described in detail as an embodiment of the present invention.

[0422] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0423] System program processing flow

[0424] Step 1: Data collection

[0425] The user wears an EEG cap and focuses their thoughts and emotions.

[0426] 1. The user puts on the EEG cap.

[0427] Input: EEG cap worn on user's head

[0428] Output: Begins collecting user's brainwave data in real time.

[0429] 2. The device collects the user's brainwave data in real time.

[0430] Operation: Data measured by the EEG cap is transferred to the terminal and saved in memory as initial data.

[0431] Input: User's EEG data

[0432] Output: Initial EEG data

[0433] Step 2: Data Preprocessing

[0434] 1. Preprocess the EEG data collected by the device.

[0435] What it does: Applies a noise reduction filter to remove unwanted noise from EEG data.

[0436] Input: Initial EEG data

[0437] Output: Noise-removed EEG data

[0438] 2. The device extracts specific features.

[0439] How it works: It uses Python's NumPy and SciPy libraries to extract specific patterns such as alpha and beta waves.

[0440] Input: Noise-removed EEG data

[0441] Output: Data with extracted features

[0442] Step 3: Encrypt and send data

[0443] 1. The device encrypts the data.

[0444] How it works: Encrypts the feature-extracted data using the AES encryption algorithm.

[0445] Input: Data with extracted features

[0446] Output: Encrypted data

[0447] 2. The device sends the encrypted data to a remote server.

[0448] How it works: Sends encrypted data to the server using the HTTPS protocol.

[0449] Input: Encrypted data

[0450] Output: Data sent to server completed

[0451] Step 4: Data analysis

[0452] 1. The server receives the encrypted data.

[0453] How it works: The server receives the data sent and decrypts it using the AES decryption algorithm.

[0454] Input: Encrypted data

[0455] Output: Decrypted data

[0456] 2. The server analyzes the data using its analysis engine.

[0457] How it works: Uses an AI analysis engine (e.g. TensorFlow, PyTorch) to identify the user's thoughts from EEG data.

[0458] Input: Decrypted data

[0459] Output: What the user is thinking

[0460] 3. The server uses the emotion engine to identify the emotional state.

[0461] How it works: Uses an emotion recognition engine (e.g., Affectiva, IBM Watson) to analyze the user's emotional state.

[0462] Input: User's EEG data

[0463] Output: User's emotional state

[0464] Step 5: Image Creation and Customization

[0465] 1. The server generates an image based on the analysis results.

[0466] How it works: It uses generative AI models (e.g., DALL·E, StyleGAN) to generate tangible images based on what the user is thinking.

[0467] Input: What the user is thinking

[0468] Output: The concrete image generated

[0469] 2. The server customizes the image based on the analysis results of the emotion engine.

[0470] How it works: Based on the results of emotion analysis, it adjusts the color and composition of the image to create a more personalized image.

[0471] Input: Generated concrete image, user's emotional state

[0472] Output: Customized image

[0473] Step 6: Sending and displaying image data

[0474] 1. The server encrypts the generated image and sends it to the user's device.

[0475] What it does: The resulting image is encrypted and sent, again using the AES encryption algorithm.

[0476] Input: Customized image

[0477] Output: Encrypted image data

[0478] 2. The device receives and decrypts the encrypted image data.

[0479] What it does: Uses the AES decryption algorithm to decrypt any encrypted data received.

[0480] Input: Encrypted image data

[0481] Output: Decoded image data

[0482] 3. The terminal displays the decrypted image on the display.

[0483] What it does: Displays the decoded image on a display device (e.g., monitor, VR headset).

[0484] Input: Decoded image data

[0485] Output: The image displayed on the screen

[0486] (Application example 2)

[0487] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0488] In recent years, there has been a demand for customization based on individual user needs and conditions to improve the consumer experience in virtual stores. However, current systems have difficulty in real-time adaptive customization based on the user's emotions and thoughts, limiting the improvement of consumer satisfaction. In addition, there has been a lack of effective means to provide the optimal products and environment that users desire. New technologies that can solve this problem and dramatically improve the user experience are needed.

[0489] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a device that collects the user's electroencephalogram data in real time, a device that preprocesses the electroencephalogram data and extracts specific features, and a device that transmits the preprocessed electroencephalogram data to a remote server. This enables optimal customization within the virtual store based on the user's thoughts and emotional state.

[0490] A "user" is a person who uses the system to enjoy an experience within a virtual store.

[0491] "EEG data" is a measurement of the electrical signals generated by the user's brain.

[0492] A "collection device" is a device for measuring and collecting EEG data in real time.

[0493] "Preprocessing" refers to the process of removing noise and extracting features from collected EEG data.

[0494] A "feature" is a specific index or pattern extracted from electroencephalogram data.

[0495] The "remote server" is a server at a remote location that receives and analyzes pre-processed EEG data.

[0496] The "analysis device" is a device that runs on a remote server and analyzes pre-processed EEG data to identify the user's thoughts and emotional state.

[0497] "Emotional state" refers to the mental and psychological state of a user as determined from their brainwave data.

[0498] "Concrete images" are visual representations that are generated based on the user's thoughts and emotional state.

[0499] "Customization" refers to individually adjusting the display content and environment within a virtual store according to the identified emotional state.

[0500] A "transmitting device" is a device for transmitting the generated image to a user's terminal.

[0501] A "display device" is a device for displaying images received at a user's terminal on a display.

[0502] A "display" is a screen or monitor used to visually display information on a user's device.

[0503] A "virtual store" refers to an environment in which products can be viewed and purchased in a virtual space just as if they were in a physical store.

[0504] MODE FOR CARRYING OUT THE INVENTION

[0505] This invention allows users to enjoy a customized virtual store experience based on their electroencephalogram (EEG) data. This system collects the user's EEG data in real time, extracts features, and then transmits the data to a remote server for analysis and image generation. The generated image is then transmitted to the user's device, where an adaptively customized virtual store is displayed.

[0506] 1. System program generation

[0507] The program for this system is constructed using the following hardware and software.

[0508] 2. Hardware and Software

[0509] Hardware used:

[0510] EEG Cap: Emotiv Insight

[0511] HMD: Oculus Quest 2

[0512] Software used:

[0513] Data analysis: TensorFlow

[0514] Emotion engine: Affectiva SDK

[0515] 3. Program processing explanation

[0516] Users wear an EEG cap and access the virtual store through an HMD. The EEG cap collects the user's brainwave data in real time and performs preprocessing such as noise reduction and filtering. The preprocessed data is encrypted and transmitted to a remote server via the HMD terminal.

[0517] The server uses TensorFlow to analyze the received EEG data to identify the user's thoughts and emotional state. The emotional state is then analyzed in detail using the Affectiva SDK. Based on the analysis results, a specific image that matches the user's emotional state is generated. This image generation process uses a generative AI model.

[0518] The generated images reflect the user's emotional state and adaptively customize the virtual store environment and product display. The customized images are transmitted from the remote server to the HMD and displayed on the user's HMD display.

[0519] 4. Example of a program and a prompt

[0520] Example 1: Stress-reducing shopping

[0521] 1. The user is feeling stressed and thinks about "relaxation."

[0522] 2. The device analyzes your thoughts and emotions. "You're feeling stressed."

[0523] 3. The server customizes the virtual store environment with a "relaxation" theme (e.g., warm lighting and relaxing music).

[0524] 4. Through the HMD, users can enjoy a shopping experience in a relaxing environment.

[0525] Example prompt:

[0526] "Creates a relaxing environment when the user is feeling stressed"

[0527] Example 2: Shopping to improve concentration

[0528] 1. Users imagine a "quiet forest" to help them concentrate.

[0529] 2. The device analyzes your thoughts and emotions. "I want to improve my concentration."

[0530] 3. The server recreates a quiet forest scene in the virtual store, which promotes concentration.

[0531] 4. Through the HMD, users can experience shopping in a quiet environment where they can concentrate.

[0532] Example prompt:

[0533] "Create a quiet environment to help users concentrate better"

[0534] This system enables a highly personalized virtual store experience based on the user's emotions and thoughts.

[0535] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0536] Step 1:

[0537] The user wears an EEG cap and accesses the virtual store through the HMD. The EEG cap collects the user's EEG data in real time. The input is the user's EEG data, and the output is the collected raw EEG data.

[0538] Step 2:

[0539] The EEG data collected by the device is preprocessed by performing noise removal and filtering. The input is raw EEG data, and the output is EEG data containing noise-removed features. Data processing involves noise filtering and feature extraction.

[0540] Step 3:

[0541] The terminal encrypts the preprocessed EEG data and sends it to a remote server. The input is the preprocessed EEG data, and the output is the encrypted data. The specific operation is to protect the data using an encryption algorithm.

[0542] Step 4:

[0543] The server decrypts the received preprocessed encrypted data and analyzes it using TensorFlow to identify the user's thoughts and emotional state. The input is the decrypted EEG data, and the output is the identified thoughts and emotional state. Data processing involves analysis using a machine learning model.

[0544] Step 5:

[0545] The server uses the Affectiva SDK to perform detailed emotion analysis based on the identified thoughts and emotional state, and generates specific images based on the results. The input is the identified thoughts and emotional state, and the output is the generated specific image. A generative AI model is used to create an image that matches the user's emotions.

[0546] Step 6:

[0547] The server sends the generated image to the user's device. Before sending, adaptive customization can be performed. The input is the generated image, and the output is the customized image. Adaptive customization can include, for example, changing the color tone or layout.

[0548] Step 7:

[0549] The terminal decodes the received image and displays it on the HMD display. The input is customized image data, and the output is visual information displayed on the HMD. Specifically, it performs visualization processing on the display.

[0550] In this way, users can enjoy a highly personalized virtual store experience based on their emotions and thoughts.

[0551] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0552] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0553] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0554] [Second embodiment]

[0555] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0556] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0557] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0558] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0559] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0560] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0561] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0562] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0563] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0564] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0565] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0566] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0567] The present invention relates to a system that acquires images such as a scene envisioned by a user from electroencephalogram data in real time and displays them as concrete images on a display. The configuration and operation of this system will be specifically described below.

[0568] System Configuration

[0569] 1. User device (terminal)

[0570] Users wearing an EEG cap or EEG measuring device provide brainwave data.

[0571] The device measures brain waves in real time and collects initial data.

[0572] 2. Intermediate processing device (terminal)

[0573] After initial collection of EEG data, preprocessing is performed to remove noise and filter it, and to extract features.

[0574] The pre-processed data is encrypted and sent to a remote server.

[0575] 3. Remote Analysis Server (Server)

[0576] The received brainwave data is analyzed to determine what the user was thinking.

[0577] Using AI (artificial intelligence) technology, concrete images are generated based on the content of thoughts.

[0578] 4. Generated image transmission device (server)

[0579] The generated image data is sent to the user's terminal.

[0580] 5. Display Device (Terminal)

[0581] The received image data is displayed on the user's terminal.

[0582] Program processing flow (natural language explanation)

[0583] 1. Data Collection and Preprocessing

[0584] The user wears an EEG cap and conjures up a specific image.

[0585] The device collects the user's brainwave data in real time and stores the initial data in memory.

[0586] The device performs noise removal and filtering to extract specific features.

[0587] 2. Data transmission

[0588] The device encrypts the pre-processed EEG data and transmits it securely to a remote server.

[0589] 3. Data Analysis

[0590] The server analyzes the received data and identifies what the user was thinking, using AI models such as neural networks.

[0591] The server generates specific images based on the identified thoughts. For example, if you think of a seascape, it generates a detailed image corresponding to it.

[0592] 4. Sending and displaying the generated image

[0593] The server transmits the generated image data to the user's terminal.

[0594] The image data received by the terminal is displayed on the display at an appropriate resolution.

[0595] Specific examples

[0596] Example 1: A user with physical limitations visualizes a scene

[0597] 1. User A, who has physical limitations, imagines a scene of a quiet lake.

[0598] 2. The device collects Person A's EEG data and performs noise removal and feature extraction.

[0599] 3. The device sends the preprocessed data to the remote server.

[0600] 4. The server analyzes the data and recognizes that Person A is thinking of a lake.

[0601] 5. The server uses image generation AI to convert the tranquil lake scene into a concrete image.

[0602] 6. The server sends the generated image to Mr. A's device.

[0603] 7. The device displays the received image on the screen, showing the lake scenery that Mr. A had in mind.

[0604] Example 2: Researchers visualize thoughts by analyzing subjects' brain waves

[0605] 1. User research subject B imagines a complex geometric pattern.

[0606] 2. The device collects Mr. B's EEG data and performs preprocessing.

[0607] 3. The device sends the preprocessed data to the remote server.

[0608] 4. The server analyzes the data and recognizes that Person B is thinking of a geometric pattern.

[0609] 5. The server generates a geometric pattern based on the analysis results.

[0610] 6. The server sends the generated image data to the researcher's computer.

[0611] 7. The device displays geometric patterns on a screen, allowing researchers to observe what the subject is thinking.

[0612] In this way, the present invention generates a concrete image of a user's thoughts and images from electroencephalogram data, providing a new means of communication.

[0613] The processing flow will be explained below.

[0614] Step 1:

[0615] The user wears an EEG cap and conjures up a specific image.

[0616] The EEG cap is worn properly and the user is made aware of the scenery or image they have in mind.

[0617] Step 2:

[0618] The device collects the user's brainwave data in real time.

[0619] An electroencephalogram (EEG) measuring device captures the user's brainwave activity and stores the data in memory.

[0620] Step 3:

[0621] The device preprocesses the collected EEG data.

[0622] Clean up the data through a noise removal filter.

[0623] Features are extracted using algorithms such as the fast Fourier transform (FFT).

[0624] Step 4:

[0625] The terminal encrypts the pre-processed data and transmits it to a remote server.

[0626] Data is encrypted using secure communication protocols (e.g. SSL / TLS).

[0627] Sends the encrypted data to the remote server and waits for an acknowledgment (ACK) from the server.

[0628] Step 5:

[0629] The server analyzes the received brain wave data.

[0630] Use AI models (e.g., deep learning neural networks) to identify what users are thinking.

[0631] Based on the analysis results, the image or scenery that the user has in mind is identified.

[0632] Step 6:

[0633] The server generates a concrete image based on the identified thought content.

[0634] Generative AI (e.g., Generative Adversarial Network, GAN) is used to create specific images based on specific content.

[0635] The image is subjected to high resolution and filtering processing as necessary.

[0636] Step 7:

[0637] The server sends the generated image to the user's terminal.

[0638] Convert the generated image data into an appropriate format (e.g., JPEG, PNG).

[0639] The data is compressed and encrypted before being sent to the user's terminal.

[0640] Step 8:

[0641] The device receives the transmitted image and displays it on the screen.

[0642] Decompress and decrypt the received data.

[0643] The image is adjusted to match the display resolution and settings and displayed in real time.

[0644] Through these steps, the scenery or image the user has in mind is reproduced on the display as a concrete image.

[0645] Example 1

[0646] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0647] With current technology, it is difficult to display a user's thoughts and images as concrete images in real time. Furthermore, there are security concerns regarding the processing and transmission of EEG data, and it is also difficult to customize the generated images according to the user's emotional state. Therefore, there is a need for the development of technology that can quickly and safely visualize a user's thoughts.

[0648] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0649] In this invention, the server includes means for encrypting the preprocessed EEG data and transmitting it to a remote server, means for analyzing the preprocessed EEG data using artificial intelligence technology in the remote server to identify the content of the user's thoughts, and means for generating concrete images based on the content of the thoughts, thereby enabling the images that the user has in mind to be analyzed quickly and securely and visually embodied.

[0650] "User" refers to an individual who uses the system and provides EEG data.

[0651] "EEG Data" refers to electronic signal data acquired by a user's EEG cap or EEG measuring device.

[0652] "Means of collecting data in real time" refers to technology that uses an EEG cap or measuring device to instantly obtain a user's brainwave data.

[0653] "Preprocessing" refers to the process of removing noise and filtering the acquired EEG data to extract specific features.

[0654] "Specific features" refer to important patterns and indicators in EEG data, which serve as basic data for analyzing the user's thoughts.

[0655] "Encryption means" refers to technology that converts data to make it difficult for third parties to decipher, in order to ensure the security of brainwave data.

[0656] "Remote Server" refers to a server located at a remote location that receives, stores, and analyzes data sent from a user's device.

[0657] "Artificial intelligence technology" refers to technology that uses machine learning, deep learning, etc. to analyze data and draw conclusions and predictions.

[0658] "Thought content" refers to what the user specifically imagines or thinks about in their head.

[0659] "Means for generating concrete images" refers to technology that uses an image generation AI model to generate visual images based on the user's thoughts.

[0660] A "prompt sentence" refers to a sentence used as an instruction or input to an image-generating AI model.

[0661] "Means for displaying on a display" refers to the technology used to visually display images generated on the terminal on a screen.

[0662] "Emotional state" refers to the user's psychological or emotional state derived from EEG data.

[0663] The present invention relates to a system that acquires an image that a user has in mind from electroencephalogram data in real time and displays it as a concrete image. The configuration and operation of this system will be specifically described below.

[0664] System Configuration

[0665] 1. User device (terminal)

[0666] The user provides the brainwave data by wearing an EEG cap or EEG measurement device, which typically connects to a terminal using Bluetooth or other wireless communication technology.

[0667] The device measures brain waves in real time and collects initial data.

[0668] 2. Intermediate processing device (terminal)

[0669] After initial collection of EEG data, preprocessing is performed to remove noise and filter it, and to extract features. Specifically, the FFT (Fast Fourier Transform) algorithm is used.

[0670] The preprocessed data is encrypted and sent to a remote server using AES (Advanced Encryption Standard).

[0671] 3. Remote Analysis Server (Server)

[0672] The received EEG data is analyzed to identify what the user was thinking, using deep learning models (e.g., TensorFlow or PyTorch).

[0673] Based on the identified thought content, an image generation AI model (e.g., Stable Diffusion or DALL-E) is used to generate concrete images.

[0674] 4. Generated image transmission device (server)

[0675] The generated image data is then re-encrypted and sent to the user's device using the HTTPS protocol.

[0676] 5. Display Device (Terminal)

[0677] The received image data is decoded on the user's device and displayed on the display at an appropriate resolution.

[0678] Program processing flow

[0679] 1. The user wears the EEG cap and imagines a specific image. For example, the user imagines a "tranquil lake scene."

[0680] 2. The device collects the user's brainwave data in real time and stores the initial data in memory.

[0681] 3. The device uses an FFT algorithm to remove noise and filter the data, and extract specific features.

[0682] 4. The device encrypts the preprocessed EEG data with AES and transmits it to a remote server using the HTTPS protocol.

[0683] 5. The server uses a deep learning model to analyze the received EEG data and determine that the user is thinking of a "lake scene."

[0684] 6. The server uses an image generation AI model to generate a detailed image corresponding to the "lake scene."

[0685] 7. The server re-encrypts the generated image data and sends it to the user's device using the HTTPS protocol.

[0686] 8. The device decrypts the encrypted data and displays it on the display in full HD resolution.

[0687] Specific examples

[0688] Example 1: A user with physical limitations visualizes a scene

[0689] 1. A user with a physical limitation imagines a scene of a tranquil lake.

[0690] 2. The device collects the user's brainwave data and performs preprocessing.

[0691] 3. The device encrypts the preprocessed data with AES and sends it to the remote server.

[0692] 4. The server uses a deep learning model to analyze the data and determine that the user is thinking of a lake.

[0693] 5. The server uses image generation AI to convert the tranquil lake scene into a concrete image.

[0694] 6. The server re-encrypts the generated image and sends it to the user's device.

[0695] 7. The device decodes the image and displays it on the screen, allowing the user to visually confirm the lakescape they had in mind.

[0696] Prompt Sentence Examples

[0697] "Show me the image generated as a result of imagining a tranquil lake scene."

[0698] "Generate an image based on the results of EEG data analysis when imagining a complex geometric pattern."

[0699] In this way, the present invention makes it possible to materialize the user's thoughts and images from electroencephalogram data into concrete images and display them on a display.

[0700] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0701] Step 1:

[0702] The user wears an EEG cap and conjures up a specific image.

[0703] Specific operation: The user wears an EEG cap, sits in a chair, and relaxes while imagining a "lake scene."

[0704] Input: An image imagined by the user in a relaxed state.

[0705] Output: EEG data collected in real time.

[0706] Step 2:

[0707] The device collects the user's brainwave data in real time and stores it in memory.

[0708] Specific operation: The signal from the EEG cap is transmitted to the terminal via wireless communication technology (e.g., Bluetooth), and initial data is stored in the memory.

[0709] Input: Radio signal from EEG cap.

[0710] Output: Initial EEG data stored in memory.

[0711] Step 3:

[0712] The device performs noise removal and filtering on the EEG data to extract specific features.

[0713] Specific operation: Noise removal and filtering are performed using an FFT (Fast Fourier Transform) algorithm, and specific features such as alpha waves and beta waves are extracted.

[0714] Input: Initial EEG data stored in memory.

[0715] Output: Preprocessed EEG data with extracted features.

[0716] Step 4:

[0717] The device encrypts the preprocessed EEG data and transmits it to a remote server.

[0718] What it does: Encrypts data using AES (Advanced Encryption Standard) and sends it to a remote server via the HTTPS protocol.

[0719] Input: Preprocessed EEG data with extracted features.

[0720] Output: Encrypted EEG data is sent to a remote server.

[0721] Step 5:

[0722] The server receives the encrypted data, decrypts it, and then analyzes it using a deep learning model to determine what the user is thinking.

[0723] Specific operation: The received data is decrypted using AES, and then the user's thoughts are analyzed using a deep learning model (e.g., TensorFlow or PyTorch).

[0724] Input: Encrypted EEG data.

[0725] Output: Decoded data and analyzed thought (e.g., "lakescape").

[0726] Step 6:

[0727] The server uses an image generation AI model to generate specific images based on the identified thought content.

[0728] Specific operation: Generate an image equivalent to a "lake scene" using an image generation AI model (e.g., Stable Diffusion or DALL-E).

[0729] Input: Parsed thoughts.

[0730] Output: The concrete image generated.

[0731] Step 7:

[0732] The server re-encrypts the generated image data and sends it to the user's terminal.

[0733] Specific operation: The generated image data is encrypted with AES and sent to the user's device via the HTTPS protocol.

[0734] Input: The concrete image to be generated.

[0735] Output: The encrypted image data is sent to the user's device.

[0736] Step 8:

[0737] The terminal decodes the received image data and displays it on the display at the appropriate resolution.

[0738] Specific operation: The received encrypted data is decrypted using AES and displayed on the display in full HD resolution.

[0739] Input: Encrypted image data.

[0740] Output: The concrete image (picture) shown on the display.

[0741] (Application example 1)

[0742] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0743] Traditional methods have made it difficult to efficiently and accurately share work and repair procedures with other engineers within a factory. This has led to communication errors, work delays, and reduced productivity. Furthermore, the lack of a way to directly visualize engineers' thinking has made it difficult for experienced engineers to share their knowledge effectively.

[0744] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0745] In this invention, the server includes means for preprocessing the user's electroencephalogram data and extracting specific features, means for analyzing the preprocessed electroencephalogram data and identifying the user's thoughts, and means for generating concrete images based on the thoughts, thereby enabling the engineer's thoughts to be embodied and shared with other engineers in real time.

[0746] "User" refers to the entity that provides the EEG data, and is primarily a technician who performs work and repairs within the factory.

[0747] "EEG data" refers to data collected by an EEG measuring device when a user has a particular thought or image in their mind.

[0748] "Preprocessing" refers to data processing means for removing noise from electroencephalogram data and extracting specific features.

[0749] "Remote server" refers to a server device located remotely for receiving and analyzing electroencephalogram data.

[0750] A "neural network" is a type of artificial intelligence technology and a deep learning model used to analyze EEG data and generate images.

[0751] "Concrete imagery" refers to visual images and figures generated based on analyzed EEG data.

[0752] "Terminal" refers to equipment, including devices such as computers and tablets used by users and technicians.

[0753] A "display" is a display device installed in a terminal, and is a device for visually displaying the generated specific image.

[0754] "Sharing" refers to the viewing and use of the generated concrete image in real time among multiple engineers.

[0755] The present invention is a system for efficiently and accurately sharing work and repair procedures with other engineers in a factory, and a specific embodiment thereof will be described.

[0756] System Configuration

[0757] Hardware

[0758] 1. EEG Cap

[0759] This device is worn by the user and collects brainwave data in real time. It is used by engineers working in factories.

[0760] 2. EEG measurement device

[0761] This device works in conjunction with the EEG cap and transmits collected EEG data to a terminal.

[0762] 3. Terminal

[0763] A general computing device such as a PC or tablet used by engineers, and is mainly used for data preprocessing and display.

[0764] 4. Remote Server

[0765] This is a server installed outside the factory that analyzes EEG data and generates images.

[0766] software

[0767] 1. Preprocessing software

[0768] It runs on the device and is responsible for removing noise from EEG data and extracting specific features.

[0769] 2. AI analysis software

[0770] It runs on a remote server and uses artificial intelligence techniques such as neural networks to analyze brainwave data and generate concrete images.

[0771] 3. Image display application

[0772] It is an application that runs on the terminal and displays the generated image to the user.

[0773] System Operation Description

[0774] Data Collection Phase

[0775] The user wears an EEG cap and thinks about a specific work procedure or repair method. The EEG measurement device collects brainwave data in real time and transmits it to a terminal. Pre-processing software on the terminal removes noise and filters the data, and extracts features.

[0776] Data transmission phase

[0777] After preprocessing, the EEG data is encrypted on the device and securely transmitted to a remote server, ensuring data security.

[0778] Data analysis phase

[0779] The remote server uses AI analysis software to analyze the received EEG data, using deep learning models such as neural networks to identify what the user is thinking and generate specific images based on that.

[0780] Image Generation and Display Phase

[0781] The generated images are then sent back to the terminal from the remote server and displayed on the terminal's image display application, allowing the user to visualize the work or repair procedures they envision as concrete images that can be shared with other technicians.

[0782] Specific examples

[0783] When performing machine maintenance

[0784] Engineer A puts on an EEG cap and thinks about the maintenance procedures for a specific machine. The terminal collects A's EEG data and performs preprocessing. The preprocessed data is sent to a remote server, where AI analysis software analyzes it and generates an image of the maintenance procedures. The generated image is sent to Engineer A's terminal and shared with other engineers. This process makes work more efficient.

[0785] Prompt Sentence Examples

[0786] "Analyze EEG data to generate a detailed image of a specific repair procedure for a machine."

[0787] As described above, the present invention provides a system that enables factory engineers to visualize and efficiently share their thoughts using electroencephalogram data.

[0788] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0789] Step 1:

[0790] The user wears the EEG cap and thinks about a specific work procedure or repair method. As input, the user's brainwave data is collected in real time. As output, the real-time brainwave signal is sent to the EEG measurement device. Specifically, the EEG cap detects the user's brainwaves and sends the signal to the EEG measurement device.

[0791] Step 2:

[0792] The terminal receives EEG data in real time from the EEG measurement device and performs preprocessing. The input is the collected EEG data. The output is preprocessed data with noise removed and features extracted. Specifically, the preprocessing software applies a noise removal filter and executes a feature extraction algorithm.

[0793] Step 3:

[0794] The device encrypts the preprocessed data and sends it to a remote server. The input is the preprocessed EEG data. The output is the encrypted data sent to the remote server. Specifically, encryption software on the device encrypts the preprocessed data and transfers the data to the server using a secure communication protocol.

[0795] Step 4:

[0796] The server receives and decrypts the encrypted data. The input is the encrypted EEG data. The output is the decrypted data. The specific operation is to decrypt the data using a secure key management system and prepare it for data analysis.

[0797] Step 5:

[0798] The server analyzes the decoded data using AI analysis software. The input is the decoded EEG data. The output is the user's thoughts. Specifically, the AI ​​analysis software analyzes the data using neural networks and generative AI models to identify the user's thoughts.

[0799] Step 6:

[0800] The server generates a specific image based on the identified thought content. The input is data that identifies the thought content. The output is the generated specific image. As a specific action, the generative AI model generates a detailed image based on the analysis results. An example of a prompt statement would be, "Analyze specific repair procedures for a certain machine from EEG data and generate a detailed image."

[0801] Step 7:

[0802] The server sends the generated image data back to the terminal. The generated image data is the input. The image data is sent to the terminal as the output. In concrete terms, the sending software on the server transfers the generated image data to the terminal.

[0803] Step 8:

[0804] The terminal receives the image data and displays it in the image display application. The input is the transmitted image data. The output is a specific image displayed on the display. The specific operation is that the image display application interprets the image data and displays it on the display device.

[0805] This allows engineers to share their thoughts as concrete images with other engineers, improving work efficiency.

[0806] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0807] This invention relates to a system that interprets the user's thoughts and emotional state based on their electroencephalogram data, generates specific images based on this, and displays them on a display. In particular, by combining it with an emotion engine, it becomes possible to generate images that take the user's emotional state into consideration. The configuration and operation of this system will be specifically described.

[0808] System Configuration

[0809] 1. User device (terminal)

[0810] Users wear an EEG cap or EEG measuring device and provide brainwave data.

[0811] The device measures brain waves in real time and collects initial data.

[0812] 2. Intermediate processing device (terminal)

[0813] After initial collection of EEG data, preprocessing such as noise removal is performed to extract specific features.

[0814] After preprocessing, the data is encrypted and sent to a remote server.

[0815] 3. Remote Analysis Server (Server)

[0816] The user's thoughts are analyzed based on the received brain wave data.

[0817] AI technology converts the thoughts into concrete images.

[0818] An emotion engine is used to recognize the user's emotional state.

[0819] 4. Generated image transmission device (server)

[0820] The generated image is sent to the user's device.

[0821] Customize images based on information from the emotion engine.

[0822] 5. Display Device (Terminal)

[0823] The image data received by the user's terminal is decoded and displayed on the screen.

[0824] Program processing flow (natural language explanation)

[0825] 1. Data Collection and Preprocessing

[0826] The user wears an EEG cap to sense their thoughts and emotions.

[0827] The device collects the user's brainwave data in real time and stores the initial data in memory.

[0828] The device performs noise removal and filtering to extract specific features.

[0829] 2. Data transmission

[0830] The device encrypts the pre-processed EEG data and transmits it to a remote server.

[0831] 3. Data Analysis

[0832] The server analyzes the received brainwave data and identifies what the user was thinking.

[0833] At the same time, the server uses an emotion engine to determine the user's emotional state.

[0834] 4. Emotion-based image generation

[0835] The server uses the analysis results of the emotion engine to generate specific images that correspond to the user's emotional state.

[0836] 5. Sending and displaying the generated image

[0837] When the server sends the generated image to the user's device, it adaptively customizes the image based on information from the emotion engine.

[0838] The terminal decompresses and decodes the received image data and displays it on the display.

[0839] Specific examples

[0840] Example 1: Landscape imagery for stress reduction

[0841] 1. The user visualizes a relaxing beach scene while feeling stressed at work.

[0842] 2. The device collects EEG data, removes noise, and extracts features.

[0843] 3. The device sends the preprocessed data to the remote server.

[0844] 4. The server analyzes the data and determines that the user is thinking of a relaxing beach scene.

[0845] 5. At the same time, the server uses an emotion engine to detect when the user is in a high stress state.

[0846] 6. The server uses AI to generate a beach scene with colors and composition that enhance the relaxing effect.

[0847] 7. The server sends the generated image to the user's device, and the adaptively customized beach scene is displayed on the screen.

[0848] Example 2: Imagery to improve concentration

[0849] 1. Imagine a forest scene that helps users improve their focus while studying for exams.

[0850] 2. The device collects and preprocesses EEG data.

[0851] 3. The device sends the preprocessed data to the remote server.

[0852] 4. The server analyzes the data and determines that the user is imagining a forest scene to improve concentration.

[0853] 5. At the same time, the server detects the user's concentration level using an emotion engine.

[0854] 6. The server uses AI to generate a forest landscape suitable for maintaining concentration.

[0855] 7. The server sends the generated image to the user's device, and a forest scene that encourages concentration is displayed on the screen.

[0856] In this way, by combining emotion engines, it becomes possible to generate images that take into account not only the user's thoughts but also their emotional state, providing a more personalized experience.

[0857] The processing flow will be explained below.

[0858] Step 1:

[0859] The user wears an EEG cap and conjures up a specific image.

[0860] The user wears an EEG cap on their head and is asked to consciously visualize a specific scene or image.

[0861] Step 2:

[0862] The device collects the user's brainwave data in real time.

[0863] The electroencephalogram measuring device captures the user's electroencephalogram signals and transmits the signal data to the terminal.

[0864] Step 3:

[0865] The device preprocesses the collected EEG data.

[0866] Data cleansing removes noise and improves data quality.

[0867] Features are extracted using algorithms such as the fast Fourier transform (FFT).

[0868] Step 4:

[0869] The terminal encrypts the pre-processed data and transmits it to a remote server.

[0870] The EEG data from which the features have been extracted is encrypted using the SSL / TLS protocol.

[0871] Securely transmit encrypted data over the Internet to a remote server.

[0872] Step 5:

[0873] The server analyzes the received brain wave data.

[0874] It uses AI models (such as deep learning neural networks) to analyze data and identify what users are thinking.

[0875] Step 6:

[0876] The server uses an emotion engine to recognize the user's emotional state.

[0877] The emotion engine uses the received data to identify the user's stress level, concentration level, and type of emotion.

[0878] Step 7:

[0879] The server generates concrete images based on the identified thoughts and emotional states.

[0880] Using image generation AI (e.g., GAN), specific images are generated based on the identified thought content and emotional state.

[0881] The image's color, brightness, and composition are adaptively adjusted to match the user's emotional state.

[0882] Step 8:

[0883] The server sends the generated image to the user's terminal.

[0884] The generated image data is compressed and converted into an appropriate format (e.g., JPEG, PNG).

[0885] The data is encrypted using the SSL / TLS protocol and sent to the user's device.

[0886] Step 9:

[0887] The device receives the transmitted image and displays it on the screen.

[0888] The received image data is decompressed and decoded.

[0889] The image is adjusted to fit the display resolution and displayed in real time.

[0890] This allows images based on the scenery and emotions imagined by the user to be embodied and displayed on the screen.

[0891] Example 2

[0892] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0893] Conventional EEG data analysis systems have difficulty accurately grasping a user's emotional state and thoughts, resulting in images that often do not match the user's expectations. Furthermore, insufficient real-time data collection and processing can result in a poor user experience. Furthermore, insufficient privacy protection for collected data raises concerns about the risk of information leaks. A system that can resolve these issues and generate images that are both accurate and optimized for the user is needed.

[0894] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0895] In this invention, the server includes means for analyzing the preprocessed electroencephalogram data to identify the user's thoughts, means for generating specific images based on the thoughts, and means for identifying the user's emotional state using an emotion recognition engine and customizing the images according to the emotional state, thereby enabling the generation of personalized images that accurately reflect the user's thoughts and emotional state.

[0896] "User" means a person who provides EEG data and uses the system.

[0897] "EEG data" refers to data that measures a user's brainwave activity in real time.

[0898] "Preprocessing" refers to the process of removing noise from collected EEG data and extracting specific features.

[0899] "Features" are important data points or patterns extracted from preprocessed EEG data.

[0900] "Encryption" is the process of transforming data using a specific algorithm in order to transmit it securely.

[0901] A "remote server" is a server that can receive and process data via the Internet.

[0902] "Analysis" refers to data processing to identify the user's thoughts and emotional state based on brainwave data.

[0903] A "concrete image" is an image or figure that visually expresses the user's thoughts.

[0904] An "emotion recognition engine" is software or algorithms that analyze a user's emotional state based on brainwave data.

[0905] "Customization" refers to the process of adjusting the color tone and composition of the generated image according to the user's particular emotional state.

[0906] "Decryption" is the process of restoring encrypted data to its original state.

[0907] "Display" means a display device that allows a user to visually view the generated image.

[0908] This invention relates to a system that interprets the user's thoughts and emotional state based on their electroencephalogram data, generates specific images based on this, and displays them on a display. In particular, by combining it with an emotion engine, it is possible to generate images that take the user's emotional state into consideration. The configuration and operation of this system will be specifically explained.

[0909] System Configuration

[0910] 1. User device (terminal)

[0911] The user wears an EEG cap or EEG measurement device (e.g., Emotiv Epoc+) and provides brainwave data.

[0912] The device is used to measure brain waves in real time and collect initial data, which is collected using software such as Emotiv Pro and MATLAB.

[0913] 2. Intermediate processing device (terminal)

[0914] After initial collection of EEG data, preprocessing such as noise removal is performed to extract specific features (e.g., alpha waves, beta waves).

[0915] For preprocessing, we use Python libraries such as NumPy and SciPy.

[0916] 3. Remote Analysis Server (Server)

[0917] The user's thoughts are analyzed based on the received brain wave data.

[0918] Use an AI analysis engine (e.g., TensorFlow, PyTorch) to convert thought content into concrete images.

[0919] Recognize the user's emotional state using an emotion engine (e.g., Affectiva, IBM Watson).

[0920] 4. Generated image transmission device (server)

[0921] The generated image is sent to the user's device.

[0922] Adaptively customize images based on sentiment analysis results.

[0923] 5. Display Device (Terminal)

[0924] The image data received by the user's device is decoded and displayed on a monitor or VR headset.

[0925] Specific examples

[0926] Example 1: Landscape imagery for stress reduction

[0927] 1. The user visualizes a relaxing beach scene while feeling stressed at work.

[0928] 2. The device collects EEG data, removes noise, and extracts features.

[0929] 3. The device sends the preprocessed data to the remote server.

[0930] 4. The server analyzes the data and determines that the user is thinking of a relaxing beach scene.

[0931] 5. At the same time, the server uses an emotion engine to detect when the user is in a high stress state.

[0932] 6. The server uses AI to generate a relaxing beach scene.

[0933] 7. The server sends the generated image to the user's terminal and displays it on the display.

[0934] Prompt Sentence Examples

[0935] "Suppose a user is under high stress at work, but imagines a relaxing beach scene. In this state, a customized beach scene that maximizes the relaxation effect is generated and displayed on the display."

[0936] Example 2: Imagery to improve concentration

[0937] 1. Imagine a forest scene that helps users improve their focus while studying for exams.

[0938] 2. The device collects EEG data, removes noise, and extracts features.

[0939] 3. The device sends the preprocessed data to the remote server.

[0940] 4. The server analyzes the data and determines that the user is imagining a forest scene to improve concentration.

[0941] 5. At the same time, the server detects the user's concentration level using an emotion engine.

[0942] 6. The server uses AI to generate a forest landscape to help you maintain concentration.

[0943] 7. The server sends the generated image to the user's terminal and displays it on the display.

[0944] Prompt Sentence Examples

[0945] "Imagine a tranquil forest scene that helps users concentrate better while studying for an exam. In this state, we generate a customized forest scene that maximizes concentration and display it on the display."

[0946] Thus, a system for generating and displaying personalized images based on a user's thought content and emotional state has been described in detail as an embodiment of the present invention.

[0947] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0948] System program processing flow

[0949] Step 1: Data collection

[0950] The user wears an EEG cap and focuses their thoughts and emotions.

[0951] 1. The user puts on the EEG cap.

[0952] Input: EEG cap worn on user's head

[0953] Output: Begins collecting user's brainwave data in real time.

[0954] 2. The device collects the user's brainwave data in real time.

[0955] Operation: Data measured by the EEG cap is transferred to the terminal and saved in memory as initial data.

[0956] Input: User's EEG data

[0957] Output: Initial EEG data

[0958] Step 2: Data Preprocessing

[0959] 1. Preprocess the EEG data collected by the device.

[0960] What it does: Applies a noise reduction filter to remove unwanted noise from EEG data.

[0961] Input: Initial EEG data

[0962] Output: Noise-removed EEG data

[0963] 2. The device extracts specific features.

[0964] How it works: It uses Python's NumPy and SciPy libraries to extract specific patterns such as alpha and beta waves.

[0965] Input: Noise-removed EEG data

[0966] Output: Data with extracted features

[0967] Step 3: Encrypt and send data

[0968] 1. The device encrypts the data.

[0969] How it works: Encrypts the feature-extracted data using the AES encryption algorithm.

[0970] Input: Data with extracted features

[0971] Output: Encrypted data

[0972] 2. The device sends the encrypted data to a remote server.

[0973] How it works: Sends encrypted data to the server using the HTTPS protocol.

[0974] Input: Encrypted data

[0975] Output: Data sent to server completed

[0976] Step 4: Data analysis

[0977] 1. The server receives the encrypted data.

[0978] How it works: The server receives the data sent and decrypts it using the AES decryption algorithm.

[0979] Input: Encrypted data

[0980] Output: Decrypted data

[0981] 2. The server analyzes the data using its analysis engine.

[0982] How it works: Uses an AI analysis engine (e.g. TensorFlow, PyTorch) to identify the user's thoughts from EEG data.

[0983] Input: Decrypted data

[0984] Output: What the user is thinking

[0985] 3. The server uses the emotion engine to identify the emotional state.

[0986] How it works: Uses an emotion recognition engine (e.g., Affectiva, IBM Watson) to analyze the user's emotional state.

[0987] Input: User's EEG data

[0988] Output: User's emotional state

[0989] Step 5: Image Creation and Customization

[0990] 1. The server generates an image based on the analysis results.

[0991] How it works: It uses generative AI models (e.g., DALL·E, StyleGAN) to generate tangible images based on what the user is thinking.

[0992] Input: What the user is thinking

[0993] Output: The concrete image generated

[0994] 2. The server customizes the image based on the analysis results of the emotion engine.

[0995] How it works: Based on the results of emotion analysis, it adjusts the color and composition of the image to create a more personalized image.

[0996] Input: Generated concrete image, user's emotional state

[0997] Output: Customized image

[0998] Step 6: Sending and displaying image data

[0999] 1. The server encrypts the generated image and sends it to the user's device.

[1000] What it does: The resulting image is encrypted and sent, again using the AES encryption algorithm.

[1001] Input: Customized image

[1002] Output: Encrypted image data

[1003] 2. The device receives and decrypts the encrypted image data.

[1004] What it does: Uses the AES decryption algorithm to decrypt any encrypted data received.

[1005] Input: Encrypted image data

[1006] Output: Decoded image data

[1007] 3. The terminal displays the decrypted image on the display.

[1008] What it does: Displays the decoded image on a display device (e.g., monitor, VR headset).

[1009] Input: Decoded image data

[1010] Output: The image displayed on the screen

[1011] (Application example 2)

[1012] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1013] In recent years, there has been a demand for customization based on individual user needs and conditions to improve the consumer experience in virtual stores. However, current systems have difficulty in real-time adaptive customization based on the user's emotions and thoughts, limiting the improvement of consumer satisfaction. In addition, there has been a lack of effective means to provide the optimal products and environment that users desire. New technologies that can solve this problem and dramatically improve the user experience are needed.

[1014] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a device that collects the user's electroencephalogram data in real time, a device that preprocesses the electroencephalogram data and extracts specific features, and a device that transmits the preprocessed electroencephalogram data to a remote server. This enables optimal customization within the virtual store based on the user's thoughts and emotional state.

[1015] A "user" is a person who uses the system to enjoy an experience within a virtual store.

[1016] "EEG data" is a measurement of the electrical signals generated by the user's brain.

[1017] A "collection device" is a device for measuring and collecting EEG data in real time.

[1018] "Preprocessing" refers to the process of removing noise and extracting features from collected EEG data.

[1019] A "feature" is a specific index or pattern extracted from electroencephalogram data.

[1020] The "remote server" is a server at a remote location that receives and analyzes pre-processed EEG data.

[1021] The "analysis device" is a device that runs on a remote server and analyzes pre-processed EEG data to identify the user's thoughts and emotional state.

[1022] "Emotional state" refers to the mental and psychological state of a user as determined from their brainwave data.

[1023] "Concrete images" are visual representations that are generated based on the user's thoughts and emotional state.

[1024] "Customization" refers to individually adjusting the display content and environment within a virtual store according to the identified emotional state.

[1025] A "transmitting device" is a device for transmitting the generated image to a user's terminal.

[1026] A "display device" is a device for displaying images received at a user's terminal on a display.

[1027] A "display" is a screen or monitor used to visually display information on a user's device.

[1028] A "virtual store" refers to an environment in which products can be viewed and purchased in a virtual space just as if they were in a physical store.

[1029] MODE FOR CARRYING OUT THE INVENTION

[1030] This invention allows users to enjoy a customized virtual store experience based on their electroencephalogram (EEG) data. This system collects the user's EEG data in real time, extracts features, and then transmits the data to a remote server for analysis and image generation. The generated image is then transmitted to the user's device, where an adaptively customized virtual store is displayed.

[1031] 1. System program generation

[1032] The program for this system is constructed using the following hardware and software.

[1033] 2. Hardware and Software

[1034] Hardware used:

[1035] EEG Cap: Emotiv Insight

[1036] HMD: Oculus Quest 2

[1037] Software used:

[1038] Data analysis: TensorFlow

[1039] Emotion engine: Affectiva SDK

[1040] 3. Program processing explanation

[1041] Users wear an EEG cap and access the virtual store through an HMD. The EEG cap collects the user's brainwave data in real time and performs preprocessing such as noise reduction and filtering. The preprocessed data is encrypted and transmitted to a remote server via the HMD terminal.

[1042] The server uses TensorFlow to analyze the received EEG data to identify the user's thoughts and emotional state. The emotional state is then analyzed in detail using the Affectiva SDK. Based on the analysis results, a specific image that matches the user's emotional state is generated. This image generation process uses a generative AI model.

[1043] The generated images reflect the user's emotional state and adaptively customize the virtual store environment and product display. The customized images are transmitted from the remote server to the HMD and displayed on the user's HMD display.

[1044] 4. Example of a program and a prompt

[1045] Example 1: Stress-reducing shopping

[1046] 1. The user is feeling stressed and thinks about "relaxation."

[1047] 2. The device analyzes your thoughts and emotions. "You're feeling stressed."

[1048] 3. The server customizes the virtual store environment with a "relaxation" theme (e.g., warm lighting and relaxing music).

[1049] 4. Through the HMD, users can enjoy a shopping experience in a relaxing environment.

[1050] Example prompt:

[1051] "Creates a relaxing environment when the user is feeling stressed"

[1052] Example 2: Shopping to improve concentration

[1053] 1. Users imagine a "quiet forest" to help them concentrate.

[1054] 2. The device analyzes your thoughts and emotions. "I want to improve my concentration."

[1055] 3. The server recreates a quiet forest scene in the virtual store, which promotes concentration.

[1056] 4. Through the HMD, users can experience shopping in a quiet environment where they can concentrate.

[1057] Example prompt:

[1058] "Create a quiet environment to help users concentrate better"

[1059] This system enables a highly personalized virtual store experience based on the user's emotions and thoughts.

[1060] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1061] Step 1:

[1062] The user wears an EEG cap and accesses the virtual store through the HMD. The EEG cap collects the user's EEG data in real time. The input is the user's EEG data, and the output is the collected raw EEG data.

[1063] Step 2:

[1064] The EEG data collected by the device is preprocessed by performing noise removal and filtering. The input is raw EEG data, and the output is EEG data containing noise-removed features. Data processing involves noise filtering and feature extraction.

[1065] Step 3:

[1066] The terminal encrypts the preprocessed EEG data and sends it to a remote server. The input is the preprocessed EEG data, and the output is the encrypted data. The specific operation is to protect the data using an encryption algorithm.

[1067] Step 4:

[1068] The server decrypts the received preprocessed encrypted data and analyzes it using TensorFlow to identify the user's thoughts and emotional state. The input is the decrypted EEG data, and the output is the identified thoughts and emotional state. Data processing involves analysis using a machine learning model.

[1069] Step 5:

[1070] The server uses the Affectiva SDK to perform detailed emotion analysis based on the identified thoughts and emotional state, and generates specific images based on the results. The input is the identified thoughts and emotional state, and the output is the generated specific image. A generative AI model is used to create an image that matches the user's emotions.

[1071] Step 6:

[1072] The server sends the generated image to the user's device. Before sending, adaptive customization can be performed. The input is the generated image, and the output is the customized image. Adaptive customization can include, for example, changing the color tone or layout.

[1073] Step 7:

[1074] The terminal decodes the received image and displays it on the HMD display. The input is customized image data, and the output is visual information displayed on the HMD. Specifically, it performs visualization processing on the display.

[1075] In this way, users can enjoy a highly personalized virtual store experience based on their emotions and thoughts.

[1076] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1077] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1078] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1079] [Third embodiment]

[1080] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1081] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1082] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1083] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1084] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1085] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1086] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1087] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1088] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1089] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1090] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1091] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1092] The present invention relates to a system that acquires images such as a scene envisioned by a user from electroencephalogram data in real time and displays them as concrete images on a display. The configuration and operation of this system will be specifically described below.

[1093] System Configuration

[1094] 1. User device (terminal)

[1095] Users wearing an EEG cap or EEG measuring device provide brainwave data.

[1096] The device measures brain waves in real time and collects initial data.

[1097] 2. Intermediate processing device (terminal)

[1098] After initial collection of EEG data, preprocessing is performed to remove noise and filter it, and to extract features.

[1099] The pre-processed data is encrypted and sent to a remote server.

[1100] 3. Remote Analysis Server (Server)

[1101] The received brainwave data is analyzed to determine what the user was thinking.

[1102] Using AI (artificial intelligence) technology, concrete images are generated based on the content of thoughts.

[1103] 4. Generated image transmission device (server)

[1104] The generated image data is sent to the user's terminal.

[1105] 5. Display Device (Terminal)

[1106] The received image data is displayed on the user's terminal.

[1107] Program processing flow (natural language explanation)

[1108] 1. Data Collection and Preprocessing

[1109] The user wears an EEG cap and conjures up a specific image.

[1110] The device collects the user's brainwave data in real time and stores the initial data in memory.

[1111] The device performs noise removal and filtering to extract specific features.

[1112] 2. Data transmission

[1113] The device encrypts the pre-processed EEG data and transmits it securely to a remote server.

[1114] 3. Data Analysis

[1115] The server analyzes the received data and identifies what the user was thinking, using AI models such as neural networks.

[1116] The server generates specific images based on the identified thoughts. For example, if you think of a seascape, it generates a detailed image corresponding to it.

[1117] 4. Sending and displaying the generated image

[1118] The server transmits the generated image data to the user's terminal.

[1119] The image data received by the terminal is displayed on the display at an appropriate resolution.

[1120] Specific examples

[1121] Example 1: A user with physical limitations visualizes a scene

[1122] 1. User A, who has physical limitations, imagines a scene of a quiet lake.

[1123] 2. The device collects Person A's EEG data and performs noise removal and feature extraction.

[1124] 3. The device sends the preprocessed data to the remote server.

[1125] 4. The server analyzes the data and recognizes that Person A is thinking of a lake.

[1126] 5. The server uses image generation AI to convert the tranquil lake scene into a concrete image.

[1127] 6. The server sends the generated image to Mr. A's device.

[1128] 7. The device displays the received image on the screen, showing the lake scenery that Mr. A had in mind.

[1129] Example 2: Researchers visualize thoughts by analyzing subjects' brain waves

[1130] 1. User research subject B imagines a complex geometric pattern.

[1131] 2. The device collects Mr. B's EEG data and performs preprocessing.

[1132] 3. The device sends the preprocessed data to the remote server.

[1133] 4. The server analyzes the data and recognizes that Person B is thinking of a geometric pattern.

[1134] 5. The server generates a geometric pattern based on the analysis results.

[1135] 6. The server sends the generated image data to the researcher's computer.

[1136] 7. The device displays geometric patterns on a screen, allowing researchers to observe what the subject is thinking.

[1137] In this way, the present invention generates a concrete image of a user's thoughts and images from electroencephalogram data, providing a new means of communication.

[1138] The processing flow will be explained below.

[1139] Step 1:

[1140] The user wears an EEG cap and conjures up a specific image.

[1141] The EEG cap is worn properly and the user is made aware of the scenery or image they have in mind.

[1142] Step 2:

[1143] The device collects the user's brainwave data in real time.

[1144] An electroencephalogram (EEG) measuring device captures the user's brainwave activity and stores the data in memory.

[1145] Step 3:

[1146] The device preprocesses the collected EEG data.

[1147] Clean up the data through a noise removal filter.

[1148] Features are extracted using algorithms such as the fast Fourier transform (FFT).

[1149] Step 4:

[1150] The terminal encrypts the pre-processed data and transmits it to a remote server.

[1151] Data is encrypted using secure communication protocols (e.g. SSL / TLS).

[1152] Sends the encrypted data to the remote server and waits for an acknowledgment (ACK) from the server.

[1153] Step 5:

[1154] The server analyzes the received brain wave data.

[1155] Use AI models (e.g., deep learning neural networks) to identify what users are thinking.

[1156] Based on the analysis results, the image or scenery that the user has in mind is identified.

[1157] Step 6:

[1158] The server generates a concrete image based on the identified thought content.

[1159] Generative AI (e.g., Generative Adversarial Network, GAN) is used to create specific images based on specific content.

[1160] The image is subjected to high resolution and filtering processing as necessary.

[1161] Step 7:

[1162] The server sends the generated image to the user's terminal.

[1163] Convert the generated image data into an appropriate format (e.g., JPEG, PNG).

[1164] The data is compressed and encrypted before being sent to the user's terminal.

[1165] Step 8:

[1166] The device receives the transmitted image and displays it on the screen.

[1167] Decompress and decrypt the received data.

[1168] The image is adjusted to match the display resolution and settings and displayed in real time.

[1169] Through these steps, the scenery or image the user has in mind is reproduced on the display as a concrete image.

[1170] Example 1

[1171] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1172] With current technology, it is difficult to display a user's thoughts and images as concrete images in real time. Furthermore, there are security concerns regarding the processing and transmission of EEG data, and it is also difficult to customize the generated images according to the user's emotional state. Therefore, there is a need for the development of technology that can quickly and safely visualize a user's thoughts.

[1173] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1174] In this invention, the server includes means for encrypting the preprocessed EEG data and transmitting it to a remote server, means for analyzing the preprocessed EEG data using artificial intelligence technology in the remote server to identify the content of the user's thoughts, and means for generating concrete images based on the content of the thoughts, thereby enabling the images that the user has in mind to be analyzed quickly and securely and visually embodied.

[1175] "User" refers to an individual who uses the system and provides EEG data.

[1176] "EEG Data" refers to electronic signal data acquired by a user's EEG cap or EEG measuring device.

[1177] "Means of collecting data in real time" refers to technology that uses an EEG cap or measuring device to instantly obtain a user's brainwave data.

[1178] "Preprocessing" refers to the process of removing noise and filtering the acquired EEG data to extract specific features.

[1179] "Specific features" refer to important patterns and indicators in EEG data, which serve as basic data for analyzing the user's thoughts.

[1180] "Encryption means" refers to technology that converts data to make it difficult for third parties to decipher, in order to ensure the security of brainwave data.

[1181] "Remote Server" refers to a server located at a remote location that receives, stores, and analyzes data sent from a user's device.

[1182] "Artificial intelligence technology" refers to technology that uses machine learning, deep learning, etc. to analyze data and draw conclusions and predictions.

[1183] "Thought content" refers to what the user specifically imagines or thinks about in their head.

[1184] "Means for generating concrete images" refers to technology that uses an image generation AI model to generate visual images based on the user's thoughts.

[1185] A "prompt sentence" refers to a sentence used as an instruction or input to an image-generating AI model.

[1186] "Means for displaying on a display" refers to the technology used to visually display images generated on the terminal on a screen.

[1187] "Emotional state" refers to the user's psychological or emotional state derived from EEG data.

[1188] The present invention relates to a system that acquires an image that a user has in mind from electroencephalogram data in real time and displays it as a concrete image. The configuration and operation of this system will be specifically described below.

[1189] System Configuration

[1190] 1. User device (terminal)

[1191] The user provides the brainwave data by wearing an EEG cap or EEG measurement device, which typically connects to a terminal using Bluetooth or other wireless communication technology.

[1192] The device measures brain waves in real time and collects initial data.

[1193] 2. Intermediate processing device (terminal)

[1194] After initial collection of EEG data, preprocessing is performed to remove noise and filter it, and to extract features. Specifically, the FFT (Fast Fourier Transform) algorithm is used.

[1195] The preprocessed data is encrypted and sent to a remote server using AES (Advanced Encryption Standard).

[1196] 3. Remote Analysis Server (Server)

[1197] The received EEG data is analyzed to identify what the user was thinking, using deep learning models (e.g., TensorFlow or PyTorch).

[1198] Based on the identified thought content, an image generation AI model (e.g., Stable Diffusion or DALL-E) is used to generate concrete images.

[1199] 4. Generated image transmission device (server)

[1200] The generated image data is then re-encrypted and sent to the user's device using the HTTPS protocol.

[1201] 5. Display Device (Terminal)

[1202] The received image data is decoded on the user's device and displayed on the display at an appropriate resolution.

[1203] Program processing flow

[1204] 1. The user wears the EEG cap and imagines a specific image. For example, the user imagines a "tranquil lake scene."

[1205] 2. The device collects the user's brainwave data in real time and stores the initial data in memory.

[1206] 3. The device uses an FFT algorithm to remove noise and filter the data, and extract specific features.

[1207] 4. The device encrypts the preprocessed EEG data with AES and transmits it to a remote server using the HTTPS protocol.

[1208] 5. The server uses a deep learning model to analyze the received EEG data and determine that the user is thinking of a "lake scene."

[1209] 6. The server uses an image generation AI model to generate a detailed image corresponding to the "lake scene."

[1210] 7. The server re-encrypts the generated image data and sends it to the user's device using the HTTPS protocol.

[1211] 8. The device decrypts the encrypted data and displays it on the display in full HD resolution.

[1212] Specific examples

[1213] Example 1: A user with physical limitations visualizes a scene

[1214] 1. A user with a physical limitation imagines a scene of a tranquil lake.

[1215] 2. The device collects the user's brainwave data and performs preprocessing.

[1216] 3. The device encrypts the preprocessed data with AES and sends it to the remote server.

[1217] 4. The server uses a deep learning model to analyze the data and determine that the user is thinking of a lake.

[1218] 5. The server uses image generation AI to convert the tranquil lake scene into a concrete image.

[1219] 6. The server re-encrypts the generated image and sends it to the user's device.

[1220] 7. The device decodes the image and displays it on the screen, allowing the user to visually confirm the lakescape they had in mind.

[1221] Prompt Sentence Examples

[1222] "Show me the image generated as a result of imagining a tranquil lake scene."

[1223] "Generate an image based on the results of EEG data analysis when imagining a complex geometric pattern."

[1224] In this way, the present invention makes it possible to materialize the user's thoughts and images from electroencephalogram data into concrete images and display them on a display.

[1225] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1226] Step 1:

[1227] The user wears an EEG cap and conjures up a specific image.

[1228] Specific operation: The user wears an EEG cap, sits in a chair, and relaxes while imagining a "lake scene."

[1229] Input: An image imagined by the user in a relaxed state.

[1230] Output: EEG data collected in real time.

[1231] Step 2:

[1232] The device collects the user's brainwave data in real time and stores it in memory.

[1233] Specific operation: The signal from the EEG cap is transmitted to the terminal via wireless communication technology (e.g., Bluetooth), and initial data is stored in the memory.

[1234] Input: Radio signal from EEG cap.

[1235] Output: Initial EEG data stored in memory.

[1236] Step 3:

[1237] The device performs noise removal and filtering on the EEG data to extract specific features.

[1238] Specific operation: Noise removal and filtering are performed using an FFT (Fast Fourier Transform) algorithm, and specific features such as alpha waves and beta waves are extracted.

[1239] Input: Initial EEG data stored in memory.

[1240] Output: Preprocessed EEG data with extracted features.

[1241] Step 4:

[1242] The device encrypts the preprocessed EEG data and transmits it to a remote server.

[1243] What it does: Encrypts data using AES (Advanced Encryption Standard) and sends it to a remote server via the HTTPS protocol.

[1244] Input: Preprocessed EEG data with extracted features.

[1245] Output: Encrypted EEG data is sent to a remote server.

[1246] Step 5:

[1247] The server receives the encrypted data, decrypts it, and then analyzes it using a deep learning model to determine what the user is thinking.

[1248] Specific operation: The received data is decrypted using AES, and then the user's thoughts are analyzed using a deep learning model (e.g., TensorFlow or PyTorch).

[1249] Input: Encrypted EEG data.

[1250] Output: Decoded data and analyzed thought (e.g., "lakescape").

[1251] Step 6:

[1252] The server uses an image generation AI model to generate specific images based on the identified thought content.

[1253] Specific operation: Generate an image equivalent to a "lake scene" using an image generation AI model (e.g., Stable Diffusion or DALL-E).

[1254] Input: Parsed thoughts.

[1255] Output: The concrete image generated.

[1256] Step 7:

[1257] The server re-encrypts the generated image data and sends it to the user's terminal.

[1258] Specific operation: The generated image data is encrypted with AES and sent to the user's device via the HTTPS protocol.

[1259] Input: The concrete image to be generated.

[1260] Output: The encrypted image data is sent to the user's device.

[1261] Step 8:

[1262] The terminal decodes the received image data and displays it on the display at the appropriate resolution.

[1263] Specific operation: The received encrypted data is decrypted using AES and displayed on the display in full HD resolution.

[1264] Input: Encrypted image data.

[1265] Output: The concrete image (picture) shown on the display.

[1266] (Application example 1)

[1267] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1268] Traditional methods have made it difficult to efficiently and accurately share work and repair procedures with other engineers within a factory. This has led to communication errors, work delays, and reduced productivity. Furthermore, the lack of a way to directly visualize engineers' thinking has made it difficult for experienced engineers to share their knowledge effectively.

[1269] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1270] In this invention, the server includes means for preprocessing the user's electroencephalogram data and extracting specific features, means for analyzing the preprocessed electroencephalogram data and identifying the user's thoughts, and means for generating concrete images based on the thoughts, thereby enabling the engineer's thoughts to be embodied and shared with other engineers in real time.

[1271] "User" refers to the entity that provides the EEG data, and is primarily a technician who performs work and repairs within the factory.

[1272] "EEG data" refers to data collected by an EEG measuring device when a user has a particular thought or image in their mind.

[1273] "Preprocessing" refers to data processing means for removing noise from electroencephalogram data and extracting specific features.

[1274] "Remote server" refers to a server device located remotely for receiving and analyzing electroencephalogram data.

[1275] A "neural network" is a type of artificial intelligence technology and a deep learning model used to analyze EEG data and generate images.

[1276] "Concrete imagery" refers to visual images and figures generated based on analyzed EEG data.

[1277] "Terminal" refers to equipment, including devices such as computers and tablets used by users and technicians.

[1278] A "display" is a display device installed in a terminal, and is a device for visually displaying the generated specific image.

[1279] "Sharing" refers to the viewing and use of the generated concrete image in real time among multiple engineers.

[1280] The present invention is a system for efficiently and accurately sharing work and repair procedures with other engineers in a factory, and a specific embodiment thereof will be described.

[1281] System Configuration

[1282] Hardware

[1283] 1. EEG Cap

[1284] This device is worn by the user and collects brainwave data in real time. It is used by engineers working in factories.

[1285] 2. EEG measurement device

[1286] This device works in conjunction with the EEG cap and transmits collected EEG data to a terminal.

[1287] 3. Terminal

[1288] A general computing device such as a PC or tablet used by engineers, and is mainly used for data preprocessing and display.

[1289] 4. Remote Server

[1290] This is a server installed outside the factory that analyzes EEG data and generates images.

[1291] software

[1292] 1. Preprocessing software

[1293] It runs on the device and is responsible for removing noise from EEG data and extracting specific features.

[1294] 2. AI analysis software

[1295] It runs on a remote server and uses artificial intelligence techniques such as neural networks to analyze brainwave data and generate concrete images.

[1296] 3. Image display application

[1297] It is an application that runs on the terminal and displays the generated image to the user.

[1298] System Operation Description

[1299] Data Collection Phase

[1300] The user wears an EEG cap and thinks about a specific work procedure or repair method. The EEG measurement device collects brainwave data in real time and transmits it to a terminal. Pre-processing software on the terminal removes noise and filters the data, and extracts features.

[1301] Data transmission phase

[1302] After preprocessing, the EEG data is encrypted on the device and securely transmitted to a remote server, ensuring data security.

[1303] Data analysis phase

[1304] The remote server uses AI analysis software to analyze the received EEG data, using deep learning models such as neural networks to identify what the user is thinking and generate specific images based on that.

[1305] Image Generation and Display Phase

[1306] The generated images are then sent back to the terminal from the remote server and displayed on the terminal's image display application, allowing the user to visualize the work or repair procedures they envision as concrete images that can be shared with other technicians.

[1307] Specific examples

[1308] When performing machine maintenance

[1309] Engineer A puts on an EEG cap and thinks about the maintenance procedures for a specific machine. The terminal collects A's EEG data and performs preprocessing. The preprocessed data is sent to a remote server, where AI analysis software analyzes it and generates an image of the maintenance procedures. The generated image is sent to Engineer A's terminal and shared with other engineers. This process makes work more efficient.

[1310] Prompt Sentence Examples

[1311] "Analyze EEG data to generate a detailed image of a specific repair procedure for a machine."

[1312] As described above, the present invention provides a system that enables factory engineers to visualize and efficiently share their thoughts using electroencephalogram data.

[1313] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1314] Step 1:

[1315] The user wears the EEG cap and thinks about a specific work procedure or repair method. As input, the user's brainwave data is collected in real time. As output, the real-time brainwave signal is sent to the EEG measurement device. Specifically, the EEG cap detects the user's brainwaves and sends the signal to the EEG measurement device.

[1316] Step 2:

[1317] The terminal receives EEG data in real time from the EEG measurement device and performs preprocessing. The input is the collected EEG data. The output is preprocessed data with noise removed and features extracted. Specifically, the preprocessing software applies a noise removal filter and executes a feature extraction algorithm.

[1318] Step 3:

[1319] The device encrypts the preprocessed data and sends it to a remote server. The input is the preprocessed EEG data. The output is the encrypted data sent to the remote server. Specifically, encryption software on the device encrypts the preprocessed data and transfers the data to the server using a secure communication protocol.

[1320] Step 4:

[1321] The server receives and decrypts the encrypted data. The input is the encrypted EEG data. The output is the decrypted data. The specific operation is to decrypt the data using a secure key management system and prepare it for data analysis.

[1322] Step 5:

[1323] The server analyzes the decoded data using AI analysis software. The input is the decoded EEG data. The output is the user's thoughts. Specifically, the AI ​​analysis software analyzes the data using neural networks and generative AI models to identify the user's thoughts.

[1324] Step 6:

[1325] The server generates a specific image based on the identified thought content. The input is data that identifies the thought content. The output is the generated specific image. As a specific action, the generative AI model generates a detailed image based on the analysis results. An example of a prompt statement would be, "Analyze specific repair procedures for a certain machine from EEG data and generate a detailed image."

[1326] Step 7:

[1327] The server sends the generated image data back to the terminal. The generated image data is the input. The image data is sent to the terminal as the output. In concrete terms, the sending software on the server transfers the generated image data to the terminal.

[1328] Step 8:

[1329] The terminal receives the image data and displays it in the image display application. The input is the transmitted image data. The output is a specific image displayed on the display. The specific operation is that the image display application interprets the image data and displays it on the display device.

[1330] This allows engineers to share their thoughts as concrete images with other engineers, improving work efficiency.

[1331] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1332] This invention relates to a system that interprets the user's thoughts and emotional state based on their electroencephalogram data, generates specific images based on this, and displays them on a display. In particular, by combining it with an emotion engine, it becomes possible to generate images that take the user's emotional state into consideration. The configuration and operation of this system will be specifically described.

[1333] System Configuration

[1334] 1. User device (terminal)

[1335] Users wear an EEG cap or EEG measuring device and provide brainwave data.

[1336] The device measures brain waves in real time and collects initial data.

[1337] 2. Intermediate processing device (terminal)

[1338] After initial collection of EEG data, preprocessing such as noise removal is performed to extract specific features.

[1339] After preprocessing, the data is encrypted and sent to a remote server.

[1340] 3. Remote Analysis Server (Server)

[1341] The user's thoughts are analyzed based on the received brain wave data.

[1342] AI technology converts the thoughts into concrete images.

[1343] An emotion engine is used to recognize the user's emotional state.

[1344] 4. Generated image transmission device (server)

[1345] The generated image is sent to the user's device.

[1346] Customize images based on information from the emotion engine.

[1347] 5. Display Device (Terminal)

[1348] The image data received by the user's terminal is decoded and displayed on the screen.

[1349] Program processing flow (natural language explanation)

[1350] 1. Data Collection and Preprocessing

[1351] The user wears an EEG cap to sense their thoughts and emotions.

[1352] The device collects the user's brainwave data in real time and stores the initial data in memory.

[1353] The device performs noise removal and filtering to extract specific features.

[1354] 2. Data transmission

[1355] The device encrypts the pre-processed EEG data and transmits it to a remote server.

[1356] 3. Data Analysis

[1357] The server analyzes the received brainwave data and identifies what the user was thinking.

[1358] At the same time, the server uses an emotion engine to determine the user's emotional state.

[1359] 4. Emotion-based image generation

[1360] The server uses the analysis results of the emotion engine to generate specific images that correspond to the user's emotional state.

[1361] 5. Sending and displaying the generated image

[1362] When the server sends the generated image to the user's device, it adaptively customizes the image based on information from the emotion engine.

[1363] The terminal decompresses and decodes the received image data and displays it on the display.

[1364] Specific examples

[1365] Example 1: Landscape imagery for stress reduction

[1366] 1. The user visualizes a relaxing beach scene while feeling stressed at work.

[1367] 2. The device collects EEG data, removes noise, and extracts features.

[1368] 3. The device sends the preprocessed data to the remote server.

[1369] 4. The server analyzes the data and determines that the user is thinking of a relaxing beach scene.

[1370] 5. At the same time, the server uses an emotion engine to detect when the user is in a high stress state.

[1371] 6. The server uses AI to generate a beach scene with colors and composition that enhance the relaxing effect.

[1372] 7. The server sends the generated image to the user's device, and the adaptively customized beach scene is displayed on the screen.

[1373] Example 2: Imagery to improve concentration

[1374] 1. Imagine a forest scene that helps users improve their focus while studying for exams.

[1375] 2. The device collects and preprocesses EEG data.

[1376] 3. The device sends the preprocessed data to the remote server.

[1377] 4. The server analyzes the data and determines that the user is imagining a forest scene to improve concentration.

[1378] 5. At the same time, the server detects the user's concentration level using an emotion engine.

[1379] 6. The server uses AI to generate a forest landscape suitable for maintaining concentration.

[1380] 7. The server sends the generated image to the user's device, and a forest scene that encourages concentration is displayed on the screen.

[1381] In this way, by combining emotion engines, it becomes possible to generate images that take into account not only the user's thoughts but also their emotional state, providing a more personalized experience.

[1382] The processing flow will be explained below.

[1383] Step 1:

[1384] The user wears an EEG cap and conjures up a specific image.

[1385] The user wears an EEG cap on their head and is asked to consciously visualize a specific scene or image.

[1386] Step 2:

[1387] The device collects the user's brainwave data in real time.

[1388] The electroencephalogram measuring device captures the user's electroencephalogram signals and transmits the signal data to the terminal.

[1389] Step 3:

[1390] The device preprocesses the collected EEG data.

[1391] Data cleansing removes noise and improves data quality.

[1392] Features are extracted using algorithms such as the fast Fourier transform (FFT).

[1393] Step 4:

[1394] The terminal encrypts the pre-processed data and transmits it to a remote server.

[1395] The EEG data from which the features have been extracted is encrypted using the SSL / TLS protocol.

[1396] Securely transmit encrypted data over the Internet to a remote server.

[1397] Step 5:

[1398] The server analyzes the received brain wave data.

[1399] It uses AI models (such as deep learning neural networks) to analyze data and identify what users are thinking.

[1400] Step 6:

[1401] The server uses an emotion engine to recognize the user's emotional state.

[1402] The emotion engine uses the received data to identify the user's stress level, concentration level, and type of emotion.

[1403] Step 7:

[1404] The server generates concrete images based on the identified thoughts and emotional states.

[1405] Using image generation AI (e.g., GAN), specific images are generated based on the identified thought content and emotional state.

[1406] The image's color, brightness, and composition are adaptively adjusted to match the user's emotional state.

[1407] Step 8:

[1408] The server sends the generated image to the user's terminal.

[1409] The generated image data is compressed and converted into an appropriate format (e.g., JPEG, PNG).

[1410] The data is encrypted using the SSL / TLS protocol and sent to the user's device.

[1411] Step 9:

[1412] The device receives the transmitted image and displays it on the screen.

[1413] The received image data is decompressed and decoded.

[1414] The image is adjusted to fit the display resolution and displayed in real time.

[1415] This allows images based on the scenery and emotions imagined by the user to be embodied and displayed on the screen.

[1416] Example 2

[1417] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1418] Conventional EEG data analysis systems have difficulty accurately grasping a user's emotional state and thoughts, resulting in images that often do not match the user's expectations. Furthermore, insufficient real-time data collection and processing can result in a poor user experience. Furthermore, insufficient privacy protection for collected data raises concerns about the risk of information leaks. A system that can resolve these issues and generate images that are both accurate and optimized for the user is needed.

[1419] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1420] In this invention, the server includes means for analyzing the preprocessed electroencephalogram data to identify the user's thoughts, means for generating specific images based on the thoughts, and means for identifying the user's emotional state using an emotion recognition engine and customizing the images according to the emotional state, thereby enabling the generation of personalized images that accurately reflect the user's thoughts and emotional state.

[1421] "User" means a person who provides EEG data and uses the system.

[1422] "EEG data" refers to data that measures a user's brainwave activity in real time.

[1423] "Preprocessing" refers to the process of removing noise from collected EEG data and extracting specific features.

[1424] "Features" are important data points or patterns extracted from preprocessed EEG data.

[1425] "Encryption" is the process of transforming data using a specific algorithm in order to transmit it securely.

[1426] A "remote server" is a server that can receive and process data via the Internet.

[1427] "Analysis" refers to data processing to identify the user's thoughts and emotional state based on brainwave data.

[1428] A "concrete image" is an image or figure that visually expresses the user's thoughts.

[1429] An "emotion recognition engine" is software or algorithms that analyze a user's emotional state based on brainwave data.

[1430] "Customization" refers to the process of adjusting the color tone and composition of the generated image according to the user's particular emotional state.

[1431] "Decryption" is the process of restoring encrypted data to its original state.

[1432] "Display" means a display device that allows a user to visually view the generated image.

[1433] This invention relates to a system that interprets the user's thoughts and emotional state based on their electroencephalogram data, generates specific images based on this, and displays them on a display. In particular, by combining it with an emotion engine, it is possible to generate images that take the user's emotional state into consideration. The configuration and operation of this system will be specifically explained.

[1434] System Configuration

[1435] 1. User device (terminal)

[1436] The user wears an EEG cap or EEG measurement device (e.g., Emotiv Epoc+) and provides brainwave data.

[1437] The device is used to measure brain waves in real time and collect initial data, which is collected using software such as Emotiv Pro and MATLAB.

[1438] 2. Intermediate processing device (terminal)

[1439] After initial collection of EEG data, preprocessing such as noise removal is performed to extract specific features (e.g., alpha waves, beta waves).

[1440] For preprocessing, we use Python libraries such as NumPy and SciPy.

[1441] 3. Remote Analysis Server (Server)

[1442] The user's thoughts are analyzed based on the received brain wave data.

[1443] Use an AI analysis engine (e.g., TensorFlow, PyTorch) to convert thought content into concrete images.

[1444] Recognize the user's emotional state using an emotion engine (e.g., Affectiva, IBM Watson).

[1445] 4. Generated image transmission device (server)

[1446] The generated image is sent to the user's device.

[1447] Adaptively customize images based on sentiment analysis results.

[1448] 5. Display Device (Terminal)

[1449] The image data received by the user's device is decoded and displayed on a monitor or VR headset.

[1450] Specific examples

[1451] Example 1: Landscape imagery for stress reduction

[1452] 1. The user visualizes a relaxing beach scene while feeling stressed at work.

[1453] 2. The device collects EEG data, removes noise, and extracts features.

[1454] 3. The device sends the preprocessed data to the remote server.

[1455] 4. The server analyzes the data and determines that the user is thinking of a relaxing beach scene.

[1456] 5. At the same time, the server uses an emotion engine to detect when the user is in a high stress state.

[1457] 6. The server uses AI to generate a relaxing beach scene.

[1458] 7. The server sends the generated image to the user's terminal and displays it on the display.

[1459] Prompt Sentence Examples

[1460] "Suppose a user is under high stress at work, but imagines a relaxing beach scene. In this state, a customized beach scene that maximizes the relaxation effect is generated and displayed on the display."

[1461] Example 2: Imagery to improve concentration

[1462] 1. Imagine a forest scene that helps users improve their focus while studying for exams.

[1463] 2. The device collects EEG data, removes noise, and extracts features.

[1464] 3. The device sends the preprocessed data to the remote server.

[1465] 4. The server analyzes the data and determines that the user is imagining a forest scene to improve concentration.

[1466] 5. At the same time, the server detects the user's concentration level using an emotion engine.

[1467] 6. The server uses AI to generate a forest landscape to help you maintain concentration.

[1468] 7. The server sends the generated image to the user's terminal and displays it on the display.

[1469] Prompt Sentence Examples

[1470] "Imagine a tranquil forest scene that helps users concentrate better while studying for an exam. In this state, we generate a customized forest scene that maximizes concentration and display it on the display."

[1471] Thus, a system for generating and displaying personalized images based on a user's thought content and emotional state has been described in detail as an embodiment of the present invention.

[1472] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1473] System program processing flow

[1474] Step 1: Data collection

[1475] The user wears an EEG cap and focuses their thoughts and emotions.

[1476] 1. The user puts on the EEG cap.

[1477] Input: EEG cap worn on user's head

[1478] Output: Begins collecting user's brainwave data in real time.

[1479] 2. The device collects the user's brainwave data in real time.

[1480] Operation: Data measured by the EEG cap is transferred to the terminal and saved in memory as initial data.

[1481] Input: User's EEG data

[1482] Output: Initial EEG data

[1483] Step 2: Data Preprocessing

[1484] 1. Preprocess the EEG data collected by the device.

[1485] What it does: Applies a noise reduction filter to remove unwanted noise from EEG data.

[1486] Input: Initial EEG data

[1487] Output: Noise-removed EEG data

[1488] 2. The device extracts specific features.

[1489] How it works: It uses Python's NumPy and SciPy libraries to extract specific patterns such as alpha and beta waves.

[1490] Input: Noise-removed EEG data

[1491] Output: Data with extracted features

[1492] Step 3: Encrypt and send data

[1493] 1. The device encrypts the data.

[1494] How it works: Encrypts the feature-extracted data using the AES encryption algorithm.

[1495] Input: Data with extracted features

[1496] Output: Encrypted data

[1497] 2. The device sends the encrypted data to a remote server.

[1498] How it works: Sends encrypted data to the server using the HTTPS protocol.

[1499] Input: Encrypted data

[1500] Output: Data sent to server completed

[1501] Step 4: Data analysis

[1502] 1. The server receives the encrypted data.

[1503] How it works: The server receives the data sent and decrypts it using the AES decryption algorithm.

[1504] Input: Encrypted data

[1505] Output: Decrypted data

[1506] 2. The server analyzes the data using its analysis engine.

[1507] How it works: Uses an AI analysis engine (e.g. TensorFlow, PyTorch) to identify the user's thoughts from EEG data.

[1508] Input: Decrypted data

[1509] Output: What the user is thinking

[1510] 3. The server uses the emotion engine to identify the emotional state.

[1511] How it works: Uses an emotion recognition engine (e.g., Affectiva, IBM Watson) to analyze the user's emotional state.

[1512] Input: User's EEG data

[1513] Output: User's emotional state

[1514] Step 5: Image Creation and Customization

[1515] 1. The server generates an image based on the analysis results.

[1516] How it works: It uses generative AI models (e.g., DALL·E, StyleGAN) to generate tangible images based on what the user is thinking.

[1517] Input: What the user is thinking

[1518] Output: The concrete image generated

[1519] 2. The server customizes the image based on the analysis results of the emotion engine.

[1520] How it works: Based on the results of emotion analysis, it adjusts the color and composition of the image to create a more personalized image.

[1521] Input: Generated concrete image, user's emotional state

[1522] Output: Customized image

[1523] Step 6: Sending and displaying image data

[1524] 1. The server encrypts the generated image and sends it to the user's device.

[1525] What it does: The resulting image is encrypted and sent, again using the AES encryption algorithm.

[1526] Input: Customized image

[1527] Output: Encrypted image data

[1528] 2. The device receives and decrypts the encrypted image data.

[1529] What it does: Uses the AES decryption algorithm to decrypt any encrypted data received.

[1530] Input: Encrypted image data

[1531] Output: Decoded image data

[1532] 3. The terminal displays the decrypted image on the display.

[1533] What it does: Displays the decoded image on a display device (e.g., monitor, VR headset).

[1534] Input: Decoded image data

[1535] Output: The image displayed on the screen

[1536] (Application example 2)

[1537] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1538] In recent years, there has been a demand for customization based on individual user needs and conditions to improve the consumer experience in virtual stores. However, current systems have difficulty in real-time adaptive customization based on the user's emotions and thoughts, limiting the improvement of consumer satisfaction. In addition, there has been a lack of effective means to provide the optimal products and environment that users desire. New technologies that can solve this problem and dramatically improve the user experience are needed.

[1539] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a device that collects the user's electroencephalogram data in real time, a device that preprocesses the electroencephalogram data and extracts specific features, and a device that transmits the preprocessed electroencephalogram data to a remote server. This enables optimal customization within the virtual store based on the user's thoughts and emotional state.

[1540] A "user" is a person who uses the system to enjoy an experience within a virtual store.

[1541] "EEG data" is a measurement of the electrical signals generated by the user's brain.

[1542] A "collection device" is a device for measuring and collecting EEG data in real time.

[1543] "Preprocessing" refers to the process of removing noise and extracting features from collected EEG data.

[1544] A "feature" is a specific index or pattern extracted from electroencephalogram data.

[1545] The "remote server" is a server at a remote location that receives and analyzes pre-processed EEG data.

[1546] The "analysis device" is a device that runs on a remote server and analyzes pre-processed EEG data to identify the user's thoughts and emotional state.

[1547] "Emotional state" refers to the mental and psychological state of a user as determined from their brainwave data.

[1548] "Concrete images" are visual representations that are generated based on the user's thoughts and emotional state.

[1549] "Customization" refers to individually adjusting the display content and environment within a virtual store according to the identified emotional state.

[1550] A "transmitting device" is a device for transmitting the generated image to a user's terminal.

[1551] A "display device" is a device for displaying images received at a user's terminal on a display.

[1552] A "display" is a screen or monitor used to visually display information on a user's device.

[1553] A "virtual store" refers to an environment in which products can be viewed and purchased in a virtual space just as if they were in a physical store.

[1554] MODE FOR CARRYING OUT THE INVENTION

[1555] This invention allows users to enjoy a customized virtual store experience based on their electroencephalogram (EEG) data. This system collects the user's EEG data in real time, extracts features, and then transmits the data to a remote server for analysis and image generation. The generated image is then transmitted to the user's device, where an adaptively customized virtual store is displayed.

[1556] 1. System program generation

[1557] The program for this system is constructed using the following hardware and software.

[1558] 2. Hardware and Software

[1559] Hardware used:

[1560] EEG Cap: Emotiv Insight

[1561] HMD: Oculus Quest 2

[1562] Software used:

[1563] Data analysis: TensorFlow

[1564] Emotion engine: Affectiva SDK

[1565] 3. Program processing explanation

[1566] Users wear an EEG cap and access the virtual store through an HMD. The EEG cap collects the user's brainwave data in real time and performs preprocessing such as noise reduction and filtering. The preprocessed data is encrypted and transmitted to a remote server via the HMD terminal.

[1567] The server uses TensorFlow to analyze the received EEG data to identify the user's thoughts and emotional state. The emotional state is then analyzed in detail using the Affectiva SDK. Based on the analysis results, a specific image that matches the user's emotional state is generated. This image generation process uses a generative AI model.

[1568] The generated images reflect the user's emotional state and adaptively customize the virtual store environment and product display. The customized images are transmitted from the remote server to the HMD and displayed on the user's HMD display.

[1569] 4. Example of a program and a prompt

[1570] Example 1: Stress-reducing shopping

[1571] 1. The user is feeling stressed and thinks about "relaxation."

[1572] 2. The device analyzes your thoughts and emotions. "You're feeling stressed."

[1573] 3. The server customizes the virtual store environment with a "relaxation" theme (e.g., warm lighting and relaxing music).

[1574] 4. Through the HMD, users can enjoy a shopping experience in a relaxing environment.

[1575] Example prompt:

[1576] "Creates a relaxing environment when the user is feeling stressed"

[1577] Example 2: Shopping to improve concentration

[1578] 1. Users imagine a "quiet forest" to help them concentrate.

[1579] 2. The device analyzes your thoughts and emotions. "I want to improve my concentration."

[1580] 3. The server recreates a quiet forest scene in the virtual store, which promotes concentration.

[1581] 4. Through the HMD, users can experience shopping in a quiet environment where they can concentrate.

[1582] Example prompt:

[1583] "Create a quiet environment to help users concentrate better"

[1584] This system enables a highly personalized virtual store experience based on the user's emotions and thoughts.

[1585] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1586] Step 1:

[1587] The user wears an EEG cap and accesses the virtual store through the HMD. The EEG cap collects the user's EEG data in real time. The input is the user's EEG data, and the output is the collected raw EEG data.

[1588] Step 2:

[1589] The EEG data collected by the device is preprocessed by performing noise removal and filtering. The input is raw EEG data, and the output is EEG data containing noise-removed features. Data processing involves noise filtering and feature extraction.

[1590] Step 3:

[1591] The terminal encrypts the preprocessed EEG data and sends it to a remote server. The input is the preprocessed EEG data, and the output is the encrypted data. The specific operation is to protect the data using an encryption algorithm.

[1592] Step 4:

[1593] The server decrypts the received preprocessed encrypted data and analyzes it using TensorFlow to identify the user's thoughts and emotional state. The input is the decrypted EEG data, and the output is the identified thoughts and emotional state. Data processing involves analysis using a machine learning model.

[1594] Step 5:

[1595] The server uses the Affectiva SDK to perform detailed emotion analysis based on the identified thoughts and emotional state, and generates specific images based on the results. The input is the identified thoughts and emotional state, and the output is the generated specific image. A generative AI model is used to create an image that matches the user's emotions.

[1596] Step 6:

[1597] The server sends the generated image to the user's device. Before sending, adaptive customization can be performed. The input is the generated image, and the output is the customized image. Adaptive customization can include, for example, changing the color tone or layout.

[1598] Step 7:

[1599] The terminal decodes the received image and displays it on the HMD display. The input is customized image data, and the output is visual information displayed on the HMD. Specifically, it performs visualization processing on the display.

[1600] In this way, users can enjoy a highly personalized virtual store experience based on their emotions and thoughts.

[1601] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1602] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1603] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1604] [Fourth embodiment]

[1605] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1606] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1607] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1608] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1609] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1610] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1611] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1612] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1613] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1614] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1615] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1616] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1617] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1618] The present invention relates to a system that acquires images such as a scene envisioned by a user from electroencephalogram data in real time and displays them as concrete images on a display. The configuration and operation of this system will be specifically described below.

[1619] System Configuration

[1620] 1. User device (terminal)

[1621] Users wearing an EEG cap or EEG measuring device provide brainwave data.

[1622] The device measures brain waves in real time and collects initial data.

[1623] 2. Intermediate processing device (terminal)

[1624] After initial collection of EEG data, preprocessing is performed to remove noise and filter it, and to extract features.

[1625] The pre-processed data is encrypted and sent to a remote server.

[1626] 3. Remote Analysis Server (Server)

[1627] The received brainwave data is analyzed to determine what the user was thinking.

[1628] Using AI (artificial intelligence) technology, concrete images are generated based on the content of thoughts.

[1629] 4. Generated image transmission device (server)

[1630] The generated image data is sent to the user's terminal.

[1631] 5. Display Device (Terminal)

[1632] The received image data is displayed on the user's terminal.

[1633] Program processing flow (natural language explanation)

[1634] 1. Data Collection and Preprocessing

[1635] The user wears an EEG cap and conjures up a specific image.

[1636] The device collects the user's brainwave data in real time and stores the initial data in memory.

[1637] The device performs noise removal and filtering to extract specific features.

[1638] 2. Data transmission

[1639] The device encrypts the pre-processed EEG data and transmits it securely to a remote server.

[1640] 3. Data Analysis

[1641] The server analyzes the received data and identifies what the user was thinking, using AI models such as neural networks.

[1642] The server generates specific images based on the identified thoughts. For example, if you think of a seascape, it generates a detailed image corresponding to it.

[1643] 4. Sending and displaying the generated image

[1644] The server transmits the generated image data to the user's terminal.

[1645] The image data received by the terminal is displayed on the display at an appropriate resolution.

[1646] Specific examples

[1647] Example 1: A user with physical limitations visualizes a scene

[1648] 1. User A, who has physical limitations, imagines a scene of a quiet lake.

[1649] 2. The device collects Person A's EEG data and performs noise removal and feature extraction.

[1650] 3. The device sends the preprocessed data to the remote server.

[1651] 4. The server analyzes the data and recognizes that Person A is thinking of a lake.

[1652] 5. The server uses image generation AI to convert the tranquil lake scene into a concrete image.

[1653] 6. The server sends the generated image to Mr. A's device.

[1654] 7. The device displays the received image on the screen, showing the lake scenery that Mr. A had in mind.

[1655] Example 2: Researchers visualize thoughts by analyzing subjects' brain waves

[1656] 1. User research subject B imagines a complex geometric pattern.

[1657] 2. The device collects Mr. B's EEG data and performs preprocessing.

[1658] 3. The device sends the preprocessed data to the remote server.

[1659] 4. The server analyzes the data and recognizes that Person B is thinking of a geometric pattern.

[1660] 5. The server generates a geometric pattern based on the analysis results.

[1661] 6. The server sends the generated image data to the researcher's computer.

[1662] 7. The device displays geometric patterns on a screen, allowing researchers to observe what the subject is thinking.

[1663] In this way, the present invention generates a concrete image of a user's thoughts and images from electroencephalogram data, providing a new means of communication.

[1664] The processing flow will be explained below.

[1665] Step 1:

[1666] The user wears an EEG cap and conjures up a specific image.

[1667] The EEG cap is worn properly and the user is made aware of the scenery or image they have in mind.

[1668] Step 2:

[1669] The device collects the user's brainwave data in real time.

[1670] An electroencephalogram (EEG) measuring device captures the user's brainwave activity and stores the data in memory.

[1671] Step 3:

[1672] The device preprocesses the collected EEG data.

[1673] Clean up the data through a noise removal filter.

[1674] Features are extracted using algorithms such as the fast Fourier transform (FFT).

[1675] Step 4:

[1676] The terminal encrypts the pre-processed data and transmits it to a remote server.

[1677] Data is encrypted using secure communication protocols (e.g. SSL / TLS).

[1678] Sends the encrypted data to the remote server and waits for an acknowledgment (ACK) from the server.

[1679] Step 5:

[1680] The server analyzes the received brain wave data.

[1681] Use AI models (e.g., deep learning neural networks) to identify what users are thinking.

[1682] Based on the analysis results, the image or scenery that the user has in mind is identified.

[1683] Step 6:

[1684] The server generates a concrete image based on the identified thought content.

[1685] Generative AI (e.g., Generative Adversarial Network, GAN) is used to create specific images based on specific content.

[1686] The image is subjected to high resolution and filtering processing as necessary.

[1687] Step 7:

[1688] The server sends the generated image to the user's terminal.

[1689] Convert the generated image data into an appropriate format (e.g., JPEG, PNG).

[1690] The data is compressed and encrypted before being sent to the user's terminal.

[1691] Step 8:

[1692] The device receives the transmitted image and displays it on the screen.

[1693] Decompress and decrypt the received data.

[1694] The image is adjusted to match the display resolution and settings and displayed in real time.

[1695] Through these steps, the scenery or image the user has in mind is reproduced on the display as a concrete image.

[1696] Example 1

[1697] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1698] With current technology, it is difficult to display a user's thoughts and images as concrete images in real time. Furthermore, there are security concerns regarding the processing and transmission of EEG data, and it is also difficult to customize the generated images according to the user's emotional state. Therefore, there is a need for the development of technology that can quickly and safely visualize a user's thoughts.

[1699] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1700] In this invention, the server includes means for encrypting the preprocessed EEG data and transmitting it to a remote server, means for analyzing the preprocessed EEG data using artificial intelligence technology in the remote server to identify the content of the user's thoughts, and means for generating concrete images based on the content of the thoughts, thereby enabling the images that the user has in mind to be analyzed quickly and securely and visually embodied.

[1701] "User" refers to an individual who uses the system and provides EEG data.

[1702] "EEG Data" refers to electronic signal data acquired by a user's EEG cap or EEG measuring device.

[1703] "Means of collecting data in real time" refers to technology that uses an EEG cap or measuring device to instantly obtain a user's brainwave data.

[1704] "Preprocessing" refers to the process of removing noise and filtering the acquired EEG data to extract specific features.

[1705] "Specific features" refer to important patterns and indicators in EEG data, which serve as basic data for analyzing the user's thoughts.

[1706] "Encryption means" refers to technology that converts data to make it difficult for third parties to decipher, in order to ensure the security of brainwave data.

[1707] "Remote Server" refers to a server located at a remote location that receives, stores, and analyzes data sent from a user's device.

[1708] "Artificial intelligence technology" refers to technology that uses machine learning, deep learning, etc. to analyze data and draw conclusions and predictions.

[1709] "Thought content" refers to what the user specifically imagines or thinks about in their head.

[1710] "Means for generating concrete images" refers to technology that uses an image generation AI model to generate visual images based on the user's thoughts.

[1711] A "prompt sentence" refers to a sentence used as an instruction or input to an image-generating AI model.

[1712] "Means for displaying on a display" refers to the technology used to visually display images generated on the terminal on a screen.

[1713] "Emotional state" refers to the user's psychological or emotional state derived from EEG data.

[1714] The present invention relates to a system that acquires an image that a user has in mind from electroencephalogram data in real time and displays it as a concrete image. The configuration and operation of this system will be specifically described below.

[1715] System Configuration

[1716] 1. User device (terminal)

[1717] The user provides the brainwave data by wearing an EEG cap or EEG measurement device, which typically connects to a terminal using Bluetooth or other wireless communication technology.

[1718] The device measures brain waves in real time and collects initial data.

[1719] 2. Intermediate processing device (terminal)

[1720] After initial collection of EEG data, preprocessing is performed to remove noise and filter it, and to extract features. Specifically, the FFT (Fast Fourier Transform) algorithm is used.

[1721] The preprocessed data is encrypted and sent to a remote server using AES (Advanced Encryption Standard).

[1722] 3. Remote Analysis Server (Server)

[1723] The received EEG data is analyzed to identify what the user was thinking, using deep learning models (e.g., TensorFlow or PyTorch).

[1724] Based on the identified thought content, an image generation AI model (e.g., Stable Diffusion or DALL-E) is used to generate concrete images.

[1725] 4. Generated image transmission device (server)

[1726] The generated image data is then re-encrypted and sent to the user's device using the HTTPS protocol.

[1727] 5. Display Device (Terminal)

[1728] The received image data is decoded on the user's device and displayed on the display at an appropriate resolution.

[1729] Program processing flow

[1730] 1. The user wears the EEG cap and imagines a specific image. For example, the user imagines a "tranquil lake scene."

[1731] 2. The device collects the user's brainwave data in real time and stores the initial data in memory.

[1732] 3. The device uses an FFT algorithm to remove noise and filter the data, and extract specific features.

[1733] 4. The device encrypts the preprocessed EEG data with AES and transmits it to a remote server using the HTTPS protocol.

[1734] 5. The server uses a deep learning model to analyze the received EEG data and determine that the user is thinking of a "lake scene."

[1735] 6. The server uses an image generation AI model to generate a detailed image corresponding to the "lake scene."

[1736] 7. The server re-encrypts the generated image data and sends it to the user's device using the HTTPS protocol.

[1737] 8. The device decrypts the encrypted data and displays it on the display in full HD resolution.

[1738] Specific examples

[1739] Example 1: A user with physical limitations visualizes a scene

[1740] 1. A user with a physical limitation imagines a scene of a tranquil lake.

[1741] 2. The device collects the user's brainwave data and performs preprocessing.

[1742] 3. The device encrypts the preprocessed data with AES and sends it to the remote server.

[1743] 4. The server uses a deep learning model to analyze the data and determine that the user is thinking of a lake.

[1744] 5. The server uses image generation AI to convert the tranquil lake scene into a concrete image.

[1745] 6. The server re-encrypts the generated image and sends it to the user's device.

[1746] 7. The device decodes the image and displays it on the screen, allowing the user to visually confirm the lakescape they had in mind.

[1747] Prompt Sentence Examples

[1748] "Show me the image generated as a result of imagining a tranquil lake scene."

[1749] "Generate an image based on the results of EEG data analysis when imagining a complex geometric pattern."

[1750] In this way, the present invention makes it possible to materialize the user's thoughts and images from electroencephalogram data into concrete images and display them on a display.

[1751] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1752] Step 1:

[1753] The user wears an EEG cap and conjures up a specific image.

[1754] Specific operation: The user wears an EEG cap, sits in a chair, and relaxes while imagining a "lake scene."

[1755] Input: An image imagined by the user in a relaxed state.

[1756] Output: EEG data collected in real time.

[1757] Step 2:

[1758] The device collects the user's brainwave data in real time and stores it in memory.

[1759] Specific operation: The signal from the EEG cap is transmitted to the terminal via wireless communication technology (e.g., Bluetooth), and initial data is stored in the memory.

[1760] Input: Radio signal from EEG cap.

[1761] Output: Initial EEG data stored in memory.

[1762] Step 3:

[1763] The device performs noise removal and filtering on the EEG data to extract specific features.

[1764] Specific operation: Noise removal and filtering are performed using an FFT (Fast Fourier Transform) algorithm, and specific features such as alpha waves and beta waves are extracted.

[1765] Input: Initial EEG data stored in memory.

[1766] Output: Preprocessed EEG data with extracted features.

[1767] Step 4:

[1768] The device encrypts the preprocessed EEG data and transmits it to a remote server.

[1769] What it does: Encrypts data using AES (Advanced Encryption Standard) and sends it to a remote server via the HTTPS protocol.

[1770] Input: Preprocessed EEG data with extracted features.

[1771] Output: Encrypted EEG data is sent to a remote server.

[1772] Step 5:

[1773] The server receives the encrypted data, decrypts it, and then analyzes it using a deep learning model to determine what the user is thinking.

[1774] Specific operation: The received data is decrypted using AES, and then the user's thoughts are analyzed using a deep learning model (e.g., TensorFlow or PyTorch).

[1775] Input: Encrypted EEG data.

[1776] Output: Decoded data and analyzed thought (e.g., "lakescape").

[1777] Step 6:

[1778] The server uses an image generation AI model to generate specific images based on the identified thought content.

[1779] Specific operation: Generate an image equivalent to a "lake scene" using an image generation AI model (e.g., Stable Diffusion or DALL-E).

[1780] Input: Parsed thoughts.

[1781] Output: The concrete image generated.

[1782] Step 7:

[1783] The server re-encrypts the generated image data and sends it to the user's terminal.

[1784] Specific operation: The generated image data is encrypted with AES and sent to the user's device via the HTTPS protocol.

[1785] Input: The concrete image to be generated.

[1786] Output: The encrypted image data is sent to the user's device.

[1787] Step 8:

[1788] The terminal decodes the received image data and displays it on the display at the appropriate resolution.

[1789] Specific operation: The received encrypted data is decrypted using AES and displayed on the display in full HD resolution.

[1790] Input: Encrypted image data.

[1791] Output: The concrete image (picture) shown on the display.

[1792] (Application example 1)

[1793] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1794] Traditional methods have made it difficult to efficiently and accurately share work and repair procedures with other engineers within a factory. This has led to communication errors, work delays, and reduced productivity. Furthermore, the lack of a way to directly visualize engineers' thinking has made it difficult for experienced engineers to share their knowledge effectively.

[1795] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1796] In this invention, the server includes means for preprocessing the user's electroencephalogram data and extracting specific features, means for analyzing the preprocessed electroencephalogram data and identifying the user's thoughts, and means for generating concrete images based on the thoughts, thereby enabling the engineer's thoughts to be embodied and shared with other engineers in real time.

[1797] "User" refers to the entity that provides the EEG data, and is primarily a technician who performs work and repairs within the factory.

[1798] "EEG data" refers to data collected by an EEG measuring device when a user has a particular thought or image in their mind.

[1799] "Preprocessing" refers to data processing means for removing noise from electroencephalogram data and extracting specific features.

[1800] "Remote server" refers to a server device located remotely for receiving and analyzing electroencephalogram data.

[1801] A "neural network" is a type of artificial intelligence technology and a deep learning model used to analyze EEG data and generate images.

[1802] "Concrete imagery" refers to visual images and figures generated based on analyzed EEG data.

[1803] "Terminal" refers to equipment, including devices such as computers and tablets used by users and technicians.

[1804] A "display" is a display device installed in a terminal, and is a device for visually displaying the generated specific image.

[1805] "Sharing" refers to the viewing and use of the generated concrete image in real time among multiple engineers.

[1806] The present invention is a system for efficiently and accurately sharing work and repair procedures with other engineers in a factory, and a specific embodiment thereof will be described.

[1807] System Configuration

[1808] Hardware

[1809] 1. EEG Cap

[1810] This device is worn by the user and collects brainwave data in real time. It is used by engineers working in factories.

[1811] 2. EEG measurement device

[1812] This device works in conjunction with the EEG cap and transmits collected EEG data to a terminal.

[1813] 3. Terminal

[1814] A general computing device such as a PC or tablet used by engineers, and is mainly used for data preprocessing and display.

[1815] 4. Remote Server

[1816] This is a server installed outside the factory that analyzes EEG data and generates images.

[1817] software

[1818] 1. Preprocessing software

[1819] It runs on the device and is responsible for removing noise from EEG data and extracting specific features.

[1820] 2. AI analysis software

[1821] It runs on a remote server and uses artificial intelligence techniques such as neural networks to analyze brainwave data and generate concrete images.

[1822] 3. Image display application

[1823] It is an application that runs on the terminal and displays the generated image to the user.

[1824] System Operation Description

[1825] Data Collection Phase

[1826] The user wears an EEG cap and thinks about a specific work procedure or repair method. The EEG measurement device collects brainwave data in real time and transmits it to a terminal. Pre-processing software on the terminal removes noise and filters the data, and extracts features.

[1827] Data transmission phase

[1828] After preprocessing, the EEG data is encrypted on the device and securely transmitted to a remote server, ensuring data security.

[1829] Data analysis phase

[1830] The remote server uses AI analysis software to analyze the received EEG data, using deep learning models such as neural networks to identify what the user is thinking and generate specific images based on that.

[1831] Image Generation and Display Phase

[1832] The generated images are then sent back to the terminal from the remote server and displayed on the terminal's image display application, allowing the user to visualize the work or repair procedures they envision as concrete images that can be shared with other technicians.

[1833] Specific examples

[1834] When performing machine maintenance

[1835] Engineer A puts on an EEG cap and thinks about the maintenance procedures for a specific machine. The terminal collects A's EEG data and performs preprocessing. The preprocessed data is sent to a remote server, where AI analysis software analyzes it and generates an image of the maintenance procedures. The generated image is sent to Engineer A's terminal and shared with other engineers. This process makes work more efficient.

[1836] Prompt Sentence Examples

[1837] "Analyze EEG data to generate a detailed image of a specific repair procedure for a machine."

[1838] As described above, the present invention provides a system that enables factory engineers to visualize and efficiently share their thoughts using electroencephalogram data.

[1839] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1840] Step 1:

[1841] The user wears the EEG cap and thinks about a specific work procedure or repair method. As input, the user's brainwave data is collected in real time. As output, the real-time brainwave signal is sent to the EEG measurement device. Specifically, the EEG cap detects the user's brainwaves and sends the signal to the EEG measurement device.

[1842] Step 2:

[1843] The terminal receives EEG data in real time from the EEG measurement device and performs preprocessing. The input is the collected EEG data. The output is preprocessed data with noise removed and features extracted. Specifically, the preprocessing software applies a noise removal filter and executes a feature extraction algorithm.

[1844] Step 3:

[1845] The device encrypts the preprocessed data and sends it to a remote server. The input is the preprocessed EEG data. The output is the encrypted data sent to the remote server. Specifically, encryption software on the device encrypts the preprocessed data and transfers the data to the server using a secure communication protocol.

[1846] Step 4:

[1847] The server receives and decrypts the encrypted data. The input is the encrypted EEG data. The output is the decrypted data. The specific operation is to decrypt the data using a secure key management system and prepare it for data analysis.

[1848] Step 5:

[1849] The server analyzes the decoded data using AI analysis software. The input is the decoded EEG data. The output is the user's thoughts. Specifically, the AI ​​analysis software analyzes the data using neural networks and generative AI models to identify the user's thoughts.

[1850] Step 6:

[1851] The server generates a specific image based on the identified thought content. The input is data that identifies the thought content. The output is the generated specific image. As a specific action, the generative AI model generates a detailed image based on the analysis results. An example of a prompt statement would be, "Analyze specific repair procedures for a certain machine from EEG data and generate a detailed image."

[1852] Step 7:

[1853] The server sends the generated image data back to the terminal. The generated image data is the input. The image data is sent to the terminal as the output. In concrete terms, the sending software on the server transfers the generated image data to the terminal.

[1854] Step 8:

[1855] The terminal receives the image data and displays it in the image display application. The input is the transmitted image data. The output is a specific image displayed on the display. The specific operation is that the image display application interprets the image data and displays it on the display device.

[1856] This allows engineers to share their thoughts as concrete images with other engineers, improving work efficiency.

[1857] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1858] This invention relates to a system that interprets the user's thoughts and emotional state based on their electroencephalogram data, generates specific images based on this, and displays them on a display. In particular, by combining it with an emotion engine, it becomes possible to generate images that take the user's emotional state into consideration. The configuration and operation of this system will be specifically described.

[1859] System Configuration

[1860] 1. User device (terminal)

[1861] Users wear an EEG cap or EEG measuring device and provide brainwave data.

[1862] The device measures brain waves in real time and collects initial data.

[1863] 2. Intermediate processing device (terminal)

[1864] After initial collection of EEG data, preprocessing such as noise removal is performed to extract specific features.

[1865] After preprocessing, the data is encrypted and sent to a remote server.

[1866] 3. Remote Analysis Server (Server)

[1867] The user's thoughts are analyzed based on the received brain wave data.

[1868] AI technology converts the thoughts into concrete images.

[1869] An emotion engine is used to recognize the user's emotional state.

[1870] 4. Generated image transmission device (server)

[1871] The generated image is sent to the user's device.

[1872] Customize images based on information from the emotion engine.

[1873] 5. Display Device (Terminal)

[1874] The image data received by the user's terminal is decoded and displayed on the screen.

[1875] Program processing flow (natural language explanation)

[1876] 1. Data Collection and Preprocessing

[1877] The user wears an EEG cap to sense their thoughts and emotions.

[1878] The device collects the user's brainwave data in real time and stores the initial data in memory.

[1879] The device performs noise removal and filtering to extract specific features.

[1880] 2. Data transmission

[1881] The device encrypts the pre-processed EEG data and transmits it to a remote server.

[1882] 3. Data Analysis

[1883] The server analyzes the received brainwave data and identifies what the user was thinking.

[1884] At the same time, the server uses an emotion engine to determine the user's emotional state.

[1885] 4. Emotion-based image generation

[1886] The server uses the analysis results of the emotion engine to generate specific images that correspond to the user's emotional state.

[1887] 5. Sending and displaying the generated image

[1888] When the server sends the generated image to the user's device, it adaptively customizes the image based on information from the emotion engine.

[1889] The terminal decompresses and decodes the received image data and displays it on the display.

[1890] Specific examples

[1891] Example 1: Landscape imagery for stress reduction

[1892] 1. The user visualizes a relaxing beach scene while feeling stressed at work.

[1893] 2. The device collects EEG data, removes noise, and extracts features.

[1894] 3. The device sends the preprocessed data to the remote server.

[1895] 4. The server analyzes the data and determines that the user is thinking of a relaxing beach scene.

[1896] 5. At the same time, the server uses an emotion engine to detect when the user is in a high stress state.

[1897] 6. The server uses AI to generate a beach scene with colors and composition that enhance the relaxing effect.

[1898] 7. The server sends the generated image to the user's device, and the adaptively customized beach scene is displayed on the screen.

[1899] Example 2: Imagery to improve concentration

[1900] 1. Imagine a forest scene that helps users improve their focus while studying for exams.

[1901] 2. The device collects and preprocesses EEG data.

[1902] 3. The device sends the preprocessed data to the remote server.

[1903] 4. The server analyzes the data and determines that the user is imagining a forest scene to improve concentration.

[1904] 5. At the same time, the server detects the user's concentration level using an emotion engine.

[1905] 6. The server uses AI to generate a forest landscape suitable for maintaining concentration.

[1906] 7. The server sends the generated image to the user's device, and a forest scene that encourages concentration is displayed on the screen.

[1907] In this way, by combining emotion engines, it becomes possible to generate images that take into account not only the user's thoughts but also their emotional state, providing a more personalized experience.

[1908] The processing flow will be explained below.

[1909] Step 1:

[1910] The user wears an EEG cap and conjures up a specific image.

[1911] The user wears an EEG cap on their head and is asked to consciously visualize a specific scene or image.

[1912] Step 2:

[1913] The device collects the user's brainwave data in real time.

[1914] The electroencephalogram measuring device captures the user's electroencephalogram signals and transmits the signal data to the terminal.

[1915] Step 3:

[1916] The device preprocesses the collected EEG data.

[1917] Data cleansing removes noise and improves data quality.

[1918] Features are extracted using algorithms such as the fast Fourier transform (FFT).

[1919] Step 4:

[1920] The terminal encrypts the pre-processed data and transmits it to a remote server.

[1921] The EEG data from which the features have been extracted is encrypted using the SSL / TLS protocol.

[1922] Securely transmit encrypted data over the Internet to a remote server.

[1923] Step 5:

[1924] The server analyzes the received brain wave data.

[1925] It uses AI models (such as deep learning neural networks) to analyze data and identify what users are thinking.

[1926] Step 6:

[1927] The server uses an emotion engine to recognize the user's emotional state.

[1928] The emotion engine uses the received data to identify the user's stress level, concentration level, and type of emotion.

[1929] Step 7:

[1930] The server generates concrete images based on the identified thoughts and emotional states.

[1931] Using image generation AI (e.g., GAN), specific images are generated based on the identified thought content and emotional state.

[1932] The image's color, brightness, and composition are adaptively adjusted to match the user's emotional state.

[1933] Step 8:

[1934] The server sends the generated image to the user's terminal.

[1935] The generated image data is compressed and converted into an appropriate format (e.g., JPEG, PNG).

[1936] The data is encrypted using the SSL / TLS protocol and sent to the user's device.

[1937] Step 9:

[1938] The device receives the transmitted image and displays it on the screen.

[1939] The received image data is decompressed and decoded.

[1940] The image is adjusted to fit the display resolution and displayed in real time.

[1941] This allows images based on the scenery and emotions imagined by the user to be embodied and displayed on the screen.

[1942] Example 2

[1943] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1944] Conventional EEG data analysis systems have difficulty accurately grasping a user's emotional state and thoughts, resulting in images that often do not match the user's expectations. Furthermore, insufficient real-time data collection and processing can result in a poor user experience. Furthermore, insufficient privacy protection for collected data raises concerns about the risk of information leaks. A system that can resolve these issues and generate images that are both accurate and optimized for the user is needed.

[1945] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1946] In this invention, the server includes means for analyzing the preprocessed electroencephalogram data to identify the user's thoughts, means for generating specific images based on the thoughts, and means for identifying the user's emotional state using an emotion recognition engine and customizing the images according to the emotional state, thereby enabling the generation of personalized images that accurately reflect the user's thoughts and emotional state.

[1947] "User" means a person who provides EEG data and uses the system.

[1948] "EEG data" refers to data that measures a user's brainwave activity in real time.

[1949] "Preprocessing" refers to the process of removing noise from collected EEG data and extracting specific features.

[1950] "Features" are important data points or patterns extracted from preprocessed EEG data.

[1951] "Encryption" is the process of transforming data using a specific algorithm in order to transmit it securely.

[1952] A "remote server" is a server that can receive and process data via the Internet.

[1953] "Analysis" refers to data processing to identify the user's thoughts and emotional state based on brainwave data.

[1954] A "concrete image" is an image or figure that visually expresses the user's thoughts.

[1955] An "emotion recognition engine" is software or algorithms that analyze a user's emotional state based on brainwave data.

[1956] "Customization" refers to the process of adjusting the color tone and composition of the generated image according to the user's particular emotional state.

[1957] "Decryption" is the process of restoring encrypted data to its original state.

[1958] "Display" means a display device that allows a user to visually view the generated image.

[1959] This invention relates to a system that interprets the user's thoughts and emotional state based on their electroencephalogram data, generates specific images based on this, and displays them on a display. In particular, by combining it with an emotion engine, it is possible to generate images that take the user's emotional state into consideration. The configuration and operation of this system will be specifically explained.

[1960] System Configuration

[1961] 1. User device (terminal)

[1962] The user wears an EEG cap or EEG measurement device (e.g., Emotiv Epoc+) and provides brainwave data.

[1963] The device is used to measure brain waves in real time and collect initial data, which is collected using software such as Emotiv Pro and MATLAB.

[1964] 2. Intermediate processing device (terminal)

[1965] After initial collection of EEG data, preprocessing such as noise removal is performed to extract specific features (e.g., alpha waves, beta waves).

[1966] For preprocessing, we use Python libraries such as NumPy and SciPy.

[1967] 3. Remote Analysis Server (Server)

[1968] The user's thoughts are analyzed based on the received brain wave data.

[1969] Use an AI analysis engine (e.g., TensorFlow, PyTorch) to convert thought content into concrete images.

[1970] Recognize the user's emotional state using an emotion engine (e.g., Affectiva, IBM Watson).

[1971] 4. Generated image transmission device (server)

[1972] The generated image is sent to the user's device.

[1973] Adaptively customize images based on sentiment analysis results.

[1974] 5. Display Device (Terminal)

[1975] The image data received by the user's device is decoded and displayed on a monitor or VR headset.

[1976] Specific examples

[1977] Example 1: Landscape imagery for stress reduction

[1978] 1. The user visualizes a relaxing beach scene while feeling stressed at work.

[1979] 2. The device collects EEG data, removes noise, and extracts features.

[1980] 3. The device sends the preprocessed data to the remote server.

[1981] 4. The server analyzes the data and determines that the user is thinking of a relaxing beach scene.

[1982] 5. At the same time, the server uses an emotion engine to detect when the user is in a high stress state.

[1983] 6. The server uses AI to generate a relaxing beach scene.

[1984] 7. The server sends the generated image to the user's terminal and displays it on the display.

[1985] Prompt Sentence Examples

[1986] "Suppose a user is under high stress at work, but imagines a relaxing beach scene. In this state, a customized beach scene that maximizes the relaxation effect is generated and displayed on the display."

[1987] Example 2: Imagery to improve concentration

[1988] 1. Imagine a forest scene that helps users improve their focus while studying for exams.

[1989] 2. The device collects EEG data, removes noise, and extracts features.

[1990] 3. The device sends the preprocessed data to the remote server.

[1991] 4. The server analyzes the data and determines that the user is imagining a forest scene to improve concentration.

[1992] 5. At the same time, the server detects the user's concentration level using an emotion engine.

[1993] 6. The server uses AI to generate a forest landscape to help you maintain concentration.

[1994] 7. The server sends the generated image to the user's terminal and displays it on the display.

[1995] Prompt Sentence Examples

[1996] "Imagine a tranquil forest scene that helps users concentrate better while studying for an exam. In this state, we generate a customized forest scene that maximizes concentration and display it on the display."

[1997] Thus, a system for generating and displaying personalized images based on a user's thought content and emotional state has been described in detail as an embodiment of the present invention.

[1998] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1999] System program processing flow

[2000] Step 1: Data collection

[2001] The user wears an EEG cap and focuses their thoughts and emotions.

[2002] 1. The user puts on the EEG cap.

[2003] Input: EEG cap worn on user's head

[2004] Output: Begins collecting user's brainwave data in real time.

[2005] 2. The device collects the user's brainwave data in real time.

[2006] Operation: Data measured by the EEG cap is transferred to the terminal and saved in memory as initial data.

[2007] Input: User's EEG data

[2008] Output: Initial EEG data

[2009] Step 2: Data Preprocessing

[2010] 1. Preprocess the EEG data collected by the device.

[2011] What it does: Applies a noise reduction filter to remove unwanted noise from EEG data.

[2012] Input: Initial EEG data

[2013] Output: Noise-removed EEG data

[2014] 2. The device extracts specific features.

[2015] How it works: It uses Python's NumPy and SciPy libraries to extract specific patterns such as alpha and beta waves.

[2016] Input: Noise-removed EEG data

[2017] Output: Data with extracted features

[2018] Step 3: Encrypt and send data

[2019] 1. The device encrypts the data.

[2020] How it works: Encrypts the feature-extracted data using the AES encryption algorithm.

[2021] Input: Data with extracted features

[2022] Output: Encrypted data

[2023] 2. The device sends the encrypted data to a remote server.

[2024] How it works: Sends encrypted data to the server using the HTTPS protocol.

[2025] Input: Encrypted data

[2026] Output: Data sent to server completed

[2027] Step 4: Data analysis

[2028] 1. The server receives the encrypted data.

[2029] How it works: The server receives the data sent and decrypts it using the AES decryption algorithm.

[2030] Input: Encrypted data

[2031] Output: Decrypted data

[2032] 2. The server analyzes the data using its analysis engine.

[2033] How it works: Uses an AI analysis engine (e.g. TensorFlow, PyTorch) to identify the user's thoughts from EEG data.

[2034] Input: Decrypted data

[2035] Output: What the user is thinking

[2036] 3. The server uses the emotion engine to identify the emotional state.

[2037] How it works: Uses an emotion recognition engine (e.g., Affectiva, IBM Watson) to analyze the user's emotional state.

[2038] Input: User's EEG data

[2039] Output: User's emotional state

[2040] Step 5: Image Creation and Customization

[2041] 1. The server generates an image based on the analysis results.

[2042] How it works: It uses generative AI models (e.g., DALL·E, StyleGAN) to generate tangible images based on what the user is thinking.

[2043] Input: What the user is thinking

[2044] Output: The concrete image generated

[2045] 2. The server customizes the image based on the analysis results of the emotion engine.

[2046] How it works: Based on the results of emotion analysis, it adjusts the color and composition of the image to create a more personalized image.

[2047] Input: Generated concrete image, user's emotional state

[2048] Output: Customized image

[2049] Step 6: Sending and displaying image data

[2050] 1. The server encrypts the generated image and sends it to the user's device.

[2051] What it does: The resulting image is encrypted and sent, again using the AES encryption algorithm.

[2052] Input: Customized image

[2053] Output: Encrypted image data

[2054] 2. The device receives and decrypts the encrypted image data.

[2055] What it does: Uses the AES decryption algorithm to decrypt any encrypted data received.

[2056] Input: Encrypted image data

[2057] Output: Decoded image data

[2058] 3. The terminal displays the decrypted image on the display.

[2059] What it does: Displays the decoded image on a display device (e.g., monitor, VR headset).

[2060] Input: Decoded image data

[2061] Output: The image displayed on the screen

[2062] (Application example 2)

[2063] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2064] In recent years, there has been a demand for customization based on individual user needs and conditions to improve the consumer experience in virtual stores. However, current systems have difficulty in real-time adaptive customization based on the user's emotions and thoughts, limiting the improvement of consumer satisfaction. In addition, there has been a lack of effective means to provide the optimal products and environment that users desire. New technologies that can solve this problem and dramatically improve the user experience are needed.

[2065] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a device that collects the user's electroencephalogram data in real time, a device that preprocesses the electroencephalogram data and extracts specific features, and a device that transmits the preprocessed electroencephalogram data to a remote server. This enables optimal customization within the virtual store based on the user's thoughts and emotional state.

[2066] A "user" is a person who uses the system to enjoy an experience within a virtual store.

[2067] "EEG data" is a measurement of the electrical signals generated by the user's brain.

[2068] A "collection device" is a device for measuring and collecting EEG data in real time.

[2069] "Preprocessing" refers to the process of removing noise and extracting features from collected EEG data.

[2070] A "feature" is a specific index or pattern extracted from electroencephalogram data.

[2071] The "remote server" is a server at a remote location that receives and analyzes pre-processed EEG data.

[2072] The "analysis device" is a device that runs on a remote server and analyzes pre-processed EEG data to identify the user's thoughts and emotional state.

[2073] "Emotional state" refers to the mental and psychological state of a user as determined from their brainwave data.

[2074] "Concrete images" are visual representations that are generated based on the user's thoughts and emotional state.

[2075] "Customization" refers to individually adjusting the display content and environment within a virtual store according to the identified emotional state.

[2076] A "transmitting device" is a device for transmitting the generated image to a user's terminal.

[2077] A "display device" is a device for displaying images received at a user's terminal on a display.

[2078] A "display" is a screen or monitor used to visually display information on a user's device.

[2079] A "virtual store" refers to an environment in which products can be viewed and purchased in a virtual space just as if they were in a physical store.

[2080] MODE FOR CARRYING OUT THE INVENTION

[2081] This invention allows users to enjoy a customized virtual store experience based on their electroencephalogram (EEG) data. This system collects the user's EEG data in real time, extracts features, and then transmits the data to a remote server for analysis and image generation. The generated image is then transmitted to the user's device, where an adaptively customized virtual store is displayed.

[2082] 1. System program generation

[2083] The program for this system is constructed using the following hardware and software.

[2084] 2. Hardware and Software

[2085] Hardware used:

[2086] EEG Cap: Emotiv Insight

[2087] HMD: Oculus Quest 2

[2088] Software used:

[2089] Data analysis: TensorFlow

[2090] Emotion engine: Affectiva SDK

[2091] 3. Program processing explanation

[2092] Users wear an EEG cap and access the virtual store through an HMD. The EEG cap collects the user's brainwave data in real time and performs preprocessing such as noise reduction and filtering. The preprocessed data is encrypted and transmitted to a remote server via the HMD terminal.

[2093] The server uses TensorFlow to analyze the received EEG data to identify the user's thoughts and emotional state. The emotional state is then analyzed in detail using the Affectiva SDK. Based on the analysis results, a specific image that matches the user's emotional state is generated. This image generation process uses a generative AI model.

[2094] The generated images reflect the user's emotional state and adaptively customize the virtual store environment and product display. The customized images are transmitted from the remote server to the HMD and displayed on the user's HMD display.

[2095] 4. Example of a program and a prompt

[2096] Example 1: Stress-reducing shopping

[2097] 1. The user is feeling stressed and thinks about "relaxation."

[2098] 2. The device analyzes your thoughts and emotions. "You're feeling stressed."

[2099] 3. The server customizes the virtual store environment with a "relaxation" theme (e.g., warm lighting and relaxing music).

[2100] 4. Through the HMD, users can enjoy a shopping experience in a relaxing environment.

[2101] Example prompt:

[2102] "Creates a relaxing environment when the user is feeling stressed"

[2103] Example 2: Shopping to improve concentration

[2104] 1. Users imagine a "quiet forest" to help them concentrate.

[2105] 2. The device analyzes your thoughts and emotions. "I want to improve my concentration."

[2106] 3. The server recreates a quiet forest scene in the virtual store, which promotes concentration.

[2107] 4. Through the HMD, users can experience shopping in a quiet environment where they can concentrate.

[2108] Example prompt:

[2109] "Create a quiet environment to help users concentrate better"

[2110] This system enables a highly personalized virtual store experience based on the user's emotions and thoughts.

[2111] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2112] Step 1:

[2113] The user wears an EEG cap and accesses the virtual store through the HMD. The EEG cap collects the user's EEG data in real time. The input is the user's EEG data, and the output is the collected raw EEG data.

[2114] Step 2:

[2115] The EEG data collected by the device is preprocessed by performing noise removal and filtering. The input is raw EEG data, and the output is EEG data containing noise-removed features. Data processing involves noise filtering and feature extraction.

[2116] Step 3:

[2117] The terminal encrypts the preprocessed EEG data and sends it to a remote server. The input is the preprocessed EEG data, and the output is the encrypted data. The specific operation is to protect the data using an encryption algorithm.

[2118] Step 4:

[2119] The server decrypts the received preprocessed encrypted data and analyzes it using TensorFlow to identify the user's thoughts and emotional state. The input is the decrypted EEG data, and the output is the identified thoughts and emotional state. Data processing involves analysis using a machine learning model.

[2120] Step 5:

[2121] The server uses the Affectiva SDK to perform detailed emotion analysis based on the identified thoughts and emotional state, and generates specific images based on the results. The input is the identified thoughts and emotional state, and the output is the generated specific image. A generative AI model is used to create an image that matches the user's emotions.

[2122] Step 6:

[2123] The server sends the generated image to the user's device. Before sending, adaptive customization can be performed. The input is the generated image, and the output is the customized image. Adaptive customization can include, for example, changing the color tone or layout.

[2124] Step 7:

[2125] The terminal decodes the received image and displays it on the HMD display. The input is customized image data, and the output is visual information displayed on the HMD. Specifically, it performs visualization processing on the display.

[2126] In this way, users can enjoy a highly personalized virtual store experience based on their emotions and thoughts.

[2127] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2128] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2129] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[2130] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2131] FIG. 9 illustrates an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and behaviors arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[2132] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[2133] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[2134] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[2135] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[2136] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[2137] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[2138] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[2139] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[2140] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[2141] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[2142] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[2143] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[2144] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[2145] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[2146] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[2147] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[2148] The following is further disclosed regarding the above embodiment.

[2149] (Claim 1)

[2150] a means for collecting the user's brainwave data in real time;

[2151] means for preprocessing the electroencephalogram data and extracting specific features;

[2152] means for transmitting the preprocessed electroencephalogram data to a remote server;

[2153] means for analyzing the preprocessed electroencephalogram data in the remote server and identifying the content of the user's thoughts;

[2154] A means for generating a concrete image based on the thought content;

[2155] means for transmitting the generated image to a user's terminal;

[2156] means for displaying the transmitted image on a display at said terminal;

[2157] A system including:

[2158] (Claim 2)

[2159] 10. The system of claim 1, wherein the generated image is generated using artificial intelligence techniques.

[2160] (Claim 3)

[2161] 10. The system of claim 1, further comprising means for analyzing the pre-processed electroencephalogram data to identify an emotional state of the user and customizing the imagery according to the emotional state.

[2162] "Example 1"

[2163] (Claim 1)

[2164] a means for collecting the user's brainwave data in real time;

[2165] means for preprocessing the electroencephalogram data and extracting specific features;

[2166] means for encrypting and transmitting the preprocessed EEG data to a remote server;

[2167] means for analyzing the pre-processed electroencephalogram data using artificial intelligence technology in the remote server to identify the content of the user's thoughts;

[2168] A means for generating a concrete image based on the thought content;

[2169] means for transmitting the generated image to a user's terminal;

[2170] means for displaying the transmitted image on a display in the terminal;

[2171] A system including:

[2172] (Claim 2)

[2173] 10. The system of claim 1, wherein the generated image is generated using a prompt sentence.

[2174] (Claim 3)

[2175] 10. The system of claim 1, further comprising means for analyzing the pre-processed electroencephalogram data to identify an emotional state of the user and customizing the imagery according to the emotional state.

[2176] "Application Example 1"

[2177] (Claim 1)

[2178] a means for collecting the user's brainwave data in real time;

[2179] means for preprocessing the electroencephalogram data and extracting specific features;

[2180] means for transmitting the preprocessed electroencephalogram data to a remote server;

[2181] means for analyzing the preprocessed electroencephalogram data in the remote server and identifying the content of the user's thoughts;

[2182] A means for generating a concrete image based on the thought content;

[2183] A means for transmitting the generated image to the technician's terminal and sharing it among the technicians;

[2184] means for displaying the transmitted image on a display at said terminal;

[2185] A system including:

[2186] (Claim 2)

[2187] 10. The system of claim 1, wherein the generated image is generated using artificial intelligence techniques including neural networks.

[2188] (Claim 3)

[2189] 10. The system of claim 1, further comprising means for analyzing the pre-processed electroencephalogram data to identify an emotional state of the user and customizing the imagery according to the emotional state.

[2190] "Example 2: Combining Emotion Engines"

[2191] (Claim 1)

[2192] a means for collecting the user's brainwave data in real time;

[2193] means for preprocessing the electroencephalogram data and extracting specific features;

[2194] means for encrypting and transmitting the preprocessed EEG data to a remote server;

[2195] means for analyzing the preprocessed electroencephalogram data in the remote server and identifying the content of the user's thoughts;

[2196] A means of generating concrete images based on the thought content;

[2197] means for identifying a user's emotional state using an emotion recognition engine and customizing the imagery according to the emotional state;

[2198] A means for encrypting the generated image and sending it to the user's device;

[2199] means at said terminal for decoding the transmitted image and displaying it on a display;

[2200] A system including:

[2201] (Claim 2)

[2202] 10. The system of claim 1, wherein the generated image is generated using artificial intelligence techniques.

[2203] (Claim 3)

[2204] 10. The system of claim 1, further comprising means for analyzing the pre-processed electroencephalogram data to identify an emotional state of the user and customizing the imagery according to the emotional state.

[2205] "Application example 2 when combining emotion engines"

[2206] procedure:

[2207] 2. Extracting technically novel parts from the application examples

[2208] One novel application, "Emotion-Driven Virtual Shopper (EDVS)," is that it identifies the user's emotional state in real time based on their brainwave data and customizes the virtual store experience (product display, environment) accordingly.

[2209] 3. Creating new inventions

[2210] The new invention we came up with is specifically designed for user experience in virtual stores, incorporating emotional state identification and customization.

[2211] 5. Generalization of proper nouns and names and hypernyms

[2212] Proper names and proper titles are converted into common names, and nouns are converted into superordinate expressions.

[2213] (Claim 1)

[2214] A device that collects the user's brainwave data in real time,

[2215] a device for preprocessing the electroencephalogram data and extracting specific features;

[2216] a device for transmitting the preprocessed EEG data to a remote server;

[2217] a device in the remote server for analyzing the pre-processed EEG data to identify the user's thoughts and emotional state;

[2218] a device for generating a specific image based on the thought content and emotional state and customizing the display in the virtual store;

[2219] a device for transmitting the generated image to a user's terminal;

[2220] a device at said terminal for displaying the transmitted image on a display;

[2221] A system including:

[2222] (Claim 2)

[2223] 10. The system of claim 1, wherein the generated imagery is generated using artificial intelligence techniques to customize the user experience.

[2224] (Claim 3)

[2225] 10. The system of claim 1, further comprising a device for analyzing the pre-processed electroencephalogram data to identify a user's emotional state and customizing the virtual store environment and product display according to the emotional state. [Explanation of symbols]

[2226] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. a means for collecting the user's brainwave data in real time; means for preprocessing the electroencephalogram data and extracting specific features; means for transmitting the preprocessed electroencephalogram data to a remote server; means for analyzing the preprocessed electroencephalogram data in the remote server and identifying the content of the user's thoughts; A means for generating a concrete image based on the thought content; means for transmitting the generated image to a user's terminal; means for displaying the transmitted image on a display at said terminal; A system including:

2. 10. The system of claim 1, wherein the generated image is generated using artificial intelligence techniques.

3. 10. The system of claim 1, further comprising means for analyzing the pre-processed electroencephalogram data to identify an emotional state of the user and customizing the imagery according to the emotional state.

Images