System

Abstract

A system is provided.SOLUTION: A system comprising: means for collecting brainwave data acquired during sleep; means for processing said brainwave data and converting dream content to natural language text; means for converting said natural language text to visual data; means for storing and making said natural language text and said visual data accessible to a user; and means for allowing a user to search and save said stored data as favorites.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In modern society, many people face the problem of being unable to remember their dreams. It is also difficult to record, review, and analyze the content of dreams. This poses a major obstacle for people interested in self-understanding and exploring psychological states. Furthermore, there are limited means to share or re-experience the content of dreams with others. Conventional technology has not provided a satisfactory solution to these challenges. [Means for solving the problem]

[0005] To solve the above problems, the present invention provides the following means. First, the system includes means for collecting EEG data acquired during sleep. It also includes means for processing the EEG data and converting the dream content into natural language text. It also includes means for converting the natural language text into visual data, thereby generating a concrete image of the dream content. It also includes means for saving the generated natural language text and visual data and making them accessible to users. It also provides means for users to search the saved data and save it as favorites, allowing users to easily look back on and re-experience past dreams. It also provides a system that further includes pre-processing means for removing noise from the EEG data and means for providing users with a re-experience based on the natural language text and visual data, thereby realizing a comprehensive solution that integrates the recording, analysis, sharing, and re-experiencing of dreams.

[0006] "Electroencephalogram data" are signals indicative of the brain's electrical activity measured on the scalp using electrodes.

[0007] "Natural language text" refers to sentences and character strings written in a language format that humans use on a daily basis.

[0008] "Visual data" refers to data formats that can be visually recognized, such as images and videos.

[0009] "Means of collection" is a general term for the devices and methods used to obtain and store the target data.

[0010] "Processing means" is a collective term for a set of operations or devices that analyze, transform, or condition collected data into a particular form.

[0011] "Storage means" is a general term for methods and devices for storing processed data on a storage medium and making it accessible at a later time.

[0012] "Search and storage methods" refers to the interfaces and methods used by users to locate specific data and retain it as a record.

[0013] "Preprocessing means" is a general term for operations and devices used to remove unnecessary noise from collected raw data and prepare it into an analyzable form.

[0014] "Means for providing a re-experience" is a general term for methods and devices used to present saved data to the user again and recreate the original experience. [Brief explanation of the drawings]

[0015] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11]FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0016] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0017] First, the terms used in the following description will be explained.

[0018] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0019] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0020] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0021] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0023] [First embodiment]

[0024] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0025] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0028] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0031] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0035] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0036] This invention relates to a system that collects a user's brainwave data while they sleep and converts the contents of their dreams into text and visualizations. Through this system, users can search, save, and re-experience their own dreams. The specific program processing required to implement this system is described below.

[0037] System configuration

[0038] Terminal

[0039] The terminal is a smartphone or tablet that connects to the EEG recording device used by the user, and an application is installed on the terminal to collect EEG data and send it to a server.

[0040] server

[0041] The server plays a central role in receiving EEG data, preprocessing, converting dreams into text, visualizing them, and storing and providing the data.

[0042] Program processing

[0043] 1. Dream Data Collection

[0044] Device:

[0045] Before sleep, users wear an EEG recording device, which collects real-time brainwave data and transmits it to a smartphone via Bluetooth or Wi-Fi.

[0046] server:

[0047] The server receives the brainwave data sent from the smartphone and stores it in a database.

[0048] 2. Translating the dream into text

[0049] server:

[0050] The collected EEG data is processed to remove unwanted noise. The preprocessed data is then fed into a text generation model (e.g., GPT-4) to convert the dream content into natural language text. The generated text is temporarily stored and provided to the user upon request.

[0051] 3. Visualize your dreams

[0052] server:

[0053] The textual content of dreams is input into an image generation model (e.g., DALL-E) and converted into concrete visual data. The generated visual data and the corresponding text data are associated and stored in a database.

[0054] 4. Searching for and re-experiencing dreams

[0055] User:

[0056] Through the dedicated application, you can use the interface to search for past dreams, choose filter options such as date, theme, or characters, and then select a specific dream from the search results to view its details.

[0057] Device:

[0058] The server sends the user's search criteria to retrieve the corresponding dream data, and displays the retrieved dream text and visual data to the user.

[0059] 5. Favorites feature

[0060] User:

[0061] Click the button to save the dream you like as a favorite.

[0062] Device:

[0063] Sends requests to the server regarding the registration and management of favorites.

[0064] server:

[0065] Based on the user's request, specific dreams are added to the user's "favorite list" and saved and updated in the database.

[0066] Specific example of the process

[0067] 1. User A goes to sleep at night:

[0068] User A wears an EEG recording device, which records his / her brain waves while he / she sleeps. The device sends the EEG data to a server via his / her smartphone.

[0069] 2. User A wakes up and launches the app:

[0070] The server processes the received EEG data and converts the dream content into natural language text using a text generation model.

[0071] 3. The server visualizes the dream text:

[0072] The server inputs the generated text into an image generation model to generate images depicting specific scenes from the dream.

[0073] 4. User A searches for dreams in the app:

[0074] User A searches for "dreams featuring family" and selects a dream they have had in the past from the list to check the details.

[0075] 5. Add your favorite dreams to your favorites:

[0076] User A saves the dream in a favorites list so that he can re-experience it at any time.

[0077] The above is an embodiment of the present invention.

[0078] The processing flow will be explained below.

[0079] Step 1:

[0080] User:

[0081] Before sleep, participants wear an electroencephalography device, which is used to record the brain's electrical activity while they sleep.

[0082] Step 2:

[0083] Device:

[0084] The EEG recording device begins collecting the user's brainwave data, which is then transmitted in real time to a smartphone via Bluetooth or Wi-Fi.

[0085] Step 3:

[0086] server:

[0087] The device receives brainwave data sent from the smartphone and stores it in a database, thereby recording the brainwave data accumulated while the user is sleeping.

[0088] Step 4:

[0089] server:

[0090] Preprocess the stored EEG data to remove noise, e.g., eye artifacts and ambient electrical interference.

[0091] Step 5:

[0092] server:

[0093] The preprocessed EEG data is fed into a text generation model (such as GPT-4) to convert the dream content into natural language text, which then describes the dream content specifically based on the EEG patterns.

[0094] Step 6:

[0095] server:

[0096] Temporarily store the generated text and prepare it for later delivery upon user request.

[0097] Step 7:

[0098] server:

[0099] The textual content of the dream is passed to an image generation model (such as DALL-E) to generate detailed visual data. For example, the text "A dragon flying under a blue sky" is converted into a concrete image.

[0100] Step 8:

[0101] server:

[0102] The generated visual data is associated with the corresponding text data and stored in a database, so that the dream text and visual data are stored in an integrated manner.

[0103] Step 9:

[0104] User:

[0105] Open the app and search for a specific dream, for example, "a dream featuring family members" or "a dream on a specific date."

[0106] Step 10:

[0107] Device:

[0108] The search criteria entered by the user are sent to the server, which then searches the database based on the criteria and retrieves the relevant dream data.

[0109] Step 11:

[0110] server:

[0111] The text and visual data of the relevant dreams are retrieved from the search results and sent to the device, which then selects dreams that match the search criteria.

[0112] Step 12:

[0113] Device:

[0114] The received search results are displayed to the user, who can then view and check the details.

[0115] Step 13:

[0116] User:

[0117] If you want to add a dream you like to your favorites list, click the "Favorites" button.

[0118] Step 14:

[0119] Device:

[0120] A request to add the dream to the favorites list is sent to the server. The request includes the dream ID and user information.

[0121] Step 15:

[0122] server:

[0123] Based on the request, the specified dream is added to the user's "favorite list." The dream ID and user ID related information are stored in the database.

[0124] Step 16:

[0125] Device:

[0126] Displaying the updated favorites list to the user for easy access.

[0127] The above are the specific processing steps of the program of the present invention.

[0128] Example 1

[0129] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0130] The challenge is to efficiently collect EEG data acquired during sleep, and then remove noise, convert it into natural language text, and visualize it so that users can clearly understand the content of their dreams. Furthermore, a system is needed that allows users to easily search the generated dream data, save it as their favorite dream, and re-experience it.

[0131] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0132] In this invention, the server includes means for collecting EEG data acquired during sleep, preprocessing means for removing noise from EEG signals, and generative model means for converting the preprocessed EEG data into natural language text. This allows the user to accurately understand the dreams they had during sleep and visually understand the content of their dreams as text and visual data. The server also includes database means for storing the natural language text and visual data and making them accessible to users, and interface means for allowing users to search and save the stored data as favorites. This allows users to easily search their dreams and re-experience their favorite dreams.

[0133] "EEG data" refers to digital signals that record brain wave activity during sleep.

[0134] "Preprocessing" refers to processing to remove noise from collected EEG data and extract pure EEG patterns.

[0135] A "generative model" is an artificial intelligence model that generates natural language text or visual data based on collected data. Specifically, it refers to a text generation model that generates natural language text and an image generation model that generates images based on text.

[0136] "Database" means a storage device that stores and manages collected and generated data and makes it accessible to users.

[0137] An "interface" refers to the operation screen or input device that allows the user to directly interact with the system.

[0138] "Search" is a feature that allows users to search for saved dream data using specific keywords or conditions.

[0139] "Favorites" is a feature that allows users to save their favorite dream data in a list for easy access later.

[0140] "Re-experience" is a function that allows users to revisit and experience the content of their dreams using text and visual data from previously generated dreams.

[0141] "Server" refers to the computer system that processes EEG data, runs the generative model, stores and retrieves data, and provides the user interface.

[0142] "Terminal" refers to the device used by the user to collect EEG data, specifically the EEG recording device and the smartphone or tablet connected to it.

[0143] This invention is a system that collects the brainwave data of users while they sleep, converts the contents of their dreams into text in natural language, and visualizes them. Through this system, users can search, save, and re-experience their own dreams.

[0144] System configuration

[0145] Device:

[0146] The terminal is a smartphone or tablet that connects to the EEG recording device used by the user. An application is installed on the terminal to collect EEG data and send it to a server. The EEG recording device transfers the data to the smartphone via Bluetooth or Wi-Fi.

[0147] server:

[0148] The server plays a central role in receiving EEG data, preprocessing, converting dreams into text, visualizing them, and storing and providing the data. Specifically, the server performs the following functions:

[0149] 1. Receiving and storing data:

[0150] The server receives the brainwave data sent from the smartphone, checks the integrity of the data, and then stores it in a database.

[0151] 2. Pretreatment:

[0152] Remove noise from the received EEG data and extract pure EEG patterns by filtering high and low frequencies.

[0153] 3. Text Generation:

[0154] The preprocessed EEG data is fed into a text generation model (e.g., GPT-4), which analyzes the EEG patterns and generates natural language text based on them.

[0155] 4. Visual Generation:

[0156] The generated text is input into an image generation model (e.g., DALL-E) to generate visual data depicting a specific scene.

[0157] 5. Data storage and provision:

[0158] The generated natural language text and visual data are associated and stored in a database, and are made available to users upon request.

[0159] 6. Retrieve and re-experience:

[0160] Allow users to search for dreams by specific criteria. For example, users can search for dreams using filters such as date, theme, characters, etc. From the search results, users can select a specific dream to view details and relive it.

[0161] 7. Favorites feature:

[0162] Users can save their favorite dreams in a "favorites list" for easy access later.

[0163] Specific examples

[0164] 1. User A goes to sleep at night:

[0165] The user wears an EEG recording device, which collects real-time brainwave data, which is then transmitted via Bluetooth to a smartphone and then sent to a server.

[0166] 2. User A wakes up and launches the app:

[0167] The server processes the received EEG data, removes noise, and converts the dream content into natural language text using a text generation model.

[0168] 3. The server visualizes the dream text:

[0169] The server inputs the generated text into an image generation model to generate images depicting specific scenes from the dream.

[0170] 4. User A searches for dreams in the app:

[0171] User A searches for "dreams featuring family" and selects a dream they have had in the past from the list to check the details.

[0172] 5. Add your favorite dreams to your favorites:

[0173] User A can save the dream to a favorites list and re-experience it at any time.

[0174] Prompt Sentence Examples

[0175] For example, the prompt for transcribing a dream might look like this:

[0176] Based on the user's brainwave data, please explain the content of this dream in natural language.

[0177] Also, the prompt for visualizing the dream is as follows:

[0178] Based on the text below, please create an image that depicts a specific scene.

[0179] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0180] Step 1:

[0181] EEG data collection

[0182] Device: The user wears an EEG recording device, which collects real-time EEG data during sleep and transmits it to a smartphone via Bluetooth or Wi-Fi.

[0183] Input: User's brainwave data

[0184] Output: EEG data sent to a smartphone

[0185] How it works: The EEG recording device captures EEG signals and converts them into digital signals. The smartphone application receives the data.

[0186] Step 2:

[0187] Receiving EEG data

[0188] Server: Receives EEG data sent from the smartphone. The server checks the integrity of the data packets and stores them accurately in a database.

[0189] Input: EEG data sent from a smartphone

[0190] Output: EEG data stored on the server

[0191] Specific operation: The server receives the data, performs error checking, and stores it in the database.

[0192] Step 3:

[0193] Pre-processing for noise removal

[0194] Server: Removes unnecessary noise from the received EEG data, filtering high and low frequencies to extract pure EEG patterns.

[0195] Input: Received EEG data

[0196] Output: Preprocessed EEG data with noise removed

[0197] Specific operations: Runs a filtering algorithm to reduce noise. Sends preprocessed data to the next processing step.

[0198] Step 4:

[0199] Text Generation

[0200] Server: The preprocessed EEG data is fed into a text generation model (e.g., GPT-4), which analyzes the EEG patterns and converts the dream content into natural language text.

[0201] Input: Preprocessed EEG data

[0202] Output: Generated natural language text

[0203] Specific operation: Calls the GPT-4 model, generates text based on EEG patterns, and temporarily stores the generated text.

[0204] Step 5:

[0205] Visual Generation

[0206] Server: The generated text is input into an image generation model (e.g., DALL-E) to generate visual data depicting a specific scene.

[0207] Input: Generated natural language text

[0208] Output: The generated visual data

[0209] Specific operation: Calls the DALL-E model, generates an image based on the text, and saves the generated image in association with the text.

[0210] Step 6:

[0211] Data storage and provision

[0212] Server: Stores the generated text and visual data in a database in an associated manner and provides this data when requested by the user.

[0213] Input: Generated natural language text and visual data

[0214] Output: Textual and visual data stored in a database

[0215] Specific operation: Stores data in a database and retrieves and provides data upon user request.

[0216] Step 7:

[0217] Retrieving and re-experiencing dreams

[0218] Users: Use the application to use the interface to search for past dreams, using filters such as date, theme, and characters, and then select a specific dream to view more details.

[0219] Input: User's search query (e.g. date, subject)

[0220] Output: Text and visual data of the dream displayed as search results

[0221] Specific operation: A query is sent to the server based on the conditions entered by the user, and the server searches for data that matches the conditions and provides it to the terminal.

[0222] Step 8:

[0223] Favorites feature

[0224] User: Click the button to save the dream you like as a favorite.

[0225] Input: User-selected dream data

[0226] Output: Dream data added to the user's "favorite list"

[0227] Specific operation: The click operation is detected and the data of the selected dream is sent to the server. The server updates the database and adds the corresponding dream to the user's "favorite list."

[0228] (Application example 1)

[0229] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0230] Conventional systems that analyze EEG data during sleep only allow users to check the content of their dreams, but to further improve users' sleep experiences, there was a need for a function that could suggest relaxation products and services based on the obtained dream data.Furthermore, there was no system that could provide users with a personalized relaxation experience based on such data.

[0231] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0232] In this invention, the server includes means for collecting EEG data acquired during sleep, means for processing the EEG data and converting the content of dreams into natural language text, and means for converting the natural language text into visual data. This not only enables users to check the content of their dreams in detail as text and visual data, but also makes it possible to suggest relaxation products and services tailored to individual needs based on the data.

[0233] "Sleep" means the time while the user is resting.

[0234] "Electroencephalogram Data" means data regarding the electrical activity of a user's brain obtained using an electroencephalogram recording device.

[0235] "Natural language text" is written information expressed in a natural language in a form that can be understood by a user.

[0236] "Visual data" is data in a format that can be visually recognized, such as images or videos.

[0237] "Storage" means the act of retaining data for an extended period of time and making it readily accessible when needed.

[0238] "Search" refers to the act of a user examining data in a database to find specific information.

[0239] "Favorites" refers to a feature that allows users to easily access their favorite data again.

[0240] "Relaxation products" are any products that users use for the purpose of relaxation.

[0241] "Services" refers to various relaxation experiences and functions provided to Users.

[0242] The present invention relates to a system that collects electroencephalogram data of a user while they are sleeping, converts the content of their dreams into text and visualizes them based on the data, and suggests relaxation products and services. An embodiment of the system will be described in detail below.

[0243] System configuration

[0244] This system consists of the following main elements:

[0245] 1. EEG recording devices and terminals

[0246] Before sleep, users wear an EEG recording device, which collects real-time brainwave data while they sleep and transmits the data to a smartphone or tablet via Bluetooth or Wi-Fi.

[0247] 2. Server

[0248] The server is the central unit that receives the EEG data and performs the following processes.

[0249] 3. Preprocessing of EEG data

[0250] The server receives the transmitted EEG data and performs preprocessing to remove noise, using Python and libraries such as NumPy and Pandas.

[0251] 4. Texting dreams

[0252] The preprocessed EEG data is then fed into a text generation model (e.g., GPT-4) to convert the dream content into natural language text.

[0253] 5. Visualize your dreams

[0254] The generated natural language text is input into an image generation model (e.g., DALL-E), and the dream content is embodied as visual data.

[0255] 6. Data storage and access

[0256] The server saves the generated natural language text and visual data and stores it in a database for user access.

[0257] 7. Data search and favorites

[0258] Users can search for past dreams through a dedicated application, using filter options such as date, theme, and characters, and can save specific dreams as favorites.

[0259] 8. Proposal of relaxation products and services

[0260] The server then uses the stored data to suggest relaxation products and services, a process that involves algorithms that analyze text and visual data to recommend products and services tailored to the user.

[0261] Specific examples

[0262] In a specific example of implementing this system, User A wears an EEG recording device at night, and EEG data is recorded while he or she sleeps. When User A launches the app the next morning, the server processes the received EEG data and converts the dream content into natural language using a text generation model. The generated text is then input into a visualization model to visualize specific scenes from the dream. Furthermore, based on this data, relaxation products (e.g., aroma oils and massage chairs) and services (e.g., massage sessions and yoga classes) are suggested.

[0263] Prompt Sentence Examples

[0264] An example of a prompt for the generative AI model is, "Please describe in text the content of the dream generated based on the user's brainwave data."

[0265] By implementing the present invention in this manner, the user can not only understand the content of their dreams in detail, but also enjoy a personalized relaxation experience.

[0266] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0267] Step 1:

[0268] EEG data collection

[0269] The user wears an EEG recording device before going to sleep. The EEG recording device collects EEG data in real time while the user sleeps and transmits the data to a device (smartphone or tablet) via Bluetooth or Wi-Fi. The input is the raw EEG data collected by the EEG recording device, and the output is the data transmitted to the device.

[0270] Step 2:

[0271] EEG data transfer

[0272] The terminal transfers the received EEG data to the server. The input here is the EEG data received by the terminal, and the output is that data is sent to the server.

[0273] Step 3:

[0274] Preprocessing of EEG data

[0275] The server preprocesses the received EEG data and removes noise. Specifically, it uses Python's NumPy and Pandas libraries to filter the data and remove unnecessary noise. The input is the raw EEG data received by the server, and the output is the preprocessed, clean EEG data.

[0276] Step 4:

[0277] Dream text

[0278] The server inputs the preprocessed EEG data into a text generation model (e.g., GPT-4) and converts the dream content into natural language text. The input is the preprocessed EEG data, and the output is the generated natural language text. The prompt sentence used is, "Please describe in text the dream content generated based on the user's EEG data."

[0279] Step 5:

[0280] Dream visualization

[0281] The server inputs the generated natural language text into an image generation model (e.g., DALL-E) to convert the dream content into visual data. The input is the generated natural language text, and the output is the generated visual data.

[0282] Step 6:

[0283] Data storage

[0284] The server stores the generated natural language text and visual data in a database. The input is the generated natural language text and visual data, and the output is that the data is stored in the database.

[0285] Step 7:

[0286] Data search and favorites

[0287] Users can search for past dreams through a dedicated application, using filter options such as date, theme, and characters to search and save specific dreams as favorites. The input is the search criteria specified by the user, and the output is the natural language text and visual data results that match the criteria.

[0288] Step 8:

[0289] Proposal of relaxation products and services

[0290] The server proposes relaxation products and services based on the stored data. The input is the stored natural language text and visual data, and the output is the proposed information for relaxation products and services based on the analysis results.

[0291] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0292] This invention relates to a system that converts the content of a user's dreams into natural language text and visual data using electroencephalogram data acquired during sleep, and further combines it with an emotion engine to analyze and provide the user's emotions. Specific program processing for implementing this system is described below.

[0293] System configuration

[0294] Terminal

[0295] The device is an EEG recording device used by the user, and a terminal such as a smartphone or tablet. An application is installed on the terminal to collect EEG data and send it to a server.

[0296] server

[0297] The server receives the EEG data and plays a central role in preprocessing, dream textualization, visualization, emotion analysis, and data storage and provision.

[0298] Program processing

[0299] 1. Dream Data Collection

[0300] Terminal

[0301] Before sleep, users wear an EEG recording device, which collects real-time brainwave data and transmits it to a smartphone via Bluetooth or Wi-Fi.

[0302] server

[0303] The device receives brainwave data sent from the smartphone and stores it in a database, thereby recording the brainwave data accumulated while the user is sleeping.

[0304] 2. Translating the dream into text

[0305] server

[0306] The stored EEG data is preprocessed to remove noise, such as eye artifacts and ambient electrical interference. The preprocessed EEG data is then fed into a text generation model (e.g., GPT-4) to convert the dream content into natural language text. The generated text is temporarily stored and made available to the user upon request.

[0307] 3. Visualize your dreams

[0308] server

[0309] The textual content of a dream is input into an image generation model (such as DALL-E) to generate detailed visual data. For example, the text "A dragon flying under a blue sky" is converted into a concrete image. The generated visual data and the corresponding text data are associated and stored in a database.

[0310] 4. Emotion analysis using an emotion engine

[0311] server

[0312] An emotion engine is applied to the generated natural language text and visual data. The emotion engine analyzes the user's emotions based on the content of the dream and saves the results. For example, it determines that the user is likely to have felt emotions such as "happiness," "fear," or "surprise" based on the content of the dream.

[0313] 5. Searching for and Re-experiencing Dreams

[0314] User

[0315] Through the dedicated application, you use the interface to search for past dreams, choose filter options such as date, theme, characters, or emotional state, and then select a specific dream from the search results to view its details.

[0316] Terminal

[0317] The system sends the user's search criteria to the server and retrieves the corresponding dream data. The retrieved dream text and visual data are then displayed to the user. Sentiment analysis results are also provided.

[0318] 6. Favorites feature

[0319] User

[0320] If you want to add a dream you like to your favorites list, click the "Favorites" button.

[0321] Terminal

[0322] A request to add the dream to the favorites list is sent to the server. The request includes the dream ID and user information.

[0323] server

[0324] Based on the request, the specified dream is added to the user's "favorites list." The dream ID and user ID related information are saved in the database. The updated favorites list is notified to the device and reflected to the user.

[0325] Specific example of the process

[0326] 1. User A goes to sleep at night

[0327] User A wears an EEG recording device, which records his / her brain waves while he / she sleeps. The device sends the EEG data to a server via his / her smartphone.

[0328] 2. User A launches the app after waking up

[0329] The server processes the received EEG data and converts the dream content into natural language text using a text generation model.

[0330] 3. The server visualizes the dream text

[0331] The server inputs the generated text into an image generation model to generate images depicting specific scenes from the dream.

[0332] 4. Emotion analysis using an emotion engine

[0333] The server applies an emotion engine to the dream text and visual data to analyze the user's emotional state.

[0334] 5. User A searches for dreams in the app

[0335] User A searches for "dreams about family members" and selects a dream they have had in the past from the list to check the details. At the same time, the results of the emotion analysis are also provided.

[0336] 6. Add your favorite dreams to your favorites

[0337] User A adds the dream to his favorites list so he can re-experience it at any time.

[0338] The above is an embodiment of the present invention.

[0339] The processing flow will be explained below.

[0340] Step 1:

[0341] User:

[0342] Before sleep, participants wear an electroencephalography (EEG) device, which records their brain's electrical activity while they sleep.

[0343] Step 2:

[0344] Device:

[0345] The EEG recording device collects the user's brainwave data in real time and transmits the data to a smartphone via Bluetooth or Wi-Fi.

[0346] Step 3:

[0347] server:

[0348] The device receives brainwave data sent from the smartphone and stores it in a database, thereby recording the brainwave data accumulated while the user is sleeping.

[0349] Step 4:

[0350] server:

[0351] The stored EEG data is preprocessed to remove noise, for example, eye artifacts and ambient electrical interference.

[0352] Step 5:

[0353] server:

[0354] The preprocessed EEG data is fed into a text generation model (such as GPT-4) to convert the dream content into natural language text, which then details the dream content based on the EEG patterns.

[0355] Step 6:

[0356] server:

[0357] Temporarily store the generated natural language text and prepare it for later serving upon user request.

[0358] Step 7:

[0359] server:

[0360] The textual content of a dream is passed to an image generation model (such as DALL-E) to generate specific visual data. For example, an image is generated based on the text "A dragon flying under a blue sky."

[0361] Step 8:

[0362] server:

[0363] The generated visual data is associated with the corresponding text data and stored in a database, so that the dream text and visual data are stored in an integrated manner.

[0364] Step 9:

[0365] server:

[0366] An emotion engine is applied to the generated natural language text and visual data, which analyzes the user's emotions based on the content of the dream and stores the results.

[0367] Step 10:

[0368] User:

[0369] Open the dedicated application and use the interface to search for a specific dream, choosing filter options such as date, theme, characters, or emotional state.

[0370] Step 11:

[0371] Device:

[0372] The search criteria entered by the user are sent to the server, which then searches the database based on the criteria and retrieves the relevant dream data.

[0373] Step 12:

[0374] server:

[0375] The text and visual data of the dream that corresponds to the search result are retrieved and sent to the device, along with the results of the emotion analysis.

[0376] Step 13:

[0377] Device:

[0378] The received search results are then displayed to the user, including the dream text, visual data, and sentiment analysis results.

[0379] Step 14:

[0380] User:

[0381] If you want to add a dream you like to your favorites list, click the "Favorites" button.

[0382] Step 15:

[0383] Device:

[0384] A request to add the dream to the favorites list is sent to the server. The request includes the dream ID and user information.

[0385] Step 16:

[0386] server:

[0387] Based on the request, the specified dream is added to the user's "favorite list." The dream ID and user ID related information are stored in the database.

[0388] Step 17:

[0389] Device:

[0390] Displaying the updated favorites list to the user for easy access.

[0391] The above are the specific processing steps of the program of the present invention.

[0392] Example 2

[0393] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0394] In modern society, many people are unable to sleep soundly due to stress and anxiety, which in turn affects their health and quality of life. It is particularly important to understand that dreams during sleep reflect our mental state and underlying emotions, but memorizing and interpreting these dreams is not easy. Conventional technologies require specialized knowledge and equipment to analyze EEG data acquired during sleep and understand the content of dreams, making them inaccessible to average users. Furthermore, there are few support systems that visualize dream content and perform emotional analysis. This leaves users unable to understand their emotional state and re-experience their dreams.

[0395] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0396] In this invention, the server includes means for processing electroencephalogram data acquired during sleep and removing noise, means for converting the content of dreams based on the electroencephalogram data into natural language text, means for converting the natural language text into visual data, means for analyzing the user's emotions using the natural language text and visual data, means for allowing the user to search the stored data and save it as a favorite, and means for providing the user with a re-experience based on the natural language text and visual data. This allows the user to easily understand the content of their dreams and visually re-experience those dreams, and further allows the user to analyze their emotional state and use it to improve their daily life and mental health.

[0397] "Electroencephalogram data" is the data obtained by measuring the user's brain activity and recording patterns of electrical potential fluctuations.

[0398] The "pre-processing means" refers to a device or algorithm that performs processing to remove noise such as eye artifacts and ambient electrical interference from EEG data.

[0399] "Natural language text" refers to text data expressed in a language format that can be understood by humans.

[0400] "Visual data" refers to image and video data generated based on natural language text.

[0401] "Emotional Analysis Means" means a device or algorithm for analyzing and classifying a user's emotional state using natural language text and visual data reflecting the content of a dream.

[0402] "Search means" refers to a device or algorithm that allows a user to search and retrieve stored data based on specific criteria.

[0403] A "favorites tool" is a device or algorithm that allows users to save their favorite dream data and make it easier to access again.

[0404] "Means for re-experiencing" refers to a means of providing a user with the ability to re-experience a dream they have had in the past, using text and visual data from that dream.

[0405] The present invention relates to a system for converting dream content into natural language text and visual data using EEG data acquired while a user is asleep, and for further emotional analysis. This system is implemented using an EEG recording device, a terminal such as a smartphone or tablet, and a server.

[0406] System configuration

[0407] Terminal

[0408] Before sleep, users wear an EEG recording device, which transmits real-time EEG data via Bluetooth or Wi-Fi to a smartphone or tablet with a dedicated application installed. This application collects EEG data and sends it to a server.

[0409] server

[0410] The server plays a central role in receiving EEG data, preprocessing, converting dream content into text, visualization, emotion analysis, and storing and providing the data. The server performs the following specific processing steps:

[0411] 1. Pretreatment

[0412] The server performs pre-processing on the received EEG data, using Fourier transform and filtering techniques to remove noise, eye artifacts, and ambient electrical interference. The pre-processed data is temporarily stored in an intermediate database.

[0413] 2. Translating the dream into text

[0414] A text generation model (e.g., GPT-4) is used to convert the preprocessed EEG data into natural language text. The model converts dream content into natural language text based on brain activity patterns. The generated text is stored in a database.

[0415] 3. Visualizing the content of your dreams

[0416] The generated text data is input into an image generation model (e.g., DALL-E) to generate detailed visual data. For example, the text "A dragon flying under a blue sky" is converted into a concrete image. The visual data and the corresponding text data are associated and stored in a database.

[0417] 4. Emotion Analysis

[0418] Using the generated natural language text and visual data, the server applies an emotion analysis engine. The engine analyzes the user's emotions based on the content of the dream and classifies the results into categories such as "happiness," "fear," and "surprise." The emotion analysis results are stored in a database along with the text and visual data.

[0419] 5. Data provision and search

[0420] Through a dedicated application, users can use an interface to search for past dreams. They can select filter options such as date, theme, characters, and emotional state, and then select a specific dream from the search results to view details. When users enter search criteria, the device sends the search criteria to the server, which retrieves the corresponding dream data. The search results are sent to the device and displayed to the user in the dedicated application.

[0421] 6. Favorites feature

[0422] The user clicks the "Favorite" button to add a dream they like to their favorite list. The device sends a "Favorite" request to the server, adding the specified dream to the user's "Favorite List." The updated list is saved in the database and notified to the device.

[0423] Specific usage examples

[0424] User A goes to sleep at night

[0425] User A wears the EEG recording device and launches the dedicated smartphone application. The application starts collecting EEG data and transmits the data to the server in real time.

[0426] User A starts the app after waking up

[0427] The server preprocesses the received EEG data to remove noise, and then uses a text generation model to convert the dream content into natural language text.

[0428] The server visualizes the dream text.

[0429] Text data is input into an image generation model to generate specific visual data.

[0430] Emotion analysis using an emotion engine

[0431] The server applies a sentiment analysis engine to the dream text and visual data to generate a result that analyzes the user's sentiment.

[0432] User A searches for dreams on the app

[0433] User A searches for "dreams involving family members" using the filter and selects a past dream from the list. The results are displayed as text, visual, and sentiment analysis results.

[0434] Add your favorite dreams to your favorites

[0435] User A adds a specific dream to his favorites list, and the list is updated.

[0436] Prompt Sentence Examples

[0437] "Please translate last night's dream into words."

[0438] "I'd like to see a visual of a dream where my family appears."

[0439] "Search for past dreams that have scared you."

[0440] This concludes the embodiment of the present invention. Using this system, users can gain a deeper understanding of their dreams, visually re-experience them, and analyze their emotions.

[0441] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0442] Program processing flow

[0443] Step 1: Collecting and transmitting EEG data

[0444] 1. The user wears an EEG recording device before going to sleep.

[0445] 2. The terminal receives real-time EEG data from the EEG recording device via Bluetooth or Wi-Fi.

[0446] 3. A dedicated application installed on the device collects brainwave data.

[0447] 4. The device sends the collected EEG data to the relay server.

[0448] Input: EEG data obtained from an EEG recording device

[0449] Output: Raw EEG data sent to a server

[0450] Step 2: Preprocessing the EEG data

[0451] 1. The server preprocesses the received EEG data.

[0452] 2. The server uses Fourier transform and filtering techniques to remove noise, specifically eye artifacts and ambient electrical interference.

[0453] 3. The server temporarily stores the preprocessed EEG data in an intermediate database.

[0454] Input: Raw EEG data

[0455] Output: Preprocessed EEG data

[0456] Step 3: Translate the dream

[0457] 1. The server inputs the preprocessed EEG data into a text generation model (e.g., GPT-4).

[0458] 2. The server uses the model to convert the dream content from the EEG data into natural language text.

[0459] 3. The server stores the generated natural language text in a database.

[0460] Input: Preprocessed EEG data

[0461] Output: Generated natural language text

[0462] Step 4: Visualize your dream

[0463] 1. The server inputs the generated natural language text into an image generation model (e.g., DALL-E).

[0464] 2. The server reads the text content and generates the corresponding visual data.

[0465] 3. The server stores the visual data and the corresponding text data in a database.

[0466] Input: Natural language text

[0467] Output: The generated visual data

[0468] Step 5: Sentiment Analysis

[0469] 1. The server inputs natural language text and visual data into the sentiment analysis engine.

[0470] 2. The server uses the engine to analyze the user's emotions based on the content of the dream.

[0471] 3. The server stores the sentiment analysis results along with the text and visual data in a database.

[0472] Input: Natural language text and visual data

[0473] Output: Emotion analysis results

[0474] Step 6: Search for and re-experience the dream

[0475] 1. An interface that allows users to search for dreams through a dedicated application.

[0476] 2. The user selects a filter option such as date, theme, character, or emotional state.

[0477] 3. The device sends the search criteria to the server.

[0478] 4. The server retrieves data that matches the search criteria from the database.

[0479] 5. The device displays the text and visual data of the dream to the user.

[0480] Input: User search criteria

[0481] Output: Text, visual, and sentiment analysis data as search results

[0482] Step 7: Favorites function

[0483] 1. The user clicks the "Favorite" button to add the dream to their favorites list.

[0484] 2. The device sends a "favorite" request to the server.

[0485] 3. The server adds the specified dream to the user's "favorite list."

[0486] 4. The server saves the updated favorites list in the database and notifies the device.

[0487] Input: "Favorite" request (yume ID and user information)

[0488] Output: Updated favorites list

[0489] Example prompt

[0490] "Please translate last night's dream into words."

[0491] "I'd like to see a visual of a dream where my family appears."

[0492] "Search for past dreams that have scared you."

[0493] (Application example 2)

[0494] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0495] It is known that the sleep quality and mental state of factory workers have a significant impact on the next day's work efficiency and the risk of making mistakes. However, current technology lacks the means to specifically understand and analyze the sleep patterns and dream content of operators and to predict and manage work efficiency and risks based on this information. Therefore, there is a need for methods to improve factory productivity and safety.

[0496] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting biosignal data acquired during sleep, means for processing the biosignal data and converting the content of dreams into natural language text, means for converting the natural language text into visual data, means for saving the natural language text and the visual data and making them accessible to users, means for allowing users to search and save the saved data as preferences, and means for analyzing the emotional state of the user and predicting and managing work efficiency and the risk of errors. This makes it possible to analyze the sleep and mental states of operators in detail and improve work efficiency and safety in factories.

[0497] "Biological signal data" refers to data that indicates the user's physiological state, such as brain waves, heart rate, and respiratory rate.

[0498] "Natural language text" is text data expressed in a language format commonly used by humans.

[0499] "Visual data" is data that provides information visually, such as images or videos.

[0500] The term "user" refers to a person who uses the system of the present invention, that is, an operator or a user.

[0501] "Emotional state" refers to information that indicates the user's emotions, such as happiness, fear, surprise, etc.

[0502] "Work efficiency" is an index that indicates the progress and productivity of work within a certain period of time.

[0503] "Risk of mistakes occurring" is an indicator that indicates the possibility of mistakes or accidents occurring during work.

[0504] "Noise" is unwanted data or interference mixed into the signal of interest.

[0505] "Preprocessing" is a series of processing operations that shape, clean, and transform raw data prior to primary data processing.

[0506] A "server" is a computer system that receives, stores, processes, and distributes data.

[0507] This invention relates to a system that analyzes biosignal data acquired while an operator is sleeping, converts the data into natural language text and visual data, and further analyzes the operator's emotional state to predict and manage work efficiency and the risk of errors.

[0508] System configuration

[0509] Terminal

[0510] It includes a biosignal recording device used by the operator and a terminal such as a smartphone or tablet, on which an application is installed to collect biosignal data and send it to a server.

[0511] server

[0512] The server receives biosignal data and plays a central role in preprocessing, dream text conversion, visualization, emotion analysis, and data storage and provision.

[0513] Program processing and hardware / software details

[0514] 1. Data Collection and Preprocessing

[0515] Terminal

[0516] Before going to sleep, the operator wears a biosignal recording device (e.g., an electroencephalograph), which collects biosignal data in real time and transmits it to a smartphone via Bluetooth or Wi-Fi.

[0517] server

[0518] The system receives biosignal data sent from the smartphone and stores it in a database. This records the biosignal data accumulated while the operator is sleeping. As a pre-processing step, noise is removed using a Butterworth filter or similar.

[0519] 2. Translating the dream into text

[0520] server

[0521] The preprocessed biosignal data is fed into a text generation model (e.g., GPT-4) to convert the dream content into natural language text, which is temporarily stored and made available upon user request.

[0522] 3. Dream visualization

[0523] server

[0524] The textual content of a dream is input into an image generation model (e.g., DALL-E) to generate detailed visual data. For example, the text "A dragon flying under a blue sky" is converted into a concrete image. The generated visual data and the corresponding text data are associated and stored in a database.

[0525] 4. Emotion analysis using an emotion engine

[0526] server

[0527] An emotion engine is applied to the generated natural language text and visual data. The emotion engine analyzes the operator's emotions based on the content of the dream and saves the results. For example, it determines that the operator is likely to have felt emotions such as "happiness," "fear," or "surprise" based on the content of the dream.

[0528] 5. Leveraging Data

[0529] server

[0530] The system integrates the sleep quality and emotional state of operators to analyze work efficiency and the risk of mistakes, and provides the prediction results to robots in the factory, which then make it possible to adjust the day's work and mitigate risks.

[0531] Specific example of the process

[0532] 1. Operators fall asleep at night

[0533] The operator wears a biosignal recording device, which records biosignals while the patient sleeps. The device then transmits the biosignal data to a server via a smartphone.

[0534] 2. The operator launches the app after waking up

[0535] The server processes the received biosignal data and converts the dream content into natural language text using a text generation model.

[0536] 3. The server visualizes the dream text.

[0537] The server inputs the generated text into an image generation model to generate images depicting specific scenes from the dream.

[0538] 4. Emotion analysis using an emotion engine

[0539] The server applies an emotion engine to the dream text and visual data to analyze the operator's emotional state.

[0540] 5. The operator checks your dreams using the app

[0541] The operator searches for "dreams involving family members," selects a dream from a list of past dreams, and checks the details. At the same time, the operator is also provided with the results of an emotional analysis.

[0542] 6. Utilizing the results

[0543] Based on the data collected and analyzed by the server, it predicts work efficiency and the risk of errors, and provides information to factory robots.

[0544] Examples of prompt statements

[0545] "In a dream, the operator saw a dragon flying under a blue sky."

[0546] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0547] Step 1:

[0548] Terminal

[0549] Before sleep, the operator wears a biosignal recording device, which collects biosignal data such as brain waves and heart rate in real time and transmits the data to a smartphone via Bluetooth or Wi-Fi, thereby obtaining biosignal data as input.

[0550] Step 2:

[0551] server

[0552] The system receives biosignal data sent from a smartphone and stores it in a database. The received data is raw and contains noise, and this becomes the input data. Next, preprocessing is performed to remove noise using a Butterworth filter or similar, and clean biosignal data is output.

[0553] Step 3:

[0554] server

[0555] The preprocessed biosignal data is input into a text generation model (e.g., GPT-4). The model analyzes the input data and converts the dream content into natural language text. The output text data is in the form of, for example, "A dragon flying under a blue sky."

[0556] Step 4:

[0557] server

[0558] The generated natural language text is input into an image generation model (e.g., DALL-E). The model generates detailed visual data (images) based on the text data. For example, the text "A dragon flying under a blue sky" is converted into a concrete image. The output image is stored in a database.

[0559] Step 5:

[0560] server

[0561] An emotion engine is applied to the generated natural language text and visual data. The emotion engine analyzes the input text and image data and outputs the operator's emotional state as "happiness," "fear," "surprise," etc. The analysis results are stored in a database.

[0562] Step 6:

[0563] server

[0564] The system analyzes the operator's work efficiency and risk of mistakes by integrating their sleep quality and emotional state. The server refers to statistical data and past records and performs data calculations to predict risks. The output prediction results are provided to the factory robots.

[0565] Step 7:

[0566] User

[0567] After waking up, the operator can check the content of their dream through a dedicated application. The saved data can be searched and viewed through the app. The user can select "dreams featuring family members" to check the text and images, and the emotional analysis results are also displayed.

[0568] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0569] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0570] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0571] [Second embodiment]

[0572] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0573] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0574] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0575] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0576] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0577] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0578] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0579] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0580] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0581] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0582] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0583] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0584] This invention relates to a system that collects a user's brainwave data while they sleep and converts the contents of their dreams into text and visualizations. Through this system, users can search, save, and re-experience their own dreams. The specific program processing required to implement this system is described below.

[0585] System configuration

[0586] Terminal

[0587] The terminal is a smartphone or tablet that connects to the EEG recording device used by the user, and an application is installed on the terminal to collect EEG data and send it to a server.

[0588] server

[0589] The server plays a central role in receiving EEG data, preprocessing, converting dreams into text, visualizing them, and storing and providing the data.

[0590] Program processing

[0591] 1. Dream Data Collection

[0592] Device:

[0593] Before sleep, users wear an EEG recording device, which collects real-time brainwave data and transmits it to a smartphone via Bluetooth or Wi-Fi.

[0594] server:

[0595] The server receives the brainwave data sent from the smartphone and stores it in a database.

[0596] 2. Translating the dream into text

[0597] server:

[0598] The collected EEG data is processed to remove unwanted noise. The preprocessed data is then fed into a text generation model (e.g., GPT-4) to convert the dream content into natural language text. The generated text is temporarily stored and provided to the user upon request.

[0599] 3. Visualize your dreams

[0600] server:

[0601] The textual content of dreams is input into an image generation model (e.g., DALL-E) and converted into concrete visual data. The generated visual data and the corresponding text data are associated and stored in a database.

[0602] 4. Searching for and re-experiencing dreams

[0603] User:

[0604] Through the dedicated application, you can use the interface to search for past dreams, choose filter options such as date, theme, or characters, and then select a specific dream from the search results to view its details.

[0605] Device:

[0606] The server sends the user's search criteria to retrieve the corresponding dream data, and displays the retrieved dream text and visual data to the user.

[0607] 5. Favorites feature

[0608] User:

[0609] Click the button to save the dream you like as a favorite.

[0610] Device:

[0611] Sends requests to the server regarding the registration and management of favorites.

[0612] server:

[0613] Based on the user's request, specific dreams are added to the user's "favorite list" and saved and updated in the database.

[0614] Specific example of the process

[0615] 1. User A goes to sleep at night:

[0616] User A wears an EEG recording device, which records his / her brain waves while he / she sleeps. The device sends the EEG data to a server via his / her smartphone.

[0617] 2. User A wakes up and launches the app:

[0618] The server processes the received EEG data and converts the dream content into natural language text using a text generation model.

[0619] 3. The server visualizes the dream text:

[0620] The server inputs the generated text into an image generation model to generate images depicting specific scenes from the dream.

[0621] 4. User A searches for dreams in the app:

[0622] User A searches for "dreams featuring family" and selects a dream they have had in the past from the list to check the details.

[0623] 5. Add your favorite dreams to your favorites:

[0624] User A saves the dream in a favorites list so that he can re-experience it at any time.

[0625] The above is an embodiment of the present invention.

[0626] The processing flow will be explained below.

[0627] Step 1:

[0628] User:

[0629] Before sleep, participants wear an electroencephalography device, which is used to record the brain's electrical activity while they sleep.

[0630] Step 2:

[0631] Device:

[0632] The EEG recording device begins collecting the user's brainwave data, which is then transmitted in real time to a smartphone via Bluetooth or Wi-Fi.

[0633] Step 3:

[0634] server:

[0635] The device receives brainwave data sent from the smartphone and stores it in a database, thereby recording the brainwave data accumulated while the user is sleeping.

[0636] Step 4:

[0637] server:

[0638] Preprocess the stored EEG data to remove noise, e.g., eye artifacts and ambient electrical interference.

[0639] Step 5:

[0640] server:

[0641] The preprocessed EEG data is fed into a text generation model (such as GPT-4) to convert the dream content into natural language text, which then describes the dream content specifically based on the EEG patterns.

[0642] Step 6:

[0643] server:

[0644] Temporarily store the generated text and prepare it for later delivery upon user request.

[0645] Step 7:

[0646] server:

[0647] The textual content of the dream is passed to an image generation model (such as DALL-E) to generate detailed visual data. For example, the text "A dragon flying under a blue sky" is converted into a concrete image.

[0648] Step 8:

[0649] server:

[0650] The generated visual data is associated with the corresponding text data and stored in a database, so that the dream text and visual data are stored in an integrated manner.

[0651] Step 9:

[0652] User:

[0653] Open the app and search for a specific dream, for example, "a dream featuring family members" or "a dream on a specific date."

[0654] Step 10:

[0655] Device:

[0656] The search criteria entered by the user are sent to the server, which then searches the database based on the criteria and retrieves the relevant dream data.

[0657] Step 11:

[0658] server:

[0659] The text and visual data of the relevant dreams are retrieved from the search results and sent to the device, which then selects dreams that match the search criteria.

[0660] Step 12:

[0661] Device:

[0662] The received search results are displayed to the user, who can then view and check the details.

[0663] Step 13:

[0664] User:

[0665] If you want to add a dream you like to your favorites list, click the "Favorites" button.

[0666] Step 14:

[0667] Device:

[0668] A request to add the dream to the favorites list is sent to the server. The request includes the dream ID and user information.

[0669] Step 15:

[0670] server:

[0671] Based on the request, the specified dream is added to the user's "favorite list." The dream ID and user ID related information are stored in the database.

[0672] Step 16:

[0673] Device:

[0674] Displaying the updated favorites list to the user for easy access.

[0675] The above are the specific processing steps of the program of the present invention.

[0676] Example 1

[0677] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0678] The challenge is to efficiently collect EEG data acquired during sleep, and then remove noise, convert it into natural language text, and visualize it so that users can clearly understand the content of their dreams. Furthermore, a system is needed that allows users to easily search the generated dream data, save it as their favorite dream, and re-experience it.

[0679] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0680] In this invention, the server includes means for collecting EEG data acquired during sleep, preprocessing means for removing noise from EEG signals, and generative model means for converting the preprocessed EEG data into natural language text. This allows the user to accurately understand the dreams they had during sleep and visually understand the content of their dreams as text and visual data. The server also includes database means for storing the natural language text and visual data and making them accessible to users, and interface means for allowing users to search and save the stored data as favorites. This allows users to easily search their dreams and re-experience their favorite dreams.

[0681] "EEG data" refers to digital signals that record brain wave activity during sleep.

[0682] "Preprocessing" refers to processing to remove noise from collected EEG data and extract pure EEG patterns.

[0683] A "generative model" is an artificial intelligence model that generates natural language text or visual data based on collected data. Specifically, it refers to a text generation model that generates natural language text and an image generation model that generates images based on text.

[0684] "Database" means a storage device that stores and manages collected and generated data and makes it accessible to users.

[0685] An "interface" refers to the operation screen or input device that allows the user to directly interact with the system.

[0686] "Search" is a feature that allows users to search for saved dream data using specific keywords or conditions.

[0687] "Favorites" is a feature that allows users to save their favorite dream data in a list for easy access later.

[0688] "Re-experience" is a function that allows users to revisit and experience the content of their dreams using text and visual data from previously generated dreams.

[0689] "Server" refers to the computer system that processes EEG data, runs the generative model, stores and retrieves data, and provides the user interface.

[0690] "Terminal" refers to the device used by the user to collect EEG data, specifically the EEG recording device and the smartphone or tablet connected to it.

[0691] This invention is a system that collects the brainwave data of users while they sleep, converts the contents of their dreams into text in natural language, and visualizes them. Through this system, users can search, save, and re-experience their own dreams.

[0692] System configuration

[0693] Device:

[0694] The terminal is a smartphone or tablet that connects to the EEG recording device used by the user. An application is installed on the terminal to collect EEG data and send it to a server. The EEG recording device transfers the data to the smartphone via Bluetooth or Wi-Fi.

[0695] server:

[0696] The server plays a central role in receiving EEG data, preprocessing, converting dreams into text, visualizing them, and storing and providing the data. Specifically, the server performs the following functions:

[0697] 1. Receiving and storing data:

[0698] The server receives the brainwave data sent from the smartphone, checks the integrity of the data, and then stores it in a database.

[0699] 2. Pretreatment:

[0700] Remove noise from the received EEG data and extract pure EEG patterns by filtering high and low frequencies.

[0701] 3. Text Generation:

[0702] The preprocessed EEG data is fed into a text generation model (e.g., GPT-4), which analyzes the EEG patterns and generates natural language text based on them.

[0703] 4. Visual Generation:

[0704] The generated text is input into an image generation model (e.g., DALL-E) to generate visual data depicting a specific scene.

[0705] 5. Data storage and provision:

[0706] The generated natural language text and visual data are associated and stored in a database, and are made available to users upon request.

[0707] 6. Retrieve and re-experience:

[0708] Allow users to search for dreams by specific criteria. For example, users can search for dreams using filters such as date, theme, characters, etc. From the search results, users can select a specific dream to view details and relive it.

[0709] 7. Favorites feature:

[0710] Users can save their favorite dreams in a "favorites list" for easy access later.

[0711] Specific examples

[0712] 1. User A goes to sleep at night:

[0713] The user wears an EEG recording device, which collects real-time brainwave data, which is then transmitted via Bluetooth to a smartphone and then sent to a server.

[0714] 2. User A wakes up and launches the app:

[0715] The server processes the received EEG data, removes noise, and converts the dream content into natural language text using a text generation model.

[0716] 3. The server visualizes the dream text:

[0717] The server inputs the generated text into an image generation model to generate images depicting specific scenes from the dream.

[0718] 4. User A searches for dreams in the app:

[0719] User A searches for "dreams featuring family" and selects a dream they have had in the past from the list to check the details.

[0720] 5. Add your favorite dreams to your favorites:

[0721] User A can save the dream to a favorites list and re-experience it at any time.

[0722] Prompt Sentence Examples

[0723] For example, the prompt for transcribing a dream might look like this:

[0724] Based on the user's brainwave data, please explain the content of this dream in natural language.

[0725] Also, the prompt for visualizing the dream is as follows:

[0726] Based on the text below, please create an image that depicts a specific scene.

[0727] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0728] Step 1:

[0729] EEG data collection

[0730] Device: The user wears an EEG recording device, which collects real-time EEG data during sleep and transmits it to a smartphone via Bluetooth or Wi-Fi.

[0731] Input: User's brainwave data

[0732] Output: EEG data sent to a smartphone

[0733] How it works: The EEG recording device captures EEG signals and converts them into digital signals. The smartphone application receives the data.

[0734] Step 2:

[0735] Receiving EEG data

[0736] Server: Receives EEG data sent from the smartphone. The server checks the integrity of the data packets and stores them accurately in a database.

[0737] Input: EEG data sent from a smartphone

[0738] Output: EEG data stored on the server

[0739] Specific operation: The server receives the data, performs error checking, and stores it in the database.

[0740] Step 3:

[0741] Pre-processing for noise removal

[0742] Server: Removes unnecessary noise from the received EEG data, filtering high and low frequencies to extract pure EEG patterns.

[0743] Input: Received EEG data

[0744] Output: Preprocessed EEG data with noise removed

[0745] Specific operations: Runs a filtering algorithm to reduce noise. Sends preprocessed data to the next processing step.

[0746] Step 4:

[0747] Text Generation

[0748] Server: The preprocessed EEG data is fed into a text generation model (e.g., GPT-4), which analyzes the EEG patterns and converts the dream content into natural language text.

[0749] Input: Preprocessed EEG data

[0750] Output: Generated natural language text

[0751] Specific operation: Calls the GPT-4 model, generates text based on EEG patterns, and temporarily stores the generated text.

[0752] Step 5:

[0753] Visual Generation

[0754] Server: The generated text is input into an image generation model (e.g., DALL-E) to generate visual data depicting a specific scene.

[0755] Input: Generated natural language text

[0756] Output: The generated visual data

[0757] Specific operation: Calls the DALL-E model, generates an image based on the text, and saves the generated image in association with the text.

[0758] Step 6:

[0759] Data storage and provision

[0760] Server: Stores the generated text and visual data in a database in an associated manner and provides this data when requested by the user.

[0761] Input: Generated natural language text and visual data

[0762] Output: Textual and visual data stored in a database

[0763] Specific operation: Stores data in a database and retrieves and provides data upon user request.

[0764] Step 7:

[0765] Retrieving and re-experiencing dreams

[0766] Users: Use the application to use the interface to search for past dreams, using filters such as date, theme, and characters, and then select a specific dream to view more details.

[0767] Input: User's search query (e.g. date, subject)

[0768] Output: Text and visual data of the dream displayed as search results

[0769] Specific operation: A query is sent to the server based on the conditions entered by the user, and the server searches for data that matches the conditions and provides it to the terminal.

[0770] Step 8:

[0771] Favorites feature

[0772] User: Click the button to save the dream you like as a favorite.

[0773] Input: User-selected dream data

[0774] Output: Dream data added to the user's "favorite list"

[0775] Specific operation: The click operation is detected and the data of the selected dream is sent to the server. The server updates the database and adds the corresponding dream to the user's "favorite list."

[0776] (Application example 1)

[0777] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0778] Conventional systems that analyze EEG data during sleep only allow users to check the content of their dreams, but to further improve users' sleep experiences, there was a need for a function that could suggest relaxation products and services based on the obtained dream data.Furthermore, there was no system that could provide users with a personalized relaxation experience based on such data.

[0779] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0780] In this invention, the server includes means for collecting EEG data acquired during sleep, means for processing the EEG data and converting the content of dreams into natural language text, and means for converting the natural language text into visual data. This not only enables users to check the content of their dreams in detail as text and visual data, but also makes it possible to suggest relaxation products and services tailored to individual needs based on the data.

[0781] "Sleep" means the time while the user is resting.

[0782] "Electroencephalogram Data" means data regarding the electrical activity of a user's brain obtained using an electroencephalogram recording device.

[0783] "Natural language text" is written information expressed in a natural language in a form that can be understood by a user.

[0784] "Visual data" is data in a format that can be visually recognized, such as images or videos.

[0785] "Storage" means the act of retaining data for an extended period of time and making it readily accessible when needed.

[0786] "Search" refers to the act of a user examining data in a database to find specific information.

[0787] "Favorites" refers to a feature that allows users to easily access their favorite data again.

[0788] "Relaxation products" are any products that users use for the purpose of relaxation.

[0789] "Services" refers to various relaxation experiences and functions provided to Users.

[0790] The present invention relates to a system that collects electroencephalogram data of a user while they are sleeping, converts the content of their dreams into text and visualizes them based on the data, and suggests relaxation products and services. An embodiment of the system will be described in detail below.

[0791] System configuration

[0792] This system consists of the following main elements:

[0793] 1. EEG recording devices and terminals

[0794] Before sleep, users wear an EEG recording device, which collects real-time brainwave data while they sleep and transmits the data to a smartphone or tablet via Bluetooth or Wi-Fi.

[0795] 2. Server

[0796] The server is the central unit that receives the EEG data and performs the following processes.

[0797] 3. Preprocessing of EEG data

[0798] The server receives the transmitted EEG data and performs preprocessing to remove noise, using Python and libraries such as NumPy and Pandas.

[0799] 4. Texting dreams

[0800] The preprocessed EEG data is then fed into a text generation model (e.g., GPT-4) to convert the dream content into natural language text.

[0801] 5. Visualize your dreams

[0802] The generated natural language text is input into an image generation model (e.g., DALL-E), and the dream content is embodied as visual data.

[0803] 6. Data storage and access

[0804] The server saves the generated natural language text and visual data and stores it in a database for user access.

[0805] 7. Data search and favorites

[0806] Users can search for past dreams through a dedicated application, using filter options such as date, theme, and characters, and can save specific dreams as favorites.

[0807] 8. Proposal of relaxation products and services

[0808] The server then uses the stored data to suggest relaxation products and services, a process that involves algorithms that analyze text and visual data to recommend products and services tailored to the user.

[0809] Specific examples

[0810] In a specific example of implementing this system, User A wears an EEG recording device at night, and EEG data is recorded while he or she sleeps. When User A launches the app the next morning, the server processes the received EEG data and converts the dream content into natural language using a text generation model. The generated text is then input into a visualization model to visualize specific scenes from the dream. Furthermore, based on this data, relaxation products (e.g., aroma oils and massage chairs) and services (e.g., massage sessions and yoga classes) are suggested.

[0811] Prompt Sentence Examples

[0812] An example of a prompt for the generative AI model is, "Please describe in text the content of the dream generated based on the user's brainwave data."

[0813] By implementing the present invention in this manner, the user can not only understand the content of their dreams in detail, but also enjoy a personalized relaxation experience.

[0814] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0815] Step 1:

[0816] EEG data collection

[0817] The user wears an EEG recording device before going to sleep. The EEG recording device collects EEG data in real time while the user sleeps and transmits the data to a device (smartphone or tablet) via Bluetooth or Wi-Fi. The input is the raw EEG data collected by the EEG recording device, and the output is the data transmitted to the device.

[0818] Step 2:

[0819] EEG data transfer

[0820] The terminal transfers the received EEG data to the server. The input here is the EEG data received by the terminal, and the output is that data is sent to the server.

[0821] Step 3:

[0822] Preprocessing of EEG data

[0823] The server preprocesses the received EEG data and removes noise. Specifically, it uses Python's NumPy and Pandas libraries to filter the data and remove unnecessary noise. The input is the raw EEG data received by the server, and the output is the preprocessed, clean EEG data.

[0824] Step 4:

[0825] Dream text

[0826] The server inputs the preprocessed EEG data into a text generation model (e.g., GPT-4) and converts the dream content into natural language text. The input is the preprocessed EEG data, and the output is the generated natural language text. The prompt sentence used is, "Please describe in text the dream content generated based on the user's EEG data."

[0827] Step 5:

[0828] Dream visualization

[0829] The server inputs the generated natural language text into an image generation model (e.g., DALL-E) to convert the dream content into visual data. The input is the generated natural language text, and the output is the generated visual data.

[0830] Step 6:

[0831] Data storage

[0832] The server stores the generated natural language text and visual data in a database. The input is the generated natural language text and visual data, and the output is that the data is stored in the database.

[0833] Step 7:

[0834] Data search and favorites

[0835] Users can search for past dreams through a dedicated application, using filter options such as date, theme, and characters to search and save specific dreams as favorites. The input is the search criteria specified by the user, and the output is the natural language text and visual data results that match the criteria.

[0836] Step 8:

[0837] Proposal of relaxation products and services

[0838] The server proposes relaxation products and services based on the stored data. The input is the stored natural language text and visual data, and the output is the proposed information for relaxation products and services based on the analysis results.

[0839] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0840] This invention relates to a system that converts the content of a user's dreams into natural language text and visual data using electroencephalogram data acquired during sleep, and further combines it with an emotion engine to analyze and provide the user's emotions. Specific program processing for implementing this system is described below.

[0841] System configuration

[0842] Terminal

[0843] The device is an EEG recording device used by the user, and a terminal such as a smartphone or tablet. An application is installed on the terminal to collect EEG data and send it to a server.

[0844] server

[0845] The server receives the EEG data and plays a central role in preprocessing, dream textualization, visualization, emotion analysis, and data storage and provision.

[0846] Program processing

[0847] 1. Dream Data Collection

[0848] Terminal

[0849] Before sleep, users wear an EEG recording device, which collects real-time brainwave data and transmits it to a smartphone via Bluetooth or Wi-Fi.

[0850] server

[0851] The device receives brainwave data sent from the smartphone and stores it in a database, thereby recording the brainwave data accumulated while the user is sleeping.

[0852] 2. Translating the dream into text

[0853] server

[0854] The stored EEG data is preprocessed to remove noise, such as eye artifacts and ambient electrical interference. The preprocessed EEG data is then fed into a text generation model (e.g., GPT-4) to convert the dream content into natural language text. The generated text is temporarily stored and made available to the user upon request.

[0855] 3. Visualize your dreams

[0856] server

[0857] The textual content of a dream is input into an image generation model (such as DALL-E) to generate detailed visual data. For example, the text "A dragon flying under a blue sky" is converted into a concrete image. The generated visual data and the corresponding text data are associated and stored in a database.

[0858] 4. Emotion analysis using an emotion engine

[0859] server

[0860] An emotion engine is applied to the generated natural language text and visual data. The emotion engine analyzes the user's emotions based on the content of the dream and saves the results. For example, it determines that the user is likely to have felt emotions such as "happiness," "fear," or "surprise" based on the content of the dream.

[0861] 5. Searching for and Re-experiencing Dreams

[0862] User

[0863] Through the dedicated application, you use the interface to search for past dreams, choose filter options such as date, theme, characters, or emotional state, and then select a specific dream from the search results to view its details.

[0864] Terminal

[0865] The system sends the user's search criteria to the server and retrieves the corresponding dream data. The retrieved dream text and visual data are then displayed to the user. Sentiment analysis results are also provided.

[0866] 6. Favorites feature

[0867] User

[0868] If you want to add a dream you like to your favorites list, click the "Favorites" button.

[0869] Terminal

[0870] A request to add the dream to the favorites list is sent to the server. The request includes the dream ID and user information.

[0871] server

[0872] Based on the request, the specified dream is added to the user's "favorites list." The dream ID and user ID related information are saved in the database. The updated favorites list is notified to the device and reflected to the user.

[0873] Specific example of the process

[0874] 1. User A goes to sleep at night

[0875] User A wears an EEG recording device, which records his / her brain waves while he / she sleeps. The device sends the EEG data to a server via his / her smartphone.

[0876] 2. User A launches the app after waking up

[0877] The server processes the received EEG data and converts the dream content into natural language text using a text generation model.

[0878] 3. The server visualizes the dream text

[0879] The server inputs the generated text into an image generation model to generate images depicting specific scenes from the dream.

[0880] 4. Emotion analysis using an emotion engine

[0881] The server applies an emotion engine to the dream text and visual data to analyze the user's emotional state.

[0882] 5. User A searches for dreams in the app

[0883] User A searches for "dreams about family members" and selects a dream they have had in the past from the list to check the details. At the same time, the results of the emotion analysis are also provided.

[0884] 6. Add your favorite dreams to your favorites

[0885] User A adds the dream to his favorites list so he can re-experience it at any time.

[0886] The above is an embodiment of the present invention.

[0887] The processing flow will be explained below.

[0888] Step 1:

[0889] User:

[0890] Before sleep, participants wear an electroencephalography (EEG) device, which records their brain's electrical activity while they sleep.

[0891] Step 2:

[0892] Device:

[0893] The EEG recording device collects the user's brainwave data in real time and transmits the data to a smartphone via Bluetooth or Wi-Fi.

[0894] Step 3:

[0895] server:

[0896] The device receives brainwave data sent from the smartphone and stores it in a database, thereby recording the brainwave data accumulated while the user is sleeping.

[0897] Step 4:

[0898] server:

[0899] The stored EEG data is preprocessed to remove noise, for example, eye artifacts and ambient electrical interference.

[0900] Step 5:

[0901] server:

[0902] The preprocessed EEG data is fed into a text generation model (such as GPT-4) to convert the dream content into natural language text, which then details the dream content based on the EEG patterns.

[0903] Step 6:

[0904] server:

[0905] Temporarily store the generated natural language text and prepare it for later serving upon user request.

[0906] Step 7:

[0907] server:

[0908] The textual content of a dream is passed to an image generation model (such as DALL-E) to generate specific visual data. For example, an image is generated based on the text "A dragon flying under a blue sky."

[0909] Step 8:

[0910] server:

[0911] The generated visual data is associated with the corresponding text data and stored in a database, so that the dream text and visual data are stored in an integrated manner.

[0912] Step 9:

[0913] server:

[0914] An emotion engine is applied to the generated natural language text and visual data, which analyzes the user's emotions based on the content of the dream and stores the results.

[0915] Step 10:

[0916] User:

[0917] Open the dedicated application and use the interface to search for a specific dream, choosing filter options such as date, theme, characters, or emotional state.

[0918] Step 11:

[0919] Device:

[0920] The search criteria entered by the user are sent to the server, which then searches the database based on the criteria and retrieves the relevant dream data.

[0921] Step 12:

[0922] server:

[0923] The text and visual data of the dream that corresponds to the search result are retrieved and sent to the device, along with the results of the emotion analysis.

[0924] Step 13:

[0925] Device:

[0926] The received search results are then displayed to the user, including the dream text, visual data, and sentiment analysis results.

[0927] Step 14:

[0928] User:

[0929] If you want to add a dream you like to your favorites list, click the "Favorites" button.

[0930] Step 15:

[0931] Device:

[0932] A request to add the dream to the favorites list is sent to the server. The request includes the dream ID and user information.

[0933] Step 16:

[0934] server:

[0935] Based on the request, the specified dream is added to the user's "favorite list." The dream ID and user ID related information are stored in the database.

[0936] Step 17:

[0937] Device:

[0938] Displaying the updated favorites list to the user for easy access.

[0939] The above are the specific processing steps of the program of the present invention.

[0940] Example 2

[0941] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0942] In modern society, many people are unable to sleep soundly due to stress and anxiety, which in turn affects their health and quality of life. It is particularly important to understand that dreams during sleep reflect our mental state and underlying emotions, but memorizing and interpreting these dreams is not easy. Conventional technologies require specialized knowledge and equipment to analyze EEG data acquired during sleep and understand the content of dreams, making them inaccessible to average users. Furthermore, there are few support systems that visualize dream content and perform emotional analysis. This leaves users unable to understand their emotional state and re-experience their dreams.

[0943] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0944] In this invention, the server includes means for processing electroencephalogram data acquired during sleep and removing noise, means for converting the content of dreams based on the electroencephalogram data into natural language text, means for converting the natural language text into visual data, means for analyzing the user's emotions using the natural language text and visual data, means for allowing the user to search the stored data and save it as a favorite, and means for providing the user with a re-experience based on the natural language text and visual data. This allows the user to easily understand the content of their dreams and visually re-experience those dreams, and further allows the user to analyze their emotional state and use it to improve their daily life and mental health.

[0945] "Electroencephalogram data" is the data obtained by measuring the user's brain activity and recording patterns of electrical potential fluctuations.

[0946] The "pre-processing means" refers to a device or algorithm that performs processing to remove noise such as eye artifacts and ambient electrical interference from EEG data.

[0947] "Natural language text" refers to text data expressed in a language format that can be understood by humans.

[0948] "Visual data" refers to image and video data generated based on natural language text.

[0949] "Emotional Analysis Means" means a device or algorithm for analyzing and classifying a user's emotional state using natural language text and visual data reflecting the content of a dream.

[0950] "Search means" refers to a device or algorithm that allows a user to search and retrieve stored data based on specific criteria.

[0951] A "favorites tool" is a device or algorithm that allows users to save their favorite dream data and make it easier to access again.

[0952] "Means for re-experiencing" refers to a means of providing a user with the ability to re-experience a dream they have had in the past, using text and visual data from that dream.

[0953] The present invention relates to a system for converting dream content into natural language text and visual data using EEG data acquired while a user is asleep, and for further emotional analysis. This system is implemented using an EEG recording device, a terminal such as a smartphone or tablet, and a server.

[0954] System configuration

[0955] Terminal

[0956] Before sleep, users wear an EEG recording device, which transmits real-time EEG data via Bluetooth or Wi-Fi to a smartphone or tablet with a dedicated application installed. This application collects EEG data and sends it to a server.

[0957] server

[0958] The server plays a central role in receiving EEG data, preprocessing, converting dream content into text, visualization, emotion analysis, and storing and providing the data. The server performs the following specific processing steps:

[0959] 1. Pretreatment

[0960] The server performs pre-processing on the received EEG data, using Fourier transform and filtering techniques to remove noise, eye artifacts, and ambient electrical interference. The pre-processed data is temporarily stored in an intermediate database.

[0961] 2. Translating the dream into text

[0962] A text generation model (e.g., GPT-4) is used to convert the preprocessed EEG data into natural language text. The model converts dream content into natural language text based on brain activity patterns. The generated text is stored in a database.

[0963] 3. Visualizing the content of your dreams

[0964] The generated text data is input into an image generation model (e.g., DALL-E) to generate detailed visual data. For example, the text "A dragon flying under a blue sky" is converted into a concrete image. The visual data and the corresponding text data are associated and stored in a database.

[0965] 4. Emotion Analysis

[0966] Using the generated natural language text and visual data, the server applies an emotion analysis engine. The engine analyzes the user's emotions based on the content of the dream and classifies the results into categories such as "happiness," "fear," and "surprise." The emotion analysis results are stored in a database along with the text and visual data.

[0967] 5. Data provision and search

[0968] Through a dedicated application, users can use an interface to search for past dreams. They can select filter options such as date, theme, characters, and emotional state, and then select a specific dream from the search results to view details. When users enter search criteria, the device sends the search criteria to the server, which retrieves the corresponding dream data. The search results are sent to the device and displayed to the user in the dedicated application.

[0969] 6. Favorites feature

[0970] The user clicks the "Favorite" button to add a dream they like to their favorite list. The device sends a "Favorite" request to the server, adding the specified dream to the user's "Favorite List." The updated list is saved in the database and notified to the device.

[0971] Specific usage examples

[0972] User A goes to sleep at night

[0973] User A wears the EEG recording device and launches the dedicated smartphone application. The application starts collecting EEG data and transmits the data to the server in real time.

[0974] User A starts the app after waking up

[0975] The server preprocesses the received EEG data to remove noise, and then uses a text generation model to convert the dream content into natural language text.

[0976] The server visualizes the dream text.

[0977] Text data is input into an image generation model to generate specific visual data.

[0978] Emotion analysis using an emotion engine

[0979] The server applies a sentiment analysis engine to the dream text and visual data to generate a result that analyzes the user's sentiment.

[0980] User A searches for dreams on the app

[0981] User A searches for "dreams involving family members" using the filter and selects a past dream from the list. The results are displayed as text, visual, and sentiment analysis results.

[0982] Add your favorite dreams to your favorites

[0983] User A adds a specific dream to his favorites list, and the list is updated.

[0984] Prompt Sentence Examples

[0985] "Please translate last night's dream into words."

[0986] "I'd like to see a visual of a dream where my family appears."

[0987] "Search for past dreams that have scared you."

[0988] This concludes the embodiment of the present invention. Using this system, users can gain a deeper understanding of their dreams, visually re-experience them, and analyze their emotions.

[0989] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0990] Program processing flow

[0991] Step 1: Collecting and transmitting EEG data

[0992] 1. The user wears an EEG recording device before going to sleep.

[0993] 2. The terminal receives real-time EEG data from the EEG recording device via Bluetooth or Wi-Fi.

[0994] 3. A dedicated application installed on the device collects brainwave data.

[0995] 4. The device sends the collected EEG data to the relay server.

[0996] Input: EEG data obtained from an EEG recording device

[0997] Output: Raw EEG data sent to a server

[0998] Step 2: Preprocessing the EEG data

[0999] 1. The server preprocesses the received EEG data.

[1000] 2. The server uses Fourier transform and filtering techniques to remove noise, specifically eye artifacts and ambient electrical interference.

[1001] 3. The server temporarily stores the preprocessed EEG data in an intermediate database.

[1002] Input: Raw EEG data

[1003] Output: Preprocessed EEG data

[1004] Step 3: Translate the dream

[1005] 1. The server inputs the preprocessed EEG data into a text generation model (e.g., GPT-4).

[1006] 2. The server uses the model to convert the dream content from the EEG data into natural language text.

[1007] 3. The server stores the generated natural language text in a database.

[1008] Input: Preprocessed EEG data

[1009] Output: Generated natural language text

[1010] Step 4: Visualize your dream

[1011] 1. The server inputs the generated natural language text into an image generation model (e.g., DALL-E).

[1012] 2. The server reads the text content and generates the corresponding visual data.

[1013] 3. The server stores the visual data and the corresponding text data in a database.

[1014] Input: Natural language text

[1015] Output: The generated visual data

[1016] Step 5: Sentiment Analysis

[1017] 1. The server inputs natural language text and visual data into the sentiment analysis engine.

[1018] 2. The server uses the engine to analyze the user's emotions based on the content of the dream.

[1019] 3. The server stores the sentiment analysis results along with the text and visual data in a database.

[1020] Input: Natural language text and visual data

[1021] Output: Emotion analysis results

[1022] Step 6: Search for and re-experience the dream

[1023] 1. An interface that allows users to search for dreams through a dedicated application.

[1024] 2. The user selects a filter option such as date, theme, character, or emotional state.

[1025] 3. The device sends the search criteria to the server.

[1026] 4. The server retrieves data that matches the search criteria from the database.

[1027] 5. The device displays the text and visual data of the dream to the user.

[1028] Input: User search criteria

[1029] Output: Text, visual, and sentiment analysis data as search results

[1030] Step 7: Favorites function

[1031] 1. The user clicks the "Favorite" button to add the dream to their favorites list.

[1032] 2. The device sends a "favorite" request to the server.

[1033] 3. The server adds the specified dream to the user's "favorite list."

[1034] 4. The server saves the updated favorites list in the database and notifies the device.

[1035] Input: "Favorite" request (yume ID and user information)

[1036] Output: Updated favorites list

[1037] Example prompt

[1038] "Please translate last night's dream into words."

[1039] "I'd like to see a visual of a dream where my family appears."

[1040] "Search for past dreams that have scared you."

[1041] (Application example 2)

[1042] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1043] It is known that the sleep quality and mental state of factory workers have a significant impact on the next day's work efficiency and the risk of making mistakes. However, current technology lacks the means to specifically understand and analyze the sleep patterns and dream content of operators and to predict and manage work efficiency and risks based on this information. Therefore, there is a need for methods to improve factory productivity and safety.

[1044] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting biosignal data acquired during sleep, means for processing the biosignal data and converting the content of dreams into natural language text, means for converting the natural language text into visual data, means for saving the natural language text and the visual data and making them accessible to users, means for allowing users to search and save the saved data as preferences, and means for analyzing the emotional state of the user and predicting and managing work efficiency and the risk of errors. This makes it possible to analyze the sleep and mental states of operators in detail and improve work efficiency and safety in factories.

[1045] "Biological signal data" refers to data that indicates the user's physiological state, such as brain waves, heart rate, and respiratory rate.

[1046] "Natural language text" is text data expressed in a language format commonly used by humans.

[1047] "Visual data" is data that provides information visually, such as images or videos.

[1048] The term "user" refers to a person who uses the system of the present invention, that is, an operator or a user.

[1049] "Emotional state" refers to information that indicates the user's emotions, such as happiness, fear, surprise, etc.

[1050] "Work efficiency" is an index that indicates the progress and productivity of work within a certain period of time.

[1051] "Risk of mistakes occurring" is an indicator that indicates the possibility of mistakes or accidents occurring during work.

[1052] "Noise" is unwanted data or interference mixed into the signal of interest.

[1053] "Preprocessing" is a series of processing operations that shape, clean, and transform raw data prior to primary data processing.

[1054] A "server" is a computer system that receives, stores, processes, and distributes data.

[1055] This invention relates to a system that analyzes biosignal data acquired while an operator is sleeping, converts the data into natural language text and visual data, and further analyzes the operator's emotional state to predict and manage work efficiency and the risk of errors.

[1056] System configuration

[1057] Terminal

[1058] It includes a biosignal recording device used by the operator and a terminal such as a smartphone or tablet, on which an application is installed to collect biosignal data and send it to a server.

[1059] server

[1060] The server receives biosignal data and plays a central role in preprocessing, dream text conversion, visualization, emotion analysis, and data storage and provision.

[1061] Program processing and hardware / software details

[1062] 1. Data Collection and Preprocessing

[1063] Terminal

[1064] Before going to sleep, the operator wears a biosignal recording device (e.g., an electroencephalograph), which collects biosignal data in real time and transmits it to a smartphone via Bluetooth or Wi-Fi.

[1065] server

[1066] The system receives biosignal data sent from the smartphone and stores it in a database. This records the biosignal data accumulated while the operator is sleeping. As a pre-processing step, noise is removed using a Butterworth filter or similar.

[1067] 2. Translating the dream into text

[1068] server

[1069] The preprocessed biosignal data is fed into a text generation model (e.g., GPT-4) to convert the dream content into natural language text, which is temporarily stored and made available upon user request.

[1070] 3. Dream visualization

[1071] server

[1072] The textual content of a dream is input into an image generation model (e.g., DALL-E) to generate detailed visual data. For example, the text "A dragon flying under a blue sky" is converted into a concrete image. The generated visual data and the corresponding text data are associated and stored in a database.

[1073] 4. Emotion analysis using an emotion engine

[1074] server

[1075] An emotion engine is applied to the generated natural language text and visual data. The emotion engine analyzes the operator's emotions based on the content of the dream and saves the results. For example, it determines that the operator is likely to have felt emotions such as "happiness," "fear," or "surprise" based on the content of the dream.

[1076] 5. Leveraging Data

[1077] server

[1078] The system integrates the sleep quality and emotional state of operators to analyze work efficiency and the risk of mistakes, and provides the prediction results to robots in the factory, which then make it possible to adjust the day's work and mitigate risks.

[1079] Specific example of the process

[1080] 1. Operators fall asleep at night

[1081] The operator wears a biosignal recording device, which records biosignals while the patient sleeps. The device then transmits the biosignal data to a server via a smartphone.

[1082] 2. The operator launches the app after waking up

[1083] The server processes the received biosignal data and converts the dream content into natural language text using a text generation model.

[1084] 3. The server visualizes the dream text.

[1085] The server inputs the generated text into an image generation model to generate images depicting specific scenes from the dream.

[1086] 4. Emotion analysis using an emotion engine

[1087] The server applies an emotion engine to the dream text and visual data to analyze the operator's emotional state.

[1088] 5. The operator checks your dreams using the app

[1089] The operator searches for "dreams involving family members," selects a dream from a list of past dreams, and checks the details. At the same time, the operator is also provided with the results of an emotional analysis.

[1090] 6. Utilizing the results

[1091] Based on the data collected and analyzed by the server, it predicts work efficiency and the risk of errors, and provides information to factory robots.

[1092] Examples of prompt statements

[1093] "In a dream, the operator saw a dragon flying under a blue sky."

[1094] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1095] Step 1:

[1096] Terminal

[1097] Before sleep, the operator wears a biosignal recording device, which collects biosignal data such as brain waves and heart rate in real time and transmits the data to a smartphone via Bluetooth or Wi-Fi, thereby obtaining biosignal data as input.

[1098] Step 2:

[1099] server

[1100] The system receives biosignal data sent from a smartphone and stores it in a database. The received data is raw and contains noise, and this becomes the input data. Next, preprocessing is performed to remove noise using a Butterworth filter or similar, and clean biosignal data is output.

[1101] Step 3:

[1102] server

[1103] The preprocessed biosignal data is input into a text generation model (e.g., GPT-4). The model analyzes the input data and converts the dream content into natural language text. The output text data is in the form of, for example, "A dragon flying under a blue sky."

[1104] Step 4:

[1105] server

[1106] The generated natural language text is input into an image generation model (e.g., DALL-E). The model generates detailed visual data (images) based on the text data. For example, the text "A dragon flying under a blue sky" is converted into a concrete image. The output image is stored in a database.

[1107] Step 5:

[1108] server

[1109] An emotion engine is applied to the generated natural language text and visual data. The emotion engine analyzes the input text and image data and outputs the operator's emotional state as "happiness," "fear," "surprise," etc. The analysis results are stored in a database.

[1110] Step 6:

[1111] server

[1112] The system analyzes the operator's work efficiency and risk of mistakes by integrating their sleep quality and emotional state. The server refers to statistical data and past records and performs data calculations to predict risks. The output prediction results are provided to the factory robots.

[1113] Step 7:

[1114] User

[1115] After waking up, the operator can check the content of their dream through a dedicated application. The saved data can be searched and viewed through the app. The user can select "dreams featuring family members" to check the text and images, and the emotional analysis results are also displayed.

[1116] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1117] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1118] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1119] [Third embodiment]

[1120] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1121] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1122] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1123] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1124] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1125] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1126] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1127] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1128] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1129] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1130] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1131] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1132] This invention relates to a system that collects a user's brainwave data while they sleep and converts the contents of their dreams into text and visualizations. Through this system, users can search, save, and re-experience their own dreams. The specific program processing required to implement this system is described below.

[1133] System configuration

[1134] Terminal

[1135] The terminal is a smartphone or tablet that connects to the EEG recording device used by the user, and an application is installed on the terminal to collect EEG data and send it to a server.

[1136] server

[1137] The server plays a central role in receiving EEG data, preprocessing, converting dreams into text, visualizing them, and storing and providing the data.

[1138] Program processing

[1139] 1. Dream Data Collection

[1140] Device:

[1141] Before sleep, users wear an EEG recording device, which collects real-time brainwave data and transmits it to a smartphone via Bluetooth or Wi-Fi.

[1142] server:

[1143] The server receives the brainwave data sent from the smartphone and stores it in a database.

[1144] 2. Translating the dream into text

[1145] server:

[1146] The collected EEG data is processed to remove unwanted noise. The preprocessed data is then fed into a text generation model (e.g., GPT-4) to convert the dream content into natural language text. The generated text is temporarily stored and provided to the user upon request.

[1147] 3. Visualize your dreams

[1148] server:

[1149] The textual content of dreams is input into an image generation model (e.g., DALL-E) and converted into concrete visual data. The generated visual data and the corresponding text data are associated and stored in a database.

[1150] 4. Searching for and re-experiencing dreams

[1151] User:

[1152] Through the dedicated application, you can use the interface to search for past dreams, choose filter options such as date, theme, or characters, and then select a specific dream from the search results to view its details.

[1153] Device:

[1154] The server sends the user's search criteria to retrieve the corresponding dream data, and displays the retrieved dream text and visual data to the user.

[1155] 5. Favorites feature

[1156] User:

[1157] Click the button to save the dream you like as a favorite.

[1158] Device:

[1159] Sends requests to the server regarding the registration and management of favorites.

[1160] server:

[1161] Based on the user's request, specific dreams are added to the user's "favorite list" and saved and updated in the database.

[1162] Specific example of the process

[1163] 1. User A goes to sleep at night:

[1164] User A wears an EEG recording device, which records his / her brain waves while he / she sleeps. The device sends the EEG data to a server via his / her smartphone.

[1165] 2. User A wakes up and launches the app:

[1166] The server processes the received EEG data and converts the dream content into natural language text using a text generation model.

[1167] 3. The server visualizes the dream text:

[1168] The server inputs the generated text into an image generation model to generate images depicting specific scenes from the dream.

[1169] 4. User A searches for dreams in the app:

[1170] User A searches for "dreams featuring family" and selects a dream they have had in the past from the list to check the details.

[1171] 5. Add your favorite dreams to your favorites:

[1172] User A saves the dream in a favorites list so that he can re-experience it at any time.

[1173] The above is an embodiment of the present invention.

[1174] The processing flow will be explained below.

[1175] Step 1:

[1176] User:

[1177] Before sleep, participants wear an electroencephalography device, which is used to record the brain's electrical activity while they sleep.

[1178] Step 2:

[1179] Device:

[1180] The EEG recording device begins collecting the user's brainwave data, which is then transmitted in real time to a smartphone via Bluetooth or Wi-Fi.

[1181] Step 3:

[1182] server:

[1183] The device receives brainwave data sent from the smartphone and stores it in a database, thereby recording the brainwave data accumulated while the user is sleeping.

[1184] Step 4:

[1185] server:

[1186] Preprocess the stored EEG data to remove noise, e.g., eye artifacts and ambient electrical interference.

[1187] Step 5:

[1188] server:

[1189] The preprocessed EEG data is fed into a text generation model (such as GPT-4) to convert the dream content into natural language text, which then describes the dream content specifically based on the EEG patterns.

[1190] Step 6:

[1191] server:

[1192] Temporarily store the generated text and prepare it for later delivery upon user request.

[1193] Step 7:

[1194] server:

[1195] The textual content of the dream is passed to an image generation model (such as DALL-E) to generate detailed visual data. For example, the text "A dragon flying under a blue sky" is converted into a concrete image.

[1196] Step 8:

[1197] server:

[1198] The generated visual data is associated with the corresponding text data and stored in a database, so that the dream text and visual data are stored in an integrated manner.

[1199] Step 9:

[1200] User:

[1201] Open the app and search for a specific dream, for example, "a dream featuring family members" or "a dream on a specific date."

[1202] Step 10:

[1203] Device:

[1204] The search criteria entered by the user are sent to the server, which then searches the database based on the criteria and retrieves the relevant dream data.

[1205] Step 11:

[1206] server:

[1207] The text and visual data of the relevant dreams are retrieved from the search results and sent to the device, which then selects dreams that match the search criteria.

[1208] Step 12:

[1209] Device:

[1210] The received search results are displayed to the user, who can then view and check the details.

[1211] Step 13:

[1212] User:

[1213] If you want to add a dream you like to your favorites list, click the "Favorites" button.

[1214] Step 14:

[1215] Device:

[1216] A request to add the dream to the favorites list is sent to the server. The request includes the dream ID and user information.

[1217] Step 15:

[1218] server:

[1219] Based on the request, the specified dream is added to the user's "favorite list." The dream ID and user ID related information are stored in the database.

[1220] Step 16:

[1221] Device:

[1222] Displaying the updated favorites list to the user for easy access.

[1223] The above are the specific processing steps of the program of the present invention.

[1224] Example 1

[1225] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1226] The challenge is to efficiently collect EEG data acquired during sleep, and then remove noise, convert it into natural language text, and visualize it so that users can clearly understand the content of their dreams. Furthermore, a system is needed that allows users to easily search the generated dream data, save it as their favorite dream, and re-experience it.

[1227] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1228] In this invention, the server includes means for collecting EEG data acquired during sleep, preprocessing means for removing noise from EEG signals, and generative model means for converting the preprocessed EEG data into natural language text. This allows the user to accurately understand the dreams they had during sleep and visually understand the content of their dreams as text and visual data. The server also includes database means for storing the natural language text and visual data and making them accessible to users, and interface means for allowing users to search and save the stored data as favorites. This allows users to easily search their dreams and re-experience their favorite dreams.

[1229] "EEG data" refers to digital signals that record brain wave activity during sleep.

[1230] "Preprocessing" refers to processing to remove noise from collected EEG data and extract pure EEG patterns.

[1231] A "generative model" is an artificial intelligence model that generates natural language text or visual data based on collected data. Specifically, it refers to a text generation model that generates natural language text and an image generation model that generates images based on text.

[1232] "Database" means a storage device that stores and manages collected and generated data and makes it accessible to users.

[1233] An "interface" refers to the operation screen or input device that allows the user to directly interact with the system.

[1234] "Search" is a feature that allows users to search for saved dream data using specific keywords or conditions.

[1235] "Favorites" is a feature that allows users to save their favorite dream data in a list for easy access later.

[1236] "Re-experience" is a function that allows users to revisit and experience the content of their dreams using text and visual data from previously generated dreams.

[1237] "Server" refers to the computer system that processes EEG data, runs the generative model, stores and retrieves data, and provides the user interface.

[1238] "Terminal" refers to the device used by the user to collect EEG data, specifically the EEG recording device and the smartphone or tablet connected to it.

[1239] This invention is a system that collects the brainwave data of users while they sleep, converts the contents of their dreams into text in natural language, and visualizes them. Through this system, users can search, save, and re-experience their own dreams.

[1240] System configuration

[1241] Device:

[1242] The terminal is a smartphone or tablet that connects to the EEG recording device used by the user. An application is installed on the terminal to collect EEG data and send it to a server. The EEG recording device transfers the data to the smartphone via Bluetooth or Wi-Fi.

[1243] server:

[1244] The server plays a central role in receiving EEG data, preprocessing, converting dreams into text, visualizing them, and storing and providing the data. Specifically, the server performs the following functions:

[1245] 1. Receiving and storing data:

[1246] The server receives the brainwave data sent from the smartphone, checks the integrity of the data, and then stores it in a database.

[1247] 2. Pretreatment:

[1248] Remove noise from the received EEG data and extract pure EEG patterns by filtering high and low frequencies.

[1249] 3. Text Generation:

[1250] The preprocessed EEG data is fed into a text generation model (e.g., GPT-4), which analyzes the EEG patterns and generates natural language text based on them.

[1251] 4. Visual Generation:

[1252] The generated text is input into an image generation model (e.g., DALL-E) to generate visual data depicting a specific scene.

[1253] 5. Data storage and provision:

[1254] The generated natural language text and visual data are associated and stored in a database, and are made available to users upon request.

[1255] 6. Retrieve and re-experience:

[1256] Allow users to search for dreams by specific criteria. For example, users can search for dreams using filters such as date, theme, characters, etc. From the search results, users can select a specific dream to view details and relive it.

[1257] 7. Favorites feature:

[1258] Users can save their favorite dreams in a "favorites list" for easy access later.

[1259] Specific examples

[1260] 1. User A goes to sleep at night:

[1261] The user wears an EEG recording device, which collects real-time brainwave data, which is then transmitted via Bluetooth to a smartphone and then sent to a server.

[1262] 2. User A wakes up and launches the app:

[1263] The server processes the received EEG data, removes noise, and converts the dream content into natural language text using a text generation model.

[1264] 3. The server visualizes the dream text:

[1265] The server inputs the generated text into an image generation model to generate images depicting specific scenes from the dream.

[1266] 4. User A searches for dreams in the app:

[1267] User A searches for "dreams featuring family" and selects a dream they have had in the past from the list to check the details.

[1268] 5. Add your favorite dreams to your favorites:

[1269] User A can save the dream to a favorites list and re-experience it at any time.

[1270] Prompt Sentence Examples

[1271] For example, the prompt for transcribing a dream might look like this:

[1272] Based on the user's brainwave data, please explain the content of this dream in natural language.

[1273] Also, the prompt for visualizing the dream is as follows:

[1274] Based on the text below, please create an image that depicts a specific scene.

[1275] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1276] Step 1:

[1277] EEG data collection

[1278] Device: The user wears an EEG recording device, which collects real-time EEG data during sleep and transmits it to a smartphone via Bluetooth or Wi-Fi.

[1279] Input: User's brainwave data

[1280] Output: EEG data sent to a smartphone

[1281] How it works: The EEG recording device captures EEG signals and converts them into digital signals. The smartphone application receives the data.

[1282] Step 2:

[1283] Receiving EEG data

[1284] Server: Receives EEG data sent from the smartphone. The server checks the integrity of the data packets and stores them accurately in a database.

[1285] Input: EEG data sent from a smartphone

[1286] Output: EEG data stored on the server

[1287] Specific operation: The server receives the data, performs error checking, and stores it in the database.

[1288] Step 3:

[1289] Pre-processing for noise removal

[1290] Server: Removes unnecessary noise from the received EEG data, filtering high and low frequencies to extract pure EEG patterns.

[1291] Input: Received EEG data

[1292] Output: Preprocessed EEG data with noise removed

[1293] Specific operations: Runs a filtering algorithm to reduce noise. Sends preprocessed data to the next processing step.

[1294] Step 4:

[1295] Text Generation

[1296] Server: The preprocessed EEG data is fed into a text generation model (e.g., GPT-4), which analyzes the EEG patterns and converts the dream content into natural language text.

[1297] Input: Preprocessed EEG data

[1298] Output: Generated natural language text

[1299] Specific operation: Calls the GPT-4 model, generates text based on EEG patterns, and temporarily stores the generated text.

[1300] Step 5:

[1301] Visual Generation

[1302] Server: The generated text is input into an image generation model (e.g., DALL-E) to generate visual data depicting a specific scene.

[1303] Input: Generated natural language text

[1304] Output: The generated visual data

[1305] Specific operation: Calls the DALL-E model, generates an image based on the text, and saves the generated image in association with the text.

[1306] Step 6:

[1307] Data storage and provision

[1308] Server: Stores the generated text and visual data in a database in an associated manner and provides this data when requested by the user.

[1309] Input: Generated natural language text and visual data

[1310] Output: Textual and visual data stored in a database

[1311] Specific operation: Stores data in a database and retrieves and provides data upon user request.

[1312] Step 7:

[1313] Retrieving and re-experiencing dreams

[1314] Users: Use the application to use the interface to search for past dreams, using filters such as date, theme, and characters, and then select a specific dream to view more details.

[1315] Input: User's search query (e.g. date, subject)

[1316] Output: Text and visual data of the dream displayed as search results

[1317] Specific operation: A query is sent to the server based on the conditions entered by the user, and the server searches for data that matches the conditions and provides it to the terminal.

[1318] Step 8:

[1319] Favorites feature

[1320] User: Click the button to save the dream you like as a favorite.

[1321] Input: User-selected dream data

[1322] Output: Dream data added to the user's "favorite list"

[1323] Specific operation: The click operation is detected and the data of the selected dream is sent to the server. The server updates the database and adds the corresponding dream to the user's "favorite list."

[1324] (Application example 1)

[1325] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1326] Conventional systems that analyze EEG data during sleep only allow users to check the content of their dreams, but to further improve users' sleep experiences, there was a need for a function that could suggest relaxation products and services based on the obtained dream data.Furthermore, there was no system that could provide users with a personalized relaxation experience based on such data.

[1327] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1328] In this invention, the server includes means for collecting EEG data acquired during sleep, means for processing the EEG data and converting the content of dreams into natural language text, and means for converting the natural language text into visual data. This not only enables users to check the content of their dreams in detail as text and visual data, but also makes it possible to suggest relaxation products and services tailored to individual needs based on the data.

[1329] "Sleep" means the time while the user is resting.

[1330] "Electroencephalogram Data" means data regarding the electrical activity of a user's brain obtained using an electroencephalogram recording device.

[1331] "Natural language text" is written information expressed in a natural language in a form that can be understood by a user.

[1332] "Visual data" is data in a format that can be visually recognized, such as images or videos.

[1333] "Storage" means the act of retaining data for an extended period of time and making it readily accessible when needed.

[1334] "Search" refers to the act of a user examining data in a database to find specific information.

[1335] "Favorites" refers to a feature that allows users to easily access their favorite data again.

[1336] "Relaxation products" are any products that users use for the purpose of relaxation.

[1337] "Services" refers to various relaxation experiences and functions provided to Users.

[1338] The present invention relates to a system that collects electroencephalogram data of a user while they are sleeping, converts the content of their dreams into text and visualizes them based on the data, and suggests relaxation products and services. An embodiment of the system will be described in detail below.

[1339] System configuration

[1340] This system consists of the following main elements:

[1341] 1. EEG recording devices and terminals

[1342] Before sleep, users wear an EEG recording device, which collects real-time brainwave data while they sleep and transmits the data to a smartphone or tablet via Bluetooth or Wi-Fi.

[1343] 2. Server

[1344] The server is the central unit that receives the EEG data and performs the following processes.

[1345] 3. Preprocessing of EEG data

[1346] The server receives the transmitted EEG data and performs preprocessing to remove noise, using Python and libraries such as NumPy and Pandas.

[1347] 4. Texting dreams

[1348] The preprocessed EEG data is then fed into a text generation model (e.g., GPT-4) to convert the dream content into natural language text.

[1349] 5. Visualize your dreams

[1350] The generated natural language text is input into an image generation model (e.g., DALL-E), and the dream content is embodied as visual data.

[1351] 6. Data storage and access

[1352] The server saves the generated natural language text and visual data and stores it in a database for user access.

[1353] 7. Data search and favorites

[1354] Users can search for past dreams through a dedicated application, using filter options such as date, theme, and characters, and can save specific dreams as favorites.

[1355] 8. Proposal of relaxation products and services

[1356] The server then uses the stored data to suggest relaxation products and services, a process that involves algorithms that analyze text and visual data to recommend products and services tailored to the user.

[1357] Specific examples

[1358] In a specific example of implementing this system, User A wears an EEG recording device at night, and EEG data is recorded while he or she sleeps. When User A launches the app the next morning, the server processes the received EEG data and converts the dream content into natural language using a text generation model. The generated text is then input into a visualization model to visualize specific scenes from the dream. Furthermore, based on this data, relaxation products (e.g., aroma oils and massage chairs) and services (e.g., massage sessions and yoga classes) are suggested.

[1359] Prompt Sentence Examples

[1360] An example of a prompt for the generative AI model is, "Please describe in text the content of the dream generated based on the user's brainwave data."

[1361] By implementing the present invention in this manner, the user can not only understand the content of their dreams in detail, but also enjoy a personalized relaxation experience.

[1362] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1363] Step 1:

[1364] EEG data collection

[1365] The user wears an EEG recording device before going to sleep. The EEG recording device collects EEG data in real time while the user sleeps and transmits the data to a device (smartphone or tablet) via Bluetooth or Wi-Fi. The input is the raw EEG data collected by the EEG recording device, and the output is the data transmitted to the device.

[1366] Step 2:

[1367] EEG data transfer

[1368] The terminal transfers the received EEG data to the server. The input here is the EEG data received by the terminal, and the output is that data is sent to the server.

[1369] Step 3:

[1370] Preprocessing of EEG data

[1371] The server preprocesses the received EEG data and removes noise. Specifically, it uses Python's NumPy and Pandas libraries to filter the data and remove unnecessary noise. The input is the raw EEG data received by the server, and the output is the preprocessed, clean EEG data.

[1372] Step 4:

[1373] Dream text

[1374] The server inputs the preprocessed EEG data into a text generation model (e.g., GPT-4) and converts the dream content into natural language text. The input is the preprocessed EEG data, and the output is the generated natural language text. The prompt sentence used is, "Please describe in text the dream content generated based on the user's EEG data."

[1375] Step 5:

[1376] Dream visualization

[1377] The server inputs the generated natural language text into an image generation model (e.g., DALL-E) to convert the dream content into visual data. The input is the generated natural language text, and the output is the generated visual data.

[1378] Step 6:

[1379] Data storage

[1380] The server stores the generated natural language text and visual data in a database. The input is the generated natural language text and visual data, and the output is that the data is stored in the database.

[1381] Step 7:

[1382] Data search and favorites

[1383] Users can search for past dreams through a dedicated application, using filter options such as date, theme, and characters to search and save specific dreams as favorites. The input is the search criteria specified by the user, and the output is the natural language text and visual data results that match the criteria.

[1384] Step 8:

[1385] Proposal of relaxation products and services

[1386] The server proposes relaxation products and services based on the stored data. The input is the stored natural language text and visual data, and the output is the proposed information for relaxation products and services based on the analysis results.

[1387] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1388] This invention relates to a system that converts the content of a user's dreams into natural language text and visual data using electroencephalogram data acquired during sleep, and further combines it with an emotion engine to analyze and provide the user's emotions. Specific program processing for implementing this system is described below.

[1389] System configuration

[1390] Terminal

[1391] The device is an EEG recording device used by the user, and a terminal such as a smartphone or tablet. An application is installed on the terminal to collect EEG data and send it to a server.

[1392] server

[1393] The server receives the EEG data and plays a central role in preprocessing, dream textualization, visualization, emotion analysis, and data storage and provision.

[1394] Program processing

[1395] 1. Dream Data Collection

[1396] Terminal

[1397] Before sleep, users wear an EEG recording device, which collects real-time brainwave data and transmits it to a smartphone via Bluetooth or Wi-Fi.

[1398] server

[1399] The device receives brainwave data sent from the smartphone and stores it in a database, thereby recording the brainwave data accumulated while the user is sleeping.

[1400] 2. Translating the dream into text

[1401] server

[1402] The stored EEG data is preprocessed to remove noise, such as eye artifacts and ambient electrical interference. The preprocessed EEG data is then fed into a text generation model (e.g., GPT-4) to convert the dream content into natural language text. The generated text is temporarily stored and made available to the user upon request.

[1403] 3. Visualize your dreams

[1404] server

[1405] The textual content of a dream is input into an image generation model (such as DALL-E) to generate detailed visual data. For example, the text "A dragon flying under a blue sky" is converted into a concrete image. The generated visual data and the corresponding text data are associated and stored in a database.

[1406] 4. Emotion analysis using an emotion engine

[1407] server

[1408] An emotion engine is applied to the generated natural language text and visual data. The emotion engine analyzes the user's emotions based on the content of the dream and saves the results. For example, it determines that the user is likely to have felt emotions such as "happiness," "fear," or "surprise" based on the content of the dream.

[1409] 5. Searching for and Re-experiencing Dreams

[1410] User

[1411] Through the dedicated application, you use the interface to search for past dreams, choose filter options such as date, theme, characters, or emotional state, and then select a specific dream from the search results to view its details.

[1412] Terminal

[1413] The system sends the user's search criteria to the server and retrieves the corresponding dream data. The retrieved dream text and visual data are then displayed to the user. Sentiment analysis results are also provided.

[1414] 6. Favorites feature

[1415] User

[1416] If you want to add a dream you like to your favorites list, click the "Favorites" button.

[1417] Terminal

[1418] A request to add the dream to the favorites list is sent to the server. The request includes the dream ID and user information.

[1419] server

[1420] Based on the request, the specified dream is added to the user's "favorites list." The dream ID and user ID related information are saved in the database. The updated favorites list is notified to the device and reflected to the user.

[1421] Specific example of the process

[1422] 1. User A goes to sleep at night

[1423] User A wears an EEG recording device, which records his / her brain waves while he / she sleeps. The device sends the EEG data to a server via his / her smartphone.

[1424] 2. User A launches the app after waking up

[1425] The server processes the received EEG data and converts the dream content into natural language text using a text generation model.

[1426] 3. The server visualizes the dream text

[1427] The server inputs the generated text into an image generation model to generate images depicting specific scenes from the dream.

[1428] 4. Emotion analysis using an emotion engine

[1429] The server applies an emotion engine to the dream text and visual data to analyze the user's emotional state.

[1430] 5. User A searches for dreams in the app

[1431] User A searches for "dreams about family members" and selects a dream they have had in the past from the list to check the details. At the same time, the results of the emotion analysis are also provided.

[1432] 6. Add your favorite dreams to your favorites

[1433] User A adds the dream to his favorites list so he can re-experience it at any time.

[1434] The above is an embodiment of the present invention.

[1435] The processing flow will be explained below.

[1436] Step 1:

[1437] User:

[1438] Before sleep, participants wear an electroencephalography (EEG) device, which records their brain's electrical activity while they sleep.

[1439] Step 2:

[1440] Device:

[1441] The EEG recording device collects the user's brainwave data in real time and transmits the data to a smartphone via Bluetooth or Wi-Fi.

[1442] Step 3:

[1443] server:

[1444] The device receives brainwave data sent from the smartphone and stores it in a database, thereby recording the brainwave data accumulated while the user is sleeping.

[1445] Step 4:

[1446] server:

[1447] The stored EEG data is preprocessed to remove noise, for example, eye artifacts and ambient electrical interference.

[1448] Step 5:

[1449] server:

[1450] The preprocessed EEG data is fed into a text generation model (such as GPT-4) to convert the dream content into natural language text, which then details the dream content based on the EEG patterns.

[1451] Step 6:

[1452] server:

[1453] Temporarily store the generated natural language text and prepare it for later serving upon user request.

[1454] Step 7:

[1455] server:

[1456] The textual content of a dream is passed to an image generation model (such as DALL-E) to generate specific visual data. For example, an image is generated based on the text "A dragon flying under a blue sky."

[1457] Step 8:

[1458] server:

[1459] The generated visual data is associated with the corresponding text data and stored in a database, so that the dream text and visual data are stored in an integrated manner.

[1460] Step 9:

[1461] server:

[1462] An emotion engine is applied to the generated natural language text and visual data, which analyzes the user's emotions based on the content of the dream and stores the results.

[1463] Step 10:

[1464] User:

[1465] Open the dedicated application and use the interface to search for a specific dream, choosing filter options such as date, theme, characters, or emotional state.

[1466] Step 11:

[1467] Device:

[1468] The search criteria entered by the user are sent to the server, which then searches the database based on the criteria and retrieves the relevant dream data.

[1469] Step 12:

[1470] server:

[1471] The text and visual data of the dream that corresponds to the search result are retrieved and sent to the device, along with the results of the emotion analysis.

[1472] Step 13:

[1473] Device:

[1474] The received search results are then displayed to the user, including the dream text, visual data, and sentiment analysis results.

[1475] Step 14:

[1476] User:

[1477] If you want to add a dream you like to your favorites list, click the "Favorites" button.

[1478] Step 15:

[1479] Device:

[1480] A request to add the dream to the favorites list is sent to the server. The request includes the dream ID and user information.

[1481] Step 16:

[1482] server:

[1483] Based on the request, the specified dream is added to the user's "favorite list." The dream ID and user ID related information are stored in the database.

[1484] Step 17:

[1485] Device:

[1486] Displaying the updated favorites list to the user for easy access.

[1487] The above are the specific processing steps of the program of the present invention.

[1488] Example 2

[1489] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1490] In modern society, many people are unable to sleep soundly due to stress and anxiety, which in turn affects their health and quality of life. It is particularly important to understand that dreams during sleep reflect our mental state and underlying emotions, but memorizing and interpreting these dreams is not easy. Conventional technologies require specialized knowledge and equipment to analyze EEG data acquired during sleep and understand the content of dreams, making them inaccessible to average users. Furthermore, there are few support systems that visualize dream content and perform emotional analysis. This leaves users unable to understand their emotional state and re-experience their dreams.

[1491] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1492] In this invention, the server includes means for processing electroencephalogram data acquired during sleep and removing noise, means for converting the content of dreams based on the electroencephalogram data into natural language text, means for converting the natural language text into visual data, means for analyzing the user's emotions using the natural language text and visual data, means for allowing the user to search the stored data and save it as a favorite, and means for providing the user with a re-experience based on the natural language text and visual data. This allows the user to easily understand the content of their dreams and visually re-experience those dreams, and further allows the user to analyze their emotional state and use it to improve their daily life and mental health.

[1493] "Electroencephalogram data" is the data obtained by measuring the user's brain activity and recording patterns of electrical potential fluctuations.

[1494] The "pre-processing means" refers to a device or algorithm that performs processing to remove noise such as eye artifacts and ambient electrical interference from EEG data.

[1495] "Natural language text" refers to text data expressed in a language format that can be understood by humans.

[1496] "Visual data" refers to image and video data generated based on natural language text.

[1497] "Emotional Analysis Means" means a device or algorithm for analyzing and classifying a user's emotional state using natural language text and visual data reflecting the content of a dream.

[1498] "Search means" refers to a device or algorithm that allows a user to search and retrieve stored data based on specific criteria.

[1499] A "favorites tool" is a device or algorithm that allows users to save their favorite dream data and make it easier to access again.

[1500] "Means for re-experiencing" refers to a means of providing a user with the ability to re-experience a dream they have had in the past, using text and visual data from that dream.

[1501] The present invention relates to a system for converting dream content into natural language text and visual data using EEG data acquired while a user is asleep, and for further emotional analysis. This system is implemented using an EEG recording device, a terminal such as a smartphone or tablet, and a server.

[1502] System configuration

[1503] Terminal

[1504] Before sleep, users wear an EEG recording device, which transmits real-time EEG data via Bluetooth or Wi-Fi to a smartphone or tablet with a dedicated application installed. This application collects EEG data and sends it to a server.

[1505] server

[1506] The server plays a central role in receiving EEG data, preprocessing, converting dream content into text, visualization, emotion analysis, and storing and providing the data. The server performs the following specific processing steps:

[1507] 1. Pretreatment

[1508] The server performs pre-processing on the received EEG data, using Fourier transform and filtering techniques to remove noise, eye artifacts, and ambient electrical interference. The pre-processed data is temporarily stored in an intermediate database.

[1509] 2. Translating the dream into text

[1510] A text generation model (e.g., GPT-4) is used to convert the preprocessed EEG data into natural language text. The model converts dream content into natural language text based on brain activity patterns. The generated text is stored in a database.

[1511] 3. Visualizing the content of your dreams

[1512] The generated text data is input into an image generation model (e.g., DALL-E) to generate detailed visual data. For example, the text "A dragon flying under a blue sky" is converted into a concrete image. The visual data and the corresponding text data are associated and stored in a database.

[1513] 4. Emotion Analysis

[1514] Using the generated natural language text and visual data, the server applies an emotion analysis engine. The engine analyzes the user's emotions based on the content of the dream and classifies the results into categories such as "happiness," "fear," and "surprise." The emotion analysis results are stored in a database along with the text and visual data.

[1515] 5. Data provision and search

[1516] Through a dedicated application, users can use an interface to search for past dreams. They can select filter options such as date, theme, characters, and emotional state, and then select a specific dream from the search results to view details. When users enter search criteria, the device sends the search criteria to the server, which retrieves the corresponding dream data. The search results are sent to the device and displayed to the user in the dedicated application.

[1517] 6. Favorites feature

[1518] The user clicks the "Favorite" button to add a dream they like to their favorite list. The device sends a "Favorite" request to the server, adding the specified dream to the user's "Favorite List." The updated list is saved in the database and notified to the device.

[1519] Specific usage examples

[1520] User A goes to sleep at night

[1521] User A wears the EEG recording device and launches the dedicated smartphone application. The application starts collecting EEG data and transmits the data to the server in real time.

[1522] User A starts the app after waking up

[1523] The server preprocesses the received EEG data to remove noise, and then uses a text generation model to convert the dream content into natural language text.

[1524] The server visualizes the dream text.

[1525] Text data is input into an image generation model to generate specific visual data.

[1526] Emotion analysis using an emotion engine

[1527] The server applies a sentiment analysis engine to the dream text and visual data to generate a result that analyzes the user's sentiment.

[1528] User A searches for dreams on the app

[1529] User A searches for "dreams involving family members" using the filter and selects a past dream from the list. The results are displayed as text, visual, and sentiment analysis results.

[1530] Add your favorite dreams to your favorites

[1531] User A adds a specific dream to his favorites list, and the list is updated.

[1532] Prompt Sentence Examples

[1533] "Please translate last night's dream into words."

[1534] "I'd like to see a visual of a dream where my family appears."

[1535] "Search for past dreams that have scared you."

[1536] This concludes the embodiment of the present invention. Using this system, users can gain a deeper understanding of their dreams, visually re-experience them, and analyze their emotions.

[1537] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1538] Program processing flow

[1539] Step 1: Collecting and transmitting EEG data

[1540] 1. The user wears an EEG recording device before going to sleep.

[1541] 2. The terminal receives real-time EEG data from the EEG recording device via Bluetooth or Wi-Fi.

[1542] 3. A dedicated application installed on the device collects brainwave data.

[1543] 4. The device sends the collected EEG data to the relay server.

[1544] Input: EEG data obtained from an EEG recording device

[1545] Output: Raw EEG data sent to a server

[1546] Step 2: Preprocessing the EEG data

[1547] 1. The server preprocesses the received EEG data.

[1548] 2. The server uses Fourier transform and filtering techniques to remove noise, specifically eye artifacts and ambient electrical interference.

[1549] 3. The server temporarily stores the preprocessed EEG data in an intermediate database.

[1550] Input: Raw EEG data

[1551] Output: Preprocessed EEG data

[1552] Step 3: Translate the dream

[1553] 1. The server inputs the preprocessed EEG data into a text generation model (e.g., GPT-4).

[1554] 2. The server uses the model to convert the dream content from the EEG data into natural language text.

[1555] 3. The server stores the generated natural language text in a database.

[1556] Input: Preprocessed EEG data

[1557] Output: Generated natural language text

[1558] Step 4: Visualize your dream

[1559] 1. The server inputs the generated natural language text into an image generation model (e.g., DALL-E).

[1560] 2. The server reads the text content and generates the corresponding visual data.

[1561] 3. The server stores the visual data and the corresponding text data in a database.

[1562] Input: Natural language text

[1563] Output: The generated visual data

[1564] Step 5: Sentiment Analysis

[1565] 1. The server inputs natural language text and visual data into the sentiment analysis engine.

[1566] 2. The server uses the engine to analyze the user's emotions based on the content of the dream.

[1567] 3. The server stores the sentiment analysis results along with the text and visual data in a database.

[1568] Input: Natural language text and visual data

[1569] Output: Emotion analysis results

[1570] Step 6: Search for and re-experience the dream

[1571] 1. An interface that allows users to search for dreams through a dedicated application.

[1572] 2. The user selects a filter option such as date, theme, character, or emotional state.

[1573] 3. The device sends the search criteria to the server.

[1574] 4. The server retrieves data that matches the search criteria from the database.

[1575] 5. The device displays the text and visual data of the dream to the user.

[1576] Input: User search criteria

[1577] Output: Text, visual, and sentiment analysis data as search results

[1578] Step 7: Favorites function

[1579] 1. The user clicks the "Favorite" button to add the dream to their favorites list.

[1580] 2. The device sends a "favorite" request to the server.

[1581] 3. The server adds the specified dream to the user's "favorite list."

[1582] 4. The server saves the updated favorites list in the database and notifies the device.

[1583] Input: "Favorite" request (yume ID and user information)

[1584] Output: Updated favorites list

[1585] Example prompt

[1586] "Please translate last night's dream into words."

[1587] "I'd like to see a visual of a dream where my family appears."

[1588] "Search for past dreams that have scared you."

[1589] (Application example 2)

[1590] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1591] It is known that the sleep quality and mental state of factory workers have a significant impact on the next day's work efficiency and the risk of making mistakes. However, current technology lacks the means to specifically understand and analyze the sleep patterns and dream content of operators and to predict and manage work efficiency and risks based on this information. Therefore, there is a need for methods to improve factory productivity and safety.

[1592] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting biosignal data acquired during sleep, means for processing the biosignal data and converting the content of dreams into natural language text, means for converting the natural language text into visual data, means for saving the natural language text and the visual data and making them accessible to users, means for allowing users to search and save the saved data as preferences, and means for analyzing the emotional state of the user and predicting and managing work efficiency and the risk of errors. This makes it possible to analyze the sleep and mental states of operators in detail and improve work efficiency and safety in factories.

[1593] "Biological signal data" refers to data that indicates the user's physiological state, such as brain waves, heart rate, and respiratory rate.

[1594] "Natural language text" is text data expressed in a language format commonly used by humans.

[1595] "Visual data" is data that provides information visually, such as images or videos.

[1596] The term "user" refers to a person who uses the system of the present invention, that is, an operator or a user.

[1597] "Emotional state" refers to information that indicates the user's emotions, such as happiness, fear, surprise, etc.

[1598] "Work efficiency" is an index that indicates the progress and productivity of work within a certain period of time.

[1599] "Risk of mistakes occurring" is an indicator that indicates the possibility of mistakes or accidents occurring during work.

[1600] "Noise" is unwanted data or interference mixed into the signal of interest.

[1601] "Preprocessing" is a series of processing operations that shape, clean, and transform raw data prior to primary data processing.

[1602] A "server" is a computer system that receives, stores, processes, and distributes data.

[1603] This invention relates to a system that analyzes biosignal data acquired while an operator is sleeping, converts the data into natural language text and visual data, and further analyzes the operator's emotional state to predict and manage work efficiency and the risk of errors.

[1604] System configuration

[1605] Terminal

[1606] It includes a biosignal recording device used by the operator and a terminal such as a smartphone or tablet, on which an application is installed to collect biosignal data and send it to a server.

[1607] server

[1608] The server receives biosignal data and plays a central role in preprocessing, dream text conversion, visualization, emotion analysis, and data storage and provision.

[1609] Program processing and hardware / software details

[1610] 1. Data Collection and Preprocessing

[1611] Terminal

[1612] Before going to sleep, the operator wears a biosignal recording device (e.g., an electroencephalograph), which collects biosignal data in real time and transmits it to a smartphone via Bluetooth or Wi-Fi.

[1613] server

[1614] The system receives biosignal data sent from the smartphone and stores it in a database. This records the biosignal data accumulated while the operator is sleeping. As a pre-processing step, noise is removed using a Butterworth filter or similar.

[1615] 2. Translating the dream into text

[1616] server

[1617] The preprocessed biosignal data is fed into a text generation model (e.g., GPT-4) to convert the dream content into natural language text, which is temporarily stored and made available upon user request.

[1618] 3. Dream visualization

[1619] server

[1620] The textual content of a dream is input into an image generation model (e.g., DALL-E) to generate detailed visual data. For example, the text "A dragon flying under a blue sky" is converted into a concrete image. The generated visual data and the corresponding text data are associated and stored in a database.

[1621] 4. Emotion analysis using an emotion engine

[1622] server

[1623] An emotion engine is applied to the generated natural language text and visual data. The emotion engine analyzes the operator's emotions based on the content of the dream and saves the results. For example, it determines that the operator is likely to have felt emotions such as "happiness," "fear," or "surprise" based on the content of the dream.

[1624] 5. Leveraging Data

[1625] server

[1626] The system integrates the sleep quality and emotional state of operators to analyze work efficiency and the risk of mistakes, and provides the prediction results to robots in the factory, which then make it possible to adjust the day's work and mitigate risks.

[1627] Specific example of the process

[1628] 1. Operators fall asleep at night

[1629] The operator wears a biosignal recording device, which records biosignals while the patient sleeps. The device then transmits the biosignal data to a server via a smartphone.

[1630] 2. The operator launches the app after waking up

[1631] The server processes the received biosignal data and converts the dream content into natural language text using a text generation model.

[1632] 3. The server visualizes the dream text.

[1633] The server inputs the generated text into an image generation model to generate images depicting specific scenes from the dream.

[1634] 4. Emotion analysis using an emotion engine

[1635] The server applies an emotion engine to the dream text and visual data to analyze the operator's emotional state.

[1636] 5. The operator checks your dreams using the app

[1637] The operator searches for "dreams involving family members," selects a dream from a list of past dreams, and checks the details. At the same time, the operator is also provided with the results of an emotional analysis.

[1638] 6. Utilizing the results

[1639] Based on the data collected and analyzed by the server, it predicts work efficiency and the risk of errors, and provides information to factory robots.

[1640] Examples of prompt statements

[1641] "In a dream, the operator saw a dragon flying under a blue sky."

[1642] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1643] Step 1:

[1644] Terminal

[1645] Before sleep, the operator wears a biosignal recording device, which collects biosignal data such as brain waves and heart rate in real time and transmits the data to a smartphone via Bluetooth or Wi-Fi, thereby obtaining biosignal data as input.

[1646] Step 2:

[1647] server

[1648] The system receives biosignal data sent from a smartphone and stores it in a database. The received data is raw and contains noise, and this becomes the input data. Next, preprocessing is performed to remove noise using a Butterworth filter or similar, and clean biosignal data is output.

[1649] Step 3:

[1650] server

[1651] The preprocessed biosignal data is input into a text generation model (e.g., GPT-4). The model analyzes the input data and converts the dream content into natural language text. The output text data is in the form of, for example, "A dragon flying under a blue sky."

[1652] Step 4:

[1653] server

[1654] The generated natural language text is input into an image generation model (e.g., DALL-E). The model generates detailed visual data (images) based on the text data. For example, the text "A dragon flying under a blue sky" is converted into a concrete image. The output image is stored in a database.

[1655] Step 5:

[1656] server

[1657] An emotion engine is applied to the generated natural language text and visual data. The emotion engine analyzes the input text and image data and outputs the operator's emotional state as "happiness," "fear," "surprise," etc. The analysis results are stored in a database.

[1658] Step 6:

[1659] server

[1660] The system analyzes the operator's work efficiency and risk of mistakes by integrating their sleep quality and emotional state. The server refers to statistical data and past records and performs data calculations to predict risks. The output prediction results are provided to the factory robots.

[1661] Step 7:

[1662] User

[1663] After waking up, the operator can check the content of their dream through a dedicated application. The saved data can be searched and viewed through the app. The user can select "dreams featuring family members" to check the text and images, and the emotional analysis results are also displayed.

[1664] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1665] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1666] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1667] [Fourth embodiment]

[1668] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1669] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1670] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1671] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1672] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1673] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1674] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1675] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1676] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1677] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1678] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1679] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1680] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1681] This invention relates to a system that collects a user's brainwave data while they sleep and converts the contents of their dreams into text and visualizations. Through this system, users can search, save, and re-experience their own dreams. The specific program processing required to implement this system is described below.

[1682] System configuration

[1683] Terminal

[1684] The terminal is a smartphone or tablet that connects to the EEG recording device used by the user, and an application is installed on the terminal to collect EEG data and send it to a server.

[1685] server

[1686] The server plays a central role in receiving EEG data, preprocessing, converting dreams into text, visualizing them, and storing and providing the data.

[1687] Program processing

[1688] 1. Dream Data Collection

[1689] Device:

[1690] Before sleep, users wear an EEG recording device, which collects real-time brainwave data and transmits it to a smartphone via Bluetooth or Wi-Fi.

[1691] server:

[1692] The server receives the brainwave data sent from the smartphone and stores it in a database.

[1693] 2. Translating the dream into text

[1694] server:

[1695] The collected EEG data is processed to remove unwanted noise. The preprocessed data is then fed into a text generation model (e.g., GPT-4) to convert the dream content into natural language text. The generated text is temporarily stored and provided to the user upon request.

[1696] 3. Visualize your dreams

[1697] server:

[1698] The textual content of dreams is input into an image generation model (e.g., DALL-E) and converted into concrete visual data. The generated visual data and the corresponding text data are associated and stored in a database.

[1699] 4. Searching for and re-experiencing dreams

[1700] User:

[1701] Through the dedicated application, you can use the interface to search for past dreams, choose filter options such as date, theme, or characters, and then select a specific dream from the search results to view its details.

[1702] Device:

[1703] The server sends the user's search criteria to retrieve the corresponding dream data, and displays the retrieved dream text and visual data to the user.

[1704] 5. Favorites feature

[1705] User:

[1706] Click the button to save the dream you like as a favorite.

[1707] Device:

[1708] Sends requests to the server regarding the registration and management of favorites.

[1709] server:

[1710] Based on the user's request, specific dreams are added to the user's "favorite list" and saved and updated in the database.

[1711] Specific example of the process

[1712] 1. User A goes to sleep at night:

[1713] User A wears an EEG recording device, which records his / her brain waves while he / she sleeps. The device sends the EEG data to a server via his / her smartphone.

[1714] 2. User A wakes up and launches the app:

[1715] The server processes the received EEG data and converts the dream content into natural language text using a text generation model.

[1716] 3. The server visualizes the dream text:

[1717] The server inputs the generated text into an image generation model to generate images depicting specific scenes from the dream.

[1718] 4. User A searches for dreams in the app:

[1719] User A searches for "dreams featuring family" and selects a dream they have had in the past from the list to check the details.

[1720] 5. Add your favorite dreams to your favorites:

[1721] User A saves the dream in a favorites list so that he can re-experience it at any time.

[1722] The above is an embodiment of the present invention.

[1723] The processing flow will be explained below.

[1724] Step 1:

[1725] User:

[1726] Before sleep, participants wear an electroencephalography device, which is used to record the brain's electrical activity while they sleep.

[1727] Step 2:

[1728] Device:

[1729] The EEG recording device begins collecting the user's brainwave data, which is then transmitted in real time to a smartphone via Bluetooth or Wi-Fi.

[1730] Step 3:

[1731] server:

[1732] The device receives brainwave data sent from the smartphone and stores it in a database, thereby recording the brainwave data accumulated while the user is sleeping.

[1733] Step 4:

[1734] server:

[1735] Preprocess the stored EEG data to remove noise, e.g., eye artifacts and ambient electrical interference.

[1736] Step 5:

[1737] server:

[1738] The preprocessed EEG data is fed into a text generation model (such as GPT-4) to convert the dream content into natural language text, which then describes the dream content specifically based on the EEG patterns.

[1739] Step 6:

[1740] server:

[1741] Temporarily store the generated text and prepare it for later delivery upon user request.

[1742] Step 7:

[1743] server:

[1744] The textual content of the dream is passed to an image generation model (such as DALL-E) to generate detailed visual data. For example, the text "A dragon flying under a blue sky" is converted into a concrete image.

[1745] Step 8:

[1746] server:

[1747] The generated visual data is associated with the corresponding text data and stored in a database, so that the dream text and visual data are stored in an integrated manner.

[1748] Step 9:

[1749] User:

[1750] Open the app and search for a specific dream, for example, "a dream featuring family members" or "a dream on a specific date."

[1751] Step 10:

[1752] Device:

[1753] The search criteria entered by the user are sent to the server, which then searches the database based on the criteria and retrieves the relevant dream data.

[1754] Step 11:

[1755] server:

[1756] The text and visual data of the relevant dreams are retrieved from the search results and sent to the device, which then selects dreams that match the search criteria.

[1757] Step 12:

[1758] Device:

[1759] The received search results are displayed to the user, who can then view and check the details.

[1760] Step 13:

[1761] User:

[1762] If you want to add a dream you like to your favorites list, click the "Favorites" button.

[1763] Step 14:

[1764] Device:

[1765] A request to add the dream to the favorites list is sent to the server. The request includes the dream ID and user information.

[1766] Step 15:

[1767] server:

[1768] Based on the request, the specified dream is added to the user's "favorite list." The dream ID and user ID related information are stored in the database.

[1769] Step 16:

[1770] Device:

[1771] Displaying the updated favorites list to the user for easy access.

[1772] The above are the specific processing steps of the program of the present invention.

[1773] Example 1

[1774] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1775] The challenge is to efficiently collect EEG data acquired during sleep, and then remove noise, convert it into natural language text, and visualize it so that users can clearly understand the content of their dreams. Furthermore, a system is needed that allows users to easily search the generated dream data, save it as their favorite dream, and re-experience it.

[1776] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1777] In this invention, the server includes means for collecting EEG data acquired during sleep, preprocessing means for removing noise from EEG signals, and generative model means for converting the preprocessed EEG data into natural language text. This allows the user to accurately understand the dreams they had during sleep and visually understand the content of their dreams as text and visual data. The server also includes database means for storing the natural language text and visual data and making them accessible to users, and interface means for allowing users to search and save the stored data as favorites. This allows users to easily search their dreams and re-experience their favorite dreams.

[1778] "EEG data" refers to digital signals that record brain wave activity during sleep.

[1779] "Preprocessing" refers to processing to remove noise from collected EEG data and extract pure EEG patterns.

[1780] A "generative model" is an artificial intelligence model that generates natural language text or visual data based on collected data. Specifically, it refers to a text generation model that generates natural language text and an image generation model that generates images based on text.

[1781] "Database" means a storage device that stores and manages collected and generated data and makes it accessible to users.

[1782] An "interface" refers to the operation screen or input device that allows the user to directly interact with the system.

[1783] "Search" is a feature that allows users to search for saved dream data using specific keywords or conditions.

[1784] "Favorites" is a feature that allows users to save their favorite dream data in a list for easy access later.

[1785] "Re-experience" is a function that allows users to revisit and experience the content of their dreams using text and visual data from previously generated dreams.

[1786] "Server" refers to the computer system that processes EEG data, runs the generative model, stores and retrieves data, and provides the user interface.

[1787] "Terminal" refers to the device used by the user to collect EEG data, specifically the EEG recording device and the smartphone or tablet connected to it.

[1788] This invention is a system that collects the brainwave data of users while they sleep, converts the contents of their dreams into text in natural language, and visualizes them. Through this system, users can search, save, and re-experience their own dreams.

[1789] System configuration

[1790] Device:

[1791] The terminal is a smartphone or tablet that connects to the EEG recording device used by the user. An application is installed on the terminal to collect EEG data and send it to a server. The EEG recording device transfers the data to the smartphone via Bluetooth or Wi-Fi.

[1792] server:

[1793] The server plays a central role in receiving EEG data, preprocessing, converting dreams into text, visualizing them, and storing and providing the data. Specifically, the server performs the following functions:

[1794] 1. Receiving and storing data:

[1795] The server receives the brainwave data sent from the smartphone, checks the integrity of the data, and then stores it in a database.

[1796] 2. Pretreatment:

[1797] Remove noise from the received EEG data and extract pure EEG patterns by filtering high and low frequencies.

[1798] 3. Text Generation:

[1799] The preprocessed EEG data is fed into a text generation model (e.g., GPT-4), which analyzes the EEG patterns and generates natural language text based on them.

[1800] 4. Visual Generation:

[1801] The generated text is input into an image generation model (e.g., DALL-E) to generate visual data depicting a specific scene.

[1802] 5. Data storage and provision:

[1803] The generated natural language text and visual data are associated and stored in a database, and are made available to users upon request.

[1804] 6. Retrieve and re-experience:

[1805] Allow users to search for dreams by specific criteria. For example, users can search for dreams using filters such as date, theme, characters, etc. From the search results, users can select a specific dream to view details and relive it.

[1806] 7. Favorites feature:

[1807] Users can save their favorite dreams in a "favorites list" for easy access later.

[1808] Specific examples

[1809] 1. User A goes to sleep at night:

[1810] The user wears an EEG recording device, which collects real-time brainwave data, which is then transmitted via Bluetooth to a smartphone and then sent to a server.

[1811] 2. User A wakes up and launches the app:

[1812] The server processes the received EEG data, removes noise, and converts the dream content into natural language text using a text generation model.

[1813] 3. The server visualizes the dream text:

[1814] The server inputs the generated text into an image generation model to generate images depicting specific scenes from the dream.

[1815] 4. User A searches for dreams in the app:

[1816] User A searches for "dreams featuring family" and selects a dream they have had in the past from the list to check the details.

[1817] 5. Add your favorite dreams to your favorites:

[1818] User A can save the dream to a favorites list and re-experience it at any time.

[1819] Prompt Sentence Examples

[1820] For example, the prompt for transcribing a dream might look like this:

[1821] Based on the user's brainwave data, please explain the content of this dream in natural language.

[1822] Also, the prompt for visualizing the dream is as follows:

[1823] Based on the text below, please create an image that depicts a specific scene.

[1824] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1825] Step 1:

[1826] EEG data collection

[1827] Device: The user wears an EEG recording device, which collects real-time EEG data during sleep and transmits it to a smartphone via Bluetooth or Wi-Fi.

[1828] Input: User's brainwave data

[1829] Output: EEG data sent to a smartphone

[1830] How it works: The EEG recording device captures EEG signals and converts them into digital signals. The smartphone application receives the data.

[1831] Step 2:

[1832] Receiving EEG data

[1833] Server: Receives EEG data sent from the smartphone. The server checks the integrity of the data packets and stores them accurately in a database.

[1834] Input: EEG data sent from a smartphone

[1835] Output: EEG data stored on the server

[1836] Specific operation: The server receives the data, performs error checking, and stores it in the database.

[1837] Step 3:

[1838] Pre-processing for noise removal

[1839] Server: Removes unnecessary noise from the received EEG data, filtering high and low frequencies to extract pure EEG patterns.

[1840] Input: Received EEG data

[1841] Output: Preprocessed EEG data with noise removed

[1842] Specific operations: Runs a filtering algorithm to reduce noise. Sends preprocessed data to the next processing step.

[1843] Step 4:

[1844] Text Generation

[1845] Server: The preprocessed EEG data is fed into a text generation model (e.g., GPT-4), which analyzes the EEG patterns and converts the dream content into natural language text.

[1846] Input: Preprocessed EEG data

[1847] Output: Generated natural language text

[1848] Specific operation: Calls the GPT-4 model, generates text based on EEG patterns, and temporarily stores the generated text.

[1849] Step 5:

[1850] Visual Generation

[1851] Server: The generated text is input into an image generation model (e.g., DALL-E) to generate visual data depicting a specific scene.

[1852] Input: Generated natural language text

[1853] Output: The generated visual data

[1854] Specific operation: Calls the DALL-E model, generates an image based on the text, and saves the generated image in association with the text.

[1855] Step 6:

[1856] Data storage and provision

[1857] Server: Stores the generated text and visual data in a database in an associated manner and provides this data when requested by the user.

[1858] Input: Generated natural language text and visual data

[1859] Output: Textual and visual data stored in a database

[1860] Specific operation: Stores data in a database and retrieves and provides data upon user request.

[1861] Step 7:

[1862] Retrieving and re-experiencing dreams

[1863] Users: Use the application to use the interface to search for past dreams, using filters such as date, theme, and characters, and then select a specific dream to view more details.

[1864] Input: User's search query (e.g. date, subject)

[1865] Output: Text and visual data of the dream displayed as search results

[1866] Specific operation: A query is sent to the server based on the conditions entered by the user, and the server searches for data that matches the conditions and provides it to the terminal.

[1867] Step 8:

[1868] Favorites feature

[1869] User: Click the button to save the dream you like as a favorite.

[1870] Input: User-selected dream data

[1871] Output: Dream data added to the user's "favorite list"

[1872] Specific operation: The click operation is detected and the data of the selected dream is sent to the server. The server updates the database and adds the corresponding dream to the user's "favorite list."

[1873] (Application example 1)

[1874] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1875] Conventional systems that analyze EEG data during sleep only allow users to check the content of their dreams, but to further improve users' sleep experiences, there was a need for a function that could suggest relaxation products and services based on the obtained dream data.Furthermore, there was no system that could provide users with a personalized relaxation experience based on such data.

[1876] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1877] In this invention, the server includes means for collecting EEG data acquired during sleep, means for processing the EEG data and converting the content of dreams into natural language text, and means for converting the natural language text into visual data. This not only enables users to check the content of their dreams in detail as text and visual data, but also makes it possible to suggest relaxation products and services tailored to individual needs based on the data.

[1878] "Sleep" means the time while the user is resting.

[1879] "Electroencephalogram Data" means data regarding the electrical activity of a user's brain obtained using an electroencephalogram recording device.

[1880] "Natural language text" is written information expressed in a natural language in a form that can be understood by a user.

[1881] "Visual data" is data in a format that can be visually recognized, such as images or videos.

[1882] "Storage" means the act of retaining data for an extended period of time and making it readily accessible when needed.

[1883] "Search" refers to the act of a user examining data in a database to find specific information.

[1884] "Favorites" refers to a feature that allows users to easily access their favorite data again.

[1885] "Relaxation products" are any products that users use for the purpose of relaxation.

[1886] "Services" refers to various relaxation experiences and functions provided to Users.

[1887] The present invention relates to a system that collects electroencephalogram data of a user while they are sleeping, converts the content of their dreams into text and visualizes them based on the data, and suggests relaxation products and services. An embodiment of the system will be described in detail below.

[1888] System configuration

[1889] This system consists of the following main elements:

[1890] 1. EEG recording devices and terminals

[1891] Before sleep, users wear an EEG recording device, which collects real-time brainwave data while they sleep and transmits the data to a smartphone or tablet via Bluetooth or Wi-Fi.

[1892] 2. Server

[1893] The server is the central unit that receives the EEG data and performs the following processes.

[1894] 3. Preprocessing of EEG data

[1895] The server receives the transmitted EEG data and performs preprocessing to remove noise, using Python and libraries such as NumPy and Pandas.

[1896] 4. Texting dreams

[1897] The preprocessed EEG data is then fed into a text generation model (e.g., GPT-4) to convert the dream content into natural language text.

[1898] 5. Visualize your dreams

[1899] The generated natural language text is input into an image generation model (e.g., DALL-E), and the dream content is embodied as visual data.

[1900] 6. Data storage and access

[1901] The server saves the generated natural language text and visual data and stores it in a database for user access.

[1902] 7. Data search and favorites

[1903] Users can search for past dreams through a dedicated application, using filter options such as date, theme, and characters, and can save specific dreams as favorites.

[1904] 8. Proposal of relaxation products and services

[1905] The server then uses the stored data to suggest relaxation products and services, a process that involves algorithms that analyze text and visual data to recommend products and services tailored to the user.

[1906] Specific examples

[1907] In a specific example of implementing this system, User A wears an EEG recording device at night, and EEG data is recorded while he or she sleeps. When User A launches the app the next morning, the server processes the received EEG data and converts the dream content into natural language using a text generation model. The generated text is then input into a visualization model to visualize specific scenes from the dream. Furthermore, based on this data, relaxation products (e.g., aroma oils and massage chairs) and services (e.g., massage sessions and yoga classes) are suggested.

[1908] Prompt Sentence Examples

[1909] An example of a prompt for the generative AI model is, "Please describe in text the content of the dream generated based on the user's brainwave data."

[1910] By implementing the present invention in this manner, the user can not only understand the content of their dreams in detail, but also enjoy a personalized relaxation experience.

[1911] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1912] Step 1:

[1913] EEG data collection

[1914] The user wears an EEG recording device before going to sleep. The EEG recording device collects EEG data in real time while the user sleeps and transmits the data to a device (smartphone or tablet) via Bluetooth or Wi-Fi. The input is the raw EEG data collected by the EEG recording device, and the output is the data transmitted to the device.

[1915] Step 2:

[1916] EEG data transfer

[1917] The terminal transfers the received EEG data to the server. The input here is the EEG data received by the terminal, and the output is that data is sent to the server.

[1918] Step 3:

[1919] Preprocessing of EEG data

[1920] The server preprocesses the received EEG data and removes noise. Specifically, it uses Python's NumPy and Pandas libraries to filter the data and remove unnecessary noise. The input is the raw EEG data received by the server, and the output is the preprocessed, clean EEG data.

[1921] Step 4:

[1922] Dream text

[1923] The server inputs the preprocessed EEG data into a text generation model (e.g., GPT-4) and converts the dream content into natural language text. The input is the preprocessed EEG data, and the output is the generated natural language text. The prompt sentence used is, "Please describe in text the dream content generated based on the user's EEG data."

[1924] Step 5:

[1925] Dream visualization

[1926] The server inputs the generated natural language text into an image generation model (e.g., DALL-E) to convert the dream content into visual data. The input is the generated natural language text, and the output is the generated visual data.

[1927] Step 6:

[1928] Data storage

[1929] The server stores the generated natural language text and visual data in a database. The input is the generated natural language text and visual data, and the output is that the data is stored in the database.

[1930] Step 7:

[1931] Data search and favorites

[1932] Users can search for past dreams through a dedicated application, using filter options such as date, theme, and characters to search and save specific dreams as favorites. The input is the search criteria specified by the user, and the output is the natural language text and visual data results that match the criteria.

[1933] Step 8:

[1934] Proposal of relaxation products and services

[1935] The server proposes relaxation products and services based on the stored data. The input is the stored natural language text and visual data, and the output is the proposed information for relaxation products and services based on the analysis results.

[1936] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1937] This invention relates to a system that converts the content of a user's dreams into natural language text and visual data using electroencephalogram data acquired during sleep, and further combines it with an emotion engine to analyze and provide the user's emotions. Specific program processing for implementing this system is described below.

[1938] System configuration

[1939] Terminal

[1940] The device is an EEG recording device used by the user, and a terminal such as a smartphone or tablet. An application is installed on the terminal to collect EEG data and send it to a server.

[1941] server

[1942] The server receives the EEG data and plays a central role in preprocessing, dream textualization, visualization, emotion analysis, and data storage and provision.

[1943] Program processing

[1944] 1. Dream Data Collection

[1945] Terminal

[1946] Before sleep, users wear an EEG recording device, which collects real-time brainwave data and transmits it to a smartphone via Bluetooth or Wi-Fi.

[1947] server

[1948] The device receives brainwave data sent from the smartphone and stores it in a database, thereby recording the brainwave data accumulated while the user is sleeping.

[1949] 2. Translating the dream into text

[1950] server

[1951] The stored EEG data is preprocessed to remove noise, such as eye artifacts and ambient electrical interference. The preprocessed EEG data is then fed into a text generation model (e.g., GPT-4) to convert the dream content into natural language text. The generated text is temporarily stored and made available to the user upon request.

[1952] 3. Visualize your dreams

[1953] server

[1954] The textual content of a dream is input into an image generation model (such as DALL-E) to generate detailed visual data. For example, the text "A dragon flying under a blue sky" is converted into a concrete image. The generated visual data and the corresponding text data are associated and stored in a database.

[1955] 4. Emotion analysis using an emotion engine

[1956] server

[1957] An emotion engine is applied to the generated natural language text and visual data. The emotion engine analyzes the user's emotions based on the content of the dream and saves the results. For example, it determines that the user is likely to have felt emotions such as "happiness," "fear," or "surprise" based on the content of the dream.

[1958] 5. Searching for and Re-experiencing Dreams

[1959] User

[1960] Through the dedicated application, you use the interface to search for past dreams, choose filter options such as date, theme, characters, or emotional state, and then select a specific dream from the search results to view its details.

[1961] Terminal

[1962] The system sends the user's search criteria to the server and retrieves the corresponding dream data. The retrieved dream text and visual data are then displayed to the user. Sentiment analysis results are also provided.

[1963] 6. Favorites feature

[1964] User

[1965] If you want to add a dream you like to your favorites list, click the "Favorites" button.

[1966] Terminal

[1967] A request to add the dream to the favorites list is sent to the server. The request includes the dream ID and user information.

[1968] server

[1969] Based on the request, the specified dream is added to the user's "favorites list." The dream ID and user ID related information are saved in the database. The updated favorites list is notified to the device and reflected to the user.

[1970] Specific example of the process

[1971] 1. User A goes to sleep at night

[1972] User A wears an EEG recording device, which records his / her brain waves while he / she sleeps. The device sends the EEG data to a server via his / her smartphone.

[1973] 2. User A launches the app after waking up

[1974] The server processes the received EEG data and converts the dream content into natural language text using a text generation model.

[1975] 3. The server visualizes the dream text

[1976] The server inputs the generated text into an image generation model to generate images depicting specific scenes from the dream.

[1977] 4. Emotion analysis using an emotion engine

[1978] The server applies an emotion engine to the dream text and visual data to analyze the user's emotional state.

[1979] 5. User A searches for dreams in the app

[1980] User A searches for "dreams about family members" and selects a dream they have had in the past from the list to check the details. At the same time, the results of the emotion analysis are also provided.

[1981] 6. Add your favorite dreams to your favorites

[1982] User A adds the dream to his favorites list so he can re-experience it at any time.

[1983] The above is an embodiment of the present invention.

[1984] The processing flow will be explained below.

[1985] Step 1:

[1986] User:

[1987] Before sleep, participants wear an electroencephalography (EEG) device, which records their brain's electrical activity while they sleep.

[1988] Step 2:

[1989] Device:

[1990] The EEG recording device collects the user's brainwave data in real time and transmits the data to a smartphone via Bluetooth or Wi-Fi.

[1991] Step 3:

[1992] server:

[1993] The device receives brainwave data sent from the smartphone and stores it in a database, thereby recording the brainwave data accumulated while the user is sleeping.

[1994] Step 4:

[1995] server:

[1996] The stored EEG data is preprocessed to remove noise, for example, eye artifacts and ambient electrical interference.

[1997] Step 5:

[1998] server:

[1999] The preprocessed EEG data is fed into a text generation model (such as GPT-4) to convert the dream content into natural language text, which then details the dream content based on the EEG patterns.

[2000] Step 6:

[2001] server:

[2002] Temporarily store the generated natural language text and prepare it for later serving upon user request.

[2003] Step 7:

[2004] server:

[2005] The textual content of a dream is passed to an image generation model (such as DALL-E) to generate specific visual data. For example, an image is generated based on the text "A dragon flying under a blue sky."

[2006] Step 8:

[2007] server:

[2008] The generated visual data is associated with the corresponding text data and stored in a database, so that the dream text and visual data are stored in an integrated manner.

[2009] Step 9:

[2010] server:

[2011] An emotion engine is applied to the generated natural language text and visual data, which analyzes the user's emotions based on the content of the dream and stores the results.

[2012] Step 10:

[2013] User:

[2014] Open the dedicated application and use the interface to search for a specific dream, choosing filter options such as date, theme, characters, or emotional state.

[2015] Step 11:

[2016] Device:

[2017] The search criteria entered by the user are sent to the server, which then searches the database based on the criteria and retrieves the relevant dream data.

[2018] Step 12:

[2019] server:

[2020] The text and visual data of the dream that corresponds to the search result are retrieved and sent to the device, along with the results of the emotion analysis.

[2021] Step 13:

[2022] Device:

[2023] The received search results are then displayed to the user, including the dream text, visual data, and sentiment analysis results.

[2024] Step 14:

[2025] User:

[2026] If you want to add a dream you like to your favorites list, click the "Favorites" button.

[2027] Step 15:

[2028] Device:

[2029] A request to add the dream to the favorites list is sent to the server. The request includes the dream ID and user information.

[2030] Step 16:

[2031] server:

[2032] Based on the request, the specified dream is added to the user's "favorite list." The dream ID and user ID related information are stored in the database.

[2033] Step 17:

[2034] Device:

[2035] Displaying the updated favorites list to the user for easy access.

[2036] The above are the specific processing steps of the program of the present invention.

[2037] Example 2

[2038] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2039] In modern society, many people are unable to sleep soundly due to stress and anxiety, which in turn affects their health and quality of life. It is particularly important to understand that dreams during sleep reflect our mental state and underlying emotions, but memorizing and interpreting these dreams is not easy. Conventional technologies require specialized knowledge and equipment to analyze EEG data acquired during sleep and understand the content of dreams, making them inaccessible to average users. Furthermore, there are few support systems that visualize dream content and perform emotional analysis. This leaves users unable to understand their emotional state and re-experience their dreams.

[2040] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[2041] In this invention, the server includes means for processing electroencephalogram data acquired during sleep and removing noise, means for converting the content of dreams based on the electroencephalogram data into natural language text, means for converting the natural language text into visual data, means for analyzing the user's emotions using the natural language text and visual data, means for allowing the user to search the stored data and save it as a favorite, and means for providing the user with a re-experience based on the natural language text and visual data. This allows the user to easily understand the content of their dreams and visually re-experience those dreams, and further allows the user to analyze their emotional state and use it to improve their daily life and mental health.

[2042] "Electroencephalogram data" is the data obtained by measuring the user's brain activity and recording patterns of electrical potential fluctuations.

[2043] The "pre-processing means" refers to a device or algorithm that performs processing to remove noise such as eye artifacts and ambient electrical interference from EEG data.

[2044] "Natural language text" refers to text data expressed in a language format that can be understood by humans.

[2045] "Visual data" refers to image and video data generated based on natural language text.

[2046] "Emotional Analysis Means" means a device or algorithm for analyzing and classifying a user's emotional state using natural language text and visual data reflecting the content of a dream.

[2047] "Search means" refers to a device or algorithm that allows a user to search and retrieve stored data based on specific criteria.

[2048] A "favorites tool" is a device or algorithm that allows users to save their favorite dream data and make it easier to access again.

[2049] "Means for re-experiencing" refers to a means of providing a user with the ability to re-experience a dream they have had in the past, using text and visual data from that dream.

[2050] The present invention relates to a system for converting dream content into natural language text and visual data using EEG data acquired while a user is asleep, and for further emotional analysis. This system is implemented using an EEG recording device, a terminal such as a smartphone or tablet, and a server.

[2051] System configuration

[2052] Terminal

[2053] Before sleep, users wear an EEG recording device, which transmits real-time EEG data via Bluetooth or Wi-Fi to a smartphone or tablet with a dedicated application installed. This application collects EEG data and sends it to a server.

[2054] server

[2055] The server plays a central role in receiving EEG data, preprocessing, converting dream content into text, visualization, emotion analysis, and storing and providing the data. The server performs the following specific processing steps:

[2056] 1. Pretreatment

[2057] The server performs pre-processing on the received EEG data, using Fourier transform and filtering techniques to remove noise, eye artifacts, and ambient electrical interference. The pre-processed data is temporarily stored in an intermediate database.

[2058] 2. Translating the dream into text

[2059] A text generation model (e.g., GPT-4) is used to convert the preprocessed EEG data into natural language text. The model converts dream content into natural language text based on brain activity patterns. The generated text is stored in a database.

[2060] 3. Visualizing the content of your dreams

[2061] The generated text data is input into an image generation model (e.g., DALL-E) to generate detailed visual data. For example, the text "A dragon flying under a blue sky" is converted into a concrete image. The visual data and the corresponding text data are associated and stored in a database.

[2062] 4. Emotion Analysis

[2063] Using the generated natural language text and visual data, the server applies an emotion analysis engine. The engine analyzes the user's emotions based on the content of the dream and classifies the results into categories such as "happiness," "fear," and "surprise." The emotion analysis results are stored in a database along with the text and visual data.

[2064] 5. Data provision and search

[2065] Through a dedicated application, users can use an interface to search for past dreams. They can select filter options such as date, theme, characters, and emotional state, and then select a specific dream from the search results to view details. When users enter search criteria, the device sends the search criteria to the server, which retrieves the corresponding dream data. The search results are sent to the device and displayed to the user in the dedicated application.

[2066] 6. Favorites feature

[2067] The user clicks the "Favorite" button to add a dream they like to their favorite list. The device sends a "Favorite" request to the server, adding the specified dream to the user's "Favorite List." The updated list is saved in the database and notified to the device.

[2068] Specific usage examples

[2069] User A goes to sleep at night

[2070] User A wears the EEG recording device and launches the dedicated smartphone application. The application starts collecting EEG data and transmits the data to the server in real time.

[2071] User A starts the app after waking up

[2072] The server preprocesses the received EEG data to remove noise, and then uses a text generation model to convert the dream content into natural language text.

[2073] The server visualizes the dream text.

[2074] Text data is input into an image generation model to generate specific visual data.

[2075] Emotion analysis using an emotion engine

[2076] The server applies a sentiment analysis engine to the dream text and visual data to generate a result that analyzes the user's sentiment.

[2077] User A searches for dreams on the app

[2078] User A searches for "dreams involving family members" using the filter and selects a past dream from the list. The results are displayed as text, visual, and sentiment analysis results.

[2079] Add your favorite dreams to your favorites

[2080] User A adds a specific dream to his favorites list, and the list is updated.

[2081] Prompt Sentence Examples

[2082] "Please translate last night's dream into words."

[2083] "I'd like to see a visual of a dream where my family appears."

[2084] "Search for past dreams that have scared you."

[2085] This concludes the embodiment of the present invention. Using this system, users can gain a deeper understanding of their dreams, visually re-experience them, and analyze their emotions.

[2086] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2087] Program processing flow

[2088] Step 1: Collecting and transmitting EEG data

[2089] 1. The user wears an EEG recording device before going to sleep.

[2090] 2. The terminal receives real-time EEG data from the EEG recording device via Bluetooth or Wi-Fi.

[2091] 3. A dedicated application installed on the device collects brainwave data.

[2092] 4. The device sends the collected EEG data to the relay server.

[2093] Input: EEG data obtained from an EEG recording device

[2094] Output: Raw EEG data sent to a server

[2095] Step 2: Preprocessing the EEG data

[2096] 1. The server preprocesses the received EEG data.

[2097] 2. The server uses Fourier transform and filtering techniques to remove noise, specifically eye artifacts and ambient electrical interference.

[2098] 3. The server temporarily stores the preprocessed EEG data in an intermediate database.

[2099] Input: Raw EEG data

[2100] Output: Preprocessed EEG data

[2101] Step 3: Translate the dream

[2102] 1. The server inputs the preprocessed EEG data into a text generation model (e.g., GPT-4).

[2103] 2. The server uses the model to convert the dream content from the EEG data into natural language text.

[2104] 3. The server stores the generated natural language text in a database.

[2105] Input: Preprocessed EEG data

[2106] Output: Generated natural language text

[2107] Step 4: Visualize your dream

[2108] 1. The server inputs the generated natural language text into an image generation model (e.g., DALL-E).

[2109] 2. The server reads the text content and generates the corresponding visual data.

[2110] 3. The server stores the visual data and the corresponding text data in a database.

[2111] Input: Natural language text

[2112] Output: The generated visual data

[2113] Step 5: Sentiment Analysis

[2114] 1. The server inputs natural language text and visual data into the sentiment analysis engine.

[2115] 2. The server uses the engine to analyze the user's emotions based on the content of the dream.

[2116] 3. The server stores the sentiment analysis results along with the text and visual data in a database.

[2117] Input: Natural language text and visual data

[2118] Output: Emotion analysis results

[2119] Step 6: Search for and re-experience the dream

[2120] 1. An interface that allows users to search for dreams through a dedicated application.

[2121] 2. The user selects a filter option such as date, theme, character, or emotional state.

[2122] 3. The device sends the search criteria to the server.

[2123] 4. The server retrieves data that matches the search criteria from the database.

[2124] 5. The device displays the text and visual data of the dream to the user.

[2125] Input: User search criteria

[2126] Output: Text, visual, and sentiment analysis data as search results

[2127] Step 7: Favorites function

[2128] 1. The user clicks the "Favorite" button to add the dream to their favorites list.

[2129] 2. The device sends a "favorite" request to the server.

[2130] 3. The server adds the specified dream to the user's "favorite list."

[2131] 4. The server saves the updated favorites list in the database and notifies the device.

[2132] Input: "Favorite" request (yume ID and user information)

[2133] Output: Updated favorites list

[2134] Example prompt

[2135] "Please translate last night's dream into words."

[2136] "I'd like to see a visual of a dream where my family appears."

[2137] "Search for past dreams that have scared you."

[2138] (Application example 2)

[2139] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2140] It is known that the sleep quality and mental state of factory workers have a significant impact on the next day's work efficiency and the risk of making mistakes. However, current technology lacks the means to specifically understand and analyze the sleep patterns and dream content of operators and to predict and manage work efficiency and risks based on this information. Therefore, there is a need for methods to improve factory productivity and safety.

[2141] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting biosignal data acquired during sleep, means for processing the biosignal data and converting the content of dreams into natural language text, means for converting the natural language text into visual data, means for saving the natural language text and the visual data and making them accessible to users, means for allowing users to search and save the saved data as preferences, and means for analyzing the emotional state of the user and predicting and managing work efficiency and the risk of errors. This makes it possible to analyze the sleep and mental states of operators in detail and improve work efficiency and safety in factories.

[2142] "Biological signal data" refers to data that indicates the user's physiological state, such as brain waves, heart rate, and respiratory rate.

[2143] "Natural language text" is text data expressed in a language format commonly used by humans.

[2144] "Visual data" is data that provides information visually, such as images or videos.

[2145] The term "user" refers to a person who uses the system of the present invention, that is, an operator or a user.

[2146] "Emotional state" refers to information that indicates the user's emotions, such as happiness, fear, surprise, etc.

[2147] "Work efficiency" is an index that indicates the progress and productivity of work within a certain period of time.

[2148] "Risk of mistakes occurring" is an indicator that indicates the possibility of mistakes or accidents occurring during work.

[2149] "Noise" is unwanted data or interference mixed into the signal of interest.

[2150] "Preprocessing" is a series of processing operations that shape, clean, and transform raw data prior to primary data processing.

[2151] A "server" is a computer system that receives, stores, processes, and distributes data.

[2152] This invention relates to a system that analyzes biosignal data acquired while an operator is sleeping, converts the data into natural language text and visual data, and further analyzes the operator's emotional state to predict and manage work efficiency and the risk of errors.

[2153] System configuration

[2154] Terminal

[2155] It includes a biosignal recording device used by the operator and a terminal such as a smartphone or tablet, on which an application is installed to collect biosignal data and send it to a server.

[2156] server

[2157] The server receives biosignal data and plays a central role in preprocessing, dream text conversion, visualization, emotion analysis, and data storage and provision.

[2158] Program processing and hardware / software details

[2159] 1. Data Collection and Preprocessing

[2160] Terminal

[2161] Before going to sleep, the operator wears a biosignal recording device (e.g., an electroencephalograph), which collects biosignal data in real time and transmits it to a smartphone via Bluetooth or Wi-Fi.

[2162] server

[2163] The system receives biosignal data sent from the smartphone and stores it in a database. This records the biosignal data accumulated while the operator is sleeping. As a pre-processing step, noise is removed using a Butterworth filter or similar.

[2164] 2. Translating the dream into text

[2165] server

[2166] The preprocessed biosignal data is fed into a text generation model (e.g., GPT-4) to convert the dream content into natural language text, which is temporarily stored and made available upon user request.

[2167] 3. Dream visualization

[2168] server

[2169] The textual content of a dream is input into an image generation model (e.g., DALL-E) to generate detailed visual data. For example, the text "A dragon flying under a blue sky" is converted into a concrete image. The generated visual data and the corresponding text data are associated and stored in a database.

[2170] 4. Emotion analysis using an emotion engine

[2171] server

[2172] An emotion engine is applied to the generated natural language text and visual data. The emotion engine analyzes the operator's emotions based on the content of the dream and saves the results. For example, it determines that the operator is likely to have felt emotions such as "happiness," "fear," or "surprise" based on the content of the dream.

[2173] 5. Leveraging Data

[2174] server

[2175] The system integrates the sleep quality and emotional state of operators to analyze work efficiency and the risk of mistakes, and provides the prediction results to robots in the factory, which then make it possible to adjust the day's work and mitigate risks.

[2176] Specific example of the process

[2177] 1. Operators fall asleep at night

[2178] The operator wears a biosignal recording device, which records biosignals while the patient sleeps. The device then transmits the biosignal data to a server via a smartphone.

[2179] 2. The operator launches the app after waking up

[2180] The server processes the received biosignal data and converts the dream content into natural language text using a text generation model.

[2181] 3. The server visualizes the dream text.

[2182] The server inputs the generated text into an image generation model to generate images depicting specific scenes from the dream.

[2183] 4. Emotion analysis using an emotion engine

[2184] The server applies an emotion engine to the dream text and visual data to analyze the operator's emotional state.

[2185] 5. The operator checks your dreams using the app

[2186] The operator searches for "dreams involving family members," selects a dream from a list of past dreams, and checks the details. At the same time, the operator is also provided with the results of an emotional analysis.

[2187] 6. Utilizing the results

[2188] Based on the data collected and analyzed by the server, it predicts work efficiency and the risk of errors, and provides information to factory robots.

[2189] Examples of prompt statements

[2190] "In a dream, the operator saw a dragon flying under a blue sky."

[2191] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2192] Step 1:

[2193] Terminal

[2194] Before sleep, the operator wears a biosignal recording device, which collects biosignal data such as brain waves and heart rate in real time and transmits the data to a smartphone via Bluetooth or Wi-Fi, thereby obtaining biosignal data as input.

[2195] Step 2:

[2196] server

[2197] The system receives biosignal data sent from a smartphone and stores it in a database. The received data is raw and contains noise, and this becomes the input data. Next, preprocessing is performed to remove noise using a Butterworth filter or similar, and clean biosignal data is output.

[2198] Step 3:

[2199] server

[2200] The preprocessed biosignal data is input into a text generation model (e.g., GPT-4). The model analyzes the input data and converts the dream content into natural language text. The output text data is in the form of, for example, "A dragon flying under a blue sky."

[2201] Step 4:

[2202] server

[2203] The generated natural language text is input into an image generation model (e.g., DALL-E). The model generates detailed visual data (images) based on the text data. For example, the text "A dragon flying under a blue sky" is converted into a concrete image. The output image is stored in a database.

[2204] Step 5:

[2205] server

[2206] An emotion engine is applied to the generated natural language text and visual data. The emotion engine analyzes the input text and image data and outputs the operator's emotional state as "happiness," "fear," "surprise," etc. The analysis results are stored in a database.

[2207] Step 6:

[2208] server

[2209] The system analyzes the operator's work efficiency and risk of mistakes by integrating their sleep quality and emotional state. The server refers to statistical data and past records and performs data calculations to predict risks. The output prediction results are provided to the factory robots.

[2210] Step 7:

[2211] User

[2212] After waking up, the operator can check the content of their dream through a dedicated application. The saved data can be searched and viewed through the app. The user can select "dreams featuring family members" to check the text and images, and the emotional analysis results are also displayed.

[2213] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2214] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2215] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[2216] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2217] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[2218] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[2219] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[2220] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[2221] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[2222] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[2223] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[2224] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[2225] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[2226] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[2227] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[2228] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[2229] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[2230] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[2231] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[2232] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[2233] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[2234] The following is further disclosed regarding the above embodiment.

[2235] (Claim 1)

[2236] means for collecting electroencephalogram data obtained during sleep;

[2237] means for processing the EEG data and converting dream content into natural language text;

[2238] means for converting said natural language text into visual data;

[2239] means for storing and making accessible to a user said natural language text and said visual data;

[2240] means for allowing a user to search and save said stored data as favorites;

[2241] A system including:

[2242] (Claim 2)

[2243] 10. The system of claim 1, further comprising a pre-processing means for removing noise from the electroencephalogram data.

[2244] (Claim 3)

[2245] 10. The system of claim 1, further comprising means for providing a re-experience to a user based on natural language text and visual data.

[2246] "Example 1"

[2247] (Claim 1)

[2248] means for collecting electroencephalogram data obtained during sleep;

[2249] a pre-processing means for processing the electroencephalogram data and removing noise from the electroencephalogram signal;

[2250] a generative model means for converting the preprocessed EEG data into natural language text;

[2251] a generative model means for converting the natural language text into visual data;

[2252] database means for storing said natural language text and said visual data and making them accessible to users;

[2253] interface means that allow a user to search and save the stored data as favorites;

[2254] A system including:

[2255] (Claim 2)

[2256] The system of claim 1 , further comprising a pre-processing means for removing noise from the electroencephalogram data.

[2257] (Claim 3)

[2258] 10. The system of claim 1, further comprising means for providing a re-experience to a user based on natural language text and visual data.

[2259] "Application Example 1"

[2260] (Claim 1)

[2261] means for collecting electroencephalogram data obtained during sleep;

[2262] means for processing the EEG data and converting dream content into natural language text;

[2263] means for converting said natural language text into visual data;

[2264] means for storing and making accessible to a user said natural language text and said visual data;

[2265] means for allowing a user to search and save said stored data as favorites;

[2266] A means for proposing relaxation products and services based on the stored data.

[2267] A system including:

[2268] (Claim 2)

[2269] 10. The system of claim 1, further comprising a pre-processing means for removing noise from the electroencephalogram data.

[2270] (Claim 3)

[2271] 10. The system of claim 1, further comprising means for providing a re-experience to a user based on natural language text and visual data.

[2272] "Example 2: Combining Emotion Engines"

[2273] (Claim 1)

[2274] means for collecting electroencephalogram data obtained during sleep;

[2275] means for processing the EEG data and converting dream content into natural language text;

[2276] means for converting said natural language text into visual data;

[2277] means for analyzing a user's sentiment using the natural language text and the visual data;

[2278] means for allowing a user to search and save said stored data as favorites;

[2279] A system including:

[2280] (Claim 2)

[2281] 10. The system of claim 1, further comprising a pre-processing means for removing noise from the electroencephalogram data.

[2282] (Claim 3)

[2283] 10. The system of claim 1, further comprising means for providing a re-experience to a user based on natural language text and visual data.

[2284] "Application example 2 when combining emotion engines"

[2285] (Claim 1)

[2286] means for collecting biosignal data obtained during sleep;

[2287] means for processing the biosignal data and converting dream content into natural language text;

[2288] means for converting said natural language text into visual data;

[2289] means for storing and making accessible to a user said natural language text and said visual data;

[2290] means for enabling a user to save said stored data as searches and preferences;

[2291] A means for analyzing the emotional state of the user and predicting and managing work efficiency and the risk of mistakes.

[2292] A system including:

[2293] (Claim 2)

[2294] 10. The system of claim 1, further comprising a pre-processing means for removing noise from the biosignal data.

[2295] (Claim 3)

[2296] 10. The system of claim 1, further comprising means for providing a re-experience to a user based on natural language text and visual data. [Explanation of symbols]

[2297] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. means for collecting electroencephalogram data obtained during sleep; means for processing the EEG data and converting dream content into natural language text; means for converting said natural language text into visual data; means for storing and making accessible to a user said natural language text and said visual data; means for allowing a user to search and save said stored data as favorites; A system including:

2. The system of claim 1 , further comprising pre-processing means for removing noise from the electroencephalogram data.

3. The system of claim 1 , further comprising means for providing a re-experience to a user based on natural language text and visual data.

Images