System

Abstract

A system is provided.SOLUTION: A system comprising: means for collecting user email, social network service, and chat application data via a network interface; means for pre-processing the collected data to remove noise data; means for training a generative model to learn user tone, expression, and thought patterns using the pre-processed data; means for generating responses that reproduce user tone, expression, and thought patterns using the trained generative model; and means for providing the generated responses over a network.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In modern society, much of communication has become digitalized, and as a result, personal wishes, preferences, and behavioral patterns are being stored as digital data. However, if the deceased did not leave detailed instructions regarding inheritance or asset management during their lifetime, their surviving family members have no way to confirm their wishes. This often makes it difficult for surviving family members to understand and appropriately process the wishes of the deceased. Furthermore, many surviving family members wish to maintain communication with their deceased family members. Given this background, there is a need for a system that can generate a fixed copy of the deceased's identity based on their digital data, allowing surviving family members to communicate with the deceased even after their death. [Means for solving the problem]

[0005] To solve the above problems, we propose a system that includes a means for collecting data from a user's email, social network service, and chat application via a network interface, a means for preprocessing the collected data and removing noise data, a means for using the preprocessed data to train a generative model that learns the user's tone of voice, expressions, and thought patterns, a means for using the trained generative model to generate responses that reproduce the user's tone of voice, expressions, and thought patterns, and a means for providing the generated responses via a network. This provides a means for surviving family members to communicate with the deceased and confirm their wishes even after their death.

[0006] A "network interface" is the hardware or software means by which a device communicates with a network.

[0007] A "user" is an individual who uses the system and provides their own digital data.

[0008] "Email" refers to a message sent or received electronically, typically communication via email services.

[0009] A "social network service" is a platform where people can share information and communicate via the Internet.

[0010] A "chat application" is software for exchanging text messages in real time.

[0011] "Data" refers to the information and message content sent or received by users via email, social networking services, and chat applications.

[0012] "Preprocessing" is the process of converting collected data into a form that is easier to analyze and removing noise.

[0013] "Noise data" is data that is not useful or potentially misleading for analysis or learning.

[0014] A "generative model" is an algorithm or machine learning model for generating new data from given data.

[0015] "Training" is the process of having a generative model learn from data.

[0016] "Tone" refers to a speaker's particular way of speaking, language, and style.

[0017] "Expression" is the way we communicate thoughts and feelings with specific words and phrases.

[0018] A "thought pattern" is a unique tendency for a particular person to think and make decisions.

[0019] A "response" is a text message that a generative model outputs in response to a particular input.

[0020] "Providing" refers to making the system-generated response available to the user or family members. [Brief explanation of the drawings]

[0021] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0022] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0023] First, the terms used in the following description will be explained.

[0024] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0025] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0026] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0027] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0028] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0029] [First embodiment]

[0030] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0031] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0032] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0033] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0034] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0035] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0036] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0037] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0038] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0039] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0040] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0041] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0042] This invention relates to a system that utilizes a user's digital data to reproduce the user's intentions and expressions even when the user is absent or has passed away. This system includes a process that collects data via a network interface, trains a generative model, and generates responses based on the results.

[0043] A specific embodiment of this system will be described below.

[0044] System configuration

[0045] This system consists of users, terminals, and servers.

[0046] User registration and data collection

[0047] 1. User Registration

[0048] A user accesses the "Will AI Service" using a device (such as a smartphone or PC) and enters personal information such as name, email address, and phone number into the new registration form. The device then sends this information to the server, which creates an account for the user.

[0049] The device then asks the user for permission to access data from email, social networking services, and chat applications. If consent is granted, the device obtains authentication information from each service and begins collecting data.

[0050] 2. Data Collection

[0051] The device periodically collects past message data from each connected email, social networking service, and chat application, including, for example, emails sent and received by the user, social media posts, and chat history.

[0052] The collected data is encrypted and transferred to a server via secure communications.

[0053] Data Preprocessing and Filtering

[0054] 3. Data Preprocessing

[0055] The server decrypts the encrypted data and maps it to the corresponding user ID.

[0056] The server performs preprocessing to convert the data into a format that is easier to analyze. Specifically, the text data of emails and chats is divided into sentences and metadata (date and time, sender, recipient, etc.) is added.

[0057] 4. Data Filtering

[0058] The server filters out noise data, for example, filtering out spam emails and advertising messages, eliminating unnecessary data.

[0059] The filtered data is saved as a dataset for training a generative model.

[0060] Training a generative model

[0061] 5. Training the model

[0062] The server uses the preprocessed data to train a generative model (e.g., a GPT model). During the training process, the model learns the user's unique tone, expressions, and thought patterns.

[0063] Once training is complete, the server optimizes and stores the generative model.

[0064] Post-death care

[0065] 6. Response Generation

[0066] After the death of a user, the family members access the "Will AI Service" using a device and enter specific authentication information. Once authentication is complete, the family members can send questions or messages to the server.

[0067] The server inputs the received question into a trained model to generate a response that reproduces the user's tone and expression.

[0068] 7. Response Provision

[0069] The server sends the generated response to the family's device, where they can read it and check the deceased's wishes and messages.

[0070] Specific examples

[0071] For example, if a user had expressed a wish to "distribute my assets equally" during their lifetime, a response reflecting this wish will be generated from the collected data and the learning results of the generative model. When a surviving family member asks, "How should I distribute my inheritance?", the server generates the response "I want my assets to be distributed equally" and provides it to the surviving family member.

[0072] The above is an example of a specific embodiment of this system. By utilizing the user's digital data, the wishes and expressions of the deceased can be reproduced even after death, making it easier for the bereaved to maintain communication with the deceased.

[0073] The processing flow will be explained below.

[0074] Step 1:

[0075] The user accesses the "Will AI Service" using a device and enters personal information into the new registration form.

[0076] The device sends this information to a server, which creates an account for the user.

[0077] Step 2:

[0078] The device will ask the user for permission to access data from email, social networking services, and chat applications.

[0079] Once the user agrees, the device will obtain the necessary authentication information from each service and begin collecting data.

[0080] Step 3:

[0081] The device periodically collects past message data from email, social networking services, and chat applications.

[0082] The collected data is encrypted and transmitted to a server via secure communication.

[0083] Step 4:

[0084] The server receives the encrypted data, decrypts it and maps it to the corresponding user ID.

[0085] The server performs preprocessing to convert the data into a format that is easy to analyze.

[0086] Step 5:

[0087] The server cleans the collected data by removing noise data and filtering out spam and advertising messages.

[0088] The cleaned data is then saved as a dataset for training generative models.

[0089] Step 6:

[0090] The server uses the preprocessed data to train a generative model.

[0091] This training process involves learning the user's tone of voice, expressions, and thought patterns.

[0092] Step 7:

[0093] The server optimizes the trained generative model and stores the model that reflects the user's characteristics.

[0094] Step 8:

[0095] After the user's death, the surviving family members can use a device to access the "Will AI Service" and enter specific authentication information (such as answers to security questions).

[0096] Once authentication is complete, the family members can send questions or messages to the server.

[0097] Step 9:

[0098] The server inputs the received question into a trained model to generate a response that replicates the user's tone and expressions.

[0099] Step 10:

[0100] The server then sends the generated response to the family member's device.

[0101] The bereaved family will review this and understand the wishes and messages of the deceased.

[0102] Through the above processing steps, the "Will AI Service" can utilize the user's digital data to effectively communicate the user's wishes even after death.

[0103] Example 1

[0104] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0105] There has been no system that can convey a user's wishes and expressions to their surviving family members even after their absence or death. Providing such a system would enable a user's wishes and emotions to be reproduced after death, making it easier for surviving family members to maintain communication with the deceased. It is also necessary to securely collect, preprocess, and filter user data, and utilize generative AI models to reproduce the user's characteristics.

[0106] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0107] In this invention, the server includes means for collecting data from a user's electronic messaging service, social networking platform, and text communication application via a network interface, means for preprocessing the collected data and removing noise data, means for training a generative AI model that learns the user's tone of voice, expressions, and thought patterns using the preprocessed data, means for generating responses that reproduce the user's tone of voice, expressions, and thought patterns using the trained generative AI model, means for providing the generated responses via a network, means for a user to input specific authentication information via a terminal, means for a surviving family member who has completed authentication to send a question or message, means for generating responses that reflect the user's intentions and expressions using the trained generative AI model, and means for transmitting the generated responses to the surviving family member's terminal. This allows the user's digital data to be utilized to reproduce the user's intentions and emotions even after the user's death, enabling the surviving family member to maintain communication with the deceased.

[0108] A "network interface" is an interface for sending and receiving data via various networks.

[0109] "Electronic messaging service" means a service for sending and receiving digital messages, such as email.

[0110] A "social networking platform" is an online platform that enables users to connect and share information with friends and acquaintances.

[0111] A "text communication application" is an application for chatting and sending and receiving messages.

[0112] "Means for collecting data" refers to means consisting of software and hardware for collecting digital data of users.

[0113] The "means for preprocessing and removing noise data" refers to a means for filtering out unnecessary parts from collected data and converting it into a form that is easy to analyze.

[0114] A "generative AI model" is an artificial intelligence model that learns from specific data and reproduces the user's tone of voice, expressions, and thought patterns.

[0115] The "means for generating a response" is a means for generating a response that reflects the characteristics of the user using a trained generative AI model.

[0116] The "means for inputting specific authentication information" refers to a means for inputting authentication information required when a user accesses a service.

[0117] "Means for sending questions or messages" refers to means by which the user or the bereaved family members can send questions or messages to the server.

[0118] The "means for transmitting a response to a terminal of a bereaved family member" is a means for transmitting the generated response to a terminal used by the bereaved family member.

[0119] This invention relates to a system that utilizes a user's digital data to reproduce the user's intentions and expressions even when the user is absent or after death. This system is composed of a user, a terminal, and a server.

[0120] User registration and data collection

[0121] User Registration

[0122] A user accesses the "Will AI Service" using a device (such as a smartphone or PC) and enters personal information such as name, email address, and phone number into a new registration form. The device sends this information to a server, which creates an account for the user. The device then requests permission from the user to access data from email, social networking services, and chat applications. If consent is granted, the device obtains authentication information from each service and begins collecting data.

[0123] Data collection

[0124] We periodically collect historical message data from each electronic messaging service, social networking platform, and text communication application to which your device is connected. This data may include, for example, emails sent and received by you, social media posts, and chat history. The collected data is encrypted and transmitted to our servers via secure communications.

[0125] Data Preprocessing and Filtering

[0126] Data Preprocessing

[0127] The server decrypts the encrypted data and maps it to the corresponding user ID. The server then performs preprocessing to convert the data into a format that is easier to analyze. Specifically, the server splits the text data of emails and chats into sentences and adds metadata (date and time, sender, recipient, etc.).

[0128] Data Filtering

[0129] The server filters out noise data such as spam emails and advertising messages, and the filtered data is stored as a training dataset for generative AI models.

[0130] Training a generative model

[0131] Training the model

[0132] The server uses the preprocessed data to train a generative AI model (such as a GPT model). During this process, it learns the user's unique tone, expressions, and thought patterns. Once training is complete, the server optimizes and saves the generative AI model.

[0133] Response generation and delivery

[0134] Authenticate and enter your question

[0135] After the death of a user, the family members access the "Will AI Service" using a device and enter specific authentication information. Once authentication is complete, the family members can send questions or messages to the server.

[0136] Response Generation

[0137] The server inputs the received questions into a trained generative AI model to generate responses that replicate the user's tone and expressions.

[0138] Response provision

[0139] The server sends the generated response to the family's device, where they can read it and confirm the deceased's wishes and messages.

[0140] Examples and prompts

[0141] For example, if a user had expressed a wish to "distribute my assets equally" during their lifetime, a response reflecting this wish will be generated from the collected data and the learning results of the generative AI model. When a surviving family member asks, "How should I distribute my inheritance?", the server will generate the response "I want my assets to be distributed equally" and provide it to the surviving family member.

[0142] Example prompt:

[0143] User: How should assets be distributed?

[0144] Server: I want my assets to be divided equally.

[0145] The above is a specific embodiment of this system. By utilizing digital data to reproduce the user's intentions and emotions, it becomes possible for the bereaved to maintain communication with the deceased.

[0146] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0147] Step 1: User enters personal information into a sign-up form

[0148] Specific actions

[0149] The user accesses the "Will AI Service" using a device (smartphone or PC) and enters personal information such as name, email address, and phone number into the new registration form.

[0150] input

[0151] Personal information such as your name, email address, and phone number.

[0152] Data processing or data calculation

[0153] The terminal transmits the entered personal information to the server.

[0154] output

[0155] The user's personal information is registered on the server and an account is created for the user.

[0156] Step 2: Obtaining data access permission

[0157] Specific actions

[0158] The device will ask the user for permission to access data from email, social networking services, and chat applications.

[0159] input

[0160] User consent information.

[0161] Data processing or data calculation

[0162] If the user agrees, the device obtains authentication information from each service.

[0163] output

[0164] The obtained credentials.

[0165] Step 3: Performing data collection

[0166] Specific actions

[0167] Periodically collect historical message data from each electronic messaging service, social networking platform, and text communication application to which the device is connected.

[0168] input

[0169] User message data from each service.

[0170] Data processing or data calculation

[0171] The device encrypts the data.

[0172] output

[0173] The encrypted message data.

[0174] Step 4: Encrypt and transfer data

[0175] Specific actions

[0176] The encrypted data is transferred to the server via secure communication.

[0177] input

[0178] The encrypted message data.

[0179] Data processing or data calculation

[0180] Data transfer via secure communications.

[0181] output

[0182] Encrypted data stored on the server.

[0183] Step 5: Decrypt and preprocess the data

[0184] Specific actions

[0185] The server decrypts the encrypted data and maps it to the corresponding user ID.

[0186] input

[0187] Encrypted data stored on the server.

[0188] Data processing or data calculation

[0189] Preprocessing is performed to convert the data into a form that is easier to analyze, and text data from emails and chats is divided into sentences and metadata is added.

[0190] output

[0191] Decrypted data and preprocessed data with metadata added.

[0192] Step 6: Remove noise data

[0193] Specific actions

[0194] The server filters out noise data such as spam emails and advertising messages.

[0195] input

[0196] Preprocessed data.

[0197] Data processing or data calculation

[0198] Spam filtering and noise data removal.

[0199] output

[0200] Clean data with noise removed.

[0201] Step 7: Training the generative AI model

[0202] Specific actions

[0203] The server uses the preprocessed data to train a generative AI model (e.g., a GPT model).

[0204] input

[0205] Clean data with noise removed.

[0206] Data processing or data calculation

[0207] The generative AI model learns the user's unique tone of voice, expressions, and thought patterns.

[0208] output

[0209] A trained generative AI model.

[0210] Step 8: Authenticate and enter your questions

[0211] Specific actions

[0212] After the user's death, the surviving family members use a device to access the "Will AI Service" and enter specific authentication information.

[0213] input

[0214] Family member credentials.

[0215] Data processing or data calculation

[0216] Once authentication is complete, the family members can send questions or messages to the server.

[0217] output

[0218] Questions and messages from bereaved families whose authentication has been completed.

[0219] Step 9: Question processing and response generation

[0220] Specific actions

[0221] The server inputs the received questions into a trained generative AI model to generate responses that replicate the user's tone and expressions.

[0222] input

[0223] Questions and messages from the bereaved families.

[0224] Data processing or data calculation

[0225] The generative AI model generates responses that reflect the user's characteristics.

[0226] output

[0227] The generated response.

[0228] Step 10: Provide a response

[0229] Specific actions

[0230] The server sends the generated response to the terminal of the bereaved family member.

[0231] input

[0232] The generated response.

[0233] Data processing or data calculation

[0234] Sends the response over secure communications.

[0235] output

[0236] The response arrived on the family member's device.

[0237] (Application example 1)

[0238] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0239] In today's digital society, there is a demand for technology that can reproduce a user's intentions and expressions even when the user is absent or has died. However, conventional technologies have difficulty accurately reproducing a user's tone of voice, expressions, and thought patterns, making it difficult to provide personalized responses tailored to each individual user. In particular, in virtual stores, it is necessary to improve customer satisfaction by retaining a user's unique response patterns and automating customer responses.

[0240] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0241] In this invention, the server includes means for collecting data from a user's email, social media services, and real-time messaging applications via a network interface, means for preprocessing the collected data and removing noise data, means for training a generative AI model that learns the user's tone of voice, expressions, and thought patterns using the preprocessed data, means for generating responses that reproduce the user's tone of voice, expressions, and thought patterns using the trained generative AI model, means for providing the generated responses via a network, means for reproducing the user's will and expressions even when the user is absent or after death, and means for learning the user's unique response patterns and providing personalized customer service in a virtual store. This makes it possible to accurately reproduce the user's unique responses even when the user is absent or after death, and to automate personalized customer service in the virtual store.

[0242] A "network interface" is a communication means that allows for the collection, transmission, and reception of digital data.

[0243] A "user" is an entity, such as an individual or a corporation, that registers and provides information.

[0244] "Data collection methods" are mechanisms for obtaining information from email, social media services, real-time messaging applications, etc.

[0245] "Preprocessing" refers to the process of converting collected data into a format that is easy to analyze and removing noise data.

[0246] "Noise data" is unnecessary data or information that reduces the accuracy of analysis or processing.

[0247] A "generative AI model" is an artificial intelligence model that learns from training data and reproduces the user's tone of voice and expressions.

[0248] A "training method" is the process of training a generative AI model using preprocessed data.

[0249] A "response generation means" is a mechanism for generating responses that reflect the user's characteristics using a trained generative AI model.

[0250] The "means for providing via a network" refers to a means for transmitting the generated response to a user or other terminal via a communication network such as the Internet.

[0251] "Means to reproduce the user's will and expression" refers to a function that generates responses by imitating the user's unique words, actions, and intentions even when the user is absent or after death.

[0252] A "virtual store" is a virtual store that offers products and services over the Internet.

[0253] "Personalization" means providing optimal responses and services based on the characteristics and preferences of individual users.

[0254] This invention relates to a system that utilizes a user's digital data to reproduce their will and expressions even when they are absent or have passed away. This system mainly collects data via a network interface, preprocesses the collected data, and trains a generative AI model to learn the user's characteristics and generate responses based on the results.

[0255] The system basically consists of users, terminals, and servers.

[0256] 1. User Registration and Data Collection

[0257] A user accesses the system using a device (such as a smartphone or personal computer), enters personal information into a new registration form, and creates an account. The device then asks the user for permission to access data from email, social media, and real-time messaging applications. If the user agrees, the device obtains authentication information from each service and begins collecting data.

[0258] 2. Data preprocessing and filtering

[0259] The server receives the encrypted data, decrypts it, and maps it to a user ID. Next, the server performs data preprocessing to convert the data into a format that is easier to analyze. Specifically, the server breaks down emails and chat messages into sentences and adds metadata (date, time, sender, recipient, etc.). Spam filtering is then performed to remove noise data.

[0260] 3. Training the generative AI model

[0261] The server uses the preprocessed data to train a generative AI model (e.g., GPT-2). During the training process, the user's unique tone, expressions, and thought patterns are learned. Once training is complete, the generative model is optimized and saved.

[0262] 4. What to do when the user is absent

[0263] When the user is absent or after death, the trained generative AI model is utilized to handle customer service in the virtual store. Specifically, the surviving family members or customers access the system using their devices and send questions or messages. The server inputs the questions into the trained model and generates a response that reproduces the user's characteristics. This response is then sent to the surviving family members or customers' devices.

[0264] Hardware and software used

[0265] The system uses the following hardware and software:

[0266] Hardware: Server with GPU (e.g. NVIDIA Tesla)

[0267] Software: Python, transformers library, GPT-2 model

[0268] Specific examples

[0269] For example, if a user had expressed a wish to "divide my assets equally" during their lifetime, a response reflecting this wish will be generated from the collected data and the learning results of the generative model. When a family member asks, "How should I distribute my inheritance?", the server generates a response such as "I want my assets to be divided equally" and provides it to the family member. Similarly, when a customer in a virtual store asks, "What are today's recommended products?", an answer reflecting the user's unique response patterns that have been learned will be provided.

[0270] Prompt Sentence Examples

[0271] Generate a response when a user asks, "What's your featured product today?"

[0272] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0273] Step 1:

[0274] A user accesses the system using a terminal and creates an account by entering personal information such as name, email address, and phone number in a new registration form. The terminal then sends this personal information to the server, generating an account for the user.

[0275] Step 2:

[0276] The device asks the user for permission to access data from email, social media services, and real-time messaging applications. If the user agrees, the device obtains authentication information from each service and begins continuously collecting data.

[0277] Input: User consent, authentication information

[0278] Output: Collected data (emails, social media posts, message history)

[0279] Step 3:

[0280] The device encrypts the data it collects and sends it over secure connections to a server, which receives the data, decrypts it, and maps it to a user ID.

[0281] Input: Encrypted data

[0282] Output: Decrypted data, data mapped to user ID

[0283] Step 4:

[0284] The server preprocesses the decrypted data and converts it into a format that is easy to analyze. Specifically, it divides it into sentences and adds metadata (date, time, sender, recipient, etc.). Spam filtering is also performed to remove noise data.

[0285] Input: Decrypted data

[0286] Output: Preprocessed data, denoised data

[0287] Step 5:

[0288] The server uses the preprocessed data to train a generative AI model (e.g., GPT-2), which learns the user's unique tone, expressions, and thought patterns, and then optimizes and saves the generative AI model.

[0289] Input: Preprocessed data

[0290] Output: A trained generative AI model

[0291] Step 6:

[0292] When the user is absent or after death, the server utilizes the generative AI model to handle customer service in the virtual store. The server receives questions and messages from customers (or their families) and inputs this information into the generative AI model.

[0293] Input: Questions or messages from the customer (or their family)

[0294] Output: The prompts fed into the AI ​​model

[0295] Step 7:

[0296] The server uses a generative AI model to generate responses that mimic the user's tone and expressions, creating an appropriate response to the question.

[0297] Input: The prompt entered into the AI ​​model

[0298] Output: The generated response

[0299] Step 8:

[0300] The server then sends the generated response to the customer's (or the bereaved family's) terminal via the network, enabling personalized responses that reflect the user's characteristics.

[0301] Input: Generated response

[0302] Output: Response sent to the customer's (or their family's) device

[0303] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0304] This invention relates to a system that utilizes a user's digital data to reproduce the user's intentions and expressions even when the user is absent or has passed away. This system includes a process for collecting data via a network interface, training a generative model, and generating responses based on the results. It also combines a function that recognizes the user's emotions using an emotion engine and generates responses using that information.

[0305] A specific embodiment of this system will be described below.

[0306] System configuration

[0307] This system consists of a user, a terminal, a server, and an emotion engine.

[0308] User registration and data collection

[0309] 1. User Registration

[0310] A user accesses the "Will AI Service" using a device (such as a smartphone or PC) and enters personal information such as name, email address, and phone number into the new registration form. The device then sends this information to the server, which creates an account for the user.

[0311] The device then asks the user for permission to access data from email, social networking services, and chat applications. If consent is granted, the device obtains authentication information from each service and begins collecting data.

[0312] 2. Data Collection

[0313] The device periodically collects past message data from each connected email, social networking service, and chat application, including, for example, emails sent and received by the user, social media posts, and chat history.

[0314] The collected data is encrypted and transferred to a server via secure communications.

[0315] Data Preprocessing and Filtering

[0316] 3. Data Preprocessing

[0317] The server decrypts the encrypted data and maps it to the corresponding user ID.

[0318] The server performs preprocessing to convert the data into a format that is easier to analyze. Specifically, the text data of emails and chats is divided into sentences and metadata (date and time, sender, recipient, etc.) is added.

[0319] 4. Data Filtering

[0320] The server filters out noise data, for example, filtering out spam emails and advertising messages, eliminating unnecessary data.

[0321] The filtered data is saved as a dataset for training a generative model.

[0322] Using the Emotion Engine

[0323] 5. Emotion extraction

[0324] The emotion engine performs emotion analysis on each message in the collected data and assigns an emotion label (e.g., joy, sadness, anger, etc.).

[0325] The emotion engine generates a dataset with emotion labels and sends it to the server.

[0326] Training a generative model

[0327] 6. Training the model

[0328] The server trains the generative model using pre-processed emotion-labeled data. During the training process, the model learns emotions along with the user's unique tone, expressions, and thought patterns.

[0329] Once training is complete, the server optimizes and stores the generative model.

[0330] Post-death care

[0331] 7. Response Generation

[0332] After the death of a user, the family members access the "Will AI Service" using a device and enter specific authentication information. Once authentication is complete, the family members can send questions or messages to the server.

[0333] The server inputs the received question into a trained generative model to generate a response that reflects the user's tone, expression, and even emotion.

[0334] 8. Response Provision

[0335] The server sends the generated response to the device of the bereaved family, who can read it and confirm the deceased's wishes and messages along with their feelings.

[0336] Specific examples

[0337] For example, if a user had expressed a wish to "divide my assets equally" during his or her lifetime, the system would record this wish along with emotional data. After the user's death, if the surviving family members ask, "How should I distribute my inheritance?", the server would respond with "I want my assets to be divided equally," incorporating the user's tone and emotions, and provide this to the surviving family members.

[0338] The above is an example of a specific embodiment of this system. This system utilizes the user's digital data and emotions to provide a means for effectively communicating their wishes even after death.

[0339] The processing flow will be explained below.

[0340] Step 1:

[0341] The user accesses the "Will AI Service" using a device and enters personal information (such as name, email address, and phone number) into the new registration form.

[0342] The device sends this information to a server, which creates an account for the user.

[0343] Step 2:

[0344] The device will ask the user for permission to access data from email, social networking services, and chat applications.

[0345] Once the user agrees, the device will obtain authentication information from each service and begin collecting data.

[0346] Step 3:

[0347] It periodically collects past message data from email, social networking services, and chat applications to which the device belongs.

[0348] The collected data is encrypted and transmitted to a server via secure communication.

[0349] Step 4:

[0350] The server receives the encrypted data, decrypts it and maps it to the corresponding user ID.

[0351] The server performs preprocessing to convert the data into a format that is easy to analyze.

[0352] Step 5:

[0353] The server cleans the collected data by removing noise data and filtering out spam and advertising messages.

[0354] The cleaned data is then saved as a dataset for training generative models.

[0355] Step 6:

[0356] The sentiment engine performs sentiment analysis on each message in the collected data and assigns an emotional label to it.

[0357] The emotion engine generates a dataset with emotion labels and sends it to the server.

[0358] Step 7:

[0359] The server trains a generative model using preprocessed data with emotion labels.

[0360] This training involves learning the user's unique tone of voice, expressions, thought patterns and emotions.

[0361] Step 8:

[0362] After the server completes the training, it optimizes the generated model and saves the model that reflects the user's characteristics.

[0363] Step 9:

[0364] After the user's death, the surviving family members can access the "Will AI Service" using a device and enter specific authentication information (e.g., answers to security questions).

[0365] Once authentication is complete, the family members can send questions or messages to the server.

[0366] Step 10:

[0367] The server receives questions and feeds them into a trained model to generate responses that reflect the user's tone, expression, and even emotion.

[0368] Step 11:

[0369] The server then sends the generated response to the family member's device.

[0370] The bereaved family will review this and emotionally understand the wishes and messages of the deceased.

[0371] Through the above processing steps, the "Will AI Service" can utilize the user's digital data and emotions to effectively communicate the user's wishes even after death.

[0372] Example 2

[0373] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0374] Conventional systems have had difficulty reproducing a user's wishes and expressions when the user is absent or after death. Furthermore, they lacked the technology to generate responses that include the user's emotions, resulting in low emotional reproducibility. This has resulted in a lack of a means to convey the wishes and emotions of the deceased to their surviving family members.

[0375] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0376] In this invention, the server includes means for collecting data from users' emails, social network systems, and messaging systems via a network interface, means for decrypting the collected data, mapping it to a user ID, preprocessing it, and removing noise data, means for training a generative model using the preprocessed data to learn the user's tone of voice, expressions, thought patterns, and emotions, means for generating responses that reproduce the user's tone of voice, expressions, thought patterns, and emotions using the trained generative model, and means for providing the generated responses via a network. This makes it possible to reproduce the will and emotions of the deceased and provide emotional messages to the bereaved.

[0377] A "network interface" is a communication means used to collect data from a user's email, social networking system, and messaging system.

[0378] "User's email" refers to electronic message data sent and received by a user over the Internet.

[0379] A "social network system" is a platform through which users interact with others and share messages and posts.

[0380] A "messaging system" is an application that allows users to send and receive text messages in real time.

[0381] "Data collection" is the process of obtaining this message data from users' email, social networking systems, and messaging systems.

[0382] "Decryption" is the process of restoring encrypted data to its original, readable state.

[0383] "User ID mapping" refers to associating collected data with the corresponding user's identifier.

[0384] "Preprocessing" refers to the stage where data is converted into an analyzable form, specifically a process that includes dividing the data into sentence units and adding metadata.

[0385] "Noise data" is data that is not necessary for analysis, and includes spam and advertising messages.

[0386] A "generative model" is a model that uses machine learning techniques to learn a user's tone, expressions, thought patterns, and emotions from training data.

[0387] "Training" is the process of training a generative model using preprocessed data.

[0388] An "emotion label" is a tag that indicates the type of emotion attached to collected data (e.g., joy, sadness, anger).

[0389] "Response generation" is the process of using a trained generative model to create responses that replicate the user's tone, expressions, thought patterns, and emotions.

[0390] "Providing" means sending the generated response in an appropriate format to the terminal of the bereaved family member.

[0391] A "terminal" is a device (e.g., smartphone, computer) that a user or family member uses to access the system, enter data, and receive results.

[0392] An "emotion engine" is a machine learning algorithm or software for extracting emotions from collected data and assigning emotion labels.

[0393] This invention relates to a system that utilizes a user's digital data to reproduce the user's intentions and expressions even when the user is absent or has passed away. This system includes a process for collecting data via a network interface, training a generative model, and generating responses based on the results. It also combines a function for recognizing the user's emotions using an emotion engine and generating responses using that information.

[0394] System configuration

[0395] This system mainly consists of a user, a terminal, a server, and an emotion engine.

[0396] User registration and data collection

[0397] 1. User Registration

[0398] A user accesses the "Will AI Service" using a device (such as a smartphone or PC) and enters personal information such as name, email address, and phone number into a new registration form. The device sends this information to a server, which creates an account for the user. The device then asks the user for permission to access data from email, social network systems, and messaging systems. If consent is granted, the device obtains authentication information from each service and begins collecting data.

[0399] 2. Data Collection

[0400] The device periodically collects past message data from each connected email, social network, and messaging system. This data includes, for example, emails sent and received by the user, social media posts, and chat history. The collected data is encrypted and transmitted to a server via secure communications.

[0401] Data Preprocessing and Filtering

[0402] 3. Data Preprocessing

[0403] The server decrypts the encrypted data and maps it to the corresponding user ID. The server then performs preprocessing to convert the data into a format that is easier to analyze. Specifically, the server splits the text data of emails and chats into sentences and adds metadata (date and time, sender, recipient, etc.).

[0404] 4. Data Filtering

[0405] The server filters out noise data, for example, filtering out spam emails and advertising messages, and stores the filtered data as a dataset for training the generative model.

[0406] Using the Emotion Engine

[0407] 5. Emotion extraction

[0408] The emotion engine performs emotion analysis on each message in the collected data and assigns an emotion label (e.g., joy, sadness, anger, etc.). The emotion engine generates a dataset with the emotion labels and sends it to the server.

[0409] Training a generative model

[0410] 6. Training the model

[0411] The server trains the generative model using preprocessed emotion-labeled data. During the training process, the model learns emotions along with the user's unique tone, expressions, and thought patterns. Once training is complete, the server optimizes and saves the generative model.

[0412] Post-death care

[0413] 7. Response Generation

[0414] After a user's death, the surviving family members access the "Will AI Service" using their devices and enter specific authentication information. Once authentication is complete, the family members send questions or messages to the server. The server then inputs the received questions into a pre-trained generative model to generate responses that reflect the user's tone, expression, and even emotion.

[0415] 8. Response Provision

[0416] The server sends the generated response to the device of the bereaved family, who can read it and confirm the deceased's wishes and messages along with their feelings.

[0417] Specific examples

[0418] For example, if a user had expressed a wish to "distribute my assets equally" during their lifetime, the system would record this wish along with emotional data. After the death, if the surviving family members ask, "How should I distribute my inheritance?", the server would respond by including the user's tone and emotions, and provide the family with the message, "I want my assets to be distributed equally."

[0419] Prompt Sentence Examples

[0420] How should I distribute my inheritance?

[0421] The above is an example of a specific embodiment of this system. This system utilizes the user's digital data and emotions to provide a means for effectively communicating their wishes even after death.

[0422] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0423] Step 1:

[0424] User Registration

[0425] A user accesses the Will AI Service website or app using a device (smartphone or PC). The user enters personal information such as name, email address, and phone number into the new registration form. The device then sends this information to the server, creating an account for the user. The input is personal information, and the output is the creation of a user account.

[0426] Step 2:

[0427] Obtaining data access permissions

[0428] The device asks the user for permission to access data from email, social network systems, and messaging systems. The user confirms and agrees to the permission. The input of this procedure is the permission request, and the output is the user's consent confirmation.

[0429] Step 3:

[0430] Data collection

[0431] Using the user's authentication information, the device collects past message data from each email, social network system, and messaging system. This data includes emails sent and received by the user, social media posts, and chat history. The collected data is encrypted and transmitted to a server via secure communication. The input to this process is authentication information and connection data, and the output is encrypted data.

[0432] Step 4:

[0433] Data Decoding and Preprocessing

[0434] The server receives the encrypted data and decrypts it. It then maps it to the corresponding user ID, splits the email or chat text data into sentences, and adds metadata (date, time, sender, recipient, etc.). The input to this step is the encrypted data and the user ID, and the output is the preprocessed data.

[0435] Step 5:

[0436] Data Filtering

[0437] The server filters out noise data from the preprocessed data. Specifically, it filters out spam emails and advertising messages, leaving only the necessary data. The input of this step is the preprocessed data, and the output is the filtered dataset.

[0438] Step 6:

[0439] emotion extraction

[0440] The server invokes the emotion engine to perform sentiment analysis on each filtered message, and assigns an emotion label (e.g., joy, sadness, anger, etc.) to each message. The inputs of this step are the filtered data and the emotion engine, and the output is emotion-labeled data.

[0441] Step 7:

[0442] Training a generative model

[0443] The server trains the generative AI model using emotion-labeled data. During this training process, it learns the user's tone, expressions, thought patterns, and emotions. Once training is complete, it optimizes and saves the generative model. The input of this step is the labeled data, and the output is the optimized generative model.

[0444] Step 8:

[0445] Identification of bereaved family members and question reception

[0446] After the user's death, the surviving family members access the "Will AI Service" using a device and enter specific authentication information. Once authentication is complete, the surviving family members send questions or messages to the server. The input of this step is the surviving family members' authentication information and questions, and the output is the completion of authentication and receipt of the questions.

[0447] Step 9:

[0448] Response Generation

[0449] The server inputs the received question into a trained generative model to generate a response that reflects the user's tone, expression, and emotion. The input for this step is the family member's question, and the output is the generated response.

[0450] Step 10:

[0451] Response provision

[0452] The server sends the generated response to the bereaved family member's device, who then confirms it. The bereaved family member can receive the user's intentions and messages along with their emotions. The input of this step is the generated response, and the output is the provision of the response to the bereaved family member's device.

[0453] This is the specific process flow of the system.

[0454] (Application example 2)

[0455] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0456] Current home and business security systems have limitations in their ability to detect abnormalities, making it difficult to respond quickly and appropriately when an emergency occurs while the user is away. Furthermore, there is a lack of systems that can automatically generate and provide countermeasures that reflect the user's intentions and intent. This creates the problem of delayed security responses.

[0457] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting data from the user's email, social network service, and chat application via a network interface, means for preprocessing the collected data and removing noise data, and means for training a generative AI model that learns the user's tone of voice, expressions, and thought patterns using the preprocessed data. This makes it possible to detect abnormalities with high accuracy even when the user is absent, and to automatically generate and provide optimal countermeasures that reflect the user's intentions and will.

[0458] A "network interface" refers to the hardware and software that connects a computer or data collection device to a network and enables data communication.

[0459] "Email" refers to a digital communication method for sending and receiving messages over the Internet or other computer networks.

[0460] "Social network service" refers to an online platform that enables people to share information and interact with each other over the Internet.

[0461] A "chat application" refers to software for exchanging messages in real time, including text, voice, and video.

[0462] "Data preprocessing" refers to the process of converting collected raw data into a format suitable for analysis and machine learning.

[0463] "Noise data" refers to data that contains information that is unnecessary or misleading for analytical or learning purposes.

[0464] A "generative AI model" is a machine learning model that learns a user's unique tone of voice and expression patterns from given data and generates new responses and content.

[0465] "Training" refers to the process of showing a machine learning model large amounts of data and having it learn its patterns and features.

[0466] "Anomaly detection" refers to techniques and processes that automatically detect data or situations that deviate from normal patterns.

[0467] A "countermeasure" is a specific course of action or proposal for dealing with a particular situation.

[0468] The present invention provides a system for automatically taking a response that reflects the user's intentions even when the user is absent in a home or business security system. The system includes the following means.

[0469] 1. User Registration and Data Collection

[0470] The server collects personal information provided by users via a network interface. Users use devices such as smartphones or PCs to register basic information such as their name, address, and household members. The server also connects with various security devices (cameras, door locks, voice assistants, etc.) and obtains access permissions to the data generated by these devices. This allows the server to centrally collect situational data both inside and outside the home.

[0471] 2. Data preprocessing and filtering

[0472] The server preprocesses the collected data and removes noise data that is unsuitable for analysis. This preprocessing includes dividing text data into sentences, converting audio data into text, and extracting important frames from video data. It also uses spam filtering to remove irrelevant and noise data.

[0473] 3. Emotion extraction using an emotion engine

[0474] The server uses an emotion engine to analyze the collected data and assigns an emotion label to each message or action, allowing it to understand the user's daily emotional patterns and identify emotional changes when an anomaly occurs.

[0475] 4. Training the generative AI model

[0476] The server uses the preprocessed data to train the generative AI model. This training process learns the user's unique tone, expressions, and thought patterns, allowing the system to reproduce the user's intentions and emotions. Anomaly detection algorithms are also trained at this stage, allowing them to distinguish between normal and abnormal.

[0477] 5. Anomaly Detection and Response Generation

[0478] If an abnormality is detected while the user is absent, the server will automatically generate the optimal countermeasure using a trained generative AI model. For example, if suspicious activity is detected, the security robot can ask, "There is suspicious behavior. Is there something I can help you with?", and the response will reproduce the user's unique tone and expressions.

[0479] 6. Response Provision

[0480] The server then sends the generated responses and countermeasures to the user's device and related security devices via the network, allowing the user to check the status of their home even when they are away from home and implement the automatically generated countermeasures as needed.

[0481] Specific examples

[0482] For example, if a user inputs a prompt question such as "There has been suspicious activity in the house recently. What should we do?" into a generative AI model, the server can generate a response such as "Suspicious activity has been detected. Would you like to contact the police?" This response reflects the user's tone of voice and thought patterns, making it possible to take appropriate action based on the user's intentions even when the user is not present.

[0483] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0484] Step 1:

[0485] The user uses the terminal to register personal information, including basic information such as name, address, and household member composition. The registered information is sent to the server via the network interface. The input data is the personal information entered by the user, and the output data is the user information stored in the database.

[0486] Step 2:

[0487] After obtaining user approval, the server collects data from each security device (camera, door lock, voice assistant, etc.). The collected data is periodically transferred to the server. The input data is the raw data obtained from the security device, and the output data is the collected data before preprocessing.

[0488] Step 3:

[0489] The server preprocesses the collected data. Specifically, it splits text data into sentences, converts audio data to text, and extracts important frames from video data. It also performs spam filtering and removes noise data. The input data is a raw data stream, and the output data is a preprocessed dataset.

[0490] Step 4:

[0491] The server analyzes the preprocessed data using an emotion engine and assigns emotion labels to each message and action. This allows the user's daily emotion patterns to be analyzed. The input data is the preprocessed data, and the output data is the data with the emotion labels.

[0492] Step 5:

[0493] The server uses the emotion-labeled data to train the generative AI model. This training process learns the user's unique tone, expressions, and thought patterns. The input data is the emotion-labeled dataset, and the output data is the trained generative AI model.

[0494] Step 6:

[0495] The server detects anomalies when the user is absent. If an anomaly is detected, a trained generative AI model is used to generate optimal countermeasures. For example, if suspicious activity is detected, the security robot will ask, "There is suspicious behavior. Is there anything we can help you with?" The input data is real-time data used to detect anomalies, and the output data is the generated countermeasures.

[0496] Step 7:

[0497] The server sends the generated countermeasures to the user's terminal and related security devices via the network. This allows the user to check the status of their home even when they are away from home and implement the automatically generated countermeasures as necessary. The input data is the generated countermeasures, and the output data is a response message notified to the user.

[0498] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0499] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0500] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0501] [Second embodiment]

[0502] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0503] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0504] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0505] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0506] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0507] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0508] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0509] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0510] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0511] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0512] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0513] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0514] This invention relates to a system that utilizes a user's digital data to reproduce the user's intentions and expressions even when the user is absent or has passed away. This system includes a process that collects data via a network interface, trains a generative model, and generates responses based on the results.

[0515] A specific embodiment of this system will be described below.

[0516] System configuration

[0517] This system consists of users, terminals, and servers.

[0518] User registration and data collection

[0519] 1. User Registration

[0520] A user accesses the "Will AI Service" using a device (such as a smartphone or PC) and enters personal information such as name, email address, and phone number into the new registration form. The device then sends this information to the server, which creates an account for the user.

[0521] The device then asks the user for permission to access data from email, social networking services, and chat applications. If consent is granted, the device obtains authentication information from each service and begins collecting data.

[0522] 2. Data Collection

[0523] The device periodically collects past message data from each connected email, social networking service, and chat application, including, for example, emails sent and received by the user, social media posts, and chat history.

[0524] The collected data is encrypted and transferred to a server via secure communications.

[0525] Data Preprocessing and Filtering

[0526] 3. Data Preprocessing

[0527] The server decrypts the encrypted data and maps it to the corresponding user ID.

[0528] The server performs preprocessing to convert the data into a format that is easier to analyze. Specifically, the text data of emails and chats is divided into sentences and metadata (date and time, sender, recipient, etc.) is added.

[0529] 4. Data Filtering

[0530] The server filters out noise data, for example, filtering out spam emails and advertising messages, eliminating unnecessary data.

[0531] The filtered data is saved as a dataset for training a generative model.

[0532] Training a generative model

[0533] 5. Training the model

[0534] The server uses the preprocessed data to train a generative model (e.g., a GPT model). During the training process, the model learns the user's unique tone, expressions, and thought patterns.

[0535] Once training is complete, the server optimizes and stores the generative model.

[0536] Post-death care

[0537] 6. Response Generation

[0538] After the death of a user, the family members access the "Will AI Service" using a device and enter specific authentication information. Once authentication is complete, the family members can send questions or messages to the server.

[0539] The server inputs the received question into a trained model to generate a response that reproduces the user's tone and expression.

[0540] 7. Response Provision

[0541] The server sends the generated response to the family's device, where they can read it and check the deceased's wishes and messages.

[0542] Specific examples

[0543] For example, if a user had expressed a wish to "distribute my assets equally" during their lifetime, a response reflecting this wish will be generated from the collected data and the learning results of the generative model. When a surviving family member asks, "How should I distribute my inheritance?", the server generates the response "I want my assets to be distributed equally" and provides it to the surviving family member.

[0544] The above is an example of a specific embodiment of this system. By utilizing the user's digital data, the wishes and expressions of the deceased can be reproduced even after death, making it easier for the bereaved to maintain communication with the deceased.

[0545] The processing flow will be explained below.

[0546] Step 1:

[0547] The user accesses the "Will AI Service" using a device and enters personal information into the new registration form.

[0548] The device sends this information to a server, which creates an account for the user.

[0549] Step 2:

[0550] The device will ask the user for permission to access data from email, social networking services, and chat applications.

[0551] Once the user agrees, the device will obtain the necessary authentication information from each service and begin collecting data.

[0552] Step 3:

[0553] The device periodically collects past message data from email, social networking services, and chat applications.

[0554] The collected data is encrypted and transmitted to a server via secure communication.

[0555] Step 4:

[0556] The server receives the encrypted data, decrypts it and maps it to the corresponding user ID.

[0557] The server performs preprocessing to convert the data into a format that is easy to analyze.

[0558] Step 5:

[0559] The server cleans the collected data by removing noise data and filtering out spam and advertising messages.

[0560] The cleaned data is then saved as a dataset for training generative models.

[0561] Step 6:

[0562] The server uses the preprocessed data to train a generative model.

[0563] This training process involves learning the user's tone of voice, expressions, and thought patterns.

[0564] Step 7:

[0565] The server optimizes the trained generative model and stores the model that reflects the user's characteristics.

[0566] Step 8:

[0567] After the user's death, the surviving family members can use a device to access the "Will AI Service" and enter specific authentication information (such as answers to security questions).

[0568] Once authentication is complete, the family members can send questions or messages to the server.

[0569] Step 9:

[0570] The server inputs the received question into a trained model to generate a response that replicates the user's tone and expressions.

[0571] Step 10:

[0572] The server then sends the generated response to the family member's device.

[0573] The bereaved family will review this and understand the wishes and messages of the deceased.

[0574] Through the above processing steps, the "Will AI Service" can utilize the user's digital data to effectively communicate the user's wishes even after death.

[0575] Example 1

[0576] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0577] There has been no system that can convey a user's wishes and expressions to their surviving family members even after their absence or death. Providing such a system would enable a user's wishes and emotions to be reproduced after death, making it easier for surviving family members to maintain communication with the deceased. It is also necessary to securely collect, preprocess, and filter user data, and utilize generative AI models to reproduce the user's characteristics.

[0578] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0579] In this invention, the server includes means for collecting data from a user's electronic messaging service, social networking platform, and text communication application via a network interface, means for preprocessing the collected data and removing noise data, means for training a generative AI model that learns the user's tone of voice, expressions, and thought patterns using the preprocessed data, means for generating responses that reproduce the user's tone of voice, expressions, and thought patterns using the trained generative AI model, means for providing the generated responses via a network, means for a user to input specific authentication information via a terminal, means for a surviving family member who has completed authentication to send a question or message, means for generating responses that reflect the user's intentions and expressions using the trained generative AI model, and means for transmitting the generated responses to the surviving family member's terminal. This allows the user's digital data to be utilized to reproduce the user's intentions and emotions even after the user's death, enabling the surviving family member to maintain communication with the deceased.

[0580] A "network interface" is an interface for sending and receiving data via various networks.

[0581] "Electronic messaging service" means a service for sending and receiving digital messages, such as email.

[0582] A "social networking platform" is an online platform that enables users to connect and share information with friends and acquaintances.

[0583] A "text communication application" is an application for chatting and sending and receiving messages.

[0584] "Means for collecting data" refers to means consisting of software and hardware for collecting digital data of users.

[0585] The "means for preprocessing and removing noise data" refers to a means for filtering out unnecessary parts from collected data and converting it into a form that is easy to analyze.

[0586] A "generative AI model" is an artificial intelligence model that learns from specific data and reproduces the user's tone of voice, expressions, and thought patterns.

[0587] The "means for generating a response" is a means for generating a response that reflects the characteristics of the user using a trained generative AI model.

[0588] The "means for inputting specific authentication information" refers to a means for inputting authentication information required when a user accesses a service.

[0589] "Means for sending questions or messages" refers to means by which the user or the bereaved family members can send questions or messages to the server.

[0590] The "means for transmitting a response to a terminal of a bereaved family member" is a means for transmitting the generated response to a terminal used by the bereaved family member.

[0591] This invention relates to a system that utilizes a user's digital data to reproduce the user's intentions and expressions even when the user is absent or after death. This system is composed of a user, a terminal, and a server.

[0592] User registration and data collection

[0593] User Registration

[0594] A user accesses the "Will AI Service" using a device (such as a smartphone or PC) and enters personal information such as name, email address, and phone number into a new registration form. The device sends this information to a server, which creates an account for the user. The device then requests permission from the user to access data from email, social networking services, and chat applications. If consent is granted, the device obtains authentication information from each service and begins collecting data.

[0595] Data collection

[0596] We periodically collect historical message data from each electronic messaging service, social networking platform, and text communication application to which your device is connected. This data may include, for example, emails sent and received by you, social media posts, and chat history. The collected data is encrypted and transmitted to our servers via secure communications.

[0597] Data Preprocessing and Filtering

[0598] Data Preprocessing

[0599] The server decrypts the encrypted data and maps it to the corresponding user ID. The server then performs preprocessing to convert the data into a format that is easier to analyze. Specifically, the server splits the text data of emails and chats into sentences and adds metadata (date and time, sender, recipient, etc.).

[0600] Data Filtering

[0601] The server filters out noise data such as spam emails and advertising messages, and the filtered data is stored as a training dataset for generative AI models.

[0602] Training a generative model

[0603] Training the model

[0604] The server uses the preprocessed data to train a generative AI model (such as a GPT model). During this process, it learns the user's unique tone, expressions, and thought patterns. Once training is complete, the server optimizes and saves the generative AI model.

[0605] Response generation and delivery

[0606] Authenticate and enter your question

[0607] After the death of a user, the family members access the "Will AI Service" using a device and enter specific authentication information. Once authentication is complete, the family members can send questions or messages to the server.

[0608] Response Generation

[0609] The server inputs the received questions into a trained generative AI model to generate responses that replicate the user's tone and expressions.

[0610] Response provision

[0611] The server sends the generated response to the family's device, where they can read it and confirm the deceased's wishes and messages.

[0612] Examples and prompts

[0613] For example, if a user had expressed a wish to "distribute my assets equally" during their lifetime, a response reflecting this wish will be generated from the collected data and the learning results of the generative AI model. When a surviving family member asks, "How should I distribute my inheritance?", the server will generate the response "I want my assets to be distributed equally" and provide it to the surviving family member.

[0614] Example prompt:

[0615] User: How should assets be distributed?

[0616] Server: I want my assets to be divided equally.

[0617] The above is a specific embodiment of this system. By utilizing digital data to reproduce the user's intentions and emotions, it becomes possible for the bereaved to maintain communication with the deceased.

[0618] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0619] Step 1: User enters personal information into a sign-up form

[0620] Specific actions

[0621] The user accesses the "Will AI Service" using a device (smartphone or PC) and enters personal information such as name, email address, and phone number into the new registration form.

[0622] input

[0623] Personal information such as your name, email address, and phone number.

[0624] Data processing or data calculation

[0625] The terminal transmits the entered personal information to the server.

[0626] output

[0627] The user's personal information is registered on the server and an account is created for the user.

[0628] Step 2: Obtaining data access permission

[0629] Specific actions

[0630] The device will ask the user for permission to access data from email, social networking services, and chat applications.

[0631] input

[0632] User consent information.

[0633] Data processing or data calculation

[0634] If the user agrees, the device obtains authentication information from each service.

[0635] output

[0636] The obtained credentials.

[0637] Step 3: Performing data collection

[0638] Specific actions

[0639] Periodically collect historical message data from each electronic messaging service, social networking platform, and text communication application to which the device is connected.

[0640] input

[0641] User message data from each service.

[0642] Data processing or data calculation

[0643] The device encrypts the data.

[0644] output

[0645] The encrypted message data.

[0646] Step 4: Encrypt and transfer data

[0647] Specific actions

[0648] The encrypted data is transferred to the server via secure communication.

[0649] input

[0650] The encrypted message data.

[0651] Data processing or data calculation

[0652] Data transfer via secure communications.

[0653] output

[0654] Encrypted data stored on the server.

[0655] Step 5: Decrypt and preprocess the data

[0656] Specific actions

[0657] The server decrypts the encrypted data and maps it to the corresponding user ID.

[0658] input

[0659] Encrypted data stored on the server.

[0660] Data processing or data calculation

[0661] Preprocessing is performed to convert the data into a form that is easier to analyze, and text data from emails and chats is divided into sentences and metadata is added.

[0662] output

[0663] Decrypted data and preprocessed data with metadata added.

[0664] Step 6: Remove noise data

[0665] Specific actions

[0666] The server filters out noise data such as spam emails and advertising messages.

[0667] input

[0668] Preprocessed data.

[0669] Data processing or data calculation

[0670] Spam filtering and noise data removal.

[0671] output

[0672] Clean data with noise removed.

[0673] Step 7: Training the generative AI model

[0674] Specific actions

[0675] The server uses the preprocessed data to train a generative AI model (e.g., a GPT model).

[0676] input

[0677] Clean data with noise removed.

[0678] Data processing or data calculation

[0679] The generative AI model learns the user's unique tone of voice, expressions, and thought patterns.

[0680] output

[0681] A trained generative AI model.

[0682] Step 8: Authenticate and enter your questions

[0683] Specific actions

[0684] After the user's death, the surviving family members use a device to access the "Will AI Service" and enter specific authentication information.

[0685] input

[0686] Family member credentials.

[0687] Data processing or data calculation

[0688] Once authentication is complete, the family members can send questions or messages to the server.

[0689] output

[0690] Questions and messages from bereaved families whose authentication has been completed.

[0691] Step 9: Question processing and response generation

[0692] Specific actions

[0693] The server inputs the received questions into a trained generative AI model to generate responses that replicate the user's tone and expressions.

[0694] input

[0695] Questions and messages from the bereaved families.

[0696] Data processing or data calculation

[0697] The generative AI model generates responses that reflect the user's characteristics.

[0698] output

[0699] The generated response.

[0700] Step 10: Provide a response

[0701] Specific actions

[0702] The server sends the generated response to the terminal of the bereaved family member.

[0703] input

[0704] The generated response.

[0705] Data processing or data calculation

[0706] Sends the response over secure communications.

[0707] output

[0708] The response arrived on the family member's device.

[0709] (Application example 1)

[0710] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0711] In today's digital society, there is a demand for technology that can reproduce a user's intentions and expressions even when the user is absent or has died. However, conventional technologies have difficulty accurately reproducing a user's tone of voice, expressions, and thought patterns, making it difficult to provide personalized responses tailored to each individual user. In particular, in virtual stores, it is necessary to improve customer satisfaction by retaining a user's unique response patterns and automating customer responses.

[0712] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0713] In this invention, the server includes means for collecting data from a user's email, social media services, and real-time messaging applications via a network interface, means for preprocessing the collected data and removing noise data, means for training a generative AI model that learns the user's tone of voice, expressions, and thought patterns using the preprocessed data, means for generating responses that reproduce the user's tone of voice, expressions, and thought patterns using the trained generative AI model, means for providing the generated responses via a network, means for reproducing the user's will and expressions even when the user is absent or after death, and means for learning the user's unique response patterns and providing personalized customer service in a virtual store. This makes it possible to accurately reproduce the user's unique responses even when the user is absent or after death, and to automate personalized customer service in the virtual store.

[0714] A "network interface" is a communication means that allows for the collection, transmission, and reception of digital data.

[0715] A "user" is an entity, such as an individual or a corporation, that registers and provides information.

[0716] "Data collection methods" are mechanisms for obtaining information from email, social media services, real-time messaging applications, etc.

[0717] "Preprocessing" refers to the process of converting collected data into a format that is easy to analyze and removing noise data.

[0718] "Noise data" is unnecessary data or information that reduces the accuracy of analysis or processing.

[0719] A "generative AI model" is an artificial intelligence model that learns from training data and reproduces the user's tone of voice and expressions.

[0720] A "training method" is the process of training a generative AI model using preprocessed data.

[0721] A "response generation means" is a mechanism for generating responses that reflect the user's characteristics using a trained generative AI model.

[0722] The "means for providing via a network" refers to a means for transmitting the generated response to a user or other terminal via a communication network such as the Internet.

[0723] "Means to reproduce the user's will and expression" refers to a function that generates responses by imitating the user's unique words, actions, and intentions even when the user is absent or after death.

[0724] A "virtual store" is a virtual store that offers products and services over the Internet.

[0725] "Personalization" means providing optimal responses and services based on the characteristics and preferences of individual users.

[0726] This invention relates to a system that utilizes a user's digital data to reproduce their will and expressions even when they are absent or have passed away. This system mainly collects data via a network interface, preprocesses the collected data, and trains a generative AI model to learn the user's characteristics and generate responses based on the results.

[0727] The system basically consists of users, terminals, and servers.

[0728] 1. User Registration and Data Collection

[0729] A user accesses the system using a device (such as a smartphone or personal computer), enters personal information into a new registration form, and creates an account. The device then asks the user for permission to access data from email, social media, and real-time messaging applications. If the user agrees, the device obtains authentication information from each service and begins collecting data.

[0730] 2. Data preprocessing and filtering

[0731] The server receives the encrypted data, decrypts it, and maps it to a user ID. Next, the server performs data preprocessing to convert the data into a format that is easier to analyze. Specifically, the server breaks down emails and chat messages into sentences and adds metadata (date, time, sender, recipient, etc.). Spam filtering is then performed to remove noise data.

[0732] 3. Training the generative AI model

[0733] The server uses the preprocessed data to train a generative AI model (e.g., GPT-2). During the training process, the user's unique tone, expressions, and thought patterns are learned. Once training is complete, the generative model is optimized and saved.

[0734] 4. What to do when the user is absent

[0735] When the user is absent or after death, the trained generative AI model is utilized to handle customer service in the virtual store. Specifically, the surviving family members or customers access the system using their devices and send questions or messages. The server inputs the questions into the trained model and generates a response that reproduces the user's characteristics. This response is then sent to the surviving family members or customers' devices.

[0736] Hardware and software used

[0737] The system uses the following hardware and software:

[0738] Hardware: Server with GPU (e.g. NVIDIA Tesla)

[0739] Software: Python, transformers library, GPT-2 model

[0740] Specific examples

[0741] For example, if a user had expressed a wish to "divide my assets equally" during their lifetime, a response reflecting this wish will be generated from the collected data and the learning results of the generative model. When a family member asks, "How should I distribute my inheritance?", the server generates a response such as "I want my assets to be divided equally" and provides it to the family member. Similarly, when a customer in a virtual store asks, "What are today's recommended products?", an answer reflecting the user's unique response patterns that have been learned will be provided.

[0742] Prompt Sentence Examples

[0743] Generate a response when a user asks, "What's your featured product today?"

[0744] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0745] Step 1:

[0746] A user accesses the system using a terminal and creates an account by entering personal information such as name, email address, and phone number in a new registration form. The terminal then sends this personal information to the server, generating an account for the user.

[0747] Step 2:

[0748] The device asks the user for permission to access data from email, social media services, and real-time messaging applications. If the user agrees, the device obtains authentication information from each service and begins continuously collecting data.

[0749] Input: User consent, authentication information

[0750] Output: Collected data (emails, social media posts, message history)

[0751] Step 3:

[0752] The device encrypts the data it collects and sends it over secure connections to a server, which receives the data, decrypts it, and maps it to a user ID.

[0753] Input: Encrypted data

[0754] Output: Decrypted data, data mapped to user ID

[0755] Step 4:

[0756] The server preprocesses the decrypted data and converts it into a format that is easy to analyze. Specifically, it divides it into sentences and adds metadata (date, time, sender, recipient, etc.). Spam filtering is also performed to remove noise data.

[0757] Input: Decrypted data

[0758] Output: Preprocessed data, denoised data

[0759] Step 5:

[0760] The server uses the preprocessed data to train a generative AI model (e.g., GPT-2), which learns the user's unique tone, expressions, and thought patterns, and then optimizes and saves the generative AI model.

[0761] Input: Preprocessed data

[0762] Output: A trained generative AI model

[0763] Step 6:

[0764] When the user is absent or after death, the server utilizes the generative AI model to handle customer service in the virtual store. The server receives questions and messages from customers (or their families) and inputs this information into the generative AI model.

[0765] Input: Questions or messages from the customer (or their family)

[0766] Output: The prompts fed into the AI ​​model

[0767] Step 7:

[0768] The server uses a generative AI model to generate responses that mimic the user's tone and expressions, creating an appropriate response to the question.

[0769] Input: The prompt entered into the AI ​​model

[0770] Output: The generated response

[0771] Step 8:

[0772] The server then sends the generated response to the customer's (or the bereaved family's) terminal via the network, enabling personalized responses that reflect the user's characteristics.

[0773] Input: Generated response

[0774] Output: Response sent to the customer's (or their family's) device

[0775] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0776] This invention relates to a system that utilizes a user's digital data to reproduce the user's intentions and expressions even when the user is absent or has passed away. This system includes a process for collecting data via a network interface, training a generative model, and generating responses based on the results. It also combines a function that recognizes the user's emotions using an emotion engine and generates responses using that information.

[0777] A specific embodiment of this system will be described below.

[0778] System configuration

[0779] This system consists of a user, a terminal, a server, and an emotion engine.

[0780] User registration and data collection

[0781] 1. User Registration

[0782] A user accesses the "Will AI Service" using a device (such as a smartphone or PC) and enters personal information such as name, email address, and phone number into the new registration form. The device then sends this information to the server, which creates an account for the user.

[0783] The device then asks the user for permission to access data from email, social networking services, and chat applications. If consent is granted, the device obtains authentication information from each service and begins collecting data.

[0784] 2. Data Collection

[0785] The device periodically collects past message data from each connected email, social networking service, and chat application, including, for example, emails sent and received by the user, social media posts, and chat history.

[0786] The collected data is encrypted and transferred to a server via secure communications.

[0787] Data Preprocessing and Filtering

[0788] 3. Data Preprocessing

[0789] The server decrypts the encrypted data and maps it to the corresponding user ID.

[0790] The server performs preprocessing to convert the data into a format that is easier to analyze. Specifically, the text data of emails and chats is divided into sentences and metadata (date and time, sender, recipient, etc.) is added.

[0791] 4. Data Filtering

[0792] The server filters out noise data, for example, filtering out spam emails and advertising messages, eliminating unnecessary data.

[0793] The filtered data is saved as a dataset for training a generative model.

[0794] Using the Emotion Engine

[0795] 5. Emotion extraction

[0796] The emotion engine performs emotion analysis on each message in the collected data and assigns an emotion label (e.g., joy, sadness, anger, etc.).

[0797] The emotion engine generates a dataset with emotion labels and sends it to the server.

[0798] Training a generative model

[0799] 6. Training the model

[0800] The server trains the generative model using pre-processed emotion-labeled data. During the training process, the model learns emotions along with the user's unique tone, expressions, and thought patterns.

[0801] Once training is complete, the server optimizes and stores the generative model.

[0802] Post-death care

[0803] 7. Response Generation

[0804] After the death of a user, the family members access the "Will AI Service" using a device and enter specific authentication information. Once authentication is complete, the family members can send questions or messages to the server.

[0805] The server inputs the received question into a trained generative model to generate a response that reflects the user's tone, expression, and even emotion.

[0806] 8. Response Provision

[0807] The server sends the generated response to the device of the bereaved family, who can read it and confirm the deceased's wishes and messages along with their feelings.

[0808] Specific examples

[0809] For example, if a user had expressed a wish to "divide my assets equally" during his or her lifetime, the system would record this wish along with emotional data. After the user's death, if the surviving family members ask, "How should I distribute my inheritance?", the server would respond with "I want my assets to be divided equally," incorporating the user's tone and emotions, and provide this to the surviving family members.

[0810] The above is an example of a specific embodiment of this system. This system utilizes the user's digital data and emotions to provide a means for effectively communicating their wishes even after death.

[0811] The processing flow will be explained below.

[0812] Step 1:

[0813] The user accesses the "Will AI Service" using a device and enters personal information (such as name, email address, and phone number) into the new registration form.

[0814] The device sends this information to a server, which creates an account for the user.

[0815] Step 2:

[0816] The device will ask the user for permission to access data from email, social networking services, and chat applications.

[0817] Once the user agrees, the device will obtain authentication information from each service and begin collecting data.

[0818] Step 3:

[0819] It periodically collects past message data from email, social networking services, and chat applications to which the device belongs.

[0820] The collected data is encrypted and transmitted to a server via secure communication.

[0821] Step 4:

[0822] The server receives the encrypted data, decrypts it and maps it to the corresponding user ID.

[0823] The server performs preprocessing to convert the data into a format that is easy to analyze.

[0824] Step 5:

[0825] The server cleans the collected data by removing noise data and filtering out spam and advertising messages.

[0826] The cleaned data is then saved as a dataset for training generative models.

[0827] Step 6:

[0828] The sentiment engine performs sentiment analysis on each message in the collected data and assigns an emotional label to it.

[0829] The emotion engine generates a dataset with emotion labels and sends it to the server.

[0830] Step 7:

[0831] The server trains a generative model using preprocessed data with emotion labels.

[0832] This training involves learning the user's unique tone of voice, expressions, thought patterns and emotions.

[0833] Step 8:

[0834] After the server completes the training, it optimizes the generated model and saves the model that reflects the user's characteristics.

[0835] Step 9:

[0836] After the user's death, the surviving family members can access the "Will AI Service" using a device and enter specific authentication information (e.g., answers to security questions).

[0837] Once authentication is complete, the family members can send questions or messages to the server.

[0838] Step 10:

[0839] The server receives questions and feeds them into a trained model to generate responses that reflect the user's tone, expression, and even emotion.

[0840] Step 11:

[0841] The server then sends the generated response to the family member's device.

[0842] The bereaved family will review this and emotionally understand the wishes and messages of the deceased.

[0843] Through the above processing steps, the "Will AI Service" can utilize the user's digital data and emotions to effectively communicate the user's wishes even after death.

[0844] Example 2

[0845] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0846] Conventional systems have had difficulty reproducing a user's wishes and expressions when the user is absent or after death. Furthermore, they lacked the technology to generate responses that include the user's emotions, resulting in low emotional reproducibility. This has resulted in a lack of a means to convey the wishes and emotions of the deceased to their surviving family members.

[0847] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0848] In this invention, the server includes means for collecting data from users' emails, social network systems, and messaging systems via a network interface, means for decrypting the collected data, mapping it to a user ID, preprocessing it, and removing noise data, means for training a generative model using the preprocessed data to learn the user's tone of voice, expressions, thought patterns, and emotions, means for generating responses that reproduce the user's tone of voice, expressions, thought patterns, and emotions using the trained generative model, and means for providing the generated responses via a network. This makes it possible to reproduce the will and emotions of the deceased and provide emotional messages to the bereaved.

[0849] A "network interface" is a communication means used to collect data from a user's email, social networking system, and messaging system.

[0850] "User's email" refers to electronic message data sent and received by a user over the Internet.

[0851] A "social network system" is a platform through which users interact with others and share messages and posts.

[0852] A "messaging system" is an application that allows users to send and receive text messages in real time.

[0853] "Data collection" is the process of obtaining this message data from users' email, social networking systems, and messaging systems.

[0854] "Decryption" is the process of restoring encrypted data to its original, readable state.

[0855] "User ID mapping" refers to associating collected data with the corresponding user's identifier.

[0856] "Preprocessing" refers to the stage where data is converted into an analyzable form, specifically a process that includes dividing the data into sentence units and adding metadata.

[0857] "Noise data" is data that is not necessary for analysis, and includes spam and advertising messages.

[0858] A "generative model" is a model that uses machine learning techniques to learn a user's tone, expressions, thought patterns, and emotions from training data.

[0859] "Training" is the process of training a generative model using preprocessed data.

[0860] An "emotion label" is a tag that indicates the type of emotion attached to collected data (e.g., joy, sadness, anger).

[0861] "Response generation" is the process of using a trained generative model to create responses that replicate the user's tone, expressions, thought patterns, and emotions.

[0862] "Providing" means sending the generated response in an appropriate format to the terminal of the bereaved family member.

[0863] A "terminal" is a device (e.g., smartphone, computer) that a user or family member uses to access the system, enter data, and receive results.

[0864] An "emotion engine" is a machine learning algorithm or software for extracting emotions from collected data and assigning emotion labels.

[0865] This invention relates to a system that utilizes a user's digital data to reproduce the user's intentions and expressions even when the user is absent or has passed away. This system includes a process for collecting data via a network interface, training a generative model, and generating responses based on the results. It also combines a function for recognizing the user's emotions using an emotion engine and generating responses using that information.

[0866] System configuration

[0867] This system mainly consists of a user, a terminal, a server, and an emotion engine.

[0868] User registration and data collection

[0869] 1. User Registration

[0870] A user accesses the "Will AI Service" using a device (such as a smartphone or PC) and enters personal information such as name, email address, and phone number into a new registration form. The device sends this information to a server, which creates an account for the user. The device then asks the user for permission to access data from email, social network systems, and messaging systems. If consent is granted, the device obtains authentication information from each service and begins collecting data.

[0871] 2. Data Collection

[0872] The device periodically collects past message data from each connected email, social network, and messaging system. This data includes, for example, emails sent and received by the user, social media posts, and chat history. The collected data is encrypted and transmitted to a server via secure communications.

[0873] Data Preprocessing and Filtering

[0874] 3. Data Preprocessing

[0875] The server decrypts the encrypted data and maps it to the corresponding user ID. The server then performs preprocessing to convert the data into a format that is easier to analyze. Specifically, the server splits the text data of emails and chats into sentences and adds metadata (date and time, sender, recipient, etc.).

[0876] 4. Data Filtering

[0877] The server filters out noise data, for example, filtering out spam emails and advertising messages, and stores the filtered data as a dataset for training the generative model.

[0878] Using the Emotion Engine

[0879] 5. Emotion extraction

[0880] The emotion engine performs emotion analysis on each message in the collected data and assigns an emotion label (e.g., joy, sadness, anger, etc.). The emotion engine generates a dataset with the emotion labels and sends it to the server.

[0881] Training a generative model

[0882] 6. Training the model

[0883] The server trains the generative model using preprocessed emotion-labeled data. During the training process, the model learns emotions along with the user's unique tone, expressions, and thought patterns. Once training is complete, the server optimizes and saves the generative model.

[0884] Post-death care

[0885] 7. Response Generation

[0886] After a user's death, the surviving family members access the "Will AI Service" using their devices and enter specific authentication information. Once authentication is complete, the family members send questions or messages to the server. The server then inputs the received questions into a pre-trained generative model to generate responses that reflect the user's tone, expression, and even emotion.

[0887] 8. Response Provision

[0888] The server sends the generated response to the device of the bereaved family, who can read it and confirm the deceased's wishes and messages along with their feelings.

[0889] Specific examples

[0890] For example, if a user had expressed a wish to "distribute my assets equally" during their lifetime, the system would record this wish along with emotional data. After the death, if the surviving family members ask, "How should I distribute my inheritance?", the server would respond by including the user's tone and emotions, and provide the family with the message, "I want my assets to be distributed equally."

[0891] Prompt Sentence Examples

[0892] How should I distribute my inheritance?

[0893] The above is an example of a specific embodiment of this system. This system utilizes the user's digital data and emotions to provide a means for effectively communicating their wishes even after death.

[0894] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0895] Step 1:

[0896] User Registration

[0897] A user accesses the Will AI Service website or app using a device (smartphone or PC). The user enters personal information such as name, email address, and phone number into the new registration form. The device then sends this information to the server, creating an account for the user. The input is personal information, and the output is the creation of a user account.

[0898] Step 2:

[0899] Obtaining data access permissions

[0900] The device asks the user for permission to access data from email, social network systems, and messaging systems. The user confirms and agrees to the permission. The input of this procedure is the permission request, and the output is the user's consent confirmation.

[0901] Step 3:

[0902] Data collection

[0903] Using the user's authentication information, the device collects past message data from each email, social network system, and messaging system. This data includes emails sent and received by the user, social media posts, and chat history. The collected data is encrypted and transmitted to a server via secure communication. The input to this process is authentication information and connection data, and the output is encrypted data.

[0904] Step 4:

[0905] Data Decoding and Preprocessing

[0906] The server receives the encrypted data and decrypts it. It then maps it to the corresponding user ID, splits the email or chat text data into sentences, and adds metadata (date, time, sender, recipient, etc.). The input to this step is the encrypted data and the user ID, and the output is the preprocessed data.

[0907] Step 5:

[0908] Data Filtering

[0909] The server filters out noise data from the preprocessed data. Specifically, it filters out spam emails and advertising messages, leaving only the necessary data. The input of this step is the preprocessed data, and the output is the filtered dataset.

[0910] Step 6:

[0911] emotion extraction

[0912] The server invokes the emotion engine to perform sentiment analysis on each filtered message, and assigns an emotion label (e.g., joy, sadness, anger, etc.) to each message. The inputs of this step are the filtered data and the emotion engine, and the output is emotion-labeled data.

[0913] Step 7:

[0914] Training a generative model

[0915] The server trains the generative AI model using emotion-labeled data. During this training process, it learns the user's tone, expressions, thought patterns, and emotions. Once training is complete, it optimizes and saves the generative model. The input of this step is the labeled data, and the output is the optimized generative model.

[0916] Step 8:

[0917] Identification of bereaved family members and question reception

[0918] After the user's death, the surviving family members access the "Will AI Service" using a device and enter specific authentication information. Once authentication is complete, the surviving family members send questions or messages to the server. The input of this step is the surviving family members' authentication information and questions, and the output is the completion of authentication and receipt of the questions.

[0919] Step 9:

[0920] Response Generation

[0921] The server inputs the received question into a trained generative model to generate a response that reflects the user's tone, expression, and emotion. The input for this step is the family member's question, and the output is the generated response.

[0922] Step 10:

[0923] Response provision

[0924] The server sends the generated response to the bereaved family member's device, who then confirms it. The bereaved family member can receive the user's intentions and messages along with their emotions. The input of this step is the generated response, and the output is the provision of the response to the bereaved family member's device.

[0925] This is the specific process flow of the system.

[0926] (Application example 2)

[0927] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0928] Current home and business security systems have limitations in their ability to detect abnormalities, making it difficult to respond quickly and appropriately when an emergency occurs while the user is away. Furthermore, there is a lack of systems that can automatically generate and provide countermeasures that reflect the user's intentions and intent. This creates the problem of delayed security responses.

[0929] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting data from the user's email, social network service, and chat application via a network interface, means for preprocessing the collected data and removing noise data, and means for training a generative AI model that learns the user's tone of voice, expressions, and thought patterns using the preprocessed data. This makes it possible to detect abnormalities with high accuracy even when the user is absent, and to automatically generate and provide optimal countermeasures that reflect the user's intentions and will.

[0930] A "network interface" refers to the hardware and software that connects a computer or data collection device to a network and enables data communication.

[0931] "Email" refers to a digital communication method for sending and receiving messages over the Internet or other computer networks.

[0932] "Social network service" refers to an online platform that enables people to share information and interact with each other over the Internet.

[0933] A "chat application" refers to software for exchanging messages in real time, including text, voice, and video.

[0934] "Data preprocessing" refers to the process of converting collected raw data into a format suitable for analysis and machine learning.

[0935] "Noise data" refers to data that contains information that is unnecessary or misleading for analytical or learning purposes.

[0936] A "generative AI model" is a machine learning model that learns a user's unique tone of voice and expression patterns from given data and generates new responses and content.

[0937] "Training" refers to the process of showing a machine learning model large amounts of data and having it learn its patterns and features.

[0938] "Anomaly detection" refers to techniques and processes that automatically detect data or situations that deviate from normal patterns.

[0939] A "countermeasure" is a specific course of action or proposal for dealing with a particular situation.

[0940] The present invention provides a system for automatically taking a response that reflects the user's intentions even when the user is absent in a home or business security system. The system includes the following means.

[0941] 1. User Registration and Data Collection

[0942] The server collects personal information provided by users via a network interface. Users use devices such as smartphones or PCs to register basic information such as their name, address, and household members. The server also connects with various security devices (cameras, door locks, voice assistants, etc.) and obtains access permissions to the data generated by these devices. This allows the server to centrally collect situational data both inside and outside the home.

[0943] 2. Data preprocessing and filtering

[0944] The server preprocesses the collected data and removes noise data that is unsuitable for analysis. This preprocessing includes dividing text data into sentences, converting audio data into text, and extracting important frames from video data. It also uses spam filtering to remove irrelevant and noise data.

[0945] 3. Emotion extraction using an emotion engine

[0946] The server uses an emotion engine to analyze the collected data and assigns an emotion label to each message or action, allowing it to understand the user's daily emotional patterns and identify emotional changes when an anomaly occurs.

[0947] 4. Training the generative AI model

[0948] The server uses the preprocessed data to train the generative AI model. This training process learns the user's unique tone, expressions, and thought patterns, allowing the system to reproduce the user's intentions and emotions. Anomaly detection algorithms are also trained at this stage, allowing them to distinguish between normal and abnormal.

[0949] 5. Anomaly Detection and Response Generation

[0950] If an abnormality is detected while the user is absent, the server will automatically generate the optimal countermeasure using a trained generative AI model. For example, if suspicious activity is detected, the security robot can ask, "There is suspicious behavior. Is there something I can help you with?", and the response will reproduce the user's unique tone and expressions.

[0951] 6. Response Provision

[0952] The server then sends the generated responses and countermeasures to the user's device and related security devices via the network, allowing the user to check the status of their home even when they are away from home and implement the automatically generated countermeasures as needed.

[0953] Specific examples

[0954] For example, if a user inputs a prompt question such as "There has been suspicious activity in the house recently. What should we do?" into a generative AI model, the server can generate a response such as "Suspicious activity has been detected. Would you like to contact the police?" This response reflects the user's tone of voice and thought patterns, making it possible to take appropriate action based on the user's intentions even when the user is not present.

[0955] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0956] Step 1:

[0957] The user uses the terminal to register personal information, including basic information such as name, address, and household member composition. The registered information is sent to the server via the network interface. The input data is the personal information entered by the user, and the output data is the user information stored in the database.

[0958] Step 2:

[0959] After obtaining user approval, the server collects data from each security device (camera, door lock, voice assistant, etc.). The collected data is periodically transferred to the server. The input data is the raw data obtained from the security device, and the output data is the collected data before preprocessing.

[0960] Step 3:

[0961] The server preprocesses the collected data. Specifically, it splits text data into sentences, converts audio data to text, and extracts important frames from video data. It also performs spam filtering and removes noise data. The input data is a raw data stream, and the output data is a preprocessed dataset.

[0962] Step 4:

[0963] The server analyzes the preprocessed data using an emotion engine and assigns emotion labels to each message and action. This allows the user's daily emotion patterns to be analyzed. The input data is the preprocessed data, and the output data is the data with the emotion labels.

[0964] Step 5:

[0965] The server uses the emotion-labeled data to train the generative AI model. This training process learns the user's unique tone, expressions, and thought patterns. The input data is the emotion-labeled dataset, and the output data is the trained generative AI model.

[0966] Step 6:

[0967] The server detects anomalies when the user is absent. If an anomaly is detected, a trained generative AI model is used to generate optimal countermeasures. For example, if suspicious activity is detected, the security robot will ask, "There is suspicious behavior. Is there anything we can help you with?" The input data is real-time data used to detect anomalies, and the output data is the generated countermeasures.

[0968] Step 7:

[0969] The server sends the generated countermeasures to the user's terminal and related security devices via the network. This allows the user to check the status of their home even when they are away from home and implement the automatically generated countermeasures as necessary. The input data is the generated countermeasures, and the output data is a response message notified to the user.

[0970] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0971] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0972] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0973] [Third embodiment]

[0974] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0975] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0976] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0977] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0978] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0979] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0980] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0981] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0982] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0983] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0984] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0985] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0986] This invention relates to a system that utilizes a user's digital data to reproduce the user's intentions and expressions even when the user is absent or has passed away. This system includes a process that collects data via a network interface, trains a generative model, and generates responses based on the results.

[0987] A specific embodiment of this system will be described below.

[0988] System configuration

[0989] This system consists of users, terminals, and servers.

[0990] User registration and data collection

[0991] 1. User Registration

[0992] A user accesses the "Will AI Service" using a device (such as a smartphone or PC) and enters personal information such as name, email address, and phone number into the new registration form. The device then sends this information to the server, which creates an account for the user.

[0993] The device then asks the user for permission to access data from email, social networking services, and chat applications. If consent is granted, the device obtains authentication information from each service and begins collecting data.

[0994] 2. Data Collection

[0995] The device periodically collects past message data from each connected email, social networking service, and chat application, including, for example, emails sent and received by the user, social media posts, and chat history.

[0996] The collected data is encrypted and transferred to a server via secure communications.

[0997] Data Preprocessing and Filtering

[0998] 3. Data Preprocessing

[0999] The server decrypts the encrypted data and maps it to the corresponding user ID.

[1000] The server performs preprocessing to convert the data into a format that is easier to analyze. Specifically, the text data of emails and chats is divided into sentences and metadata (date and time, sender, recipient, etc.) is added.

[1001] 4. Data Filtering

[1002] The server filters out noise data, for example, filtering out spam emails and advertising messages, eliminating unnecessary data.

[1003] The filtered data is saved as a dataset for training a generative model.

[1004] Training a generative model

[1005] 5. Training the model

[1006] The server uses the preprocessed data to train a generative model (e.g., a GPT model). During the training process, the model learns the user's unique tone, expressions, and thought patterns.

[1007] Once training is complete, the server optimizes and stores the generative model.

[1008] Post-death care

[1009] 6. Response Generation

[1010] After the death of a user, the family members access the "Will AI Service" using a device and enter specific authentication information. Once authentication is complete, the family members can send questions or messages to the server.

[1011] The server inputs the received question into a trained model to generate a response that reproduces the user's tone and expression.

[1012] 7. Response Provision

[1013] The server sends the generated response to the family's device, where they can read it and check the deceased's wishes and messages.

[1014] Specific examples

[1015] For example, if a user had expressed a wish to "distribute my assets equally" during their lifetime, a response reflecting this wish will be generated from the collected data and the learning results of the generative model. When a surviving family member asks, "How should I distribute my inheritance?", the server generates the response "I want my assets to be distributed equally" and provides it to the surviving family member.

[1016] The above is an example of a specific embodiment of this system. By utilizing the user's digital data, the wishes and expressions of the deceased can be reproduced even after death, making it easier for the bereaved to maintain communication with the deceased.

[1017] The processing flow will be explained below.

[1018] Step 1:

[1019] The user accesses the "Will AI Service" using a device and enters personal information into the new registration form.

[1020] The device sends this information to a server, which creates an account for the user.

[1021] Step 2:

[1022] The device will ask the user for permission to access data from email, social networking services, and chat applications.

[1023] Once the user agrees, the device will obtain the necessary authentication information from each service and begin collecting data.

[1024] Step 3:

[1025] The device periodically collects past message data from email, social networking services, and chat applications.

[1026] The collected data is encrypted and transmitted to a server via secure communication.

[1027] Step 4:

[1028] The server receives the encrypted data, decrypts it and maps it to the corresponding user ID.

[1029] The server performs preprocessing to convert the data into a format that is easy to analyze.

[1030] Step 5:

[1031] The server cleans the collected data by removing noise data and filtering out spam and advertising messages.

[1032] The cleaned data is then saved as a dataset for training generative models.

[1033] Step 6:

[1034] The server uses the preprocessed data to train a generative model.

[1035] This training process involves learning the user's tone of voice, expressions, and thought patterns.

[1036] Step 7:

[1037] The server optimizes the trained generative model and stores the model that reflects the user's characteristics.

[1038] Step 8:

[1039] After the user's death, the surviving family members can use a device to access the "Will AI Service" and enter specific authentication information (such as answers to security questions).

[1040] Once authentication is complete, the family members can send questions or messages to the server.

[1041] Step 9:

[1042] The server inputs the received question into a trained model to generate a response that replicates the user's tone and expressions.

[1043] Step 10:

[1044] The server then sends the generated response to the family member's device.

[1045] The bereaved family will review this and understand the wishes and messages of the deceased.

[1046] Through the above processing steps, the "Will AI Service" can utilize the user's digital data to effectively communicate the user's wishes even after death.

[1047] Example 1

[1048] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1049] There has been no system that can convey a user's wishes and expressions to their surviving family members even after their absence or death. Providing such a system would enable a user's wishes and emotions to be reproduced after death, making it easier for surviving family members to maintain communication with the deceased. It is also necessary to securely collect, preprocess, and filter user data, and utilize generative AI models to reproduce the user's characteristics.

[1050] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1051] In this invention, the server includes means for collecting data from a user's electronic messaging service, social networking platform, and text communication application via a network interface, means for preprocessing the collected data and removing noise data, means for training a generative AI model that learns the user's tone of voice, expressions, and thought patterns using the preprocessed data, means for generating responses that reproduce the user's tone of voice, expressions, and thought patterns using the trained generative AI model, means for providing the generated responses via a network, means for a user to input specific authentication information via a terminal, means for a surviving family member who has completed authentication to send a question or message, means for generating responses that reflect the user's intentions and expressions using the trained generative AI model, and means for transmitting the generated responses to the surviving family member's terminal. This allows the user's digital data to be utilized to reproduce the user's intentions and emotions even after the user's death, enabling the surviving family member to maintain communication with the deceased.

[1052] A "network interface" is an interface for sending and receiving data via various networks.

[1053] "Electronic messaging service" means a service for sending and receiving digital messages, such as email.

[1054] A "social networking platform" is an online platform that enables users to connect and share information with friends and acquaintances.

[1055] A "text communication application" is an application for chatting and sending and receiving messages.

[1056] "Means for collecting data" refers to means consisting of software and hardware for collecting digital data of users.

[1057] The "means for preprocessing and removing noise data" refers to a means for filtering out unnecessary parts from collected data and converting it into a form that is easy to analyze.

[1058] A "generative AI model" is an artificial intelligence model that learns from specific data and reproduces the user's tone of voice, expressions, and thought patterns.

[1059] The "means for generating a response" is a means for generating a response that reflects the characteristics of the user using a trained generative AI model.

[1060] The "means for inputting specific authentication information" refers to a means for inputting authentication information required when a user accesses a service.

[1061] "Means for sending questions or messages" refers to means by which the user or the bereaved family members can send questions or messages to the server.

[1062] The "means for transmitting a response to a terminal of a bereaved family member" is a means for transmitting the generated response to a terminal used by the bereaved family member.

[1063] This invention relates to a system that utilizes a user's digital data to reproduce the user's intentions and expressions even when the user is absent or after death. This system is composed of a user, a terminal, and a server.

[1064] User registration and data collection

[1065] User Registration

[1066] A user accesses the "Will AI Service" using a device (such as a smartphone or PC) and enters personal information such as name, email address, and phone number into a new registration form. The device sends this information to a server, which creates an account for the user. The device then requests permission from the user to access data from email, social networking services, and chat applications. If consent is granted, the device obtains authentication information from each service and begins collecting data.

[1067] Data collection

[1068] We periodically collect historical message data from each electronic messaging service, social networking platform, and text communication application to which your device is connected. This data may include, for example, emails sent and received by you, social media posts, and chat history. The collected data is encrypted and transmitted to our servers via secure communications.

[1069] Data Preprocessing and Filtering

[1070] Data Preprocessing

[1071] The server decrypts the encrypted data and maps it to the corresponding user ID. The server then performs preprocessing to convert the data into a format that is easier to analyze. Specifically, the server splits the text data of emails and chats into sentences and adds metadata (date and time, sender, recipient, etc.).

[1072] Data Filtering

[1073] The server filters out noise data such as spam emails and advertising messages, and the filtered data is stored as a training dataset for generative AI models.

[1074] Training a generative model

[1075] Training the model

[1076] The server uses the preprocessed data to train a generative AI model (such as a GPT model). During this process, it learns the user's unique tone, expressions, and thought patterns. Once training is complete, the server optimizes and saves the generative AI model.

[1077] Response generation and delivery

[1078] Authenticate and enter your question

[1079] After the death of a user, the family members access the "Will AI Service" using a device and enter specific authentication information. Once authentication is complete, the family members can send questions or messages to the server.

[1080] Response Generation

[1081] The server inputs the received questions into a trained generative AI model to generate responses that replicate the user's tone and expressions.

[1082] Response provision

[1083] The server sends the generated response to the family's device, where they can read it and confirm the deceased's wishes and messages.

[1084] Examples and prompts

[1085] For example, if a user had expressed a wish to "distribute my assets equally" during their lifetime, a response reflecting this wish will be generated from the collected data and the learning results of the generative AI model. When a surviving family member asks, "How should I distribute my inheritance?", the server will generate the response "I want my assets to be distributed equally" and provide it to the surviving family member.

[1086] Example prompt:

[1087] User: How should assets be distributed?

[1088] Server: I want my assets to be divided equally.

[1089] The above is a specific embodiment of this system. By utilizing digital data to reproduce the user's intentions and emotions, it becomes possible for the bereaved to maintain communication with the deceased.

[1090] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1091] Step 1: User enters personal information into a sign-up form

[1092] Specific actions

[1093] The user accesses the "Will AI Service" using a device (smartphone or PC) and enters personal information such as name, email address, and phone number into the new registration form.

[1094] input

[1095] Personal information such as your name, email address, and phone number.

[1096] Data processing or data calculation

[1097] The terminal transmits the entered personal information to the server.

[1098] output

[1099] The user's personal information is registered on the server and an account is created for the user.

[1100] Step 2: Obtaining data access permission

[1101] Specific actions

[1102] The device will ask the user for permission to access data from email, social networking services, and chat applications.

[1103] input

[1104] User consent information.

[1105] Data processing or data calculation

[1106] If the user agrees, the device obtains authentication information from each service.

[1107] output

[1108] The obtained credentials.

[1109] Step 3: Performing data collection

[1110] Specific actions

[1111] Periodically collect historical message data from each electronic messaging service, social networking platform, and text communication application to which the device is connected.

[1112] input

[1113] User message data from each service.

[1114] Data processing or data calculation

[1115] The device encrypts the data.

[1116] output

[1117] The encrypted message data.

[1118] Step 4: Encrypt and transfer data

[1119] Specific actions

[1120] The encrypted data is transferred to the server via secure communication.

[1121] input

[1122] The encrypted message data.

[1123] Data processing or data calculation

[1124] Data transfer via secure communications.

[1125] output

[1126] Encrypted data stored on the server.

[1127] Step 5: Decrypt and preprocess the data

[1128] Specific actions

[1129] The server decrypts the encrypted data and maps it to the corresponding user ID.

[1130] input

[1131] Encrypted data stored on the server.

[1132] Data processing or data calculation

[1133] Preprocessing is performed to convert the data into a form that is easier to analyze, and text data from emails and chats is divided into sentences and metadata is added.

[1134] output

[1135] Decrypted data and preprocessed data with metadata added.

[1136] Step 6: Remove noise data

[1137] Specific actions

[1138] The server filters out noise data such as spam emails and advertising messages.

[1139] input

[1140] Preprocessed data.

[1141] Data processing or data calculation

[1142] Spam filtering and noise data removal.

[1143] output

[1144] Clean data with noise removed.

[1145] Step 7: Training the generative AI model

[1146] Specific actions

[1147] The server uses the preprocessed data to train a generative AI model (e.g., a GPT model).

[1148] input

[1149] Clean data with noise removed.

[1150] Data processing or data calculation

[1151] The generative AI model learns the user's unique tone of voice, expressions, and thought patterns.

[1152] output

[1153] A trained generative AI model.

[1154] Step 8: Authenticate and enter your questions

[1155] Specific actions

[1156] After the user's death, the surviving family members use a device to access the "Will AI Service" and enter specific authentication information.

[1157] input

[1158] Family member credentials.

[1159] Data processing or data calculation

[1160] Once authentication is complete, the family members can send questions or messages to the server.

[1161] output

[1162] Questions and messages from bereaved families whose authentication has been completed.

[1163] Step 9: Question processing and response generation

[1164] Specific actions

[1165] The server inputs the received questions into a trained generative AI model to generate responses that replicate the user's tone and expressions.

[1166] input

[1167] Questions and messages from the bereaved families.

[1168] Data processing or data calculation

[1169] The generative AI model generates responses that reflect the user's characteristics.

[1170] output

[1171] The generated response.

[1172] Step 10: Provide a response

[1173] Specific actions

[1174] The server sends the generated response to the terminal of the bereaved family member.

[1175] input

[1176] The generated response.

[1177] Data processing or data calculation

[1178] Sends the response over secure communications.

[1179] output

[1180] The response arrived on the family member's device.

[1181] (Application example 1)

[1182] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1183] In today's digital society, there is a demand for technology that can reproduce a user's intentions and expressions even when the user is absent or has died. However, conventional technologies have difficulty accurately reproducing a user's tone of voice, expressions, and thought patterns, making it difficult to provide personalized responses tailored to each individual user. In particular, in virtual stores, it is necessary to improve customer satisfaction by retaining a user's unique response patterns and automating customer responses.

[1184] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1185] In this invention, the server includes means for collecting data from a user's email, social media services, and real-time messaging applications via a network interface, means for preprocessing the collected data and removing noise data, means for training a generative AI model that learns the user's tone of voice, expressions, and thought patterns using the preprocessed data, means for generating responses that reproduce the user's tone of voice, expressions, and thought patterns using the trained generative AI model, means for providing the generated responses via a network, means for reproducing the user's will and expressions even when the user is absent or after death, and means for learning the user's unique response patterns and providing personalized customer service in a virtual store. This makes it possible to accurately reproduce the user's unique responses even when the user is absent or after death, and to automate personalized customer service in the virtual store.

[1186] A "network interface" is a communication means that allows for the collection, transmission, and reception of digital data.

[1187] A "user" is an entity, such as an individual or a corporation, that registers and provides information.

[1188] "Data collection methods" are mechanisms for obtaining information from email, social media services, real-time messaging applications, etc.

[1189] "Preprocessing" refers to the process of converting collected data into a format that is easy to analyze and removing noise data.

[1190] "Noise data" is unnecessary data or information that reduces the accuracy of analysis or processing.

[1191] A "generative AI model" is an artificial intelligence model that learns from training data and reproduces the user's tone of voice and expressions.

[1192] A "training method" is the process of training a generative AI model using preprocessed data.

[1193] A "response generation means" is a mechanism for generating responses that reflect the user's characteristics using a trained generative AI model.

[1194] The "means for providing via a network" refers to a means for transmitting the generated response to a user or other terminal via a communication network such as the Internet.

[1195] "Means to reproduce the user's will and expression" refers to a function that generates responses by imitating the user's unique words, actions, and intentions even when the user is absent or after death.

[1196] A "virtual store" is a virtual store that offers products and services over the Internet.

[1197] "Personalization" means providing optimal responses and services based on the characteristics and preferences of individual users.

[1198] This invention relates to a system that utilizes a user's digital data to reproduce their will and expressions even when they are absent or have passed away. This system mainly collects data via a network interface, preprocesses the collected data, and trains a generative AI model to learn the user's characteristics and generate responses based on the results.

[1199] The system basically consists of users, terminals, and servers.

[1200] 1. User Registration and Data Collection

[1201] A user accesses the system using a device (such as a smartphone or personal computer), enters personal information into a new registration form, and creates an account. The device then asks the user for permission to access data from email, social media, and real-time messaging applications. If the user agrees, the device obtains authentication information from each service and begins collecting data.

[1202] 2. Data preprocessing and filtering

[1203] The server receives the encrypted data, decrypts it, and maps it to a user ID. Next, the server performs data preprocessing to convert the data into a format that is easier to analyze. Specifically, the server breaks down emails and chat messages into sentences and adds metadata (date, time, sender, recipient, etc.). Spam filtering is then performed to remove noise data.

[1204] 3. Training the generative AI model

[1205] The server uses the preprocessed data to train a generative AI model (e.g., GPT-2). During the training process, the user's unique tone, expressions, and thought patterns are learned. Once training is complete, the generative model is optimized and saved.

[1206] 4. What to do when the user is absent

[1207] When the user is absent or after death, the trained generative AI model is utilized to handle customer service in the virtual store. Specifically, the surviving family members or customers access the system using their devices and send questions or messages. The server inputs the questions into the trained model and generates a response that reproduces the user's characteristics. This response is then sent to the surviving family members or customers' devices.

[1208] Hardware and software used

[1209] The system uses the following hardware and software:

[1210] Hardware: Server with GPU (e.g. NVIDIA Tesla)

[1211] Software: Python, transformers library, GPT-2 model

[1212] Specific examples

[1213] For example, if a user had expressed a wish to "divide my assets equally" during their lifetime, a response reflecting this wish will be generated from the collected data and the learning results of the generative model. When a family member asks, "How should I distribute my inheritance?", the server generates a response such as "I want my assets to be divided equally" and provides it to the family member. Similarly, when a customer in a virtual store asks, "What are today's recommended products?", an answer reflecting the user's unique response patterns that have been learned will be provided.

[1214] Prompt Sentence Examples

[1215] Generate a response when a user asks, "What's your featured product today?"

[1216] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1217] Step 1:

[1218] A user accesses the system using a terminal and creates an account by entering personal information such as name, email address, and phone number in a new registration form. The terminal then sends this personal information to the server, generating an account for the user.

[1219] Step 2:

[1220] The device asks the user for permission to access data from email, social media services, and real-time messaging applications. If the user agrees, the device obtains authentication information from each service and begins continuously collecting data.

[1221] Input: User consent, authentication information

[1222] Output: Collected data (emails, social media posts, message history)

[1223] Step 3:

[1224] The device encrypts the data it collects and sends it over secure connections to a server, which receives the data, decrypts it, and maps it to a user ID.

[1225] Input: Encrypted data

[1226] Output: Decrypted data, data mapped to user ID

[1227] Step 4:

[1228] The server preprocesses the decrypted data and converts it into a format that is easy to analyze. Specifically, it divides it into sentences and adds metadata (date, time, sender, recipient, etc.). Spam filtering is also performed to remove noise data.

[1229] Input: Decrypted data

[1230] Output: Preprocessed data, denoised data

[1231] Step 5:

[1232] The server uses the preprocessed data to train a generative AI model (e.g., GPT-2), which learns the user's unique tone, expressions, and thought patterns, and then optimizes and saves the generative AI model.

[1233] Input: Preprocessed data

[1234] Output: A trained generative AI model

[1235] Step 6:

[1236] When the user is absent or after death, the server utilizes the generative AI model to handle customer service in the virtual store. The server receives questions and messages from customers (or their families) and inputs this information into the generative AI model.

[1237] Input: Questions or messages from the customer (or their family)

[1238] Output: The prompts fed into the AI ​​model

[1239] Step 7:

[1240] The server uses a generative AI model to generate responses that mimic the user's tone and expressions, creating an appropriate response to the question.

[1241] Input: The prompt entered into the AI ​​model

[1242] Output: The generated response

[1243] Step 8:

[1244] The server then sends the generated response to the customer's (or the bereaved family's) terminal via the network, enabling personalized responses that reflect the user's characteristics.

[1245] Input: Generated response

[1246] Output: Response sent to the customer's (or their family's) device

[1247] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1248] This invention relates to a system that utilizes a user's digital data to reproduce the user's intentions and expressions even when the user is absent or has passed away. This system includes a process for collecting data via a network interface, training a generative model, and generating responses based on the results. It also combines a function that recognizes the user's emotions using an emotion engine and generates responses using that information.

[1249] A specific embodiment of this system will be described below.

[1250] System configuration

[1251] This system consists of a user, a terminal, a server, and an emotion engine.

[1252] User registration and data collection

[1253] 1. User Registration

[1254] A user accesses the "Will AI Service" using a device (such as a smartphone or PC) and enters personal information such as name, email address, and phone number into the new registration form. The device then sends this information to the server, which creates an account for the user.

[1255] The device then asks the user for permission to access data from email, social networking services, and chat applications. If consent is granted, the device obtains authentication information from each service and begins collecting data.

[1256] 2. Data Collection

[1257] The device periodically collects past message data from each connected email, social networking service, and chat application, including, for example, emails sent and received by the user, social media posts, and chat history.

[1258] The collected data is encrypted and transferred to a server via secure communications.

[1259] Data Preprocessing and Filtering

[1260] 3. Data Preprocessing

[1261] The server decrypts the encrypted data and maps it to the corresponding user ID.

[1262] The server performs preprocessing to convert the data into a format that is easier to analyze. Specifically, the text data of emails and chats is divided into sentences and metadata (date and time, sender, recipient, etc.) is added.

[1263] 4. Data Filtering

[1264] The server filters out noise data, for example, filtering out spam emails and advertising messages, eliminating unnecessary data.

[1265] The filtered data is saved as a dataset for training a generative model.

[1266] Using the Emotion Engine

[1267] 5. Emotion extraction

[1268] The emotion engine performs emotion analysis on each message in the collected data and assigns an emotion label (e.g., joy, sadness, anger, etc.).

[1269] The emotion engine generates a dataset with emotion labels and sends it to the server.

[1270] Training a generative model

[1271] 6. Training the model

[1272] The server trains the generative model using pre-processed emotion-labeled data. During the training process, the model learns emotions along with the user's unique tone, expressions, and thought patterns.

[1273] Once training is complete, the server optimizes and stores the generative model.

[1274] Post-death care

[1275] 7. Response Generation

[1276] After the death of a user, the family members access the "Will AI Service" using a device and enter specific authentication information. Once authentication is complete, the family members can send questions or messages to the server.

[1277] The server inputs the received question into a trained generative model to generate a response that reflects the user's tone, expression, and even emotion.

[1278] 8. Response Provision

[1279] The server sends the generated response to the device of the bereaved family, who can read it and confirm the deceased's wishes and messages along with their feelings.

[1280] Specific examples

[1281] For example, if a user had expressed a wish to "divide my assets equally" during his or her lifetime, the system would record this wish along with emotional data. After the user's death, if the surviving family members ask, "How should I distribute my inheritance?", the server would respond with "I want my assets to be divided equally," incorporating the user's tone and emotions, and provide this to the surviving family members.

[1282] The above is an example of a specific embodiment of this system. This system utilizes the user's digital data and emotions to provide a means for effectively communicating their wishes even after death.

[1283] The processing flow will be explained below.

[1284] Step 1:

[1285] The user accesses the "Will AI Service" using a device and enters personal information (such as name, email address, and phone number) into the new registration form.

[1286] The device sends this information to a server, which creates an account for the user.

[1287] Step 2:

[1288] The device will ask the user for permission to access data from email, social networking services, and chat applications.

[1289] Once the user agrees, the device will obtain authentication information from each service and begin collecting data.

[1290] Step 3:

[1291] It periodically collects past message data from email, social networking services, and chat applications to which the device belongs.

[1292] The collected data is encrypted and transmitted to a server via secure communication.

[1293] Step 4:

[1294] The server receives the encrypted data, decrypts it and maps it to the corresponding user ID.

[1295] The server performs preprocessing to convert the data into a format that is easy to analyze.

[1296] Step 5:

[1297] The server cleans the collected data by removing noise data and filtering out spam and advertising messages.

[1298] The cleaned data is then saved as a dataset for training generative models.

[1299] Step 6:

[1300] The sentiment engine performs sentiment analysis on each message in the collected data and assigns an emotional label to it.

[1301] The emotion engine generates a dataset with emotion labels and sends it to the server.

[1302] Step 7:

[1303] The server trains a generative model using preprocessed data with emotion labels.

[1304] This training involves learning the user's unique tone of voice, expressions, thought patterns and emotions.

[1305] Step 8:

[1306] After the server completes the training, it optimizes the generated model and saves the model that reflects the user's characteristics.

[1307] Step 9:

[1308] After the user's death, the surviving family members can access the "Will AI Service" using a device and enter specific authentication information (e.g., answers to security questions).

[1309] Once authentication is complete, the family members can send questions or messages to the server.

[1310] Step 10:

[1311] The server receives questions and feeds them into a trained model to generate responses that reflect the user's tone, expression, and even emotion.

[1312] Step 11:

[1313] The server then sends the generated response to the family member's device.

[1314] The bereaved family will review this and emotionally understand the wishes and messages of the deceased.

[1315] Through the above processing steps, the "Will AI Service" can utilize the user's digital data and emotions to effectively communicate the user's wishes even after death.

[1316] Example 2

[1317] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1318] Conventional systems have had difficulty reproducing a user's wishes and expressions when the user is absent or after death. Furthermore, they lacked the technology to generate responses that include the user's emotions, resulting in low emotional reproducibility. This has resulted in a lack of a means to convey the wishes and emotions of the deceased to their surviving family members.

[1319] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1320] In this invention, the server includes means for collecting data from users' emails, social network systems, and messaging systems via a network interface, means for decrypting the collected data, mapping it to a user ID, preprocessing it, and removing noise data, means for training a generative model using the preprocessed data to learn the user's tone of voice, expressions, thought patterns, and emotions, means for generating responses that reproduce the user's tone of voice, expressions, thought patterns, and emotions using the trained generative model, and means for providing the generated responses via a network. This makes it possible to reproduce the will and emotions of the deceased and provide emotional messages to the bereaved.

[1321] A "network interface" is a communication means used to collect data from a user's email, social networking system, and messaging system.

[1322] "User's email" refers to electronic message data sent and received by a user over the Internet.

[1323] A "social network system" is a platform through which users interact with others and share messages and posts.

[1324] A "messaging system" is an application that allows users to send and receive text messages in real time.

[1325] "Data collection" is the process of obtaining this message data from users' email, social networking systems, and messaging systems.

[1326] "Decryption" is the process of restoring encrypted data to its original, readable state.

[1327] "User ID mapping" refers to associating collected data with the corresponding user's identifier.

[1328] "Preprocessing" refers to the stage where data is converted into an analyzable form, specifically a process that includes dividing the data into sentence units and adding metadata.

[1329] "Noise data" is data that is not necessary for analysis, and includes spam and advertising messages.

[1330] A "generative model" is a model that uses machine learning techniques to learn a user's tone, expressions, thought patterns, and emotions from training data.

[1331] "Training" is the process of training a generative model using preprocessed data.

[1332] An "emotion label" is a tag that indicates the type of emotion attached to collected data (e.g., joy, sadness, anger).

[1333] "Response generation" is the process of using a trained generative model to create responses that replicate the user's tone, expressions, thought patterns, and emotions.

[1334] "Providing" means sending the generated response in an appropriate format to the terminal of the bereaved family member.

[1335] A "terminal" is a device (e.g., smartphone, computer) that a user or family member uses to access the system, enter data, and receive results.

[1336] An "emotion engine" is a machine learning algorithm or software for extracting emotions from collected data and assigning emotion labels.

[1337] This invention relates to a system that utilizes a user's digital data to reproduce the user's intentions and expressions even when the user is absent or has passed away. This system includes a process for collecting data via a network interface, training a generative model, and generating responses based on the results. It also combines a function for recognizing the user's emotions using an emotion engine and generating responses using that information.

[1338] System configuration

[1339] This system mainly consists of a user, a terminal, a server, and an emotion engine.

[1340] User registration and data collection

[1341] 1. User Registration

[1342] A user accesses the "Will AI Service" using a device (such as a smartphone or PC) and enters personal information such as name, email address, and phone number into a new registration form. The device sends this information to a server, which creates an account for the user. The device then asks the user for permission to access data from email, social network systems, and messaging systems. If consent is granted, the device obtains authentication information from each service and begins collecting data.

[1343] 2. Data Collection

[1344] The device periodically collects past message data from each connected email, social network, and messaging system. This data includes, for example, emails sent and received by the user, social media posts, and chat history. The collected data is encrypted and transmitted to a server via secure communications.

[1345] Data Preprocessing and Filtering

[1346] 3. Data Preprocessing

[1347] The server decrypts the encrypted data and maps it to the corresponding user ID. The server then performs preprocessing to convert the data into a format that is easier to analyze. Specifically, the server splits the text data of emails and chats into sentences and adds metadata (date and time, sender, recipient, etc.).

[1348] 4. Data Filtering

[1349] The server filters out noise data, for example, filtering out spam emails and advertising messages, and stores the filtered data as a dataset for training the generative model.

[1350] Using the Emotion Engine

[1351] 5. Emotion extraction

[1352] The emotion engine performs emotion analysis on each message in the collected data and assigns an emotion label (e.g., joy, sadness, anger, etc.). The emotion engine generates a dataset with the emotion labels and sends it to the server.

[1353] Training a generative model

[1354] 6. Training the model

[1355] The server trains the generative model using preprocessed emotion-labeled data. During the training process, the model learns emotions along with the user's unique tone, expressions, and thought patterns. Once training is complete, the server optimizes and saves the generative model.

[1356] Post-death care

[1357] 7. Response Generation

[1358] After a user's death, the surviving family members access the "Will AI Service" using their devices and enter specific authentication information. Once authentication is complete, the family members send questions or messages to the server. The server then inputs the received questions into a pre-trained generative model to generate responses that reflect the user's tone, expression, and even emotion.

[1359] 8. Response Provision

[1360] The server sends the generated response to the device of the bereaved family, who can read it and confirm the deceased's wishes and messages along with their feelings.

[1361] Specific examples

[1362] For example, if a user had expressed a wish to "distribute my assets equally" during their lifetime, the system would record this wish along with emotional data. After the death, if the surviving family members ask, "How should I distribute my inheritance?", the server would respond by including the user's tone and emotions, and provide the family with the message, "I want my assets to be distributed equally."

[1363] Prompt Sentence Examples

[1364] How should I distribute my inheritance?

[1365] The above is an example of a specific embodiment of this system. This system utilizes the user's digital data and emotions to provide a means for effectively communicating their wishes even after death.

[1366] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1367] Step 1:

[1368] User Registration

[1369] A user accesses the Will AI Service website or app using a device (smartphone or PC). The user enters personal information such as name, email address, and phone number into the new registration form. The device then sends this information to the server, creating an account for the user. The input is personal information, and the output is the creation of a user account.

[1370] Step 2:

[1371] Obtaining data access permissions

[1372] The device asks the user for permission to access data from email, social network systems, and messaging systems. The user confirms and agrees to the permission. The input of this procedure is the permission request, and the output is the user's consent confirmation.

[1373] Step 3:

[1374] Data collection

[1375] Using the user's authentication information, the device collects past message data from each email, social network system, and messaging system. This data includes emails sent and received by the user, social media posts, and chat history. The collected data is encrypted and transmitted to a server via secure communication. The input to this process is authentication information and connection data, and the output is encrypted data.

[1376] Step 4:

[1377] Data Decoding and Preprocessing

[1378] The server receives the encrypted data and decrypts it. It then maps it to the corresponding user ID, splits the email or chat text data into sentences, and adds metadata (date, time, sender, recipient, etc.). The input to this step is the encrypted data and the user ID, and the output is the preprocessed data.

[1379] Step 5:

[1380] Data Filtering

[1381] The server filters out noise data from the preprocessed data. Specifically, it filters out spam emails and advertising messages, leaving only the necessary data. The input of this step is the preprocessed data, and the output is the filtered dataset.

[1382] Step 6:

[1383] emotion extraction

[1384] The server invokes the emotion engine to perform sentiment analysis on each filtered message, and assigns an emotion label (e.g., joy, sadness, anger, etc.) to each message. The inputs of this step are the filtered data and the emotion engine, and the output is emotion-labeled data.

[1385] Step 7:

[1386] Training a generative model

[1387] The server trains the generative AI model using emotion-labeled data. During this training process, it learns the user's tone, expressions, thought patterns, and emotions. Once training is complete, it optimizes and saves the generative model. The input of this step is the labeled data, and the output is the optimized generative model.

[1388] Step 8:

[1389] Identification of bereaved family members and question reception

[1390] After the user's death, the surviving family members access the "Will AI Service" using a device and enter specific authentication information. Once authentication is complete, the surviving family members send questions or messages to the server. The input of this step is the surviving family members' authentication information and questions, and the output is the completion of authentication and receipt of the questions.

[1391] Step 9:

[1392] Response Generation

[1393] The server inputs the received question into a trained generative model to generate a response that reflects the user's tone, expression, and emotion. The input for this step is the family member's question, and the output is the generated response.

[1394] Step 10:

[1395] Response provision

[1396] The server sends the generated response to the bereaved family member's device, who then confirms it. The bereaved family member can receive the user's intentions and messages along with their emotions. The input of this step is the generated response, and the output is the provision of the response to the bereaved family member's device.

[1397] This is the specific process flow of the system.

[1398] (Application example 2)

[1399] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1400] Current home and business security systems have limitations in their ability to detect abnormalities, making it difficult to respond quickly and appropriately when an emergency occurs while the user is away. Furthermore, there is a lack of systems that can automatically generate and provide countermeasures that reflect the user's intentions and intent. This creates the problem of delayed security responses.

[1401] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting data from the user's email, social network service, and chat application via a network interface, means for preprocessing the collected data and removing noise data, and means for training a generative AI model that learns the user's tone of voice, expressions, and thought patterns using the preprocessed data. This makes it possible to detect abnormalities with high accuracy even when the user is absent, and to automatically generate and provide optimal countermeasures that reflect the user's intentions and will.

[1402] A "network interface" refers to the hardware and software that connects a computer or data collection device to a network and enables data communication.

[1403] "Email" refers to a digital communication method for sending and receiving messages over the Internet or other computer networks.

[1404] "Social network service" refers to an online platform that enables people to share information and interact with each other over the Internet.

[1405] A "chat application" refers to software for exchanging messages in real time, including text, voice, and video.

[1406] "Data preprocessing" refers to the process of converting collected raw data into a format suitable for analysis and machine learning.

[1407] "Noise data" refers to data that contains information that is unnecessary or misleading for analytical or learning purposes.

[1408] A "generative AI model" is a machine learning model that learns a user's unique tone of voice and expression patterns from given data and generates new responses and content.

[1409] "Training" refers to the process of showing a machine learning model large amounts of data and having it learn its patterns and features.

[1410] "Anomaly detection" refers to techniques and processes that automatically detect data or situations that deviate from normal patterns.

[1411] A "countermeasure" is a specific course of action or proposal for dealing with a particular situation.

[1412] The present invention provides a system for automatically taking a response that reflects the user's intentions even when the user is absent in a home or business security system. The system includes the following means.

[1413] 1. User Registration and Data Collection

[1414] The server collects personal information provided by users via a network interface. Users use devices such as smartphones or PCs to register basic information such as their name, address, and household members. The server also connects with various security devices (cameras, door locks, voice assistants, etc.) and obtains access permissions to the data generated by these devices. This allows the server to centrally collect situational data both inside and outside the home.

[1415] 2. Data preprocessing and filtering

[1416] The server preprocesses the collected data and removes noise data that is unsuitable for analysis. This preprocessing includes dividing text data into sentences, converting audio data into text, and extracting important frames from video data. It also uses spam filtering to remove irrelevant and noise data.

[1417] 3. Emotion extraction using an emotion engine

[1418] The server uses an emotion engine to analyze the collected data and assigns an emotion label to each message or action, allowing it to understand the user's daily emotional patterns and identify emotional changes when an anomaly occurs.

[1419] 4. Training the generative AI model

[1420] The server uses the preprocessed data to train the generative AI model. This training process learns the user's unique tone, expressions, and thought patterns, allowing the system to reproduce the user's intentions and emotions. Anomaly detection algorithms are also trained at this stage, allowing them to distinguish between normal and abnormal.

[1421] 5. Anomaly Detection and Response Generation

[1422] If an abnormality is detected while the user is absent, the server will automatically generate the optimal countermeasure using a trained generative AI model. For example, if suspicious activity is detected, the security robot can ask, "There is suspicious behavior. Is there something I can help you with?", and the response will reproduce the user's unique tone and expressions.

[1423] 6. Response Provision

[1424] The server then sends the generated responses and countermeasures to the user's device and related security devices via the network, allowing the user to check the status of their home even when they are away from home and implement the automatically generated countermeasures as needed.

[1425] Specific examples

[1426] For example, if a user inputs a prompt question such as "There has been suspicious activity in the house recently. What should we do?" into a generative AI model, the server can generate a response such as "Suspicious activity has been detected. Would you like to contact the police?" This response reflects the user's tone of voice and thought patterns, making it possible to take appropriate action based on the user's intentions even when the user is not present.

[1427] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1428] Step 1:

[1429] The user uses the terminal to register personal information, including basic information such as name, address, and household member composition. The registered information is sent to the server via the network interface. The input data is the personal information entered by the user, and the output data is the user information stored in the database.

[1430] Step 2:

[1431] After obtaining user approval, the server collects data from each security device (camera, door lock, voice assistant, etc.). The collected data is periodically transferred to the server. The input data is the raw data obtained from the security device, and the output data is the collected data before preprocessing.

[1432] Step 3:

[1433] The server preprocesses the collected data. Specifically, it splits text data into sentences, converts audio data to text, and extracts important frames from video data. It also performs spam filtering and removes noise data. The input data is a raw data stream, and the output data is a preprocessed dataset.

[1434] Step 4:

[1435] The server analyzes the preprocessed data using an emotion engine and assigns emotion labels to each message and action. This allows the user's daily emotion patterns to be analyzed. The input data is the preprocessed data, and the output data is the data with the emotion labels.

[1436] Step 5:

[1437] The server uses the emotion-labeled data to train the generative AI model. This training process learns the user's unique tone, expressions, and thought patterns. The input data is the emotion-labeled dataset, and the output data is the trained generative AI model.

[1438] Step 6:

[1439] The server detects anomalies when the user is absent. If an anomaly is detected, a trained generative AI model is used to generate optimal countermeasures. For example, if suspicious activity is detected, the security robot will ask, "There is suspicious behavior. Is there anything we can help you with?" The input data is real-time data used to detect anomalies, and the output data is the generated countermeasures.

[1440] Step 7:

[1441] The server sends the generated countermeasures to the user's terminal and related security devices via the network. This allows the user to check the status of their home even when they are away from home and implement the automatically generated countermeasures as necessary. The input data is the generated countermeasures, and the output data is a response message notified to the user.

[1442] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1443] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1444] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1445] [Fourth embodiment]

[1446] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1447] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1448] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1449] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1450] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1451] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1452] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1453] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1454] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1455] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1456] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1457] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1458] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1459] This invention relates to a system that utilizes a user's digital data to reproduce the user's intentions and expressions even when the user is absent or has passed away. This system includes a process that collects data via a network interface, trains a generative model, and generates responses based on the results.

[1460] A specific embodiment of this system will be described below.

[1461] System configuration

[1462] This system consists of users, terminals, and servers.

[1463] User registration and data collection

[1464] 1. User Registration

[1465] A user accesses the "Will AI Service" using a device (such as a smartphone or PC) and enters personal information such as name, email address, and phone number into the new registration form. The device then sends this information to the server, which creates an account for the user.

[1466] The device then asks the user for permission to access data from email, social networking services, and chat applications. If consent is granted, the device obtains authentication information from each service and begins collecting data.

[1467] 2. Data Collection

[1468] The device periodically collects past message data from each connected email, social networking service, and chat application, including, for example, emails sent and received by the user, social media posts, and chat history.

[1469] The collected data is encrypted and transferred to a server via secure communications.

[1470] Data Preprocessing and Filtering

[1471] 3. Data Preprocessing

[1472] The server decrypts the encrypted data and maps it to the corresponding user ID.

[1473] The server performs preprocessing to convert the data into a format that is easier to analyze. Specifically, the text data of emails and chats is divided into sentences and metadata (date and time, sender, recipient, etc.) is added.

[1474] 4. Data Filtering

[1475] The server filters out noise data, for example, filtering out spam emails and advertising messages, eliminating unnecessary data.

[1476] The filtered data is saved as a dataset for training a generative model.

[1477] Training a generative model

[1478] 5. Training the model

[1479] The server uses the preprocessed data to train a generative model (e.g., a GPT model). During the training process, the model learns the user's unique tone, expressions, and thought patterns.

[1480] Once training is complete, the server optimizes and stores the generative model.

[1481] Post-death care

[1482] 6. Response Generation

[1483] After the death of a user, the family members access the "Will AI Service" using a device and enter specific authentication information. Once authentication is complete, the family members can send questions or messages to the server.

[1484] The server inputs the received question into a trained model to generate a response that reproduces the user's tone and expression.

[1485] 7. Response Provision

[1486] The server sends the generated response to the family's device, where they can read it and check the deceased's wishes and messages.

[1487] Specific examples

[1488] For example, if a user had expressed a wish to "distribute my assets equally" during their lifetime, a response reflecting this wish will be generated from the collected data and the learning results of the generative model. When a surviving family member asks, "How should I distribute my inheritance?", the server generates the response "I want my assets to be distributed equally" and provides it to the surviving family member.

[1489] The above is an example of a specific embodiment of this system. By utilizing the user's digital data, the wishes and expressions of the deceased can be reproduced even after death, making it easier for the bereaved to maintain communication with the deceased.

[1490] The processing flow will be explained below.

[1491] Step 1:

[1492] The user accesses the "Will AI Service" using a device and enters personal information into the new registration form.

[1493] The device sends this information to a server, which creates an account for the user.

[1494] Step 2:

[1495] The device will ask the user for permission to access data from email, social networking services, and chat applications.

[1496] Once the user agrees, the device will obtain the necessary authentication information from each service and begin collecting data.

[1497] Step 3:

[1498] The device periodically collects past message data from email, social networking services, and chat applications.

[1499] The collected data is encrypted and transmitted to a server via secure communication.

[1500] Step 4:

[1501] The server receives the encrypted data, decrypts it and maps it to the corresponding user ID.

[1502] The server performs preprocessing to convert the data into a format that is easy to analyze.

[1503] Step 5:

[1504] The server cleans the collected data by removing noise data and filtering out spam and advertising messages.

[1505] The cleaned data is then saved as a dataset for training generative models.

[1506] Step 6:

[1507] The server uses the preprocessed data to train a generative model.

[1508] This training process involves learning the user's tone of voice, expressions, and thought patterns.

[1509] Step 7:

[1510] The server optimizes the trained generative model and stores the model that reflects the user's characteristics.

[1511] Step 8:

[1512] After the user's death, the surviving family members can use a device to access the "Will AI Service" and enter specific authentication information (such as answers to security questions).

[1513] Once authentication is complete, the family members can send questions or messages to the server.

[1514] Step 9:

[1515] The server inputs the received question into a trained model to generate a response that replicates the user's tone and expressions.

[1516] Step 10:

[1517] The server then sends the generated response to the family member's device.

[1518] The bereaved family will review this and understand the wishes and messages of the deceased.

[1519] Through the above processing steps, the "Will AI Service" can utilize the user's digital data to effectively communicate the user's wishes even after death.

[1520] Example 1

[1521] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1522] There has been no system that can convey a user's wishes and expressions to their surviving family members even after their absence or death. Providing such a system would enable a user's wishes and emotions to be reproduced after death, making it easier for surviving family members to maintain communication with the deceased. It is also necessary to securely collect, preprocess, and filter user data, and utilize generative AI models to reproduce the user's characteristics.

[1523] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1524] In this invention, the server includes means for collecting data from a user's electronic messaging service, social networking platform, and text communication application via a network interface, means for preprocessing the collected data and removing noise data, means for training a generative AI model that learns the user's tone of voice, expressions, and thought patterns using the preprocessed data, means for generating responses that reproduce the user's tone of voice, expressions, and thought patterns using the trained generative AI model, means for providing the generated responses via a network, means for a user to input specific authentication information via a terminal, means for a surviving family member who has completed authentication to send a question or message, means for generating responses that reflect the user's intentions and expressions using the trained generative AI model, and means for transmitting the generated responses to the surviving family member's terminal. This allows the user's digital data to be utilized to reproduce the user's intentions and emotions even after the user's death, enabling the surviving family member to maintain communication with the deceased.

[1525] A "network interface" is an interface for sending and receiving data via various networks.

[1526] "Electronic messaging service" means a service for sending and receiving digital messages, such as email.

[1527] A "social networking platform" is an online platform that enables users to connect and share information with friends and acquaintances.

[1528] A "text communication application" is an application for chatting and sending and receiving messages.

[1529] "Means for collecting data" refers to means consisting of software and hardware for collecting digital data of users.

[1530] The "means for preprocessing and removing noise data" refers to a means for filtering out unnecessary parts from collected data and converting it into a form that is easy to analyze.

[1531] A "generative AI model" is an artificial intelligence model that learns from specific data and reproduces the user's tone of voice, expressions, and thought patterns.

[1532] The "means for generating a response" is a means for generating a response that reflects the characteristics of the user using a trained generative AI model.

[1533] The "means for inputting specific authentication information" refers to a means for inputting authentication information required when a user accesses a service.

[1534] "Means for sending questions or messages" refers to means by which the user or the bereaved family members can send questions or messages to the server.

[1535] The "means for transmitting a response to a terminal of a bereaved family member" is a means for transmitting the generated response to a terminal used by the bereaved family member.

[1536] This invention relates to a system that utilizes a user's digital data to reproduce the user's intentions and expressions even when the user is absent or after death. This system is composed of a user, a terminal, and a server.

[1537] User registration and data collection

[1538] User Registration

[1539] A user accesses the "Will AI Service" using a device (such as a smartphone or PC) and enters personal information such as name, email address, and phone number into a new registration form. The device sends this information to a server, which creates an account for the user. The device then requests permission from the user to access data from email, social networking services, and chat applications. If consent is granted, the device obtains authentication information from each service and begins collecting data.

[1540] Data collection

[1541] We periodically collect historical message data from each electronic messaging service, social networking platform, and text communication application to which your device is connected. This data may include, for example, emails sent and received by you, social media posts, and chat history. The collected data is encrypted and transmitted to our servers via secure communications.

[1542] Data Preprocessing and Filtering

[1543] Data Preprocessing

[1544] The server decrypts the encrypted data and maps it to the corresponding user ID. The server then performs preprocessing to convert the data into a format that is easier to analyze. Specifically, the server splits the text data of emails and chats into sentences and adds metadata (date and time, sender, recipient, etc.).

[1545] Data Filtering

[1546] The server filters out noise data such as spam emails and advertising messages, and the filtered data is stored as a training dataset for generative AI models.

[1547] Training a generative model

[1548] Training the model

[1549] The server uses the preprocessed data to train a generative AI model (such as a GPT model). During this process, it learns the user's unique tone, expressions, and thought patterns. Once training is complete, the server optimizes and saves the generative AI model.

[1550] Response generation and delivery

[1551] Authenticate and enter your question

[1552] After the death of a user, the family members access the "Will AI Service" using a device and enter specific authentication information. Once authentication is complete, the family members can send questions or messages to the server.

[1553] Response Generation

[1554] The server inputs the received questions into a trained generative AI model to generate responses that replicate the user's tone and expressions.

[1555] Response provision

[1556] The server sends the generated response to the family's device, where they can read it and confirm the deceased's wishes and messages.

[1557] Examples and prompts

[1558] For example, if a user had expressed a wish to "distribute my assets equally" during their lifetime, a response reflecting this wish will be generated from the collected data and the learning results of the generative AI model. When a surviving family member asks, "How should I distribute my inheritance?", the server will generate the response "I want my assets to be distributed equally" and provide it to the surviving family member.

[1559] Example prompt:

[1560] User: How should assets be distributed?

[1561] Server: I want my assets to be divided equally.

[1562] The above is a specific embodiment of this system. By utilizing digital data to reproduce the user's intentions and emotions, it becomes possible for the bereaved to maintain communication with the deceased.

[1563] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1564] Step 1: User enters personal information into a sign-up form

[1565] Specific actions

[1566] The user accesses the "Will AI Service" using a device (smartphone or PC) and enters personal information such as name, email address, and phone number into the new registration form.

[1567] input

[1568] Personal information such as your name, email address, and phone number.

[1569] Data processing or data calculation

[1570] The terminal transmits the entered personal information to the server.

[1571] output

[1572] The user's personal information is registered on the server and an account is created for the user.

[1573] Step 2: Obtaining data access permission

[1574] Specific actions

[1575] The device will ask the user for permission to access data from email, social networking services, and chat applications.

[1576] input

[1577] User consent information.

[1578] Data processing or data calculation

[1579] If the user agrees, the device obtains authentication information from each service.

[1580] output

[1581] The obtained credentials.

[1582] Step 3: Performing data collection

[1583] Specific actions

[1584] Periodically collect historical message data from each electronic messaging service, social networking platform, and text communication application to which the device is connected.

[1585] input

[1586] User message data from each service.

[1587] Data processing or data calculation

[1588] The device encrypts the data.

[1589] output

[1590] The encrypted message data.

[1591] Step 4: Encrypt and transfer data

[1592] Specific actions

[1593] The encrypted data is transferred to the server via secure communication.

[1594] input

[1595] The encrypted message data.

[1596] Data processing or data calculation

[1597] Data transfer via secure communications.

[1598] output

[1599] Encrypted data stored on the server.

[1600] Step 5: Decrypt and preprocess the data

[1601] Specific actions

[1602] The server decrypts the encrypted data and maps it to the corresponding user ID.

[1603] input

[1604] Encrypted data stored on the server.

[1605] Data processing or data calculation

[1606] Preprocessing is performed to convert the data into a form that is easier to analyze, and text data from emails and chats is divided into sentences and metadata is added.

[1607] output

[1608] Decrypted data and preprocessed data with metadata added.

[1609] Step 6: Remove noise data

[1610] Specific actions

[1611] The server filters out noise data such as spam emails and advertising messages.

[1612] input

[1613] Preprocessed data.

[1614] Data processing or data calculation

[1615] Spam filtering and noise data removal.

[1616] output

[1617] Clean data with noise removed.

[1618] Step 7: Training the generative AI model

[1619] Specific actions

[1620] The server uses the preprocessed data to train a generative AI model (e.g., a GPT model).

[1621] input

[1622] Clean data with noise removed.

[1623] Data processing or data calculation

[1624] The generative AI model learns the user's unique tone of voice, expressions, and thought patterns.

[1625] output

[1626] A trained generative AI model.

[1627] Step 8: Authenticate and enter your questions

[1628] Specific actions

[1629] After the user's death, the surviving family members use a device to access the "Will AI Service" and enter specific authentication information.

[1630] input

[1631] Family member credentials.

[1632] Data processing or data calculation

[1633] Once authentication is complete, the family members can send questions or messages to the server.

[1634] output

[1635] Questions and messages from bereaved families whose authentication has been completed.

[1636] Step 9: Question processing and response generation

[1637] Specific actions

[1638] The server inputs the received questions into a trained generative AI model to generate responses that replicate the user's tone and expressions.

[1639] input

[1640] Questions and messages from the bereaved families.

[1641] Data processing or data calculation

[1642] The generative AI model generates responses that reflect the user's characteristics.

[1643] output

[1644] The generated response.

[1645] Step 10: Provide a response

[1646] Specific actions

[1647] The server sends the generated response to the terminal of the bereaved family member.

[1648] input

[1649] The generated response.

[1650] Data processing or data calculation

[1651] Sends the response over secure communications.

[1652] output

[1653] The response arrived on the family member's device.

[1654] (Application example 1)

[1655] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1656] In today's digital society, there is a demand for technology that can reproduce a user's intentions and expressions even when the user is absent or has died. However, conventional technologies have difficulty accurately reproducing a user's tone of voice, expressions, and thought patterns, making it difficult to provide personalized responses tailored to each individual user. In particular, in virtual stores, it is necessary to improve customer satisfaction by retaining a user's unique response patterns and automating customer responses.

[1657] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1658] In this invention, the server includes means for collecting data from a user's email, social media services, and real-time messaging applications via a network interface, means for preprocessing the collected data and removing noise data, means for training a generative AI model that learns the user's tone of voice, expressions, and thought patterns using the preprocessed data, means for generating responses that reproduce the user's tone of voice, expressions, and thought patterns using the trained generative AI model, means for providing the generated responses via a network, means for reproducing the user's will and expressions even when the user is absent or after death, and means for learning the user's unique response patterns and providing personalized customer service in a virtual store. This makes it possible to accurately reproduce the user's unique responses even when the user is absent or after death, and to automate personalized customer service in the virtual store.

[1659] A "network interface" is a communication means that allows for the collection, transmission, and reception of digital data.

[1660] A "user" is an entity, such as an individual or a corporation, that registers and provides information.

[1661] "Data collection methods" are mechanisms for obtaining information from email, social media services, real-time messaging applications, etc.

[1662] "Preprocessing" refers to the process of converting collected data into a format that is easy to analyze and removing noise data.

[1663] "Noise data" is unnecessary data or information that reduces the accuracy of analysis or processing.

[1664] A "generative AI model" is an artificial intelligence model that learns from training data and reproduces the user's tone of voice and expressions.

[1665] A "training method" is the process of training a generative AI model using preprocessed data.

[1666] A "response generation means" is a mechanism for generating responses that reflect the user's characteristics using a trained generative AI model.

[1667] The "means for providing via a network" refers to a means for transmitting the generated response to a user or other terminal via a communication network such as the Internet.

[1668] "Means to reproduce the user's will and expression" refers to a function that generates responses by imitating the user's unique words, actions, and intentions even when the user is absent or after death.

[1669] A "virtual store" is a virtual store that offers products and services over the Internet.

[1670] "Personalization" means providing optimal responses and services based on the characteristics and preferences of individual users.

[1671] This invention relates to a system that utilizes a user's digital data to reproduce their will and expressions even when they are absent or have passed away. This system mainly collects data via a network interface, preprocesses the collected data, and trains a generative AI model to learn the user's characteristics and generate responses based on the results.

[1672] The system basically consists of users, terminals, and servers.

[1673] 1. User Registration and Data Collection

[1674] A user accesses the system using a device (such as a smartphone or personal computer), enters personal information into a new registration form, and creates an account. The device then asks the user for permission to access data from email, social media, and real-time messaging applications. If the user agrees, the device obtains authentication information from each service and begins collecting data.

[1675] 2. Data preprocessing and filtering

[1676] The server receives the encrypted data, decrypts it, and maps it to a user ID. Next, the server performs data preprocessing to convert the data into a format that is easier to analyze. Specifically, the server breaks down emails and chat messages into sentences and adds metadata (date, time, sender, recipient, etc.). Spam filtering is then performed to remove noise data.

[1677] 3. Training the generative AI model

[1678] The server uses the preprocessed data to train a generative AI model (e.g., GPT-2). During the training process, the user's unique tone, expressions, and thought patterns are learned. Once training is complete, the generative model is optimized and saved.

[1679] 4. What to do when the user is absent

[1680] When the user is absent or after death, the trained generative AI model is utilized to handle customer service in the virtual store. Specifically, the surviving family members or customers access the system using their devices and send questions or messages. The server inputs the questions into the trained model and generates a response that reproduces the user's characteristics. This response is then sent to the surviving family members or customers' devices.

[1681] Hardware and software used

[1682] The system uses the following hardware and software:

[1683] Hardware: Server with GPU (e.g. NVIDIA Tesla)

[1684] Software: Python, transformers library, GPT-2 model

[1685] Specific examples

[1686] For example, if a user had expressed a wish to "divide my assets equally" during their lifetime, a response reflecting this wish will be generated from the collected data and the learning results of the generative model. When a family member asks, "How should I distribute my inheritance?", the server generates a response such as "I want my assets to be divided equally" and provides it to the family member. Similarly, when a customer in a virtual store asks, "What are today's recommended products?", an answer reflecting the user's unique response patterns that have been learned will be provided.

[1687] Prompt Sentence Examples

[1688] Generate a response when a user asks, "What's your featured product today?"

[1689] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1690] Step 1:

[1691] A user accesses the system using a terminal and creates an account by entering personal information such as name, email address, and phone number in a new registration form. The terminal then sends this personal information to the server, generating an account for the user.

[1692] Step 2:

[1693] The device asks the user for permission to access data from email, social media services, and real-time messaging applications. If the user agrees, the device obtains authentication information from each service and begins continuously collecting data.

[1694] Input: User consent, authentication information

[1695] Output: Collected data (emails, social media posts, message history)

[1696] Step 3:

[1697] The device encrypts the data it collects and sends it over secure connections to a server, which receives the data, decrypts it, and maps it to a user ID.

[1698] Input: Encrypted data

[1699] Output: Decrypted data, data mapped to user ID

[1700] Step 4:

[1701] The server preprocesses the decrypted data and converts it into a format that is easy to analyze. Specifically, it divides it into sentences and adds metadata (date, time, sender, recipient, etc.). Spam filtering is also performed to remove noise data.

[1702] Input: Decrypted data

[1703] Output: Preprocessed data, denoised data

[1704] Step 5:

[1705] The server uses the preprocessed data to train a generative AI model (e.g., GPT-2), which learns the user's unique tone, expressions, and thought patterns, and then optimizes and saves the generative AI model.

[1706] Input: Preprocessed data

[1707] Output: A trained generative AI model

[1708] Step 6:

[1709] When the user is absent or after death, the server utilizes the generative AI model to handle customer service in the virtual store. The server receives questions and messages from customers (or their families) and inputs this information into the generative AI model.

[1710] Input: Questions or messages from the customer (or their family)

[1711] Output: The prompts fed into the AI ​​model

[1712] Step 7:

[1713] The server uses a generative AI model to generate responses that mimic the user's tone and expressions, creating an appropriate response to the question.

[1714] Input: The prompt entered into the AI ​​model

[1715] Output: The generated response

[1716] Step 8:

[1717] The server then sends the generated response to the customer's (or the bereaved family's) terminal via the network, enabling personalized responses that reflect the user's characteristics.

[1718] Input: Generated response

[1719] Output: Response sent to the customer's (or their family's) device

[1720] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1721] This invention relates to a system that utilizes a user's digital data to reproduce the user's intentions and expressions even when the user is absent or has passed away. This system includes a process for collecting data via a network interface, training a generative model, and generating responses based on the results. It also combines a function that recognizes the user's emotions using an emotion engine and generates responses using that information.

[1722] A specific embodiment of this system will be described below.

[1723] System configuration

[1724] This system consists of a user, a terminal, a server, and an emotion engine.

[1725] User registration and data collection

[1726] 1. User Registration

[1727] A user accesses the "Will AI Service" using a device (such as a smartphone or PC) and enters personal information such as name, email address, and phone number into the new registration form. The device then sends this information to the server, which creates an account for the user.

[1728] The device then asks the user for permission to access data from email, social networking services, and chat applications. If consent is granted, the device obtains authentication information from each service and begins collecting data.

[1729] 2. Data Collection

[1730] The device periodically collects past message data from each connected email, social networking service, and chat application, including, for example, emails sent and received by the user, social media posts, and chat history.

[1731] The collected data is encrypted and transferred to a server via secure communications.

[1732] Data Preprocessing and Filtering

[1733] 3. Data Preprocessing

[1734] The server decrypts the encrypted data and maps it to the corresponding user ID.

[1735] The server performs preprocessing to convert the data into a format that is easier to analyze. Specifically, the text data of emails and chats is divided into sentences and metadata (date and time, sender, recipient, etc.) is added.

[1736] 4. Data Filtering

[1737] The server filters out noise data, for example, filtering out spam emails and advertising messages, eliminating unnecessary data.

[1738] The filtered data is saved as a dataset for training a generative model.

[1739] Using the Emotion Engine

[1740] 5. Emotion extraction

[1741] The emotion engine performs emotion analysis on each message in the collected data and assigns an emotion label (e.g., joy, sadness, anger, etc.).

[1742] The emotion engine generates a dataset with emotion labels and sends it to the server.

[1743] Training a generative model

[1744] 6. Training the model

[1745] The server trains the generative model using pre-processed emotion-labeled data. During the training process, the model learns emotions along with the user's unique tone, expressions, and thought patterns.

[1746] Once training is complete, the server optimizes and stores the generative model.

[1747] Post-death care

[1748] 7. Response Generation

[1749] After the death of a user, the family members access the "Will AI Service" using a device and enter specific authentication information. Once authentication is complete, the family members can send questions or messages to the server.

[1750] The server inputs the received question into a trained generative model to generate a response that reflects the user's tone, expression, and even emotion.

[1751] 8. Response Provision

[1752] The server sends the generated response to the device of the bereaved family, who can read it and confirm the deceased's wishes and messages along with their feelings.

[1753] Specific examples

[1754] For example, if a user had expressed a wish to "divide my assets equally" during his or her lifetime, the system would record this wish along with emotional data. After the user's death, if the surviving family members ask, "How should I distribute my inheritance?", the server would respond with "I want my assets to be divided equally," incorporating the user's tone and emotions, and provide this to the surviving family members.

[1755] The above is an example of a specific embodiment of this system. This system utilizes the user's digital data and emotions to provide a means for effectively communicating their wishes even after death.

[1756] The processing flow will be explained below.

[1757] Step 1:

[1758] The user accesses the "Will AI Service" using a device and enters personal information (such as name, email address, and phone number) into the new registration form.

[1759] The device sends this information to a server, which creates an account for the user.

[1760] Step 2:

[1761] The device will ask the user for permission to access data from email, social networking services, and chat applications.

[1762] Once the user agrees, the device will obtain authentication information from each service and begin collecting data.

[1763] Step 3:

[1764] It periodically collects past message data from email, social networking services, and chat applications to which the device belongs.

[1765] The collected data is encrypted and transmitted to a server via secure communication.

[1766] Step 4:

[1767] The server receives the encrypted data, decrypts it and maps it to the corresponding user ID.

[1768] The server performs preprocessing to convert the data into a format that is easy to analyze.

[1769] Step 5:

[1770] The server cleans the collected data by removing noise data and filtering out spam and advertising messages.

[1771] The cleaned data is then saved as a dataset for training generative models.

[1772] Step 6:

[1773] The sentiment engine performs sentiment analysis on each message in the collected data and assigns an emotional label to it.

[1774] The emotion engine generates a dataset with emotion labels and sends it to the server.

[1775] Step 7:

[1776] The server trains a generative model using preprocessed data with emotion labels.

[1777] This training involves learning the user's unique tone of voice, expressions, thought patterns and emotions.

[1778] Step 8:

[1779] After the server completes the training, it optimizes the generated model and saves the model that reflects the user's characteristics.

[1780] Step 9:

[1781] After the user's death, the surviving family members can access the "Will AI Service" using a device and enter specific authentication information (e.g., answers to security questions).

[1782] Once authentication is complete, the family members can send questions or messages to the server.

[1783] Step 10:

[1784] The server receives questions and feeds them into a trained model to generate responses that reflect the user's tone, expression, and even emotion.

[1785] Step 11:

[1786] The server then sends the generated response to the family member's device.

[1787] The bereaved family will review this and emotionally understand the wishes and messages of the deceased.

[1788] Through the above processing steps, the "Will AI Service" can utilize the user's digital data and emotions to effectively communicate the user's wishes even after death.

[1789] Example 2

[1790] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1791] Conventional systems have had difficulty reproducing a user's wishes and expressions when the user is absent or after death. Furthermore, they lacked the technology to generate responses that include the user's emotions, resulting in low emotional reproducibility. This has resulted in a lack of a means to convey the wishes and emotions of the deceased to their surviving family members.

[1792] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1793] In this invention, the server includes means for collecting data from users' emails, social network systems, and messaging systems via a network interface, means for decrypting the collected data, mapping it to a user ID, preprocessing it, and removing noise data, means for training a generative model using the preprocessed data to learn the user's tone of voice, expressions, thought patterns, and emotions, means for generating responses that reproduce the user's tone of voice, expressions, thought patterns, and emotions using the trained generative model, and means for providing the generated responses via a network. This makes it possible to reproduce the will and emotions of the deceased and provide emotional messages to the bereaved.

[1794] A "network interface" is a communication means used to collect data from a user's email, social networking system, and messaging system.

[1795] "User's email" refers to electronic message data sent and received by a user over the Internet.

[1796] A "social network system" is a platform through which users interact with others and share messages and posts.

[1797] A "messaging system" is an application that allows users to send and receive text messages in real time.

[1798] "Data collection" is the process of obtaining this message data from users' email, social networking systems, and messaging systems.

[1799] "Decryption" is the process of restoring encrypted data to its original, readable state.

[1800] "User ID mapping" refers to associating collected data with the corresponding user's identifier.

[1801] "Preprocessing" refers to the stage where data is converted into an analyzable form, specifically a process that includes dividing the data into sentence units and adding metadata.

[1802] "Noise data" is data that is not necessary for analysis, and includes spam and advertising messages.

[1803] A "generative model" is a model that uses machine learning techniques to learn a user's tone, expressions, thought patterns, and emotions from training data.

[1804] "Training" is the process of training a generative model using preprocessed data.

[1805] An "emotion label" is a tag that indicates the type of emotion attached to collected data (e.g., joy, sadness, anger).

[1806] "Response generation" is the process of using a trained generative model to create responses that replicate the user's tone, expressions, thought patterns, and emotions.

[1807] "Providing" means sending the generated response in an appropriate format to the terminal of the bereaved family member.

[1808] A "terminal" is a device (e.g., smartphone, computer) that a user or family member uses to access the system, enter data, and receive results.

[1809] An "emotion engine" is a machine learning algorithm or software for extracting emotions from collected data and assigning emotion labels.

[1810] This invention relates to a system that utilizes a user's digital data to reproduce the user's intentions and expressions even when the user is absent or has passed away. This system includes a process for collecting data via a network interface, training a generative model, and generating responses based on the results. It also combines a function for recognizing the user's emotions using an emotion engine and generating responses using that information.

[1811] System configuration

[1812] This system mainly consists of a user, a terminal, a server, and an emotion engine.

[1813] User registration and data collection

[1814] 1. User Registration

[1815] A user accesses the "Will AI Service" using a device (such as a smartphone or PC) and enters personal information such as name, email address, and phone number into a new registration form. The device sends this information to a server, which creates an account for the user. The device then asks the user for permission to access data from email, social network systems, and messaging systems. If consent is granted, the device obtains authentication information from each service and begins collecting data.

[1816] 2. Data Collection

[1817] The device periodically collects past message data from each connected email, social network, and messaging system. This data includes, for example, emails sent and received by the user, social media posts, and chat history. The collected data is encrypted and transmitted to a server via secure communications.

[1818] Data Preprocessing and Filtering

[1819] 3. Data Preprocessing

[1820] The server decrypts the encrypted data and maps it to the corresponding user ID. The server then performs preprocessing to convert the data into a format that is easier to analyze. Specifically, the server splits the text data of emails and chats into sentences and adds metadata (date and time, sender, recipient, etc.).

[1821] 4. Data Filtering

[1822] The server filters out noise data, for example, filtering out spam emails and advertising messages, and stores the filtered data as a dataset for training the generative model.

[1823] Using the Emotion Engine

[1824] 5. Emotion extraction

[1825] The emotion engine performs emotion analysis on each message in the collected data and assigns an emotion label (e.g., joy, sadness, anger, etc.). The emotion engine generates a dataset with the emotion labels and sends it to the server.

[1826] Training a generative model

[1827] 6. Training the model

[1828] The server trains the generative model using preprocessed emotion-labeled data. During the training process, the model learns emotions along with the user's unique tone, expressions, and thought patterns. Once training is complete, the server optimizes and saves the generative model.

[1829] Post-death care

[1830] 7. Response Generation

[1831] After a user's death, the surviving family members access the "Will AI Service" using their devices and enter specific authentication information. Once authentication is complete, the family members send questions or messages to the server. The server then inputs the received questions into a pre-trained generative model to generate responses that reflect the user's tone, expression, and even emotion.

[1832] 8. Response Provision

[1833] The server sends the generated response to the device of the bereaved family, who can read it and confirm the deceased's wishes and messages along with their feelings.

[1834] Specific examples

[1835] For example, if a user had expressed a wish to "distribute my assets equally" during their lifetime, the system would record this wish along with emotional data. After the death, if the surviving family members ask, "How should I distribute my inheritance?", the server would respond by including the user's tone and emotions, and provide the family with the message, "I want my assets to be distributed equally."

[1836] Prompt Sentence Examples

[1837] How should I distribute my inheritance?

[1838] The above is an example of a specific embodiment of this system. This system utilizes the user's digital data and emotions to provide a means for effectively communicating their wishes even after death.

[1839] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1840] Step 1:

[1841] User Registration

[1842] A user accesses the Will AI Service website or app using a device (smartphone or PC). The user enters personal information such as name, email address, and phone number into the new registration form. The device then sends this information to the server, creating an account for the user. The input is personal information, and the output is the creation of a user account.

[1843] Step 2:

[1844] Obtaining data access permissions

[1845] The device asks the user for permission to access data from email, social network systems, and messaging systems. The user confirms and agrees to the permission. The input of this procedure is the permission request, and the output is the user's consent confirmation.

[1846] Step 3:

[1847] Data collection

[1848] Using the user's authentication information, the device collects past message data from each email, social network system, and messaging system. This data includes emails sent and received by the user, social media posts, and chat history. The collected data is encrypted and transmitted to a server via secure communication. The input to this process is authentication information and connection data, and the output is encrypted data.

[1849] Step 4:

[1850] Data Decoding and Preprocessing

[1851] The server receives the encrypted data and decrypts it. It then maps it to the corresponding user ID, splits the email or chat text data into sentences, and adds metadata (date, time, sender, recipient, etc.). The input to this step is the encrypted data and the user ID, and the output is the preprocessed data.

[1852] Step 5:

[1853] Data Filtering

[1854] The server filters out noise data from the preprocessed data. Specifically, it filters out spam emails and advertising messages, leaving only the necessary data. The input of this step is the preprocessed data, and the output is the filtered dataset.

[1855] Step 6:

[1856] emotion extraction

[1857] The server invokes the emotion engine to perform sentiment analysis on each filtered message, and assigns an emotion label (e.g., joy, sadness, anger, etc.) to each message. The inputs of this step are the filtered data and the emotion engine, and the output is emotion-labeled data.

[1858] Step 7:

[1859] Training a generative model

[1860] The server trains the generative AI model using emotion-labeled data. During this training process, it learns the user's tone, expressions, thought patterns, and emotions. Once training is complete, it optimizes and saves the generative model. The input of this step is the labeled data, and the output is the optimized generative model.

[1861] Step 8:

[1862] Identification of bereaved family members and question reception

[1863] After the user's death, the surviving family members access the "Will AI Service" using a device and enter specific authentication information. Once authentication is complete, the surviving family members send questions or messages to the server. The input of this step is the surviving family members' authentication information and questions, and the output is the completion of authentication and receipt of the questions.

[1864] Step 9:

[1865] Response Generation

[1866] The server inputs the received question into a trained generative model to generate a response that reflects the user's tone, expression, and emotion. The input for this step is the family member's question, and the output is the generated response.

[1867] Step 10:

[1868] Response provision

[1869] The server sends the generated response to the bereaved family member's device, who then confirms it. The bereaved family member can receive the user's intentions and messages along with their emotions. The input of this step is the generated response, and the output is the provision of the response to the bereaved family member's device.

[1870] This is the specific process flow of the system.

[1871] (Application example 2)

[1872] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1873] Current home and business security systems have limitations in their ability to detect abnormalities, making it difficult to respond quickly and appropriately when an emergency occurs while the user is away. Furthermore, there is a lack of systems that can automatically generate and provide countermeasures that reflect the user's intentions and intent. This creates the problem of delayed security responses.

[1874] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting data from the user's email, social network service, and chat application via a network interface, means for preprocessing the collected data and removing noise data, and means for training a generative AI model that learns the user's tone of voice, expressions, and thought patterns using the preprocessed data. This makes it possible to detect abnormalities with high accuracy even when the user is absent, and to automatically generate and provide optimal countermeasures that reflect the user's intentions and will.

[1875] A "network interface" refers to the hardware and software that connects a computer or data collection device to a network and enables data communication.

[1876] "Email" refers to a digital communication method for sending and receiving messages over the Internet or other computer networks.

[1877] "Social network service" refers to an online platform that enables people to share information and interact with each other over the Internet.

[1878] A "chat application" refers to software for exchanging messages in real time, including text, voice, and video.

[1879] "Data preprocessing" refers to the process of converting collected raw data into a format suitable for analysis and machine learning.

[1880] "Noise data" refers to data that contains information that is unnecessary or misleading for analytical or learning purposes.

[1881] A "generative AI model" is a machine learning model that learns a user's unique tone of voice and expression patterns from given data and generates new responses and content.

[1882] "Training" refers to the process of showing a machine learning model large amounts of data and having it learn its patterns and features.

[1883] "Anomaly detection" refers to techniques and processes that automatically detect data or situations that deviate from normal patterns.

[1884] A "countermeasure" is a specific course of action or proposal for dealing with a particular situation.

[1885] The present invention provides a system for automatically taking a response that reflects the user's intentions even when the user is absent in a home or business security system. The system includes the following means.

[1886] 1. User Registration and Data Collection

[1887] The server collects personal information provided by users via a network interface. Users use devices such as smartphones or PCs to register basic information such as their name, address, and household members. The server also connects with various security devices (cameras, door locks, voice assistants, etc.) and obtains access permissions to the data generated by these devices. This allows the server to centrally collect situational data both inside and outside the home.

[1888] 2. Data preprocessing and filtering

[1889] The server preprocesses the collected data and removes noise data that is unsuitable for analysis. This preprocessing includes dividing text data into sentences, converting audio data into text, and extracting important frames from video data. It also uses spam filtering to remove irrelevant and noise data.

[1890] 3. Emotion extraction using an emotion engine

[1891] The server uses an emotion engine to analyze the collected data and assigns an emotion label to each message or action, allowing it to understand the user's daily emotional patterns and identify emotional changes when an anomaly occurs.

[1892] 4. Training the generative AI model

[1893] The server uses the preprocessed data to train the generative AI model. This training process learns the user's unique tone, expressions, and thought patterns, allowing the system to reproduce the user's intentions and emotions. Anomaly detection algorithms are also trained at this stage, allowing them to distinguish between normal and abnormal.

[1894] 5. Anomaly Detection and Response Generation

[1895] If an abnormality is detected while the user is absent, the server will automatically generate the optimal countermeasure using a trained generative AI model. For example, if suspicious activity is detected, the security robot can ask, "There is suspicious behavior. Is there something I can help you with?", and the response will reproduce the user's unique tone and expressions.

[1896] 6. Response Provision

[1897] The server then sends the generated responses and countermeasures to the user's device and related security devices via the network, allowing the user to check the status of their home even when they are away from home and implement the automatically generated countermeasures as needed.

[1898] Specific examples

[1899] For example, if a user inputs a prompt question such as "There has been suspicious activity in the house recently. What should we do?" into a generative AI model, the server can generate a response such as "Suspicious activity has been detected. Would you like to contact the police?" This response reflects the user's tone of voice and thought patterns, making it possible to take appropriate action based on the user's intentions even when the user is not present.

[1900] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1901] Step 1:

[1902] The user uses the terminal to register personal information, including basic information such as name, address, and household member composition. The registered information is sent to the server via the network interface. The input data is the personal information entered by the user, and the output data is the user information stored in the database.

[1903] Step 2:

[1904] After obtaining user approval, the server collects data from each security device (camera, door lock, voice assistant, etc.). The collected data is periodically transferred to the server. The input data is the raw data obtained from the security device, and the output data is the collected data before preprocessing.

[1905] Step 3:

[1906] The server preprocesses the collected data. Specifically, it splits text data into sentences, converts audio data to text, and extracts important frames from video data. It also performs spam filtering and removes noise data. The input data is a raw data stream, and the output data is a preprocessed dataset.

[1907] Step 4:

[1908] The server analyzes the preprocessed data using an emotion engine and assigns emotion labels to each message and action. This allows the user's daily emotion patterns to be analyzed. The input data is the preprocessed data, and the output data is the data with the emotion labels.

[1909] Step 5:

[1910] The server uses the emotion-labeled data to train the generative AI model. This training process learns the user's unique tone, expressions, and thought patterns. The input data is the emotion-labeled dataset, and the output data is the trained generative AI model.

[1911] Step 6:

[1912] The server detects anomalies when the user is absent. If an anomaly is detected, a trained generative AI model is used to generate optimal countermeasures. For example, if suspicious activity is detected, the security robot will ask, "There is suspicious behavior. Is there anything we can help you with?" The input data is real-time data used to detect anomalies, and the output data is the generated countermeasures.

[1913] Step 7:

[1914] The server sends the generated countermeasures to the user's terminal and related security devices via the network. This allows the user to check the status of their home even when they are away from home and implement the automatically generated countermeasures as necessary. The input data is the generated countermeasures, and the output data is a response message notified to the user.

[1915] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1916] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1917] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1918] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1919] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1920] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1921] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1922] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1923] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1924] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1925] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1926] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1927] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1928] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1929] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1930] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1931] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1932] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1933] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1934] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1935] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1936] The following is further disclosed regarding the above embodiment.

[1937] (Claim 1)

[1938] means for collecting data of a user's email, social network service, or chat application via a network interface;

[1939] means for pre-processing the collected data and removing noise data;

[1940] means for training a generative model using the pre-processed data to learn the user's tone, expressions, and thought patterns;

[1941] means for generating responses using the trained generative model that replicate the user's tone, expressions, and thought patterns;

[1942] means for providing the generated response over a network;

[1943] A system including:

[1944] (Claim 2)

[1945] 2. The system according to claim 1, wherein the data collection means also collects purchase history data of users.

[1946] (Claim 3)

[1947] 2. The system of claim 1, wherein the pre-processing means performs spam filtering.

[1948] "Example 1"

[1949] (Claim 1)

[1950] means for collecting data from a user's electronic messaging service, social networking platform, or text communication application via the network interface;

[1951] means for pre-processing the collected data and removing noise data;

[1952] means for using the pre-processed data to train a generative AI model that learns the user's tone, expressions, and thought patterns;

[1953] means for generating responses using the trained generative AI model that replicate the user's tone, expressions, and thought patterns;

[1954] means for providing the generated response over a network;

[1955] means for a user to input specific authentication information via a terminal;

[1956] A means for verified family members to send questions or messages,

[1957] A means for generating a response that reflects the user's intentions and expressions using the trained generative AI model;

[1958] means for transmitting the generated response to a terminal of the bereaved family;

[1959] A system including:

[1960] (Claim 2)

[1961] 10. The system of claim 1, wherein the system obtains a user's data access permission and obtains authentication information from each service.

[1962] (Claim 3)

[1963] 10. The system of claim 1, wherein the collected data is encrypted and transferred to the server via secure communication.

[1964] "Application Example 1"

[1965] (Claim 1)

[1966] means for collecting user email, social media service, and real-time messaging application data via a network interface;

[1967] means for pre-processing the collected data and removing noise data;

[1968] means for using the pre-processed data to train a generative AI model that learns the user's tone, expressions, and thought patterns;

[1969] means for generating responses using the trained generative AI model that replicate the user's tone, expressions, and thought patterns;

[1970] means for providing the generated response over a network;

[1971] A means to reproduce the user's will and expression even when the user is absent or after death,

[1972] A means for learning a response pattern specific to a user in a virtual store and providing personalized customer service;

[1973] A system including:

[1974] (Claim 2)

[1975] 2. The system according to claim 1, wherein the data collection means also collects purchase history data of users.

[1976] (Claim 3)

[1977] 10. The system of claim 1, wherein the pre-processing means performs spam filtering and detection of unusual language characteristics.

[1978] "Example 2: Combining Emotion Engines"

[1979] (Claim 1)

[1980] means for collecting user email, social network system, and messaging system data via a network interface;

[1981] means for decoding the collected data, mapping it to a user ID, and performing preprocessing to remove noise data;

[1982] means for using the pre-processed data to train a generative model that learns the user's tone, expressions, thought patterns and emotions;

[1983] means for generating responses using the trained generative model that replicate a user's tone, expressions, thought patterns, and emotions;

[1984] means for providing the generated response over a network;

[1985] A system including:

[1986] (Claim 2)

[1987] 2. The system according to claim 1, wherein the data collection means also collects purchase history data of users.

[1988] (Claim 3)

[1989] 2. The system of claim 1, wherein the pre-processing means performs spam filtering.

[1990] "Application example 2 when combining emotion engines"

[1991] (Claim 1)

[1992] means for collecting data of a user's email, social network service, and chat application via a network interface;

[1993] means for pre-processing the collected data and removing noise data;

[1994] means for using the pre-processed data to train a generative AI model that learns the user's tone, expressions, and thought patterns;

[1995] means for generating responses using the trained generative AI model that replicate the user's tone, expressions, and thought patterns;

[1996] A means for detecting anomalies based on user data and generating optimal countermeasures according to the situation;

[1997] means for providing the generated response and countermeasures via a network;

[1998] A system including:

[1999] (Claim 2)

[2000] 2. The system according to claim 1, wherein the data collection means also collects purchase history data of users.

[2001] (Claim 3)

[2002] 2. The system of claim 1, wherein the pre-processing means performs spam filtering. [Explanation of symbols]

[2003] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. means for collecting data of a user's email, social network service, and chat application via a network interface; means for pre-processing the collected data and removing noise data; means for training a generative model using the pre-processed data to learn the user's tone, expressions, and thought patterns; means for generating responses using the trained generative model that replicate the user's tone, expressions, and thought patterns; means for providing the generated response over a network; A system including:

2. 2. The system according to claim 1, wherein said data collection means also collects purchase history data of users.

3. 2. The system of claim 1, wherein the pre-processing means performs spam filtering.

Images