system

Abstract

Provide a system. An information collection means for inputting information about the deceased; a model construction means for constructing a personality model of the deceased based on the information collected by the information collection means; input receiving means for receiving text input by a user; a response generating means for analyzing the text received by the input receiving means and generating a response based on the personality model; a response sending means for sending the response generated by the response generating means to a user.

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] There is a lack of means for bereaved family members, especially the elderly, to find emotional comfort through a simulated conversation with the deceased. Conventional technologies struggle to recreate natural conversations that reflect the personality and vocabulary of the deceased, making them unable to fully soothe the emotions of bereaved family members. Furthermore, no systems exist that can continuously improve and adapt to conversations with the deceased. Given these circumstances, there is a need for a system that allows bereaved family members to find emotional comfort through conversations with the deceased. [Means for solving the problem]

[0005] To solve the above problems, the present invention provides the following means. First, an information collection means is provided for inputting information about the deceased, and then a model construction means is provided for constructing a personality model of the deceased based on the information collected by the information collection means. Furthermore, an input receiving means is provided for receiving text entered by a user, and a response generation means is provided for analyzing the text received by the input receiving means and generating a response based on the personality model. Finally, a response sending means is incorporated for sending the response generated by the response generation means to the user. The response generation means generates a response using a natural language generation model that learns the unique phrases and vocabulary of the deceased, and uses natural language processing technology to analyze the user's input. This realizes a system that allows bereaved family members to have a simulated conversation with the deceased and find comfort in their hearts.

[0006] "Information collection means" refers to an interface or device for inputting or obtaining information about the deceased person from a user.

[0007] "Model construction means" refers to a device or algorithm that constructs a system model to imitate the personality and characteristics of the deceased based on information about the deceased obtained by the information collection means.

[0008] "Input receiving means" refers to a device or algorithm for receiving text data entered by a user into the system.

[0009] The "response generation means" refers to a device or algorithm that analyzes the text data received by the input receiving means and generates an appropriate response using a pre-constructed personality model.

[0010] "Response sending means" refers to a device or algorithm for sending the text response generated by the response generating means to the user.

[0011] A "personality model" refers to a data model or set of algorithms constructed to recreate the personality and characteristics of a deceased person.

[0012] A "natural language generation model" is an algorithm or system that has learned the characteristics of language used by humans on a daily basis, and refers to a technology that generates natural language responses to input text data.

[0013] "Natural language processing technology" refers to the technologies and methods that enable computers to understand, analyze, and generate natural language. [Brief explanation of the drawings]

[0014] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0015] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0016] First, the terms used in the following description will be explained.

[0017] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0018] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0019] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0020] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0021] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0022] [First embodiment]

[0023] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0024] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0025] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0026] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0027] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0028] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0029] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0030] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0031] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0032] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0033] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0034] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0035] The present invention is a system that collects information about the deceased and simulates a conversation with the deceased based on that information. This system includes an information collection means, a model construction means, an input reception means, a response generation means, and a response transmission means. The program processing flow of this system is described below.

[0036] First, the user inputs information about the deceased using the information collection tool, including photos of the deceased, voice messages, text notes, and answers to detailed questions about their personality and hobbies, which are then sent to the server and stored.

[0037] The server then uses the collected information to build a personality model of the deceased using a model building mechanism, which includes the deceased's unique phrasing and vocabulary, as well as an algorithm that learns the deceased's personality traits.

[0038] When a user logs in to the system and starts a dialogue, the input receiving means receives text data entered by the user. The user can use the system to enter what they want to talk about with the deceased in natural language.

[0039] The server analyzes the received text data using a response generation tool. The analysis is carried out using natural language processing technology to understand the context of the input text and generate an appropriate response. The server incorporates the deceased's unique vocabulary and expressions to make the dialogue with the user more natural.

[0040] The generated response is sent from the response sending means to the user's terminal via the server. A chat interface is displayed on the user's terminal, where the generated response is displayed. This allows the user to enjoy the experience of interacting with the deceased in real time.

[0041] As a concrete example, consider the case where a user types, "Dad, how was your day?" The user's device sends this input to the server. The server analyzes the input and uses a personality model of the deceased person to generate a response such as, "I'm doing well today. I'm watching over you every day." The generated response is then sent to the user's device and displayed in the chat interface. The user can view the displayed response and continue the conversation.

[0042] This system allows bereaved families to have a simulated conversation with the deceased and find comfort in their hearts. This is a particularly significant psychological support for the elderly. This invention is a groundbreaking system that heals the hearts of bereaved families through conversation with the deceased.

[0043] The processing flow will be explained below.

[0044] Step 1:

[0045] The user uses the information collection tool to enter information about the deceased, including photos of the deceased, audio recordings, text notes, and answers to detailed questions about their personality and hobbies.

[0046] Step 2:

[0047] The terminal transmits the information about the deceased person input by the user to the server.

[0048] Step 3:

[0049] The server stores the received information and uses a model building means to build a personality model of the deceased, which learns the deceased's unique phrasing, vocabulary, and other characteristics.

[0050] Step 4:

[0051] A user logs into the system and opens a chat interface to begin a conversation.

[0052] Step 5:

[0053] The user types into the chat interface what they want to start the conversation with, for example, "Dad, how was your day?"

[0054] Step 6:

[0055] The terminal transmits the text data entered by the user to the server.

[0056] Step 7:

[0057] The server receives the input and begins parsing it using a response generation method. Natural language processing techniques are used to analyze the text data and understand the context.

[0058] Step 8:

[0059] The server uses a pre-built personality model of the deceased to generate an appropriate response, taking into account the deceased's unique phrasing and vocabulary.

[0060] Step 9:

[0061] The server transmits the generated response to the user's terminal through the response transmitting means.

[0062] Step 10:

[0063] The device displays the response received from the server in the chat interface, for example, "I'm doing well today. I'm watching over you every day."

[0064] Step 11:

[0065] The user confirms the response and provides further input to continue further interaction.

[0066] Step 12:

[0067] The server stores the conversation history with the user to help continuously improve the model.

[0068] In this way, the system enriches and continuously improves the user's interaction with the deceased.

[0069] Example 1

[0070] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0071] There is a need for a system that allows bereaved family members to simulate conversations with the deceased based on information about the deceased. However, conventional systems have had difficulty generating conversations that accurately reflect the personality and characteristics of the deceased. Furthermore, the content of the conversations is unnatural, which means that the bereaved family members are not provided with sufficient healing. Furthermore, the speed of real-time conversation generation and response display is slow, which can degrade the user experience.

[0072] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0073] In this invention, the server includes an information collection means for inputting information about the deceased, a model construction means for constructing a personality model of the deceased based on the collected information, an input receiving means for receiving text data input by a user, a response generation means for analyzing the text data and generating a response, a response sending means for sending the generated response to the user in real time, and a terminal for displaying the response on a chat interface. This enables natural, real-time conversation with the deceased, and allows the bereaved family to find emotional healing through conversation with the deceased.

[0074] The "information collection means" is a means by which a user inputs information about the deceased (photos, voice messages, text notes, answers to questions, etc.) and sends it to the server.

[0075] The "model construction means" is a means for constructing a personality model of the deceased based on the information collected by the information collection means. This model includes an algorithm that learns the deceased's unique expressions, vocabulary, and personality traits.

[0076] The "input receiving means" is a means for receiving text data entered by the user, and is responsible for transmitting the user's input to the server in real time.

[0077] The "response generation means" is a means for analyzing the text data received by the input receiving means and generating an appropriate response based on a personality model. The response generation means understands the context of the text using natural language processing technology.

[0078] The "response transmitting means" is a means for transmitting the response generated by the response generating means to the user's terminal in real time.

[0079] A "terminal" is a device used by a user that displays the chat interface and shows the user the responses received.

[0080] The present invention provides a system for collecting information about a deceased person and simulating a conversation with the deceased person based on the collected information. The system includes an information collection unit, a model construction unit, an input receiving unit, a response generation unit, a response sending unit, and a terminal.

[0081] First, the user inputs information about the deceased using an information collection tool. Specific hardware used may be a device such as a PC, smartphone, or tablet. Through this information collection tool, the user inputs photos of the deceased, voice messages, text notes, and answers to detailed questions about their personality and hobbies. This data is then sent from the device to a server and stored.

[0082] The server then uses the collected information to build a personality model of the deceased using a model-building tool, such as machine learning libraries like Tensorflow® or PyTorch, which are used in natural language processing. This personality model includes algorithms that learn the deceased's unique vocabulary, vocabulary, and personality traits.

[0083] When a user logs in to the system and opens the dialogue interface, the input receiving means receives text data entered by the user. The user can enter dialogue content in natural language through the terminal. For example, the user can enter a prompt sentence such as "Dad, how was your day?"

[0084] The server receives the text data sent via the input receiving means and analyzes it using the response generating means. This analysis uses a natural language generation model such as OpenAI's GPT-3 (registered trademark). Through this analysis, the context of the input text is understood and an appropriate response is generated. In this process, a personality model of the deceased is referenced, and the deceased's unique phrasing and vocabulary are incorporated to make the dialogue more natural.

[0085] The generated response is sent from the server to the user's terminal via the response sending means. A chat interface is displayed on the terminal, where the generated response is displayed in real time, allowing the user to simulate a real-time conversation with the deceased.

[0086] For example, if a user types and sends "Dad, how was your day?", the server analyzes this input and generates a response based on the personality model, such as "I'm doing well today. I'm watching over you every day." This response is then sent to the user's device and displayed in the chat interface, allowing the user to continue the conversation.

[0087] Examples of prompt sentences include:

[0088] "Mom, what did you do today?"

[0089] "Grandpa, tell me some memories from your childhood."

[0090] "Grandma, what was your favorite dish?"

[0091] This system allows bereaved families to find comfort through conversation with the deceased, which is a great source of emotional support, especially for the elderly.

[0092] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0093] Step 1:

[0094] A user accesses the system and logs in.

[0095] Specifically, a user uses a device to access a web browser or dedicated application and enters authentication information (user ID and password) to log in. This authentication information is sent from the device to the server, which then references a database to perform authentication.

[0096] Input: User ID, Password

[0097] Output: Login success / failure message

[0098] Step 2:

[0099] The user inputs information about the deceased person using the information collection means.

[0100] Specifically, the user accesses a form to upload photos of the deceased, voice messages, text notes, answers to questions, etc. For example, by clicking the "Upload Photo" button, the user uploads a photo. This information is sent from the device to the server and stored on the server.

[0101] Input: Photos of the deceased, voice messages, text notes, and answers to questions

[0102] Output: Message that data has been saved to the server

[0103] Step 3:

[0104] The server builds a personality model based on the collected information.

[0105] Specifically, the server analyzes the stored data and uses machine learning libraries such as TensorFlow and PyTorch to build a personality model that learns the deceased's unique phrasing, vocabulary, and personality traits.

[0106] Input: Photos of the deceased, voice messages, text notes, and answers to questions

[0107] Output: Constructed personality model

[0108] Step 4:

[0109] The user logs back into the system and opens the interactive interface.

[0110] Specifically, the user clicks the "Start conversation" button, and the conversation interface appears. The user enters a message in natural language into the chat box. For example, the user enters a prompt sentence such as "Dad, how was your day?" The terminal then sends this message to the server.

[0111] Input: Natural language message

[0112] Output: Notification of completion of sending

[0113] Step 5:

[0114] The server receives the input text data and analyzes it using a response generation means.

[0115] Specifically, the server uses a natural language generation model, such as OpenAI's GPT-3, to understand and analyze the context of the input text. During this analysis process, the server also references a personality model and generates a response that incorporates the deceased's unique phrasing and vocabulary. For example, a response might be generated such as, "I'm doing well today. I'm watching over you every day."

[0116] Input: Natural language message, personality model

[0117] Output: The generated response message

[0118] Step 6:

[0119] The server transmits the generated response to the user's terminal in real time.

[0120] Specifically, the generated response message is sent from the server to the user's terminal, and the terminal displays it on the chat interface. The user can check the displayed response and re-enter the next dialogue content.

[0121] Input: The generated response message

[0122] Output: Displayed in the chat interface

[0123] The above is a detailed description of the specific processing steps of the program of the present invention, which allows the user to enjoy natural conversation with the deceased in real time and find comfort in their heart.

[0124] (Application example 1)

[0125] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0126] The present invention relates to a system that allows users to simulate memories and conversations with the deceased. Conventional memorial services mainly provide static information, limiting the means by which bereaved family members can actually experience conversations with the deceased. As a result, they are insufficient to recreate interactions with the deceased and provide emotional healing. Therefore, the present invention aims to provide a system that allows users to more realistically simulate conversations with the deceased, thereby providing a means for users to find emotional comfort through interactions with the deceased.

[0127] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0128] In this invention, the server includes an input means for inputting information about the deceased, a model construction means for constructing a personality model of the deceased based on the information collected by the input means, and an input receiving means for receiving text input by the user, thereby enabling the user to upload photos, audio, and text memos of the deceased and receive a response based on the personality traits of the deceased in real time on the display means of the smartphone application.

[0129] "Input means" refers to a means by which a user inputs information about the deceased.

[0130] The "model construction means" is a means for constructing a personality model of the deceased based on the collected information.

[0131] The "input receiving means" is a means for receiving text data input by a user.

[0132] A "response generator" is a means for analyzing the received text and generating a response based on a personality model of the deceased person.

[0133] The "response sending means" is a means for sending the generated response to the user's terminal.

[0134] "Uploading means" refers to a means by which a user can upload photos, audio, and text notes of the deceased.

[0135] The "display means" is a means for displaying the generated response on the user's terminal.

[0136] A "natural language generation model" is a model used to learn the specific phrasing and vocabulary of the deceased.

[0137] "Natural language processing technology" is a technology for analyzing user input and generating appropriate responses.

[0138] The present invention is a system that collects information about the deceased and imitates conversations with the deceased based on that information. This system collects information about the deceased from a user and builds an AI model that imitates the deceased's personality and phrasing based on that information. Specifically, the system includes the following means.

[0139] First, the user uses an input device to input information about the deceased. This information can include photos of the deceased, voice messages, text notes, and answers to detailed questions about their personality and hobbies. This information is then sent to a server and stored. An uploading method is provided for this purpose.

[0140] The server then uses a model building means to build a personality model of the deceased based on the collected information. The personality model includes algorithms that learn the deceased's unique phrasing, vocabulary, and personality traits. A natural language generation model is used to learn the deceased's phrasing and vocabulary.

[0141] When a user logs in to the system and begins a dialogue, the input receiving means is utilized to receive the text data entered by the user. The received text is sent to the server and analyzed by the response generating means. Natural language processing technology is used for the analysis, and includes the process of understanding the context of the input text and generating an appropriate response.

[0142] The response generated by the server is transmitted to the user's terminal via a response transmission means. The user's terminal is provided with a display means for displaying the generated response. This allows the user to enjoy a simulated experience of interacting with the deceased.

[0143] As a concrete example, consider the case where a user inputs "Dad, how was your day?" The user's device sends this input to the server. The server analyzes the input and uses a personality model of the deceased person to generate a response such as "I'm doing well today. I'm watching over you every day." The generated response is then sent to the user's device and displayed. This allows the user to view the displayed response and continue the conversation.

[0144] This system allows bereaved families to have a simulated conversation with the deceased and find comfort in their hearts. This is a particularly significant psychological support for the elderly. This invention is a groundbreaking system that heals the hearts of bereaved families through conversation with the deceased.

[0145] As a concrete example, the following is an example of a prompt sentence to be input to a generative AI model:

[0146] "Tell me a story from your garden."

[0147] "What book are you reading today?"

[0148] "What do you think about your mother?"

[0149] This allows the user to feel as if they are having a more realistic conversation with the deceased, which can bring about emotional healing.

[0150] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0151] Step 1:

[0152] The user uses the input means to input information about the deceased.

[0153] The information entered includes photos of the deceased, voice messages, text notes, and answers to detailed questions about their personality and hobbies.

[0154] The input information is sent to the server via uploading means and saved there.

[0155] Step 2:

[0156] Based on the information acquired by the server, the model construction means constructs a personality model of the deceased person.

[0157] To build this model, algorithms are used to analyze collected data and learn specific phrases, vocabulary, and personality traits.

[0158] A natural language generation model is used as the generative AI model.

[0159] Step 3:

[0160] A user logs into a terminal and begins a dialogue.

[0161] The text data input by the user is received by the input receiving means.

[0162] The received text is sent back to the server.

[0163] Step 4:

[0164] The server analyzes the text data received by the input receiving means.

[0165] It uses natural language processing techniques to understand the context of the input text and generate an appropriate response.

[0166] During the analysis process, processing is carried out to incorporate the deceased's unique phrasing and vocabulary to simulate more realistic dialogue.

[0167] Step 5:

[0168] The response generated by the server is transmitted to the user's terminal by the response transmitting means.

[0169] The generated response is displayed on the user's terminal through a display means.

[0170] Step 6:

[0171] The user confirms the response displayed on the terminal and continues the conversation.

[0172] This series of conversations takes place in real time, allowing users to virtually experience a conversation with the deceased and enjoy the process.

[0173] The above is a specific flow of processing by the system according to the present invention.

[0174] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0175] The present invention is a system that collects information about a deceased person and simulates a conversation with the deceased person based on that information. This system includes an information collection means, a model construction means, an input reception means, a response generation means, a response transmission means, and an emotion engine. The program processing flow of this system is described below.

[0176] First, the user inputs information about the deceased using the information collection tool, including photos of the deceased, voice messages, text notes, and answers to detailed questions about their personality and hobbies, which is then sent to the server and stored.

[0177] The server then uses the collected information to build a personality model of the deceased using a model building mechanism, which includes the deceased's unique phrasing and vocabulary, as well as an algorithm that learns the deceased's personality traits.

[0178] When a user logs in to the system and starts a dialogue, the input receiving means receives text data entered by the user. The user can use the system to enter what they want to talk about with the deceased in natural language.

[0179] The server analyzes the received text data using an emotion engine before sending it to the response generation means. The emotion engine recognizes the user's emotion from the user's input text and passes the recognition result to the response generation means. This emotion recognition allows the response generation means to generate an appropriate response that matches the user's emotional state.

[0180] The response generator uses natural language processing technology to analyze the text data and understand the context. Based on the analysis results and emotional information from the emotion engine, it incorporates the deceased's unique vocabulary and expressions to generate a more natural response for the user.

[0181] The generated response is sent to the user's terminal by the response sending means via the server. A chat interface is displayed on the user's terminal, where the generated response is displayed. This allows the user to enjoy the experience of interacting with the deceased in real time.

[0182] As a specific example, consider the case where a user inputs "Dad, how was your day?" The user's device sends this input to the server. The server receives the input, and the emotion engine analyzes the user's emotions. If this text is recognized as an emotional question, the response generation means generates a response that is considerate of the user's emotions. For example, if the emotion engine recognizes "loneliness," the response generation means generates a response such as "I'm doing well today. I'm always here for you so you don't feel lonely." The generated response is then sent to the user's device and displayed on the chat interface. The user can view the displayed response and continue the conversation.

[0183] In this way, the system provides emotional comfort by recreating the user's rich dialogue with the deceased and adjusting responses according to the user's emotional state. This is particularly valuable for elderly people. This is an innovative system that provides emotional comfort to bereaved families through dialogue with the deceased.

[0184] The processing flow will be explained below.

[0185] Step 1:

[0186] The user uses the information collection tool to enter information about the deceased, including photos of the deceased, voice messages, text notes, and answers to detailed questions about their personality and hobbies.

[0187] Step 2:

[0188] The terminal transmits the information about the deceased person input by the user to the server.

[0189] Step 3:

[0190] The server stores the received information and uses a model building means to build a personality model of the deceased, which learns the deceased's unique phrasing, vocabulary, and other characteristics.

[0191] Step 4:

[0192] A user logs into the system and opens a chat interface to begin a conversation.

[0193] Step 5:

[0194] The user types into the chat interface what they want to start the conversation with, for example, "Dad, how was your day?"

[0195] Step 6:

[0196] The terminal transmits the text data entered by the user to the server.

[0197] Step 7:

[0198] The server receives the input and uses an emotion engine to analyze the user's emotions from the input text data, for example, determining whether the user's input has emotions such as loneliness, sadness, or joy.

[0199] Step 8:

[0200] The server passes the user's emotion information recognized by the emotion engine to the response generating means.

[0201] Step 9:

[0202] The response generation means generates an appropriate response based on the emotional information and input text data, using a pre-built personality model of the deceased, taking into account the deceased's unique phrasing and vocabulary, and creating a response that is in tune with the user's emotions.

[0203] Step 10:

[0204] The server transmits the generated response to the user's terminal through the response transmitting means.

[0205] Step 11:

[0206] The device displays the response received from the server in the chat interface, for example, "I'm doing well today. I'm always by your side so you won't feel lonely."

[0207] Step 12:

[0208] The user confirms the response and provides further input to continue further interaction.

[0209] Step 13:

[0210] The server stores the conversation history with the user to help continuously improve the model.

[0211] In this way, the system provides a rich recreation of the user's interaction with the deceased, and even tailors responses based on the user's emotional state. This process not only provides comfort to the user, but also allows them to receive personalized responses.

[0212] Example 2

[0213] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0214] The problem to be solved by this invention is to provide comfort to a user by generating appropriate and natural responses that correspond to the user's emotional state when imitating a conversation with a deceased person. In particular, it is necessary to reproduce the deceased's unique phrasing and vocabulary to provide the user with a realistic conversation experience. It is also important that these responses are in tune with the user's emotions.

[0215] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes an information collection means for inputting general information about the deceased, a model construction means for constructing a personality model of the deceased based on the information collected by the information collection means, an input receiving means for receiving text input by a user, a response generation means for analyzing the text received by the input receiving means with an emotion engine and generating a response based on the emotion recognition result and the personality model, and a response sending means for sending the response generated by the response generation means to the user. This allows a natural response to be generated that matches the user's emotions, allowing the user to receive emotional comfort through a conversation with the deceased.

[0216] "General information about the deceased" is information that includes data that reflects the characteristics of the deceased, such as photographs of the deceased, voice messages, text notes, and answers about their personality and hobbies.

[0217] An "information collection tool" is any device or software that allows a user to enter information about the deceased person into the system.

[0218] A "model construction means" is a device or algorithm that constructs a personality model of the deceased person based on the collected information.

[0219] "Input receiving means" refers to a device or software that receives text entered by a user.

[0220] An "emotion engine" is software or algorithms that analyze user-entered text and recognize the user's emotional state.

[0221] The "response generation means" is a device or algorithm that generates a response to the user based on the analysis results of the emotion engine and the personality model.

[0222] The "response sending means" is a device or software that sends the generated response to the user.

[0223] A "natural language generation model" is an algorithm or machine learning model that learns the unique phrases and vocabulary of the deceased and generates natural-sounding dialogue.

[0224] "Natural language processing technology" is a technology for analyzing user input text and understanding its context and meaning.

[0225] The present invention is a system that collects information about the deceased and simulates a conversation with the deceased based on that information. This system includes an information collection means for collecting general information about the deceased, a model construction means for constructing a personality model based on the collected information, an input reception means for receiving text entered by a user, a response generation means for analyzing the entered text and generating a response based on the emotion recognition results and the personality model, and a response transmission means for transmitting the generated response to the user. The specific usage and processing content of each means are described below.

[0226] First, an information collection method is provided for users to enter general information about the deceased. The information collection method is implemented using a web interface or mobile app. This interface includes a function to upload photos of the deceased, a function to record voice messages, a field to input text notes, and a form to answer questions about personality and hobbies. This input data is sent to the server using a secure communication protocol (e.g., SSL / TLS). The submitted data is then stored in a database (e.g., a relational database or object storage) on the server.

[0227] The server then uses a model-building method to build a personality model based on the stored information. This process uses a machine learning framework (e.g., TensorFlow, PyTorch). Specifically, the server extracts the deceased's unique phrases, vocabulary, and personality traits from the collected text and audio data, and trains a natural language generation model. This model learns the deceased's unique phrases and vocabulary and is used to realistically recreate interactions with the user.

[0228] When a user starts a conversation with the system, the input receiving means receives the text input by the user. The user can freely input text through the chat interface of the system. This input data is transmitted to the server in real time.

[0229] The server then analyzes the data received by the input receiving means using an emotion engine. The emotion engine uses natural language processing technology (e.g., Sentiment Analysis API) to extract emotional information from the user's input text. This emotion analysis makes it possible to understand the user's emotional state. The analysis results are passed to the response generating means.

[0230] The response generation means generates an appropriate response that matches the user's emotional state based on the emotion analysis results and a personality model. This process uses a generative AI model (e.g., GPT-4 (registered trademark)). For example, if a user inputs "Dad, how was your day?", the emotion analysis may recognize the emotion of loneliness. Based on this information, the response generation means generates a response such as "I'm doing well today. I'm always here for you so you don't feel lonely."

[0231] Finally, the generated response is transmitted to the user's terminal through the response transmitting means, and a chat interface is displayed on the user's terminal, where the generated response is displayed in real time, thereby allowing the user to have an experience that mimics a conversation with the deceased person.

[0232] The following is a concrete example of a prompt sentence:

[0233] User input: "Dad, how was your day?"

[0234] System response: "I'm doing well today. I'll always be by your side so you won't feel lonely."

[0235] This system allows users to find comfort through conversation with the deceased, providing great emotional support for the elderly and their bereaved families.

[0236] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0237] Step 1: Gather information

[0238] Users use a web interface or mobile app to enter information about the deceased, including the ability to upload a photo of the deceased, record a voice message, enter text notes, and answer questions about their personality and hobbies.

[0239] Input: User-entered photos of the deceased, voice messages, text notes, and answers to questions.

[0240] Data processing: The server receives the uploaded photos and audio data and converts them into the appropriate format. It also receives text data.

[0241] Output: These data are sent to the server and stored.

[0242] Step 2: Save your information

[0243] The server stores the received information: photos and voice messages are stored in object storage, while text notes and answers to questions are stored in a relational database.

[0244] Input: Various information about the deceased person entered by the user.

[0245] Data processing: The server classifies the data into appropriate categories and stores them in the corresponding storage (e.g., database or object storage).

[0246] Output: The stored data can be accessed for subsequent processing.

[0247] Step 3: Model Building

[0248] The server builds a personality model of the deceased person based on the stored information, using machine learning frameworks (e.g., TensorFlow, PyTorch) for this process.

[0249] Input: Information about the deceased person stored in the database.

[0250] Data processing: The server analyzes text and audio data to extract specific phrases, vocabulary, and personality traits. The extracted data is then used to train a machine learning model.

[0251] Output: A personality model is generated that reflects the characteristics of the deceased.

[0252] Step 4: Receiving Input

[0253] The user initiates a conversation with the system. They can freely enter text through a chat interface, and the input data is sent to the server in real time.

[0254] Input: Text data that users enter into the chat interface.

[0255] Data processing: The server receives the input text data in real time and converts it into an analyzable format.

[0256] Output: The converted text data is used for sentiment analysis.

[0257] Step 5: Sentiment Analysis

[0258] The server analyzes the data received by the input receiving means using an emotion engine. Natural language processing technology (e.g., Sentiment Analysis API) is used to extract emotion information from the user's input text.

[0259] Input: User text data received in real time.

[0260] Data processing: The emotion engine analyzes the user's text data and recognizes their emotional state (e.g., joy, sadness, loneliness, etc.).

[0261] Output: Analysis of the user's emotional state.

[0262] Step 6: Response Generation

[0263] The response generator generates an appropriate response that matches the user's emotional state based on the emotion analysis results and personality model. This process uses a generative AI model (e.g., GPT-4).

[0264] Input: Analysis of the user's emotional state and a personality model of the deceased.

[0265] Data processing: The response generation means selects appropriate phrases and vocabulary that match the emotional state and generates natural-looking response text.

[0266] Output: The generated response text.

[0267] Step 7: Send response

[0268] The generated response is sent to the user's terminal via the response sending means and displayed on the chat interface.

[0269] Input: The generated response text.

[0270] Data processing: Formatting the response text for sending to the user's device.

[0271] Output: The response text that will be displayed to the user in their chat interface.

[0272] This process allows the system to provide users with a natural dialogue experience, enabling realistic conversations using the deceased person's unique vocabulary and expressions.

[0273] (Application example 2)

[0274] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0275] Conventional memorial systems simply store information about the deceased, but do not allow bereaved families to relive conversations with the deceased. This leaves users feeling disconnected from the deceased and unable to receive sufficient emotional comfort. Furthermore, conventional memorial systems are unable to generate responses that reflect the user's emotional state, resulting in a lack of naturalness and intimacy in the conversation. This results in insufficient emotional support for bereaved families.

[0276] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0277] In this invention, the server includes an information collection means for inputting information about the deceased, a model construction means for constructing a personality model of the deceased based on the information collected by the information collection means, an input receiving means for receiving text input by a user, an emotion analysis means for recognizing an emotional state, a response generation means for generating a response based on the emotional state recognized by the emotion analysis means and the personality model, a response sending means for sending the response generated by the response generation means to the user, and a means for providing a chat interface to the user terminal, thereby making it possible to more naturally reproduce a conversation with the deceased and generate a response appropriate to the emotional state of the user.

[0278] "Information gathering means" refers to means for inputting information about the deceased, such as photos, voice messages, text notes, and answers about their personality and hobbies.

[0279] The "model construction means" is a means for constructing a personality model of the deceased based on the information collected by the information collection means.

[0280] The "input receiving means" is a means for receiving text data input by the user.

[0281] The "emotion analysis means" is a means for analyzing and recognizing emotions from the user's text received by the input receiving means.

[0282] The "response generation means" is a means for generating a response to the user based on the emotional state of the user recognized by the emotion analysis means and a personality model.

[0283] The "response transmitting means" is a means for transmitting the response generated by the response generating means to the user's terminal.

[0284] A "chat interface" is a means of providing an interface that allows a user to enjoy real-time conversation with a deceased person.

[0285] A "generative AI model" is an artificial intelligence model that uses natural language generation technology to learn the unique phrases and vocabulary of the deceased and generate responses to user input.

[0286] A "prompt sentence" is an input sentence that a generative AI model uses to generate a response, and is a sentence that includes the user's input and emotional state.

[0287] The system for realizing this invention is a system for collecting information about the deceased and providing psychological support to a user by simulating a conversation with the deceased. The system mainly includes an information collection means, a model construction means, an input receiving means, an emotion analysis means, a response generation means, a response sending means, and a chat interface.

[0288] First, information about the deceased is collected using the user's device. Using the information collection tool, the user inputs photos of the deceased, voice messages, text notes, and answers to detailed questions about their personality and hobbies. This information is then sent to the server and stored.

[0289] The server then uses the collected information to build a personality model using a model-building method. This process uses a generative AI model to learn the deceased person's unique phrasing, vocabulary, and personality traits. The built personality model is then used to generate responses.

[0290] When a user logs in to the system and starts a dialogue, the input receiving means receives text data entered by the user. This text data is sent to the server and analyzed by the emotion analysis means. The emotion analysis means recognizes the user's emotion from the user's input text and passes the result to the response generation means.

[0291] The response generation means generates a response based on the analyzed emotional state and personality model. In this process, a generative AI model is used to generate a natural and friendly response appropriate to the user's input. The generated response is sent to the user's device by the response sending means via the server. A chat interface is displayed on the user's device, where the generated response is displayed.

[0292] As a concrete example, consider a case where a user inputs "Dad, how was your day?" If the emotion analysis means recognizes that this question is emotional, the response generation means generates a response such as "I'm doing well today. I'm always here for you so you don't feel lonely." This response is displayed in the chat interface and can be viewed by the user.

[0293] Example prompt sentence:

[0294] Father: Dad, how was your day? (Emotion: lonely)

[0295] The system is built primarily using Python and OpenAI APIs, and provides a familiar conversational experience that takes into account the user's emotional state and the personality traits of the deceased. Through the system, users can re-create conversations with the deceased and receive emotional support.

[0296] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0297] Step 1:

[0298] Using the information gathering tool, photos of the deceased, voice messages, text notes, and answers to detailed questions about their personality and hobbies are entered.

[0299] Input: Information about the deceased person entered by the user (photos, voice messages, text notes, answers to questions).

[0300] Output: Data about the deceased person that is sent to and stored on a server.

[0301] Specifically, the necessary information is collected through an input form provided on the user's terminal screen.

[0302] Step 2:

[0303] Based on the information collected by the information collecting means, the server uses the model building means to build a personality model of the deceased person.

[0304] Input: Data about the deceased collected and stored in Step 1.

[0305] Output: Personality model of the deceased person (generative AI model).

[0306] Specifically, the server uses the OpenAI API to train a natural language generation model based on the input data.

[0307] Step 3:

[0308] When a user logs in to the system and starts a dialogue, the input receiving means receives text data input by the user.

[0309] Input: Text data that the user types into the terminal.

[0310] Output: The user's text data sent to the server.

[0311] Specifically, the user's terminal transmits the input text to the server in real time.

[0312] Step 4:

[0313] The server analyzes the text received by the input receiving means and recognizes the user's emotions by the emotion analysis means.

[0314] Input: User's text data received in step 3.

[0315] Output: The analyzed emotional state of the user.

[0316] Specifically, the server uses natural language processing technology (such as OpenAI's emotion analysis API) to analyze emotions from text data.

[0317] Step 5:

[0318] The response generating means generates a response based on the emotional state recognized by the emotion analyzing means and the personality model.

[0319] Input: Emotional state recognized by the emotion analysis means, personality model constructed by the model construction means.

[0320] Output: The generated response text.

[0321] Specifically, the server uses a generative AI model to generate an appropriate response based on the user's input and emotions.

[0322] Step 6:

[0323] The server transmits the response generated by the response generating means to the user.

[0324] Input: The response text generated in step 5.

[0325] Output: The response text sent to the user's terminal.

[0326] Specifically, the server transmits the generated response in real time to the user's chat interface.

[0327] Step 7:

[0328] The user's terminal displays the sent response in a chat interface, allowing the user to continue the dialogue.

[0329] Input: The response text sent in step 6.

[0330] Output: The response text that will be displayed in the chat interface.

[0331] As a specific operation, the user's terminal displays the response text in a specific UI (user interface) component, allowing the user to view it and provide further input.

[0332] In this way, the entire system works in cooperation, allowing the user to enjoy a real-time interactive experience with the deceased.

[0333] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0334] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0335] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0336] [Second embodiment]

[0337] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0338] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0339] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0340] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0341] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0342] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0343] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0344] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0345] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0346] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0347] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0348] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0349] The present invention is a system that collects information about the deceased and simulates a conversation with the deceased based on that information. This system includes an information collection means, a model construction means, an input reception means, a response generation means, and a response transmission means. The program processing flow of this system is described below.

[0350] First, the user inputs information about the deceased using the information collection tool, including photos of the deceased, voice messages, text notes, and answers to detailed questions about their personality and hobbies, which are then sent to the server and stored.

[0351] The server then uses the collected information to build a personality model of the deceased using a model building mechanism, which includes the deceased's unique phrasing and vocabulary, as well as an algorithm that learns the deceased's personality traits.

[0352] When a user logs in to the system and starts a dialogue, the input receiving means receives text data entered by the user. The user can use the system to enter what they want to talk about with the deceased in natural language.

[0353] The server analyzes the received text data using a response generation tool. The analysis is carried out using natural language processing technology to understand the context of the input text and generate an appropriate response. The server incorporates the deceased's unique vocabulary and expressions to make the dialogue with the user more natural.

[0354] The generated response is sent from the response sending means to the user's terminal via the server. A chat interface is displayed on the user's terminal, where the generated response is displayed. This allows the user to enjoy the experience of interacting with the deceased in real time.

[0355] As a concrete example, consider the case where a user types, "Dad, how was your day?" The user's device sends this input to the server. The server analyzes the input and uses a personality model of the deceased person to generate a response such as, "I'm doing well today. I'm watching over you every day." The generated response is then sent to the user's device and displayed in the chat interface. The user can view the displayed response and continue the conversation.

[0356] This system allows bereaved families to have a simulated conversation with the deceased and find comfort in their hearts. This is a particularly significant psychological support for the elderly. This invention is a groundbreaking system that heals the hearts of bereaved families through conversation with the deceased.

[0357] The processing flow will be explained below.

[0358] Step 1:

[0359] The user uses the information collection tool to enter information about the deceased, including photos of the deceased, audio recordings, text notes, and answers to detailed questions about their personality and hobbies.

[0360] Step 2:

[0361] The terminal transmits the information about the deceased person input by the user to the server.

[0362] Step 3:

[0363] The server stores the received information and uses a model building means to build a personality model of the deceased, which learns the deceased's unique phrasing, vocabulary, and other characteristics.

[0364] Step 4:

[0365] A user logs into the system and opens a chat interface to begin a conversation.

[0366] Step 5:

[0367] The user types into the chat interface what they want to start the conversation with, for example, "Dad, how was your day?"

[0368] Step 6:

[0369] The terminal transmits the text data entered by the user to the server.

[0370] Step 7:

[0371] The server receives the input and begins parsing it using a response generation method. Natural language processing techniques are used to analyze the text data and understand the context.

[0372] Step 8:

[0373] The server uses a pre-built personality model of the deceased to generate an appropriate response, taking into account the deceased's unique phrasing and vocabulary.

[0374] Step 9:

[0375] The server transmits the generated response to the user's terminal through the response transmitting means.

[0376] Step 10:

[0377] The device displays the response received from the server in the chat interface, for example, "I'm doing well today. I'm watching over you every day."

[0378] Step 11:

[0379] The user confirms the response and provides further input to continue further interaction.

[0380] Step 12:

[0381] The server stores the conversation history with the user to help continuously improve the model.

[0382] In this way, the system enriches and continuously improves the user's interaction with the deceased.

[0383] Example 1

[0384] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0385] There is a need for a system that allows bereaved family members to simulate conversations with the deceased based on information about the deceased. However, conventional systems have had difficulty generating conversations that accurately reflect the personality and characteristics of the deceased. Furthermore, the content of the conversations is unnatural, which means that the bereaved family members are not provided with sufficient healing. Furthermore, the speed of real-time conversation generation and response display is slow, which can degrade the user experience.

[0386] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0387] In this invention, the server includes an information collection means for inputting information about the deceased, a model construction means for constructing a personality model of the deceased based on the collected information, an input receiving means for receiving text data input by a user, a response generation means for analyzing the text data and generating a response, a response sending means for sending the generated response to the user in real time, and a terminal for displaying the response on a chat interface. This enables natural, real-time conversation with the deceased, and allows the bereaved family to find emotional healing through conversation with the deceased.

[0388] The "information collection means" is a means by which a user inputs information about the deceased (photos, voice messages, text notes, answers to questions, etc.) and sends it to the server.

[0389] The "model construction means" is a means for constructing a personality model of the deceased based on the information collected by the information collection means. This model includes an algorithm that learns the deceased's unique expressions, vocabulary, and personality traits.

[0390] The "input receiving means" is a means for receiving text data entered by the user, and is responsible for transmitting the user's input to the server in real time.

[0391] The "response generation means" is a means for analyzing the text data received by the input receiving means and generating an appropriate response based on a personality model. The response generation means understands the context of the text using natural language processing technology.

[0392] The "response transmitting means" is a means for transmitting the response generated by the response generating means to the user's terminal in real time.

[0393] A "terminal" is a device used by a user that displays the chat interface and shows the user the responses received.

[0394] The present invention provides a system for collecting information about a deceased person and simulating a conversation with the deceased person based on the collected information. The system includes an information collection unit, a model construction unit, an input receiving unit, a response generation unit, a response sending unit, and a terminal.

[0395] First, the user inputs information about the deceased using an information collection tool. Specific hardware used may be a device such as a PC, smartphone, or tablet. Through this information collection tool, the user inputs photos of the deceased, voice messages, text notes, and answers to detailed questions about their personality and hobbies. This data is then sent from the device to a server and stored.

[0396] The server then uses the collected information to build a personality model of the deceased using model-building tools, such as machine learning libraries like TensorFlow and PyTorch, which are used in natural language processing. This personality model includes algorithms that learn the deceased's unique phrasing, vocabulary, and personality traits.

[0397] When a user logs in to the system and opens the dialogue interface, the input receiving means receives text data entered by the user. The user can enter dialogue content in natural language through the terminal. For example, the user can enter a prompt sentence such as "Dad, how was your day?"

[0398] The server receives the text data sent via the input receiving means and analyzes it using the response generating means. This analysis uses a natural language generation model such as OpenAI's GPT-3. Through this analysis, the context of the input text is understood and an appropriate response is generated. In this process, a personality model of the deceased is referenced, and the deceased's unique phrasing and vocabulary are incorporated to make the dialogue more natural.

[0399] The generated response is sent from the server to the user's terminal via the response sending means. A chat interface is displayed on the terminal, where the generated response is displayed in real time, allowing the user to simulate a real-time conversation with the deceased.

[0400] For example, if a user types and sends "Dad, how was your day?", the server analyzes this input and generates a response based on the personality model, such as "I'm doing well today. I'm watching over you every day." This response is then sent to the user's device and displayed in the chat interface, allowing the user to continue the conversation.

[0401] Examples of prompt sentences include:

[0402] "Mom, what did you do today?"

[0403] "Grandpa, tell me some memories from your childhood."

[0404] "Grandma, what was your favorite dish?"

[0405] This system allows bereaved families to find comfort through conversation with the deceased, which is a great source of emotional support, especially for the elderly.

[0406] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0407] Step 1:

[0408] A user accesses the system and logs in.

[0409] Specifically, a user uses a device to access a web browser or dedicated application and enters authentication information (user ID and password) to log in. This authentication information is sent from the device to the server, which then references a database to perform authentication.

[0410] Input: User ID, Password

[0411] Output: Login success / failure message

[0412] Step 2:

[0413] The user inputs information about the deceased person using the information collection means.

[0414] Specifically, the user accesses a form to upload photos of the deceased, voice messages, text notes, answers to questions, etc. For example, by clicking the "Upload Photo" button, the user uploads a photo. This information is sent from the device to the server and stored on the server.

[0415] Input: Photos of the deceased, voice messages, text notes, and answers to questions

[0416] Output: Message that data has been saved to the server

[0417] Step 3:

[0418] The server builds a personality model based on the collected information.

[0419] Specifically, the server analyzes the stored data and uses machine learning libraries such as TensorFlow and PyTorch to build a personality model that learns the deceased's unique phrasing, vocabulary, and personality traits.

[0420] Input: Photos of the deceased, voice messages, text notes, and answers to questions

[0421] Output: Constructed personality model

[0422] Step 4:

[0423] The user logs back into the system and opens the interactive interface.

[0424] Specifically, the user clicks the "Start conversation" button, and the conversation interface appears. The user enters a message in natural language into the chat box. For example, the user enters a prompt sentence such as "Dad, how was your day?" The terminal then sends this message to the server.

[0425] Input: Natural language message

[0426] Output: Notification of completion of sending

[0427] Step 5:

[0428] The server receives the input text data and analyzes it using a response generation means.

[0429] Specifically, the server uses a natural language generation model, such as OpenAI's GPT-3, to understand and analyze the context of the input text. During this analysis process, the server also references a personality model and generates a response that incorporates the deceased's unique phrasing and vocabulary. For example, a response might be generated such as, "I'm doing well today. I'm watching over you every day."

[0430] Input: Natural language message, personality model

[0431] Output: The generated response message

[0432] Step 6:

[0433] The server transmits the generated response to the user's terminal in real time.

[0434] Specifically, the generated response message is sent from the server to the user's terminal, and the terminal displays it on the chat interface. The user can check the displayed response and re-enter the next dialogue content.

[0435] Input: The generated response message

[0436] Output: Displayed in the chat interface

[0437] The above is a detailed description of the specific processing steps of the program of the present invention, which allows the user to enjoy natural conversation with the deceased in real time and find comfort in their heart.

[0438] (Application example 1)

[0439] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0440] The present invention relates to a system that allows users to simulate memories and conversations with the deceased. Conventional memorial services mainly provide static information, limiting the means by which bereaved family members can actually experience conversations with the deceased. As a result, they are insufficient to recreate interactions with the deceased and provide emotional healing. Therefore, the present invention aims to provide a system that allows users to more realistically simulate conversations with the deceased, thereby providing a means for users to find emotional comfort through interactions with the deceased.

[0441] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0442] In this invention, the server includes an input means for inputting information about the deceased, a model construction means for constructing a personality model of the deceased based on the information collected by the input means, and an input receiving means for receiving text input by the user, thereby enabling the user to upload photos, audio, and text memos of the deceased and receive a response based on the personality traits of the deceased in real time on the display means of the smartphone application.

[0443] "Input means" refers to a means by which a user inputs information about the deceased.

[0444] The "model construction means" is a means for constructing a personality model of the deceased based on the collected information.

[0445] The "input receiving means" is a means for receiving text data input by a user.

[0446] A "response generator" is a means for analyzing the received text and generating a response based on a personality model of the deceased person.

[0447] The "response sending means" is a means for sending the generated response to the user's terminal.

[0448] "Uploading means" refers to a means by which a user can upload photos, audio, and text notes of the deceased.

[0449] The "display means" is a means for displaying the generated response on the user's terminal.

[0450] A "natural language generation model" is a model used to learn the specific phrasing and vocabulary of the deceased.

[0451] "Natural language processing technology" is a technology for analyzing user input and generating appropriate responses.

[0452] The present invention is a system that collects information about the deceased and imitates conversations with the deceased based on that information. This system collects information about the deceased from a user and builds an AI model that imitates the deceased's personality and phrasing based on that information. Specifically, the system includes the following means.

[0453] First, the user uses an input device to input information about the deceased. This information can include photos of the deceased, voice messages, text notes, and answers to detailed questions about their personality and hobbies. This information is then sent to a server and stored. An uploading method is provided for this purpose.

[0454] The server then uses a model building means to build a personality model of the deceased based on the collected information. The personality model includes algorithms that learn the deceased's unique phrasing, vocabulary, and personality traits. A natural language generation model is used to learn the deceased's phrasing and vocabulary.

[0455] When a user logs in to the system and begins a dialogue, the input receiving means is utilized to receive the text data entered by the user. The received text is sent to the server and analyzed by the response generating means. Natural language processing technology is used for the analysis, and includes the process of understanding the context of the input text and generating an appropriate response.

[0456] The response generated by the server is transmitted to the user's terminal via a response transmission means. The user's terminal is provided with a display means for displaying the generated response. This allows the user to enjoy a simulated experience of interacting with the deceased.

[0457] As a concrete example, consider the case where a user inputs "Dad, how was your day?" The user's device sends this input to the server. The server analyzes the input and uses a personality model of the deceased person to generate a response such as "I'm doing well today. I'm watching over you every day." The generated response is then sent to the user's device and displayed. This allows the user to view the displayed response and continue the conversation.

[0458] This system allows bereaved families to have a simulated conversation with the deceased and find comfort in their hearts. This is a particularly significant psychological support for the elderly. This invention is a groundbreaking system that heals the hearts of bereaved families through conversation with the deceased.

[0459] As a concrete example, the following is an example of a prompt sentence to be input to a generative AI model:

[0460] "Tell me a story from your garden."

[0461] "What book are you reading today?"

[0462] "What do you think about your mother?"

[0463] This allows the user to feel as if they are having a more realistic conversation with the deceased, which can bring about emotional healing.

[0464] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0465] Step 1:

[0466] The user uses the input means to input information about the deceased.

[0467] The information entered includes photos of the deceased, voice messages, text notes, and answers to detailed questions about their personality and hobbies.

[0468] The input information is sent to the server via uploading means and saved there.

[0469] Step 2:

[0470] Based on the information acquired by the server, the model construction means constructs a personality model of the deceased person.

[0471] To build this model, algorithms are used to analyze collected data and learn specific phrases, vocabulary, and personality traits.

[0472] A natural language generation model is used as the generative AI model.

[0473] Step 3:

[0474] A user logs into a terminal and begins a dialogue.

[0475] The text data input by the user is received by the input receiving means.

[0476] The received text is sent back to the server.

[0477] Step 4:

[0478] The server analyzes the text data received by the input receiving means.

[0479] It uses natural language processing techniques to understand the context of the input text and generate an appropriate response.

[0480] During the analysis process, processing is carried out to incorporate the deceased's unique phrasing and vocabulary to simulate more realistic dialogue.

[0481] Step 5:

[0482] The response generated by the server is transmitted to the user's terminal by the response transmitting means.

[0483] The generated response is displayed on the user's terminal through a display means.

[0484] Step 6:

[0485] The user confirms the response displayed on the terminal and continues the conversation.

[0486] This series of conversations takes place in real time, allowing users to virtually experience a conversation with the deceased and enjoy the process.

[0487] The above is a specific flow of processing by the system according to the present invention.

[0488] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0489] The present invention is a system that collects information about a deceased person and simulates a conversation with the deceased person based on that information. This system includes an information collection means, a model construction means, an input reception means, a response generation means, a response transmission means, and an emotion engine. The program processing flow of this system is described below.

[0490] First, the user inputs information about the deceased using the information collection tool, including photos of the deceased, voice messages, text notes, and answers to detailed questions about their personality and hobbies, which is then sent to the server and stored.

[0491] The server then uses the collected information to build a personality model of the deceased using a model building mechanism, which includes the deceased's unique phrasing and vocabulary, as well as an algorithm that learns the deceased's personality traits.

[0492] When a user logs in to the system and starts a dialogue, the input receiving means receives text data entered by the user. The user can use the system to enter what they want to talk about with the deceased in natural language.

[0493] The server analyzes the received text data using an emotion engine before sending it to the response generation means. The emotion engine recognizes the user's emotion from the user's input text and passes the recognition result to the response generation means. This emotion recognition allows the response generation means to generate an appropriate response that matches the user's emotional state.

[0494] The response generator uses natural language processing technology to analyze the text data and understand the context. Based on the analysis results and emotional information from the emotion engine, it incorporates the deceased's unique vocabulary and expressions to generate a more natural response for the user.

[0495] The generated response is sent to the user's terminal by the response sending means via the server. A chat interface is displayed on the user's terminal, where the generated response is displayed. This allows the user to enjoy the experience of interacting with the deceased in real time.

[0496] As a specific example, consider the case where a user inputs "Dad, how was your day?" The user's device sends this input to the server. The server receives the input, and the emotion engine analyzes the user's emotions. If this text is recognized as an emotional question, the response generation means generates a response that is considerate of the user's emotions. For example, if the emotion engine recognizes "loneliness," the response generation means generates a response such as "I'm doing well today. I'm always here for you so you don't feel lonely." The generated response is then sent to the user's device and displayed on the chat interface. The user can view the displayed response and continue the conversation.

[0497] In this way, the system provides emotional comfort by recreating the user's rich dialogue with the deceased and adjusting responses according to the user's emotional state. This is particularly valuable for elderly people. This is an innovative system that provides emotional comfort to bereaved families through dialogue with the deceased.

[0498] The processing flow will be explained below.

[0499] Step 1:

[0500] The user uses the information collection tool to enter information about the deceased, including photos of the deceased, voice messages, text notes, and answers to detailed questions about their personality and hobbies.

[0501] Step 2:

[0502] The terminal transmits the information about the deceased person input by the user to the server.

[0503] Step 3:

[0504] The server stores the received information and uses a model building means to build a personality model of the deceased, which learns the deceased's unique phrasing, vocabulary, and other characteristics.

[0505] Step 4:

[0506] A user logs into the system and opens a chat interface to begin a conversation.

[0507] Step 5:

[0508] The user types into the chat interface what they want to start the conversation with, for example, "Dad, how was your day?"

[0509] Step 6:

[0510] The terminal transmits the text data entered by the user to the server.

[0511] Step 7:

[0512] The server receives the input and uses an emotion engine to analyze the user's emotions from the input text data, for example, determining whether the user's input has emotions such as loneliness, sadness, or joy.

[0513] Step 8:

[0514] The server passes the user's emotion information recognized by the emotion engine to the response generating means.

[0515] Step 9:

[0516] The response generation means generates an appropriate response based on the emotional information and input text data, using a pre-built personality model of the deceased, taking into account the deceased's unique phrasing and vocabulary, and creating a response that is in tune with the user's emotions.

[0517] Step 10:

[0518] The server transmits the generated response to the user's terminal through the response transmitting means.

[0519] Step 11:

[0520] The device displays the response received from the server in the chat interface, for example, "I'm doing well today. I'm always by your side so you won't feel lonely."

[0521] Step 12:

[0522] The user confirms the response and provides further input to continue further interaction.

[0523] Step 13:

[0524] The server stores the conversation history with the user to help continuously improve the model.

[0525] In this way, the system provides a rich recreation of the user's interaction with the deceased, and even tailors responses based on the user's emotional state. This process not only provides comfort to the user, but also allows them to receive personalized responses.

[0526] Example 2

[0527] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0528] The problem to be solved by this invention is to provide comfort to a user by generating appropriate and natural responses that correspond to the user's emotional state when imitating a conversation with a deceased person. In particular, it is necessary to reproduce the deceased's unique phrasing and vocabulary to provide the user with a realistic conversation experience. It is also important that these responses are in tune with the user's emotions.

[0529] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes an information collection means for inputting general information about the deceased, a model construction means for constructing a personality model of the deceased based on the information collected by the information collection means, an input receiving means for receiving text input by a user, a response generation means for analyzing the text received by the input receiving means with an emotion engine and generating a response based on the emotion recognition result and the personality model, and a response sending means for sending the response generated by the response generation means to the user. This allows a natural response to be generated that matches the user's emotions, allowing the user to receive emotional comfort through a conversation with the deceased.

[0530] "General information about the deceased" is information that includes data that reflects the characteristics of the deceased, such as photographs of the deceased, voice messages, text notes, and answers about their personality and hobbies.

[0531] An "information collection tool" is any device or software that allows a user to enter information about the deceased person into the system.

[0532] A "model construction means" is a device or algorithm that constructs a personality model of the deceased person based on the collected information.

[0533] "Input receiving means" refers to a device or software that receives text entered by a user.

[0534] An "emotion engine" is software or algorithms that analyze user-entered text and recognize the user's emotional state.

[0535] The "response generation means" is a device or algorithm that generates a response to the user based on the analysis results of the emotion engine and the personality model.

[0536] The "response sending means" is a device or software that sends the generated response to the user.

[0537] A "natural language generation model" is an algorithm or machine learning model that learns the unique phrases and vocabulary of the deceased and generates natural-sounding dialogue.

[0538] "Natural language processing technology" is a technology for analyzing user input text and understanding its context and meaning.

[0539] The present invention is a system that collects information about the deceased and simulates a conversation with the deceased based on that information. This system includes an information collection means for collecting general information about the deceased, a model construction means for constructing a personality model based on the collected information, an input reception means for receiving text entered by a user, a response generation means for analyzing the entered text and generating a response based on the emotion recognition results and the personality model, and a response transmission means for transmitting the generated response to the user. The specific usage and processing content of each means are described below.

[0540] First, an information collection method is provided for users to enter general information about the deceased. The information collection method is implemented using a web interface or mobile app. This interface includes a function to upload photos of the deceased, a function to record voice messages, a field to input text notes, and a form to answer questions about personality and hobbies. This input data is sent to the server using a secure communication protocol (e.g., SSL / TLS). The submitted data is then stored in a database (e.g., a relational database or object storage) on the server.

[0541] The server then uses a model-building method to build a personality model based on the stored information. This process uses a machine learning framework (e.g., TensorFlow, PyTorch). Specifically, the server extracts the deceased's unique phrases, vocabulary, and personality traits from the collected text and audio data, and trains a natural language generation model. This model learns the deceased's unique phrases and vocabulary and is used to realistically recreate interactions with the user.

[0542] When a user starts a conversation with the system, the input receiving means receives the text input by the user. The user can freely input text through the chat interface of the system. This input data is transmitted to the server in real time.

[0543] The server then analyzes the data received by the input receiving means using an emotion engine. The emotion engine uses natural language processing technology (e.g., Sentiment Analysis API) to extract emotional information from the user's input text. This emotion analysis makes it possible to understand the user's emotional state. The analysis results are passed to the response generating means.

[0544] The response generator generates an appropriate response that matches the user's emotional state based on the emotion analysis results and personality model. This process uses a generative AI model (e.g., GPT-4). For example, if a user inputs "Dad, how was your day?", emotion analysis may recognize the emotion of loneliness. Based on this information, the response generator generates a response such as "I'm doing well today. I'm always here for you so you don't feel lonely."

[0545] Finally, the generated response is transmitted to the user's terminal through the response transmitting means, and a chat interface is displayed on the user's terminal, where the generated response is displayed in real time, thereby allowing the user to have an experience that mimics a conversation with the deceased person.

[0546] The following is a concrete example of a prompt sentence:

[0547] User input: "Dad, how was your day?"

[0548] System response: "I'm doing well today. I'll always be by your side so you won't feel lonely."

[0549] This system allows users to find comfort through conversation with the deceased, providing great emotional support for the elderly and their bereaved families.

[0550] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0551] Step 1: Gather information

[0552] Users use a web interface or mobile app to enter information about the deceased, including the ability to upload a photo of the deceased, record a voice message, enter text notes, and answer questions about their personality and hobbies.

[0553] Input: User-entered photos of the deceased, voice messages, text notes, and answers to questions.

[0554] Data processing: The server receives the uploaded photos and audio data and converts them into the appropriate format. It also receives text data.

[0555] Output: These data are sent to the server and stored.

[0556] Step 2: Save your information

[0557] The server stores the received information: photos and voice messages are stored in object storage, while text notes and answers to questions are stored in a relational database.

[0558] Input: Various information about the deceased person entered by the user.

[0559] Data processing: The server classifies the data into appropriate categories and stores them in the corresponding storage (e.g., database or object storage).

[0560] Output: The stored data can be accessed for subsequent processing.

[0561] Step 3: Model Building

[0562] The server builds a personality model of the deceased person based on the stored information, using machine learning frameworks (e.g., TensorFlow, PyTorch) for this process.

[0563] Input: Information about the deceased person stored in the database.

[0564] Data processing: The server analyzes text and audio data to extract specific phrases, vocabulary, and personality traits. The extracted data is then used to train a machine learning model.

[0565] Output: A personality model is generated that reflects the characteristics of the deceased.

[0566] Step 4: Receiving Input

[0567] The user initiates a conversation with the system. They can freely enter text through a chat interface, and the input data is sent to the server in real time.

[0568] Input: Text data that users enter into the chat interface.

[0569] Data processing: The server receives the input text data in real time and converts it into an analyzable format.

[0570] Output: The converted text data is used for sentiment analysis.

[0571] Step 5: Sentiment Analysis

[0572] The server analyzes the data received by the input receiving means using an emotion engine. Natural language processing technology (e.g., Sentiment Analysis API) is used to extract emotion information from the user's input text.

[0573] Input: User text data received in real time.

[0574] Data processing: The emotion engine analyzes the user's text data and recognizes their emotional state (e.g., joy, sadness, loneliness, etc.).

[0575] Output: Analysis of the user's emotional state.

[0576] Step 6: Response Generation

[0577] The response generator generates an appropriate response that matches the user's emotional state based on the emotion analysis results and personality model. This process uses a generative AI model (e.g., GPT-4).

[0578] Input: Analysis of the user's emotional state and a personality model of the deceased.

[0579] Data processing: The response generation means selects appropriate phrases and vocabulary that match the emotional state and generates natural-looking response text.

[0580] Output: The generated response text.

[0581] Step 7: Send response

[0582] The generated response is sent to the user's terminal via the response sending means and displayed on the chat interface.

[0583] Input: The generated response text.

[0584] Data processing: Formatting the response text for sending to the user's device.

[0585] Output: The response text that will be displayed to the user in their chat interface.

[0586] This process allows the system to provide users with a natural dialogue experience, enabling realistic conversations using the deceased person's unique vocabulary and expressions.

[0587] (Application example 2)

[0588] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0589] Conventional memorial systems simply store information about the deceased, but do not allow bereaved families to relive conversations with the deceased. This leaves users feeling disconnected from the deceased and unable to receive sufficient emotional comfort. Furthermore, conventional memorial systems are unable to generate responses that reflect the user's emotional state, resulting in a lack of naturalness and intimacy in the conversation. This results in insufficient emotional support for bereaved families.

[0590] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0591] In this invention, the server includes an information collection means for inputting information about the deceased, a model construction means for constructing a personality model of the deceased based on the information collected by the information collection means, an input receiving means for receiving text input by a user, an emotion analysis means for recognizing an emotional state, a response generation means for generating a response based on the emotional state recognized by the emotion analysis means and the personality model, a response sending means for sending the response generated by the response generation means to the user, and a means for providing a chat interface to the user terminal, thereby making it possible to more naturally reproduce a conversation with the deceased and generate a response appropriate to the emotional state of the user.

[0592] "Information gathering means" refers to means for inputting information about the deceased, such as photos, voice messages, text notes, and answers about their personality and hobbies.

[0593] The "model construction means" is a means for constructing a personality model of the deceased based on the information collected by the information collection means.

[0594] The "input receiving means" is a means for receiving text data input by the user.

[0595] The "emotion analysis means" is a means for analyzing and recognizing emotions from the user's text received by the input receiving means.

[0596] The "response generation means" is a means for generating a response to the user based on the emotional state of the user recognized by the emotion analysis means and a personality model.

[0597] The "response transmitting means" is a means for transmitting the response generated by the response generating means to the user's terminal.

[0598] A "chat interface" is a means of providing an interface that allows a user to enjoy real-time conversation with a deceased person.

[0599] A "generative AI model" is an artificial intelligence model that uses natural language generation technology to learn the unique phrases and vocabulary of the deceased and generate responses to user input.

[0600] A "prompt sentence" is an input sentence that a generative AI model uses to generate a response, and is a sentence that includes the user's input and emotional state.

[0601] The system for realizing this invention is a system for collecting information about the deceased and providing psychological support to a user by simulating a conversation with the deceased. The system mainly includes an information collection means, a model construction means, an input receiving means, an emotion analysis means, a response generation means, a response sending means, and a chat interface.

[0602] First, information about the deceased is collected using the user's device. Using the information collection tool, the user inputs photos of the deceased, voice messages, text notes, and answers to detailed questions about their personality and hobbies. This information is then sent to the server and stored.

[0603] The server then uses the collected information to build a personality model using a model-building method. This process uses a generative AI model to learn the deceased person's unique phrasing, vocabulary, and personality traits. The built personality model is then used to generate responses.

[0604] When a user logs in to the system and starts a dialogue, the input receiving means receives text data entered by the user. This text data is sent to the server and analyzed by the emotion analysis means. The emotion analysis means recognizes the user's emotion from the user's input text and passes the result to the response generation means.

[0605] The response generation means generates a response based on the analyzed emotional state and personality model. In this process, a generative AI model is used to generate a natural and friendly response appropriate to the user's input. The generated response is sent to the user's device by the response sending means via the server. A chat interface is displayed on the user's device, where the generated response is displayed.

[0606] As a concrete example, consider a case where a user inputs "Dad, how was your day?" If the emotion analysis means recognizes that this question is emotional, the response generation means generates a response such as "I'm doing well today. I'm always here for you so you don't feel lonely." This response is displayed in the chat interface and can be viewed by the user.

[0607] Example prompt sentence:

[0608] Father: Dad, how was your day? (Emotion: lonely)

[0609] The system is built primarily using Python and OpenAI APIs, and provides a familiar conversational experience that takes into account the user's emotional state and the personality traits of the deceased. Through the system, users can re-create conversations with the deceased and receive emotional support.

[0610] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0611] Step 1:

[0612] Using the information gathering tool, photos of the deceased, voice messages, text notes, and answers to detailed questions about their personality and hobbies are entered.

[0613] Input: Information about the deceased person entered by the user (photos, voice messages, text notes, answers to questions).

[0614] Output: Data about the deceased person that is sent to and stored on a server.

[0615] Specifically, the necessary information is collected through an input form provided on the user's terminal screen.

[0616] Step 2:

[0617] Based on the information collected by the information collecting means, the server uses the model building means to build a personality model of the deceased person.

[0618] Input: Data about the deceased collected and stored in Step 1.

[0619] Output: Personality model of the deceased person (generative AI model).

[0620] Specifically, the server uses the OpenAI API to train a natural language generation model based on the input data.

[0621] Step 3:

[0622] When a user logs in to the system and starts a dialogue, the input receiving means receives text data input by the user.

[0623] Input: Text data that the user types into the terminal.

[0624] Output: The user's text data sent to the server.

[0625] Specifically, the user's terminal transmits the input text to the server in real time.

[0626] Step 4:

[0627] The server analyzes the text received by the input receiving means and recognizes the user's emotions by the emotion analysis means.

[0628] Input: User's text data received in step 3.

[0629] Output: The analyzed emotional state of the user.

[0630] Specifically, the server uses natural language processing technology (such as OpenAI's emotion analysis API) to analyze emotions from text data.

[0631] Step 5:

[0632] The response generating means generates a response based on the emotional state recognized by the emotion analyzing means and the personality model.

[0633] Input: Emotional state recognized by the emotion analysis means, personality model constructed by the model construction means.

[0634] Output: The generated response text.

[0635] Specifically, the server uses a generative AI model to generate an appropriate response based on the user's input and emotions.

[0636] Step 6:

[0637] The server transmits the response generated by the response generating means to the user.

[0638] Input: The response text generated in step 5.

[0639] Output: The response text sent to the user's terminal.

[0640] Specifically, the server transmits the generated response in real time to the user's chat interface.

[0641] Step 7:

[0642] The user's terminal displays the sent response in a chat interface, allowing the user to continue the dialogue.

[0643] Input: The response text sent in step 6.

[0644] Output: The response text that will be displayed in the chat interface.

[0645] As a specific operation, the user's terminal displays the response text in a specific UI (user interface) component, allowing the user to view it and provide further input.

[0646] In this way, the entire system works in cooperation, allowing the user to enjoy a real-time interactive experience with the deceased.

[0647] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0648] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0649] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0650] [Third embodiment]

[0651] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0652] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0653] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0654] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0655] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0656] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0657] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0658] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0659] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0660] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0661] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0662] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0663] The present invention is a system that collects information about the deceased and simulates a conversation with the deceased based on that information. This system includes an information collection means, a model construction means, an input reception means, a response generation means, and a response transmission means. The program processing flow of this system is described below.

[0664] First, the user inputs information about the deceased using the information collection tool, including photos of the deceased, voice messages, text notes, and answers to detailed questions about their personality and hobbies, which are then sent to the server and stored.

[0665] The server then uses the collected information to build a personality model of the deceased using a model building mechanism, which includes the deceased's unique phrasing and vocabulary, as well as an algorithm that learns the deceased's personality traits.

[0666] When a user logs in to the system and starts a dialogue, the input receiving means receives text data entered by the user. The user can use the system to enter what they want to talk about with the deceased in natural language.

[0667] The server analyzes the received text data using a response generation tool. The analysis is carried out using natural language processing technology to understand the context of the input text and generate an appropriate response. The server incorporates the deceased's unique vocabulary and expressions to make the dialogue with the user more natural.

[0668] The generated response is sent from the response sending means to the user's terminal via the server. A chat interface is displayed on the user's terminal, where the generated response is displayed. This allows the user to enjoy the experience of interacting with the deceased in real time.

[0669] As a concrete example, consider the case where a user types, "Dad, how was your day?" The user's device sends this input to the server. The server analyzes the input and uses a personality model of the deceased person to generate a response such as, "I'm doing well today. I'm watching over you every day." The generated response is then sent to the user's device and displayed in the chat interface. The user can view the displayed response and continue the conversation.

[0670] This system allows bereaved families to have a simulated conversation with the deceased and find comfort in their hearts. This is a particularly significant psychological support for the elderly. This invention is a groundbreaking system that heals the hearts of bereaved families through conversation with the deceased.

[0671] The processing flow will be explained below.

[0672] Step 1:

[0673] The user uses the information collection tool to enter information about the deceased, including photos of the deceased, audio recordings, text notes, and answers to detailed questions about their personality and hobbies.

[0674] Step 2:

[0675] The terminal transmits the information about the deceased person input by the user to the server.

[0676] Step 3:

[0677] The server stores the received information and uses a model building means to build a personality model of the deceased, which learns the deceased's unique phrasing, vocabulary, and other characteristics.

[0678] Step 4:

[0679] A user logs into the system and opens a chat interface to begin a conversation.

[0680] Step 5:

[0681] The user types into the chat interface what they want to start the conversation with, for example, "Dad, how was your day?"

[0682] Step 6:

[0683] The terminal transmits the text data entered by the user to the server.

[0684] Step 7:

[0685] The server receives the input and begins parsing it using a response generation method. Natural language processing techniques are used to analyze the text data and understand the context.

[0686] Step 8:

[0687] The server uses a pre-built personality model of the deceased to generate an appropriate response, taking into account the deceased's unique phrasing and vocabulary.

[0688] Step 9:

[0689] The server transmits the generated response to the user's terminal through the response transmitting means.

[0690] Step 10:

[0691] The device displays the response received from the server in the chat interface, for example, "I'm doing well today. I'm watching over you every day."

[0692] Step 11:

[0693] The user confirms the response and provides further input to continue further interaction.

[0694] Step 12:

[0695] The server stores the conversation history with the user to help continuously improve the model.

[0696] In this way, the system enriches and continuously improves the user's interaction with the deceased.

[0697] Example 1

[0698] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0699] There is a need for a system that allows bereaved family members to simulate conversations with the deceased based on information about the deceased. However, conventional systems have had difficulty generating conversations that accurately reflect the personality and characteristics of the deceased. Furthermore, the content of the conversations is unnatural, which means that the bereaved family members are not provided with sufficient healing. Furthermore, the speed of real-time conversation generation and response display is slow, which can degrade the user experience.

[0700] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0701] In this invention, the server includes an information collection means for inputting information about the deceased, a model construction means for constructing a personality model of the deceased based on the collected information, an input receiving means for receiving text data input by a user, a response generation means for analyzing the text data and generating a response, a response sending means for sending the generated response to the user in real time, and a terminal for displaying the response on a chat interface. This enables natural, real-time conversation with the deceased, and allows the bereaved family to find emotional healing through conversation with the deceased.

[0702] The "information collection means" is a means by which a user inputs information about the deceased (photos, voice messages, text notes, answers to questions, etc.) and sends it to the server.

[0703] The "model construction means" is a means for constructing a personality model of the deceased based on the information collected by the information collection means. This model includes an algorithm that learns the deceased's unique expressions, vocabulary, and personality traits.

[0704] The "input receiving means" is a means for receiving text data entered by the user, and is responsible for transmitting the user's input to the server in real time.

[0705] The "response generation means" is a means for analyzing the text data received by the input receiving means and generating an appropriate response based on a personality model. The response generation means understands the context of the text using natural language processing technology.

[0706] The "response transmitting means" is a means for transmitting the response generated by the response generating means to the user's terminal in real time.

[0707] A "terminal" is a device used by a user that displays the chat interface and shows the user the responses received.

[0708] The present invention provides a system for collecting information about a deceased person and simulating a conversation with the deceased person based on the collected information. The system includes an information collection unit, a model construction unit, an input receiving unit, a response generation unit, a response sending unit, and a terminal.

[0709] First, the user inputs information about the deceased using an information collection tool. Specific hardware used may be a device such as a PC, smartphone, or tablet. Through this information collection tool, the user inputs photos of the deceased, voice messages, text notes, and answers to detailed questions about their personality and hobbies. This data is then sent from the device to a server and stored.

[0710] The server then uses the collected information to build a personality model of the deceased using model-building tools, such as machine learning libraries like TensorFlow and PyTorch, which are used in natural language processing. This personality model includes algorithms that learn the deceased's unique phrasing, vocabulary, and personality traits.

[0711] When a user logs in to the system and opens the dialogue interface, the input receiving means receives text data entered by the user. The user can enter dialogue content in natural language through the terminal. For example, the user can enter a prompt sentence such as "Dad, how was your day?"

[0712] The server receives the text data sent via the input receiving means and analyzes it using the response generating means. This analysis uses a natural language generation model such as OpenAI's GPT-3. Through this analysis, the context of the input text is understood and an appropriate response is generated. In this process, a personality model of the deceased is referenced, and the deceased's unique phrasing and vocabulary are incorporated to make the dialogue more natural.

[0713] The generated response is sent from the server to the user's terminal via the response sending means. A chat interface is displayed on the terminal, where the generated response is displayed in real time, allowing the user to simulate a real-time conversation with the deceased.

[0714] For example, if a user types and sends "Dad, how was your day?", the server analyzes this input and generates a response based on the personality model, such as "I'm doing well today. I'm watching over you every day." This response is then sent to the user's device and displayed in the chat interface, allowing the user to continue the conversation.

[0715] Examples of prompt sentences include:

[0716] "Mom, what did you do today?"

[0717] "Grandpa, tell me some memories from your childhood."

[0718] "Grandma, what was your favorite dish?"

[0719] This system allows bereaved families to find comfort through conversation with the deceased, which is a great source of emotional support, especially for the elderly.

[0720] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0721] Step 1:

[0722] A user accesses the system and logs in.

[0723] Specifically, a user uses a device to access a web browser or dedicated application and enters authentication information (user ID and password) to log in. This authentication information is sent from the device to the server, which then references a database to perform authentication.

[0724] Input: User ID, Password

[0725] Output: Login success / failure message

[0726] Step 2:

[0727] The user inputs information about the deceased person using the information collection means.

[0728] Specifically, the user accesses a form to upload photos of the deceased, voice messages, text notes, answers to questions, etc. For example, by clicking the "Upload Photo" button, the user uploads a photo. This information is sent from the device to the server and stored on the server.

[0729] Input: Photos of the deceased, voice messages, text notes, and answers to questions

[0730] Output: Message that data has been saved to the server

[0731] Step 3:

[0732] The server builds a personality model based on the collected information.

[0733] Specifically, the server analyzes the stored data and uses machine learning libraries such as TensorFlow and PyTorch to build a personality model that learns the deceased's unique phrasing, vocabulary, and personality traits.

[0734] Input: Photos of the deceased, voice messages, text notes, and answers to questions

[0735] Output: Constructed personality model

[0736] Step 4:

[0737] The user logs back into the system and opens the interactive interface.

[0738] Specifically, the user clicks the "Start conversation" button, and the conversation interface appears. The user enters a message in natural language into the chat box. For example, the user enters a prompt sentence such as "Dad, how was your day?" The terminal then sends this message to the server.

[0739] Input: Natural language message

[0740] Output: Notification of completion of sending

[0741] Step 5:

[0742] The server receives the input text data and analyzes it using a response generation means.

[0743] Specifically, the server uses a natural language generation model, such as OpenAI's GPT-3, to understand and analyze the context of the input text. During this analysis process, the server also references a personality model and generates a response that incorporates the deceased's unique phrasing and vocabulary. For example, a response might be generated such as, "I'm doing well today. I'm watching over you every day."

[0744] Input: Natural language message, personality model

[0745] Output: The generated response message

[0746] Step 6:

[0747] The server transmits the generated response to the user's terminal in real time.

[0748] Specifically, the generated response message is sent from the server to the user's terminal, and the terminal displays it on the chat interface. The user can check the displayed response and re-enter the next dialogue content.

[0749] Input: The generated response message

[0750] Output: Displayed in the chat interface

[0751] The above is a detailed description of the specific processing steps of the program of the present invention, which allows the user to enjoy natural conversation with the deceased in real time and find comfort in their heart.

[0752] (Application example 1)

[0753] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0754] The present invention relates to a system that allows users to simulate memories and conversations with the deceased. Conventional memorial services mainly provide static information, limiting the means by which bereaved family members can actually experience conversations with the deceased. As a result, they are insufficient to recreate interactions with the deceased and provide emotional healing. Therefore, the present invention aims to provide a system that allows users to more realistically simulate conversations with the deceased, thereby providing a means for users to find emotional comfort through interactions with the deceased.

[0755] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0756] In this invention, the server includes an input means for inputting information about the deceased, a model construction means for constructing a personality model of the deceased based on the information collected by the input means, and an input receiving means for receiving text input by the user, thereby enabling the user to upload photos, audio, and text memos of the deceased and receive a response based on the personality traits of the deceased in real time on the display means of the smartphone application.

[0757] "Input means" refers to a means by which a user inputs information about the deceased.

[0758] The "model construction means" is a means for constructing a personality model of the deceased based on the collected information.

[0759] The "input receiving means" is a means for receiving text data input by a user.

[0760] A "response generator" is a means for analyzing the received text and generating a response based on a personality model of the deceased person.

[0761] The "response sending means" is a means for sending the generated response to the user's terminal.

[0762] "Uploading means" refers to a means by which a user can upload photos, audio, and text notes of the deceased.

[0763] The "display means" is a means for displaying the generated response on the user's terminal.

[0764] A "natural language generation model" is a model used to learn the specific phrasing and vocabulary of the deceased.

[0765] "Natural language processing technology" is a technology for analyzing user input and generating appropriate responses.

[0766] The present invention is a system that collects information about the deceased and imitates conversations with the deceased based on that information. This system collects information about the deceased from a user and builds an AI model that imitates the deceased's personality and phrasing based on that information. Specifically, the system includes the following means.

[0767] First, the user uses an input device to input information about the deceased. This information can include photos of the deceased, voice messages, text notes, and answers to detailed questions about their personality and hobbies. This information is then sent to a server and stored. An uploading method is provided for this purpose.

[0768] The server then uses a model building means to build a personality model of the deceased based on the collected information. The personality model includes algorithms that learn the deceased's unique phrasing, vocabulary, and personality traits. A natural language generation model is used to learn the deceased's phrasing and vocabulary.

[0769] When a user logs in to the system and begins a dialogue, the input receiving means is utilized to receive the text data entered by the user. The received text is sent to the server and analyzed by the response generating means. Natural language processing technology is used for the analysis, and includes the process of understanding the context of the input text and generating an appropriate response.

[0770] The response generated by the server is transmitted to the user's terminal via a response transmission means. The user's terminal is provided with a display means for displaying the generated response. This allows the user to enjoy a simulated experience of interacting with the deceased.

[0771] As a concrete example, consider the case where a user inputs "Dad, how was your day?" The user's device sends this input to the server. The server analyzes the input and uses a personality model of the deceased person to generate a response such as "I'm doing well today. I'm watching over you every day." The generated response is then sent to the user's device and displayed. This allows the user to view the displayed response and continue the conversation.

[0772] This system allows bereaved families to have a simulated conversation with the deceased and find comfort in their hearts. This is a particularly significant psychological support for the elderly. This invention is a groundbreaking system that heals the hearts of bereaved families through conversation with the deceased.

[0773] As a concrete example, the following is an example of a prompt sentence to be input to a generative AI model:

[0774] "Tell me a story from your garden."

[0775] "What book are you reading today?"

[0776] "What do you think about your mother?"

[0777] This allows the user to feel as if they are having a more realistic conversation with the deceased, which can bring about emotional healing.

[0778] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0779] Step 1:

[0780] The user uses the input means to input information about the deceased.

[0781] The information entered includes photos of the deceased, voice messages, text notes, and answers to detailed questions about their personality and hobbies.

[0782] The input information is sent to the server via uploading means and saved there.

[0783] Step 2:

[0784] Based on the information acquired by the server, the model construction means constructs a personality model of the deceased person.

[0785] To build this model, algorithms are used to analyze collected data and learn specific phrases, vocabulary, and personality traits.

[0786] A natural language generation model is used as the generative AI model.

[0787] Step 3:

[0788] A user logs into a terminal and begins a dialogue.

[0789] The text data input by the user is received by the input receiving means.

[0790] The received text is sent back to the server.

[0791] Step 4:

[0792] The server analyzes the text data received by the input receiving means.

[0793] It uses natural language processing techniques to understand the context of the input text and generate an appropriate response.

[0794] During the analysis process, processing is carried out to incorporate the deceased's unique phrasing and vocabulary to simulate more realistic dialogue.

[0795] Step 5:

[0796] The response generated by the server is transmitted to the user's terminal by the response transmitting means.

[0797] The generated response is displayed on the user's terminal through a display means.

[0798] Step 6:

[0799] The user confirms the response displayed on the terminal and continues the conversation.

[0800] This series of conversations takes place in real time, allowing users to virtually experience a conversation with the deceased and enjoy the process.

[0801] The above is a specific flow of processing by the system according to the present invention.

[0802] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0803] The present invention is a system that collects information about a deceased person and simulates a conversation with the deceased person based on that information. This system includes an information collection means, a model construction means, an input reception means, a response generation means, a response transmission means, and an emotion engine. The program processing flow of this system is described below.

[0804] First, the user inputs information about the deceased using the information collection tool, including photos of the deceased, voice messages, text notes, and answers to detailed questions about their personality and hobbies, which is then sent to the server and stored.

[0805] The server then uses the collected information to build a personality model of the deceased using a model building mechanism, which includes the deceased's unique phrasing and vocabulary, as well as an algorithm that learns the deceased's personality traits.

[0806] When a user logs in to the system and starts a dialogue, the input receiving means receives text data entered by the user. The user can use the system to enter what they want to talk about with the deceased in natural language.

[0807] The server analyzes the received text data using an emotion engine before sending it to the response generation means. The emotion engine recognizes the user's emotion from the user's input text and passes the recognition result to the response generation means. This emotion recognition allows the response generation means to generate an appropriate response that matches the user's emotional state.

[0808] The response generator uses natural language processing technology to analyze the text data and understand the context. Based on the analysis results and emotional information from the emotion engine, it incorporates the deceased's unique vocabulary and expressions to generate a more natural response for the user.

[0809] The generated response is sent to the user's terminal by the response sending means via the server. A chat interface is displayed on the user's terminal, where the generated response is displayed. This allows the user to enjoy the experience of interacting with the deceased in real time.

[0810] As a specific example, consider the case where a user inputs "Dad, how was your day?" The user's device sends this input to the server. The server receives the input, and the emotion engine analyzes the user's emotions. If this text is recognized as an emotional question, the response generation means generates a response that is considerate of the user's emotions. For example, if the emotion engine recognizes "loneliness," the response generation means generates a response such as "I'm doing well today. I'm always here for you so you don't feel lonely." The generated response is then sent to the user's device and displayed on the chat interface. The user can view the displayed response and continue the conversation.

[0811] In this way, the system provides emotional comfort by recreating the user's rich dialogue with the deceased and adjusting responses according to the user's emotional state. This is particularly valuable for elderly people. This is an innovative system that provides emotional comfort to bereaved families through dialogue with the deceased.

[0812] The processing flow will be explained below.

[0813] Step 1:

[0814] The user uses the information collection tool to enter information about the deceased, including photos of the deceased, voice messages, text notes, and answers to detailed questions about their personality and hobbies.

[0815] Step 2:

[0816] The terminal transmits the information about the deceased person input by the user to the server.

[0817] Step 3:

[0818] The server stores the received information and uses a model building means to build a personality model of the deceased, which learns the deceased's unique phrasing, vocabulary, and other characteristics.

[0819] Step 4:

[0820] A user logs into the system and opens a chat interface to begin a conversation.

[0821] Step 5:

[0822] The user types into the chat interface what they want to start the conversation with, for example, "Dad, how was your day?"

[0823] Step 6:

[0824] The terminal transmits the text data entered by the user to the server.

[0825] Step 7:

[0826] The server receives the input and uses an emotion engine to analyze the user's emotions from the input text data, for example, determining whether the user's input has emotions such as loneliness, sadness, or joy.

[0827] Step 8:

[0828] The server passes the user's emotion information recognized by the emotion engine to the response generating means.

[0829] Step 9:

[0830] The response generation means generates an appropriate response based on the emotional information and input text data, using a pre-built personality model of the deceased, taking into account the deceased's unique phrasing and vocabulary, and creating a response that is in tune with the user's emotions.

[0831] Step 10:

[0832] The server transmits the generated response to the user's terminal through the response transmitting means.

[0833] Step 11:

[0834] The device displays the response received from the server in the chat interface, for example, "I'm doing well today. I'm always by your side so you won't feel lonely."

[0835] Step 12:

[0836] The user confirms the response and provides further input to continue further interaction.

[0837] Step 13:

[0838] The server stores the conversation history with the user to help continuously improve the model.

[0839] In this way, the system provides a rich recreation of the user's interaction with the deceased, and even tailors responses based on the user's emotional state. This process not only provides comfort to the user, but also allows them to receive personalized responses.

[0840] Example 2

[0841] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0842] The problem to be solved by this invention is to provide comfort to a user by generating appropriate and natural responses that correspond to the user's emotional state when imitating a conversation with a deceased person. In particular, it is necessary to reproduce the deceased's unique phrasing and vocabulary to provide the user with a realistic conversation experience. It is also important that these responses are in tune with the user's emotions.

[0843] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes an information collection means for inputting general information about the deceased, a model construction means for constructing a personality model of the deceased based on the information collected by the information collection means, an input receiving means for receiving text input by a user, a response generation means for analyzing the text received by the input receiving means with an emotion engine and generating a response based on the emotion recognition result and the personality model, and a response sending means for sending the response generated by the response generation means to the user. This allows a natural response to be generated that matches the user's emotions, allowing the user to receive emotional comfort through a conversation with the deceased.

[0844] "General information about the deceased" is information that includes data that reflects the characteristics of the deceased, such as photographs of the deceased, voice messages, text notes, and answers about their personality and hobbies.

[0845] An "information collection tool" is any device or software that allows a user to enter information about the deceased person into the system.

[0846] A "model construction means" is a device or algorithm that constructs a personality model of the deceased person based on the collected information.

[0847] "Input receiving means" refers to a device or software that receives text entered by a user.

[0848] An "emotion engine" is software or algorithms that analyze user-entered text and recognize the user's emotional state.

[0849] The "response generation means" is a device or algorithm that generates a response to the user based on the analysis results of the emotion engine and the personality model.

[0850] The "response sending means" is a device or software that sends the generated response to the user.

[0851] A "natural language generation model" is an algorithm or machine learning model that learns the unique phrases and vocabulary of the deceased and generates natural-sounding dialogue.

[0852] "Natural language processing technology" is a technology for analyzing user input text and understanding its context and meaning.

[0853] The present invention is a system that collects information about the deceased and simulates a conversation with the deceased based on that information. This system includes an information collection means for collecting general information about the deceased, a model construction means for constructing a personality model based on the collected information, an input reception means for receiving text entered by a user, a response generation means for analyzing the entered text and generating a response based on the emotion recognition results and the personality model, and a response transmission means for transmitting the generated response to the user. The specific usage and processing content of each means are described below.

[0854] First, an information collection method is provided for users to enter general information about the deceased. The information collection method is implemented using a web interface or mobile app. This interface includes a function to upload photos of the deceased, a function to record voice messages, a field to input text notes, and a form to answer questions about personality and hobbies. This input data is sent to the server using a secure communication protocol (e.g., SSL / TLS). The submitted data is then stored in a database (e.g., a relational database or object storage) on the server.

[0855] The server then uses a model-building method to build a personality model based on the stored information. This process uses a machine learning framework (e.g., TensorFlow, PyTorch). Specifically, the server extracts the deceased's unique phrases, vocabulary, and personality traits from the collected text and audio data, and trains a natural language generation model. This model learns the deceased's unique phrases and vocabulary and is used to realistically recreate interactions with the user.

[0856] When a user starts a conversation with the system, the input receiving means receives the text input by the user. The user can freely input text through the chat interface of the system. This input data is transmitted to the server in real time.

[0857] The server then analyzes the data received by the input receiving means using an emotion engine. The emotion engine uses natural language processing technology (e.g., Sentiment Analysis API) to extract emotional information from the user's input text. This emotion analysis makes it possible to understand the user's emotional state. The analysis results are passed to the response generating means.

[0858] The response generator generates an appropriate response that matches the user's emotional state based on the emotion analysis results and personality model. This process uses a generative AI model (e.g., GPT-4). For example, if a user inputs "Dad, how was your day?", emotion analysis may recognize the emotion of loneliness. Based on this information, the response generator generates a response such as "I'm doing well today. I'm always here for you so you don't feel lonely."

[0859] Finally, the generated response is transmitted to the user's terminal through the response transmitting means, and a chat interface is displayed on the user's terminal, where the generated response is displayed in real time, thereby allowing the user to have an experience that mimics a conversation with the deceased person.

[0860] The following is a concrete example of a prompt sentence:

[0861] User input: "Dad, how was your day?"

[0862] System response: "I'm doing well today. I'll always be by your side so you won't feel lonely."

[0863] This system allows users to find comfort through conversation with the deceased, providing great emotional support for the elderly and their bereaved families.

[0864] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0865] Step 1: Gather information

[0866] Users use a web interface or mobile app to enter information about the deceased, including the ability to upload a photo of the deceased, record a voice message, enter text notes, and answer questions about their personality and hobbies.

[0867] Input: User-entered photos of the deceased, voice messages, text notes, and answers to questions.

[0868] Data processing: The server receives the uploaded photos and audio data and converts them into the appropriate format. It also receives text data.

[0869] Output: These data are sent to the server and stored.

[0870] Step 2: Save your information

[0871] The server stores the received information: photos and voice messages are stored in object storage, while text notes and answers to questions are stored in a relational database.

[0872] Input: Various information about the deceased person entered by the user.

[0873] Data processing: The server classifies the data into appropriate categories and stores them in the corresponding storage (e.g., database or object storage).

[0874] Output: The stored data can be accessed for subsequent processing.

[0875] Step 3: Model Building

[0876] The server builds a personality model of the deceased person based on the stored information, using machine learning frameworks (e.g., TensorFlow, PyTorch) for this process.

[0877] Input: Information about the deceased person stored in the database.

[0878] Data processing: The server analyzes text and audio data to extract specific phrases, vocabulary, and personality traits. The extracted data is then used to train a machine learning model.

[0879] Output: A personality model is generated that reflects the characteristics of the deceased.

[0880] Step 4: Receiving Input

[0881] The user initiates a conversation with the system. They can freely enter text through a chat interface, and the input data is sent to the server in real time.

[0882] Input: Text data that users enter into the chat interface.

[0883] Data processing: The server receives the input text data in real time and converts it into an analyzable format.

[0884] Output: The converted text data is used for sentiment analysis.

[0885] Step 5: Sentiment Analysis

[0886] The server analyzes the data received by the input receiving means using an emotion engine. Natural language processing technology (e.g., Sentiment Analysis API) is used to extract emotion information from the user's input text.

[0887] Input: User text data received in real time.

[0888] Data processing: The emotion engine analyzes the user's text data and recognizes their emotional state (e.g., joy, sadness, loneliness, etc.).

[0889] Output: Analysis of the user's emotional state.

[0890] Step 6: Response Generation

[0891] The response generator generates an appropriate response that matches the user's emotional state based on the emotion analysis results and personality model. This process uses a generative AI model (e.g., GPT-4).

[0892] Input: Analysis of the user's emotional state and a personality model of the deceased.

[0893] Data processing: The response generation means selects appropriate phrases and vocabulary that match the emotional state and generates natural-looking response text.

[0894] Output: The generated response text.

[0895] Step 7: Send response

[0896] The generated response is sent to the user's terminal via the response sending means and displayed on the chat interface.

[0897] Input: The generated response text.

[0898] Data processing: Formatting the response text for sending to the user's device.

[0899] Output: The response text that will be displayed to the user in their chat interface.

[0900] This process allows the system to provide users with a natural dialogue experience, enabling realistic conversations using the deceased person's unique vocabulary and expressions.

[0901] (Application example 2)

[0902] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0903] Conventional memorial systems simply store information about the deceased, but do not allow bereaved families to relive conversations with the deceased. This leaves users feeling disconnected from the deceased and unable to receive sufficient emotional comfort. Furthermore, conventional memorial systems are unable to generate responses that reflect the user's emotional state, resulting in a lack of naturalness and intimacy in the conversation. This results in insufficient emotional support for bereaved families.

[0904] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0905] In this invention, the server includes an information collection means for inputting information about the deceased, a model construction means for constructing a personality model of the deceased based on the information collected by the information collection means, an input receiving means for receiving text input by a user, an emotion analysis means for recognizing an emotional state, a response generation means for generating a response based on the emotional state recognized by the emotion analysis means and the personality model, a response sending means for sending the response generated by the response generation means to the user, and a means for providing a chat interface to the user terminal, thereby making it possible to more naturally reproduce a conversation with the deceased and generate a response appropriate to the emotional state of the user.

[0906] "Information gathering means" refers to means for inputting information about the deceased, such as photos, voice messages, text notes, and answers about their personality and hobbies.

[0907] The "model construction means" is a means for constructing a personality model of the deceased based on the information collected by the information collection means.

[0908] The "input receiving means" is a means for receiving text data input by the user.

[0909] The "emotion analysis means" is a means for analyzing and recognizing emotions from the user's text received by the input receiving means.

[0910] The "response generation means" is a means for generating a response to the user based on the emotional state of the user recognized by the emotion analysis means and a personality model.

[0911] The "response transmitting means" is a means for transmitting the response generated by the response generating means to the user's terminal.

[0912] A "chat interface" is a means of providing an interface that allows a user to enjoy real-time conversation with a deceased person.

[0913] A "generative AI model" is an artificial intelligence model that uses natural language generation technology to learn the unique phrases and vocabulary of the deceased and generate responses to user input.

[0914] A "prompt sentence" is an input sentence that a generative AI model uses to generate a response, and is a sentence that includes the user's input and emotional state.

[0915] The system for realizing this invention is a system for collecting information about the deceased and providing psychological support to a user by simulating a conversation with the deceased. The system mainly includes an information collection means, a model construction means, an input receiving means, an emotion analysis means, a response generation means, a response sending means, and a chat interface.

[0916] First, information about the deceased is collected using the user's device. Using the information collection tool, the user inputs photos of the deceased, voice messages, text notes, and answers to detailed questions about their personality and hobbies. This information is then sent to the server and stored.

[0917] The server then uses the collected information to build a personality model using a model-building method. This process uses a generative AI model to learn the deceased person's unique phrasing, vocabulary, and personality traits. The built personality model is then used to generate responses.

[0918] When a user logs in to the system and starts a dialogue, the input receiving means receives text data entered by the user. This text data is sent to the server and analyzed by the emotion analysis means. The emotion analysis means recognizes the user's emotion from the user's input text and passes the result to the response generation means.

[0919] The response generation means generates a response based on the analyzed emotional state and personality model. In this process, a generative AI model is used to generate a natural and friendly response appropriate to the user's input. The generated response is sent to the user's device by the response sending means via the server. A chat interface is displayed on the user's device, where the generated response is displayed.

[0920] As a concrete example, consider a case where a user inputs "Dad, how was your day?" If the emotion analysis means recognizes that this question is emotional, the response generation means generates a response such as "I'm doing well today. I'm always here for you so you don't feel lonely." This response is displayed in the chat interface and can be viewed by the user.

[0921] Example prompt sentence:

[0922] Father: Dad, how was your day? (Emotion: lonely)

[0923] The system is built primarily using Python and OpenAI APIs, and provides a familiar conversational experience that takes into account the user's emotional state and the personality traits of the deceased. Through the system, users can re-create conversations with the deceased and receive emotional support.

[0924] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0925] Step 1:

[0926] Using the information gathering tool, photos of the deceased, voice messages, text notes, and answers to detailed questions about their personality and hobbies are entered.

[0927] Input: Information about the deceased person entered by the user (photos, voice messages, text notes, answers to questions).

[0928] Output: Data about the deceased person that is sent to and stored on a server.

[0929] Specifically, the necessary information is collected through an input form provided on the user's terminal screen.

[0930] Step 2:

[0931] Based on the information collected by the information collecting means, the server uses the model building means to build a personality model of the deceased person.

[0932] Input: Data about the deceased collected and stored in Step 1.

[0933] Output: Personality model of the deceased person (generative AI model).

[0934] Specifically, the server uses the OpenAI API to train a natural language generation model based on the input data.

[0935] Step 3:

[0936] When a user logs in to the system and starts a dialogue, the input receiving means receives text data input by the user.

[0937] Input: Text data that the user types into the terminal.

[0938] Output: The user's text data sent to the server.

[0939] Specifically, the user's terminal transmits the input text to the server in real time.

[0940] Step 4:

[0941] The server analyzes the text received by the input receiving means and recognizes the user's emotions by the emotion analysis means.

[0942] Input: User's text data received in step 3.

[0943] Output: The analyzed emotional state of the user.

[0944] Specifically, the server uses natural language processing technology (such as OpenAI's emotion analysis API) to analyze emotions from text data.

[0945] Step 5:

[0946] The response generating means generates a response based on the emotional state recognized by the emotion analyzing means and the personality model.

[0947] Input: Emotional state recognized by the emotion analysis means, personality model constructed by the model construction means.

[0948] Output: The generated response text.

[0949] Specifically, the server uses a generative AI model to generate an appropriate response based on the user's input and emotions.

[0950] Step 6:

[0951] The server transmits the response generated by the response generating means to the user.

[0952] Input: The response text generated in step 5.

[0953] Output: The response text sent to the user's terminal.

[0954] Specifically, the server transmits the generated response in real time to the user's chat interface.

[0955] Step 7:

[0956] The user's terminal displays the sent response in a chat interface, allowing the user to continue the dialogue.

[0957] Input: The response text sent in step 6.

[0958] Output: The response text that will be displayed in the chat interface.

[0959] As a specific operation, the user's terminal displays the response text in a specific UI (user interface) component, allowing the user to view it and provide further input.

[0960] In this way, the entire system works in cooperation, allowing the user to enjoy a real-time interactive experience with the deceased.

[0961] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0962] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0963] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[0964] [Fourth embodiment]

[0965] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[0966] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0967] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0968] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[0969] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0970] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0971] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0972] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[0973] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0974] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0975] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0976] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0977] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[0978] The present invention is a system that collects information about the deceased and simulates a conversation with the deceased based on that information. This system includes an information collection means, a model construction means, an input reception means, a response generation means, and a response transmission means. The program processing flow of this system is described below.

[0979] First, the user inputs information about the deceased using the information collection tool, including photos of the deceased, voice messages, text notes, and answers to detailed questions about their personality and hobbies, which are then sent to the server and stored.

[0980] The server then uses the collected information to build a personality model of the deceased using a model building mechanism, which includes the deceased's unique phrasing and vocabulary, as well as an algorithm that learns the deceased's personality traits.

[0981] When a user logs in to the system and starts a dialogue, the input receiving means receives text data entered by the user. The user can use the system to enter what they want to talk about with the deceased in natural language.

[0982] The server analyzes the received text data using a response generation tool. The analysis is carried out using natural language processing technology to understand the context of the input text and generate an appropriate response. The server incorporates the deceased's unique vocabulary and expressions to make the dialogue with the user more natural.

[0983] The generated response is sent from the response sending means to the user's terminal via the server. A chat interface is displayed on the user's terminal, where the generated response is displayed. This allows the user to enjoy the experience of interacting with the deceased in real time.

[0984] As a concrete example, consider the case where a user types, "Dad, how was your day?" The user's device sends this input to the server. The server analyzes the input and uses a personality model of the deceased person to generate a response such as, "I'm doing well today. I'm watching over you every day." The generated response is then sent to the user's device and displayed in the chat interface. The user can view the displayed response and continue the conversation.

[0985] This system allows bereaved families to have a simulated conversation with the deceased and find comfort in their hearts. This is a particularly significant psychological support for the elderly. This invention is a groundbreaking system that heals the hearts of bereaved families through conversation with the deceased.

[0986] The processing flow will be explained below.

[0987] Step 1:

[0988] The user uses the information collection tool to enter information about the deceased, including photos of the deceased, audio recordings, text notes, and answers to detailed questions about their personality and hobbies.

[0989] Step 2:

[0990] The terminal transmits the information about the deceased person input by the user to the server.

[0991] Step 3:

[0992] The server stores the received information and uses a model building means to build a personality model of the deceased, which learns the deceased's unique phrasing, vocabulary, and other characteristics.

[0993] Step 4:

[0994] A user logs into the system and opens a chat interface to begin a conversation.

[0995] Step 5:

[0996] The user types into the chat interface what they want to start the conversation with, for example, "Dad, how was your day?"

[0997] Step 6:

[0998] The terminal transmits the text data entered by the user to the server.

[0999] Step 7:

[1000] The server receives the input and begins parsing it using a response generation method. Natural language processing techniques are used to analyze the text data and understand the context.

[1001] Step 8:

[1002] The server uses a pre-built personality model of the deceased to generate an appropriate response, taking into account the deceased's unique phrasing and vocabulary.

[1003] Step 9:

[1004] The server transmits the generated response to the user's terminal through the response transmitting means.

[1005] Step 10:

[1006] The device displays the response received from the server in the chat interface, for example, "I'm doing well today. I'm watching over you every day."

[1007] Step 11:

[1008] The user confirms the response and provides further input to continue further interaction.

[1009] Step 12:

[1010] The server stores the conversation history with the user to help continuously improve the model.

[1011] In this way, the system enriches and continuously improves the user's interaction with the deceased.

[1012] Example 1

[1013] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1014] There is a need for a system that allows bereaved family members to simulate conversations with the deceased based on information about the deceased. However, conventional systems have had difficulty generating conversations that accurately reflect the personality and characteristics of the deceased. Furthermore, the content of the conversations is unnatural, which means that the bereaved family members are not provided with sufficient healing. Furthermore, the speed of real-time conversation generation and response display is slow, which can degrade the user experience.

[1015] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1016] In this invention, the server includes an information collection means for inputting information about the deceased, a model construction means for constructing a personality model of the deceased based on the collected information, an input receiving means for receiving text data input by a user, a response generation means for analyzing the text data and generating a response, a response sending means for sending the generated response to the user in real time, and a terminal for displaying the response on a chat interface. This enables natural, real-time conversation with the deceased, and allows the bereaved family to find emotional healing through conversation with the deceased.

[1017] The "information collection means" is a means by which a user inputs information about the deceased (photos, voice messages, text notes, answers to questions, etc.) and sends it to the server.

[1018] The "model construction means" is a means for constructing a personality model of the deceased based on the information collected by the information collection means. This model includes an algorithm that learns the deceased's unique expressions, vocabulary, and personality traits.

[1019] The "input receiving means" is a means for receiving text data entered by the user, and is responsible for transmitting the user's input to the server in real time.

[1020] The "response generation means" is a means for analyzing the text data received by the input receiving means and generating an appropriate response based on a personality model. The response generation means understands the context of the text using natural language processing technology.

[1021] The "response transmitting means" is a means for transmitting the response generated by the response generating means to the user's terminal in real time.

[1022] A "terminal" is a device used by a user that displays the chat interface and shows the user the responses received.

[1023] The present invention provides a system for collecting information about a deceased person and simulating a conversation with the deceased person based on the collected information. The system includes an information collection unit, a model construction unit, an input receiving unit, a response generation unit, a response sending unit, and a terminal.

[1024] First, the user inputs information about the deceased using an information collection tool. Specific hardware used may be a device such as a PC, smartphone, or tablet. Through this information collection tool, the user inputs photos of the deceased, voice messages, text notes, and answers to detailed questions about their personality and hobbies. This data is then sent from the device to a server and stored.

[1025] The server then uses the collected information to build a personality model of the deceased using model-building tools, such as machine learning libraries like TensorFlow and PyTorch, which are used in natural language processing. This personality model includes algorithms that learn the deceased's unique phrasing, vocabulary, and personality traits.

[1026] When a user logs in to the system and opens the dialogue interface, the input receiving means receives text data entered by the user. The user can enter dialogue content in natural language through the terminal. For example, the user can enter a prompt sentence such as "Dad, how was your day?"

[1027] The server receives the text data sent via the input receiving means and analyzes it using the response generating means. This analysis uses a natural language generation model such as OpenAI's GPT-3. Through this analysis, the context of the input text is understood and an appropriate response is generated. In this process, a personality model of the deceased is referenced, and the deceased's unique phrasing and vocabulary are incorporated to make the dialogue more natural.

[1028] The generated response is sent from the server to the user's terminal via the response sending means. A chat interface is displayed on the terminal, where the generated response is displayed in real time, allowing the user to simulate a real-time conversation with the deceased.

[1029] For example, if a user types and sends "Dad, how was your day?", the server analyzes this input and generates a response based on the personality model, such as "I'm doing well today. I'm watching over you every day." This response is then sent to the user's device and displayed in the chat interface, allowing the user to continue the conversation.

[1030] Examples of prompt sentences include:

[1031] "Mom, what did you do today?"

[1032] "Grandpa, tell me some memories from your childhood."

[1033] "Grandma, what was your favorite dish?"

[1034] This system allows bereaved families to find comfort through conversation with the deceased, which is a great source of emotional support, especially for the elderly.

[1035] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1036] Step 1:

[1037] A user accesses the system and logs in.

[1038] Specifically, a user uses a device to access a web browser or dedicated application and enters authentication information (user ID and password) to log in. This authentication information is sent from the device to the server, which then references a database to perform authentication.

[1039] Input: User ID, Password

[1040] Output: Login success / failure message

[1041] Step 2:

[1042] The user inputs information about the deceased person using the information collection means.

[1043] Specifically, the user accesses a form to upload photos of the deceased, voice messages, text notes, answers to questions, etc. For example, by clicking the "Upload Photo" button, the user uploads a photo. This information is sent from the device to the server and stored on the server.

[1044] Input: Photos of the deceased, voice messages, text notes, and answers to questions

[1045] Output: Message that data has been saved to the server

[1046] Step 3:

[1047] The server builds a personality model based on the collected information.

[1048] Specifically, the server analyzes the stored data and uses machine learning libraries such as TensorFlow and PyTorch to build a personality model that learns the deceased's unique phrasing, vocabulary, and personality traits.

[1049] Input: Photos of the deceased, voice messages, text notes, and answers to questions

[1050] Output: Constructed personality model

[1051] Step 4:

[1052] The user logs back into the system and opens the interactive interface.

[1053] Specifically, the user clicks the "Start conversation" button, and the conversation interface appears. The user enters a message in natural language into the chat box. For example, the user enters a prompt sentence such as "Dad, how was your day?" The terminal then sends this message to the server.

[1054] Input: Natural language message

[1055] Output: Notification of completion of sending

[1056] Step 5:

[1057] The server receives the input text data and analyzes it using a response generation means.

[1058] Specifically, the server uses a natural language generation model, such as OpenAI's GPT-3, to understand and analyze the context of the input text. During this analysis process, the server also references a personality model and generates a response that incorporates the deceased's unique phrasing and vocabulary. For example, a response might be generated such as, "I'm doing well today. I'm watching over you every day."

[1059] Input: Natural language message, personality model

[1060] Output: The generated response message

[1061] Step 6:

[1062] The server transmits the generated response to the user's terminal in real time.

[1063] Specifically, the generated response message is sent from the server to the user's terminal, and the terminal displays it on the chat interface. The user can check the displayed response and re-enter the next dialogue content.

[1064] Input: The generated response message

[1065] Output: Displayed in the chat interface

[1066] The above is a detailed description of the specific processing steps of the program of the present invention, which allows the user to enjoy natural conversation with the deceased in real time and find comfort in their heart.

[1067] (Application example 1)

[1068] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1069] The present invention relates to a system that allows users to simulate memories and conversations with the deceased. Conventional memorial services mainly provide static information, limiting the means by which bereaved family members can actually experience conversations with the deceased. As a result, they are insufficient to recreate interactions with the deceased and provide emotional healing. Therefore, the present invention aims to provide a system that allows users to more realistically simulate conversations with the deceased, thereby providing a means for users to find emotional comfort through interactions with the deceased.

[1070] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1071] In this invention, the server includes an input means for inputting information about the deceased, a model construction means for constructing a personality model of the deceased based on the information collected by the input means, and an input receiving means for receiving text input by the user, thereby enabling the user to upload photos, audio, and text memos of the deceased and receive a response based on the personality traits of the deceased in real time on the display means of the smartphone application.

[1072] "Input means" refers to a means by which a user inputs information about the deceased.

[1073] The "model construction means" is a means for constructing a personality model of the deceased based on the collected information.

[1074] The "input receiving means" is a means for receiving text data input by a user.

[1075] A "response generator" is a means for analyzing the received text and generating a response based on a personality model of the deceased person.

[1076] The "response sending means" is a means for sending the generated response to the user's terminal.

[1077] "Uploading means" refers to a means by which a user can upload photos, audio, and text notes of the deceased.

[1078] The "display means" is a means for displaying the generated response on the user's terminal.

[1079] A "natural language generation model" is a model used to learn the specific phrasing and vocabulary of the deceased.

[1080] "Natural language processing technology" is a technology for analyzing user input and generating appropriate responses.

[1081] The present invention is a system that collects information about the deceased and imitates conversations with the deceased based on that information. This system collects information about the deceased from a user and builds an AI model that imitates the deceased's personality and phrasing based on that information. Specifically, the system includes the following means.

[1082] First, the user uses an input device to input information about the deceased. This information can include photos of the deceased, voice messages, text notes, and answers to detailed questions about their personality and hobbies. This information is then sent to a server and stored. An uploading method is provided for this purpose.

[1083] The server then uses a model building means to build a personality model of the deceased based on the collected information. The personality model includes algorithms that learn the deceased's unique phrasing, vocabulary, and personality traits. A natural language generation model is used to learn the deceased's phrasing and vocabulary.

[1084] When a user logs in to the system and begins a dialogue, the input receiving means is utilized to receive the text data entered by the user. The received text is sent to the server and analyzed by the response generating means. Natural language processing technology is used for the analysis, and includes the process of understanding the context of the input text and generating an appropriate response.

[1085] The response generated by the server is transmitted to the user's terminal via a response transmission means. The user's terminal is provided with a display means for displaying the generated response. This allows the user to enjoy a simulated experience of interacting with the deceased.

[1086] As a concrete example, consider the case where a user inputs "Dad, how was your day?" The user's device sends this input to the server. The server analyzes the input and uses a personality model of the deceased person to generate a response such as "I'm doing well today. I'm watching over you every day." The generated response is then sent to the user's device and displayed. This allows the user to view the displayed response and continue the conversation.

[1087] This system allows bereaved families to have a simulated conversation with the deceased and find comfort in their hearts. This is a particularly significant psychological support for the elderly. This invention is a groundbreaking system that heals the hearts of bereaved families through conversation with the deceased.

[1088] As a concrete example, the following is an example of a prompt sentence to be input to a generative AI model:

[1089] "Tell me a story from your garden."

[1090] "What book are you reading today?"

[1091] "What do you think about your mother?"

[1092] This allows the user to feel as if they are having a more realistic conversation with the deceased, which can bring about emotional healing.

[1093] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1094] Step 1:

[1095] The user uses the input means to input information about the deceased.

[1096] The information entered includes photos of the deceased, voice messages, text notes, and answers to detailed questions about their personality and hobbies.

[1097] The input information is sent to the server via uploading means and saved there.

[1098] Step 2:

[1099] Based on the information acquired by the server, the model construction means constructs a personality model of the deceased person.

[1100] To build this model, algorithms are used to analyze collected data and learn specific phrases, vocabulary, and personality traits.

[1101] A natural language generation model is used as the generative AI model.

[1102] Step 3:

[1103] A user logs into a terminal and begins a dialogue.

[1104] The text data input by the user is received by the input receiving means.

[1105] The received text is sent back to the server.

[1106] Step 4:

[1107] The server analyzes the text data received by the input receiving means.

[1108] It uses natural language processing techniques to understand the context of the input text and generate an appropriate response.

[1109] During the analysis process, processing is carried out to incorporate the deceased's unique phrasing and vocabulary to simulate more realistic dialogue.

[1110] Step 5:

[1111] The response generated by the server is transmitted to the user's terminal by the response transmitting means.

[1112] The generated response is displayed on the user's terminal through a display means.

[1113] Step 6:

[1114] The user confirms the response displayed on the terminal and continues the conversation.

[1115] This series of conversations takes place in real time, allowing users to virtually experience a conversation with the deceased and enjoy the process.

[1116] The above is a specific flow of processing by the system according to the present invention.

[1117] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1118] The present invention is a system that collects information about a deceased person and simulates a conversation with the deceased person based on that information. This system includes an information collection means, a model construction means, an input reception means, a response generation means, a response transmission means, and an emotion engine. The program processing flow of this system is described below.

[1119] First, the user inputs information about the deceased using the information collection tool, including photos of the deceased, voice messages, text notes, and answers to detailed questions about their personality and hobbies, which is then sent to the server and stored.

[1120] The server then uses the collected information to build a personality model of the deceased using a model building mechanism, which includes the deceased's unique phrasing and vocabulary, as well as an algorithm that learns the deceased's personality traits.

[1121] When a user logs in to the system and starts a dialogue, the input receiving means receives text data entered by the user. The user can use the system to enter what they want to talk about with the deceased in natural language.

[1122] The server analyzes the received text data using an emotion engine before sending it to the response generation means. The emotion engine recognizes the user's emotion from the user's input text and passes the recognition result to the response generation means. This emotion recognition allows the response generation means to generate an appropriate response that matches the user's emotional state.

[1123] The response generator uses natural language processing technology to analyze the text data and understand the context. Based on the analysis results and emotional information from the emotion engine, it incorporates the deceased's unique vocabulary and expressions to generate a more natural response for the user.

[1124] The generated response is sent to the user's terminal by the response sending means via the server. A chat interface is displayed on the user's terminal, where the generated response is displayed. This allows the user to enjoy the experience of interacting with the deceased in real time.

[1125] As a specific example, consider the case where a user inputs "Dad, how was your day?" The user's device sends this input to the server. The server receives the input, and the emotion engine analyzes the user's emotions. If this text is recognized as an emotional question, the response generation means generates a response that is considerate of the user's emotions. For example, if the emotion engine recognizes "loneliness," the response generation means generates a response such as "I'm doing well today. I'm always here for you so you don't feel lonely." The generated response is then sent to the user's device and displayed on the chat interface. The user can view the displayed response and continue the conversation.

[1126] In this way, the system provides emotional comfort by recreating the user's rich dialogue with the deceased and adjusting responses according to the user's emotional state. This is particularly valuable for elderly people. This is an innovative system that provides emotional comfort to bereaved families through dialogue with the deceased.

[1127] The processing flow will be explained below.

[1128] Step 1:

[1129] The user uses the information collection tool to enter information about the deceased, including photos of the deceased, voice messages, text notes, and answers to detailed questions about their personality and hobbies.

[1130] Step 2:

[1131] The terminal transmits the information about the deceased person input by the user to the server.

[1132] Step 3:

[1133] The server stores the received information and uses a model building means to build a personality model of the deceased, which learns the deceased's unique phrasing, vocabulary, and other characteristics.

[1134] Step 4:

[1135] A user logs into the system and opens a chat interface to begin a conversation.

[1136] Step 5:

[1137] The user types into the chat interface what they want to start the conversation with, for example, "Dad, how was your day?"

[1138] Step 6:

[1139] The terminal transmits the text data entered by the user to the server.

[1140] Step 7:

[1141] The server receives the input and uses an emotion engine to analyze the user's emotions from the input text data, for example, determining whether the user's input has emotions such as loneliness, sadness, or joy.

[1142] Step 8:

[1143] The server passes the user's emotion information recognized by the emotion engine to the response generating means.

[1144] Step 9:

[1145] The response generation means generates an appropriate response based on the emotional information and input text data, using a pre-built personality model of the deceased, taking into account the deceased's unique phrasing and vocabulary, and creating a response that is in tune with the user's emotions.

[1146] Step 10:

[1147] The server transmits the generated response to the user's terminal through the response transmitting means.

[1148] Step 11:

[1149] The device displays the response received from the server in the chat interface, for example, "I'm doing well today. I'm always by your side so you won't feel lonely."

[1150] Step 12:

[1151] The user confirms the response and provides further input to continue further interaction.

[1152] Step 13:

[1153] The server stores the conversation history with the user to help continuously improve the model.

[1154] In this way, the system provides a rich recreation of the user's interaction with the deceased, and even tailors responses based on the user's emotional state. This process not only provides comfort to the user, but also allows them to receive personalized responses.

[1155] Example 2

[1156] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1157] The problem to be solved by this invention is to provide comfort to a user by generating appropriate and natural responses that correspond to the user's emotional state when imitating a conversation with a deceased person. In particular, it is necessary to reproduce the deceased's unique phrasing and vocabulary to provide the user with a realistic conversation experience. It is also important that these responses are in tune with the user's emotions.

[1158] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes an information collection means for inputting general information about the deceased, a model construction means for constructing a personality model of the deceased based on the information collected by the information collection means, an input receiving means for receiving text input by a user, a response generation means for analyzing the text received by the input receiving means with an emotion engine and generating a response based on the emotion recognition result and the personality model, and a response sending means for sending the response generated by the response generation means to the user. This allows a natural response to be generated that matches the user's emotions, allowing the user to receive emotional comfort through a conversation with the deceased.

[1159] "General information about the deceased" is information that includes data that reflects the characteristics of the deceased, such as photographs of the deceased, voice messages, text notes, and answers about their personality and hobbies.

[1160] An "information collection tool" is any device or software that allows a user to enter information about the deceased person into the system.

[1161] A "model construction means" is a device or algorithm that constructs a personality model of the deceased person based on the collected information.

[1162] "Input receiving means" refers to a device or software that receives text entered by a user.

[1163] An "emotion engine" is software or algorithms that analyze user-entered text and recognize the user's emotional state.

[1164] The "response generation means" is a device or algorithm that generates a response to the user based on the analysis results of the emotion engine and the personality model.

[1165] The "response sending means" is a device or software that sends the generated response to the user.

[1166] A "natural language generation model" is an algorithm or machine learning model that learns the unique phrases and vocabulary of the deceased and generates natural-sounding dialogue.

[1167] "Natural language processing technology" is a technology for analyzing user input text and understanding its context and meaning.

[1168] The present invention is a system that collects information about the deceased and simulates a conversation with the deceased based on that information. This system includes an information collection means for collecting general information about the deceased, a model construction means for constructing a personality model based on the collected information, an input reception means for receiving text entered by a user, a response generation means for analyzing the entered text and generating a response based on the emotion recognition results and the personality model, and a response transmission means for transmitting the generated response to the user. The specific usage and processing content of each means are described below.

[1169] First, an information collection method is provided for users to enter general information about the deceased. The information collection method is implemented using a web interface or mobile app. This interface includes a function to upload photos of the deceased, a function to record voice messages, a field to input text notes, and a form to answer questions about personality and hobbies. This input data is sent to the server using a secure communication protocol (e.g., SSL / TLS). The submitted data is then stored in a database (e.g., a relational database or object storage) on the server.

[1170] The server then uses a model-building method to build a personality model based on the stored information. This process uses a machine learning framework (e.g., TensorFlow, PyTorch). Specifically, the server extracts the deceased's unique phrases, vocabulary, and personality traits from the collected text and audio data, and trains a natural language generation model. This model learns the deceased's unique phrases and vocabulary and is used to realistically recreate interactions with the user.

[1171] When a user starts a conversation with the system, the input receiving means receives the text input by the user. The user can freely input text through the chat interface of the system. This input data is transmitted to the server in real time.

[1172] The server then analyzes the data received by the input receiving means using an emotion engine. The emotion engine uses natural language processing technology (e.g., Sentiment Analysis API) to extract emotional information from the user's input text. This emotion analysis makes it possible to understand the user's emotional state. The analysis results are passed to the response generating means.

[1173] The response generator generates an appropriate response that matches the user's emotional state based on the emotion analysis results and personality model. This process uses a generative AI model (e.g., GPT-4). For example, if a user inputs "Dad, how was your day?", emotion analysis may recognize the emotion of loneliness. Based on this information, the response generator generates a response such as "I'm doing well today. I'm always here for you so you don't feel lonely."

[1174] Finally, the generated response is transmitted to the user's terminal through the response transmitting means, and a chat interface is displayed on the user's terminal, where the generated response is displayed in real time, thereby allowing the user to have an experience that mimics a conversation with the deceased person.

[1175] The following is a concrete example of a prompt sentence:

[1176] User input: "Dad, how was your day?"

[1177] System response: "I'm doing well today. I'll always be by your side so you won't feel lonely."

[1178] This system allows users to find comfort through conversation with the deceased, providing great emotional support for the elderly and their bereaved families.

[1179] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1180] Step 1: Gather information

[1181] Users use a web interface or mobile app to enter information about the deceased, including the ability to upload a photo of the deceased, record a voice message, enter text notes, and answer questions about their personality and hobbies.

[1182] Input: User-entered photos of the deceased, voice messages, text notes, and answers to questions.

[1183] Data processing: The server receives the uploaded photos and audio data and converts them into the appropriate format. It also receives text data.

[1184] Output: These data are sent to the server and stored.

[1185] Step 2: Save your information

[1186] The server stores the received information: photos and voice messages are stored in object storage, while text notes and answers to questions are stored in a relational database.

[1187] Input: Various information about the deceased person entered by the user.

[1188] Data processing: The server classifies the data into appropriate categories and stores them in the corresponding storage (e.g., database or object storage).

[1189] Output: The stored data can be accessed for subsequent processing.

[1190] Step 3: Model Building

[1191] The server builds a personality model of the deceased person based on the stored information, using machine learning frameworks (e.g., TensorFlow, PyTorch) for this process.

[1192] Input: Information about the deceased person stored in the database.

[1193] Data processing: The server analyzes text and audio data to extract specific phrases, vocabulary, and personality traits. The extracted data is then used to train a machine learning model.

[1194] Output: A personality model is generated that reflects the characteristics of the deceased.

[1195] Step 4: Receiving Input

[1196] The user initiates a conversation with the system. They can freely enter text through a chat interface, and the input data is sent to the server in real time.

[1197] Input: Text data that users enter into the chat interface.

[1198] Data processing: The server receives the input text data in real time and converts it into an analyzable format.

[1199] Output: The converted text data is used for sentiment analysis.

[1200] Step 5: Sentiment Analysis

[1201] The server analyzes the data received by the input receiving means using an emotion engine. Natural language processing technology (e.g., Sentiment Analysis API) is used to extract emotion information from the user's input text.

[1202] Input: User text data received in real time.

[1203] Data processing: The emotion engine analyzes the user's text data and recognizes their emotional state (e.g., joy, sadness, loneliness, etc.).

[1204] Output: Analysis of the user's emotional state.

[1205] Step 6: Response Generation

[1206] The response generator generates an appropriate response that matches the user's emotional state based on the emotion analysis results and personality model. This process uses a generative AI model (e.g., GPT-4).

[1207] Input: Analysis of the user's emotional state and a personality model of the deceased.

[1208] Data processing: The response generation means selects appropriate phrases and vocabulary that match the emotional state and generates natural-looking response text.

[1209] Output: The generated response text.

[1210] Step 7: Send response

[1211] The generated response is sent to the user's terminal via the response sending means and displayed on the chat interface.

[1212] Input: The generated response text.

[1213] Data processing: Formatting the response text for sending to the user's device.

[1214] Output: The response text that will be displayed to the user in their chat interface.

[1215] This process allows the system to provide users with a natural dialogue experience, enabling realistic conversations using the deceased person's unique vocabulary and expressions.

[1216] (Application example 2)

[1217] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1218] Conventional memorial systems simply store information about the deceased, but do not allow bereaved families to relive conversations with the deceased. This leaves users feeling disconnected from the deceased and unable to receive sufficient emotional comfort. Furthermore, conventional memorial systems are unable to generate responses that reflect the user's emotional state, resulting in a lack of naturalness and intimacy in the conversation. This results in insufficient emotional support for bereaved families.

[1219] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1220] In this invention, the server includes an information collection means for inputting information about the deceased, a model construction means for constructing a personality model of the deceased based on the information collected by the information collection means, an input receiving means for receiving text input by a user, an emotion analysis means for recognizing an emotional state, a response generation means for generating a response based on the emotional state recognized by the emotion analysis means and the personality model, a response sending means for sending the response generated by the response generation means to the user, and a means for providing a chat interface to the user terminal, thereby making it possible to more naturally reproduce a conversation with the deceased and generate a response appropriate to the emotional state of the user.

[1221] "Information gathering means" refers to means for inputting information about the deceased, such as photos, voice messages, text notes, and answers about their personality and hobbies.

[1222] The "model construction means" is a means for constructing a personality model of the deceased based on the information collected by the information collection means.

[1223] The "input receiving means" is a means for receiving text data input by the user.

[1224] The "emotion analysis means" is a means for analyzing and recognizing emotions from the user's text received by the input receiving means.

[1225] The "response generation means" is a means for generating a response to the user based on the emotional state of the user recognized by the emotion analysis means and a personality model.

[1226] The "response transmitting means" is a means for transmitting the response generated by the response generating means to the user's terminal.

[1227] A "chat interface" is a means of providing an interface that allows a user to enjoy real-time conversation with a deceased person.

[1228] A "generative AI model" is an artificial intelligence model that uses natural language generation technology to learn the unique phrases and vocabulary of the deceased and generate responses to user input.

[1229] A "prompt sentence" is an input sentence that a generative AI model uses to generate a response, and is a sentence that includes the user's input and emotional state.

[1230] The system for realizing this invention is a system for collecting information about the deceased and providing psychological support to a user by simulating a conversation with the deceased. The system mainly includes an information collection means, a model construction means, an input receiving means, an emotion analysis means, a response generation means, a response sending means, and a chat interface.

[1231] First, information about the deceased is collected using the user's device. Using the information collection tool, the user inputs photos of the deceased, voice messages, text notes, and answers to detailed questions about their personality and hobbies. This information is then sent to the server and stored.

[1232] The server then uses the collected information to build a personality model using a model-building method. This process uses a generative AI model to learn the deceased person's unique phrasing, vocabulary, and personality traits. The built personality model is then used to generate responses.

[1233] When a user logs in to the system and starts a dialogue, the input receiving means receives text data entered by the user. This text data is sent to the server and analyzed by the emotion analysis means. The emotion analysis means recognizes the user's emotion from the user's input text and passes the result to the response generation means.

[1234] The response generation means generates a response based on the analyzed emotional state and personality model. In this process, a generative AI model is used to generate a natural and friendly response appropriate to the user's input. The generated response is sent to the user's device by the response sending means via the server. A chat interface is displayed on the user's device, where the generated response is displayed.

[1235] As a concrete example, consider a case where a user inputs "Dad, how was your day?" If the emotion analysis means recognizes that this question is emotional, the response generation means generates a response such as "I'm doing well today. I'm always here for you so you don't feel lonely." This response is displayed in the chat interface and can be viewed by the user.

[1236] Example prompt sentence:

[1237] Father: Dad, how was your day? (Emotion: lonely)

[1238] The system is built primarily using Python and OpenAI APIs, and provides a familiar conversational experience that takes into account the user's emotional state and the personality traits of the deceased. Through the system, users can re-create conversations with the deceased and receive emotional support.

[1239] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1240] Step 1:

[1241] Using the information gathering tool, photos of the deceased, voice messages, text notes, and answers to detailed questions about their personality and hobbies are entered.

[1242] Input: Information about the deceased person entered by the user (photos, voice messages, text notes, answers to questions).

[1243] Output: Data about the deceased person that is sent to and stored on a server.

[1244] Specifically, the necessary information is collected through an input form provided on the user's terminal screen.

[1245] Step 2:

[1246] Based on the information collected by the information collecting means, the server uses the model building means to build a personality model of the deceased person.

[1247] Input: Data about the deceased collected and stored in Step 1.

[1248] Output: Personality model of the deceased person (generative AI model).

[1249] Specifically, the server uses the OpenAI API to train a natural language generation model based on the input data.

[1250] Step 3:

[1251] When a user logs in to the system and starts a dialogue, the input receiving means receives text data input by the user.

[1252] Input: Text data that the user types into the terminal.

[1253] Output: The user's text data sent to the server.

[1254] Specifically, the user's terminal transmits the input text to the server in real time.

[1255] Step 4:

[1256] The server analyzes the text received by the input receiving means and recognizes the user's emotions by the emotion analysis means.

[1257] Input: User's text data received in step 3.

[1258] Output: The analyzed emotional state of the user.

[1259] Specifically, the server uses natural language processing technology (such as OpenAI's emotion analysis API) to analyze emotions from text data.

[1260] Step 5:

[1261] The response generating means generates a response based on the emotional state recognized by the emotion analyzing means and the personality model.

[1262] Input: Emotional state recognized by the emotion analysis means, personality model constructed by the model construction means.

[1263] Output: The generated response text.

[1264] Specifically, the server uses a generative AI model to generate an appropriate response based on the user's input and emotions.

[1265] Step 6:

[1266] The server transmits the response generated by the response generating means to the user.

[1267] Input: The response text generated in step 5.

[1268] Output: The response text sent to the user's terminal.

[1269] Specifically, the server transmits the generated response in real time to the user's chat interface.

[1270] Step 7:

[1271] The user's terminal displays the sent response in a chat interface, allowing the user to continue the dialogue.

[1272] Input: The response text sent in step 6.

[1273] Output: The response text that will be displayed in the chat interface.

[1274] As a specific operation, the user's terminal displays the response text in a specific UI (user interface) component, allowing the user to view it and provide further input.

[1275] In this way, the entire system works in cooperation, allowing the user to enjoy a real-time interactive experience with the deceased.

[1276] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1277] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1278] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1279] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1280] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1281] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1282] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1283] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1284] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1285] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1286] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1287] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1288] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1289] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1290] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1291] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1292] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1293] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1294] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1295] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1296] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1297] The following is further disclosed regarding the above embodiment.

[1298] (Claim 1)

[1299] an information gathering means for inputting information about the deceased;

[1300] a model construction means for constructing a personality model of the deceased based on the information collected by the information collection means;

[1301] input receiving means for receiving text input by a user;

[1302] a response generating means for analyzing the text received by the input receiving means and generating a response based on the personality model;

[1303] a response sending means for sending the response generated by the response generating means to a user.

[1304] (Claim 2)

[1305] 2. The system of claim 1, wherein the response generation means generates responses using a natural language generation model that learns the deceased's unique phrasing and vocabulary.

[1306] (Claim 3)

[1307] 10. The system of claim 1, wherein the response generating means uses natural language processing techniques to parse the user's input.

[1308] "Example 1"

[1309] (Claim 1)

[1310] an information gathering means for inputting information about the deceased;

[1311] a model construction means for constructing a personality model of the deceased based on the information collected by the information collection means;

[1312] an input receiving means for receiving text data input by a user;

[1313] a response generating means for analyzing the text data received by the input receiving means and generating a response based on the personality model;

[1314] a response sending means for sending the response generated by the response generating means to the user in real time;

[1315] a terminal that displays the response transmitted by the response transmitting means on a chat interface;

[1316] A system including:

[1317] (Claim 2)

[1318] 2. The system of claim 1, wherein the response generation means generates responses using a natural language generation model that learns the deceased's unique phrasing and vocabulary.

[1319] (Claim 3)

[1320] 10. The system of claim 1, wherein the response generating means uses natural language processing techniques to parse the user's input.

[1321] "Application Example 1"

[1322] (Claim 1)

[1323] an input means for inputting information about the deceased;

[1324] a model construction means for constructing a personality model of the deceased based on the information collected by the input means;

[1325] input receiving means for receiving text input by a user;

[1326] a response generating means for analyzing the text received by the input receiving means and generating a response based on the personality model;

[1327] a response sending means for sending the generated response to the user's terminal;

[1328] an uploading means for uploading photos, audio, and text notes of the deceased;

[1329] a display means for displaying a response generated using the personality model on a user's terminal;

[1330] A system including:

[1331] (Claim 2)

[1332] 10. The system of claim 1, wherein the response generation means generates responses using a natural language generation model that learns the deceased's unique phrasing and vocabulary.

[1333] (Claim 3)

[1334] 10. The system of claim 1, wherein the response generation means uses natural language processing techniques to generate responses based on personality traits of the deceased.

[1335] "Example 2: Combining Emotion Engines"

[1336] (Claim 1)

[1337] an information gathering means for inputting general information about the deceased;

[1338] a model construction means for constructing a personality model of the deceased based on the information collected by the information collection means;

[1339] input receiving means for receiving text input by a user;

[1340] a response generating means for analyzing the text received by the input receiving means with an emotion engine and generating a response based on the emotion recognition result and the personality model;

[1341] a response sending means for sending the response generated by the response generating means to a user.

[1342] (Claim 2)

[1343] 2. The system of claim 1, wherein the response generation means generates responses using a natural language generation model that learns the deceased's unique phrasing and vocabulary.

[1344] (Claim 3)

[1345] 2. The system according to claim 1, wherein the response generating means uses natural language processing technology to analyze the user's input and generates a response based on the analysis result.

[1346] "Application example 2 when combining emotion engines"

[1347] (Claim 1)

[1348] an information gathering means for inputting information about the deceased;

[1349] a model construction means for constructing a personality model of the deceased based on the information collected by the information collection means;

[1350] input receiving means for receiving text input by a user;

[1351] emotion analysis means for analyzing the text received by the input receiving means and recognizing an emotional state;

[1352] a response generating means for generating a response based on the emotional state recognized by the emotion analyzing means and the personality model;

[1353] a response sending means for sending the response generated by the response generating means to a user;

[1354] and means for providing a chat interface to the user terminal.

[1355] (Claim 2)

[1356] 2. The system of claim 1, wherein the response generation means generates responses using a natural language generation model that learns the deceased's unique phrasing and vocabulary.

[1357] (Claim 3)

[1358] 2. The system of claim 1, wherein the response generation means includes means for generating a response from a user's input using a generative AI model and adjusting the response based on the prompt sentence. [Explanation of symbols]

[1359] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. an information gathering means for inputting information about the deceased; a model construction means for constructing a personality model of the deceased based on the information collected by the information collection means; input receiving means for receiving text input by a user; a response generating means for analyzing the text received by the input receiving means and generating a response based on the personality model; a response sending means for sending the response generated by the response generating means to a user.

2. 2. The system of claim 1, wherein the response generating means generates responses using a natural language generation model that is trained to the deceased person's unique phrasing and vocabulary.

3. 10. The system of claim 1, wherein the response generating means uses natural language processing techniques to analyze the user's input.

Images