System

Abstract

A system is provided.SOLUTION: The system includes an input unit for allowing the elderly person to speak the past, an analysis unit for analyzing the spoken content and extracting an important keyword, a question generation unit for generating a next question based on the extracted keyword and presenting the next question to the elderly person, a database management unit for registering and managing profile information and past photographs of the elderly person, a sentence generation unit for automatically generating an autobiographical sentence based on received information, and a format conversion unit for selecting and outputting an autobiographical style.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Reminiscing about past events is considered an effective way to prevent dementia in the elderly, but it is often difficult to do this independently. Furthermore, in elderly care facilities, the burden on care staff is heavy, making it difficult to adequately support reminiscence sessions for individual elderly people. Furthermore, systematically recording past events and compiling them into an autobiography, rather than simply recalling them, can contribute to improving the elderly's self-esteem and communication with their families. However, there is a need for a method to effectively support this process. [Means for solving the problem]

[0005] This invention provides a system that generates autobiographical text based on elderly people's recounting of past events. Specifically, the system includes an input means for elderly people to recount their past experiences, an analysis means for analyzing the recounted content and extracting important keywords, a question generation means for generating the next question based on the extracted keywords and presenting it to the elderly, a database management means for registering and managing the elderly's profile information and past photos, a text generation means for automatically generating autobiographical text based on the received information, and a format conversion means for selecting and outputting the autobiographical style. This system allows elderly people to practice reminiscence therapy through dialogue with AI and generate their autobiography, thereby preventing dementia and promoting communication with family. Furthermore, the addition of a voice recognition means and a correction management means can improve user convenience.

[0006] "Elderly" refers to adults who have reached the later stages of life, often referring to people aged 60 or over.

[0007] "Dementia prevention" is a general term for preventative measures, activities, and treatments aimed at preventing older people from developing dementia.

[0008] Reminiscence therapy is a type of therapy that achieves psychological and cognitive benefits by recalling and talking about past events and experiences.

[0009] "AI" is an abbreviation for artificial intelligence, and refers to technology that gives computers the ability to think, learn, and make decisions like humans.

[0010] "Input means" means an interface through which a user provides data to the system, including voice input and text input.

[0011] "Analysis means" refers to the programs and algorithms used to analyze input data and extract necessary information and keywords.

[0012] The "question generation means" is a program that has the function of automatically creating the next question based on the extracted keywords and presenting it to the user.

[0013] The "database management means" is a system for registering and efficiently managing data such as user profile information and past photos.

[0014] "Text generation means" refers to a program or algorithm that automatically creates text based on collected user information.

[0015] The "format conversion means" is a program that has the function of outputting the generated autobiography in a specific style or format.

[0016] "Speech recognition means" means a technology or system that converts a user's speech into text data.

[0017] The "modification management means" is a system having a function that allows a user to check and modify the generated autobiography. [Brief explanation of the drawings]

[0018] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0019] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0020] First, the terms used in the following description will be explained.

[0021] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0022] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0023] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0024] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0025] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0026] [First embodiment]

[0027] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0028] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0029] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0030] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0031] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0032] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0033] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0034] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0035] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0036] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0037] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0038] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0039] The present invention is a system that generates autobiographical text based on the recounting of past events by elderly people. This system involves the process of inputting and analyzing the content spoken by the elderly person, generating questions to be presented next, and finally compiling the autobiography.

[0040] System configuration

[0041] user

[0042] Elderly people: They are the main users of the system and talk about their own past experiences.

[0043] Family and care staff: Provide support to the older person as needed.

[0044] Terminal

[0045] Input devices: Devices that allow the older adult to input speech, including microphones and touchscreens.

[0046] Output device: A device used to display questions and generated text for confirmation. This can include a display or speaker.

[0047] server

[0048] Database: Stores and manages elderly profile information, conversations, uploaded photos and important events.

[0049] Analysis engine: Analyzes user voice data and extracts important keywords and phrases using natural language processing technology.

[0050] Question generation engine: Automatically generates the next question based on the extracted keywords.

[0051] Text generation engine: Automatically generates autobiographical text based on collected data.

[0052] Format conversion engine: Converts the generated autobiography into a specific style (documentary, romantic comedy, etc.) and outputs it.

[0053] Program processing

[0054] Initial setup and receiving user information

[0055] User: Logs in to a device and enters basic information such as date of birth and major life events.

[0056] Terminal: The entered information is encrypted and sent to the server.

[0057] Server: Validates login information and stores user profiles in a database. Also receives past photos and important events from users and stores them in a database.

[0058] Starting a conversation

[0059] Device: Present the user with an initial question, for example, "Tell me about your childhood."

[0060] User: Speaks into the device and recounts memories of the elderly person's childhood.

[0061] Device: Converts the user's voice into text data and sends it to the server.

[0062] Data analysis and question generation

[0063] Server: Analyzes the text data and extracts important keywords.

[0064] Server: Generates the next question based on the extracted keywords and sends it to the device. For example, it might ask, "What was your most memorable camping experience?"

[0065] Editing and creating autobiographies

[0066] User: Speaks further details into the device in response to the questions presented.

[0067] Device: Converts the user's speech into text and sends it back to the server.

[0068] Server: Based on the collected data, compiles autobiographical text and adds it to the autobiographical chapters.

[0069] Server: Formats the autobiography based on the style selected by the user and generates the final version.

[0070] Output and Save

[0071] Server: Generates the completed autobiography in PDF format and sends it to the device.

[0072] Device: Shows a preview version and presents it to the user for review.

[0073] User: Review the autobiography and submit a request to make corrections if necessary.

[0074] Server: Save the final version with the changes and make it available for printing or downloading.

[0075] Specific examples

[0076] Example 1: Recounting childhood memories

[0077] User: "I spent my childhood in Hokkaido, a place rich in nature."

[0078] Device: Converts this speech into text and sends it to the server.

[0079] Server: Extracts the keyword "Hokkaido, rich in nature," generates the question "What was the most memorable event at that time?" and sends it to the device.

[0080] User: "My favorite memories are going camping with my family and fishing on a big lake."

[0081] Terminal: Converts the voice into text data and sends it back to the server.

[0082] Server: Add this episode to the "Childhood Memories" chapter and generate it as text.

[0083] Example 2: Memories from school days

[0084] User: "I spent my student days in Tokyo. I had lots of adventures with my friends."

[0085] Device: Converts this speech into text and sends it to the server.

[0086] Server: Extracts the keywords "student days," "friends," and "adventure," generates a question such as "What was the most fun adventure you had at that time?" and sends it to the device.

[0087] User: "The most fun I had was climbing Mount Fuji with my friends."

[0088] Device: Converts the voice into text data and sends it back to the server.

[0089] Server: Add this episode to the chapter "Memories of Student Life" and generate it as a sentence.

[0090] In this way, a system in which AI generates autobiographical text based on past events recounted by elderly people will be useful in preventing dementia among the elderly and promoting communication with their families.

[0091] The processing flow will be explained below.

[0092] Specific processing steps of the program

[0093] Step 1: Initial setup and receiving user information

[0094] 1.1 User: Log in to the device and launch the AI ​​autobiography creation system.

[0095] Specific actions: Enter your username and password on the login screen displayed on the device and click the "Login" button.

[0096] 1.2 Device: Sends login information to server.

[0097] Specific operation: The entered login information is encrypted and sent to the server.

[0098] 1.3 Server: Verify login information and retrieve user profile data.

[0099] What it does: Retrieves user profile information (e.g., name, date of birth, major events) from a database.

[0100] 1.4 User: Register past photos and important events (topics and turning points) via the device.

[0101] Specific actions: Click the "Upload Photo" or "Add Event" button, select a file, or enter text.

[0102] 1.5 Terminal: Sends uploaded photos and entered event information to the server.

[0103] Specific operation: Transfers the selected photo files and text data to the server.

[0104] 1.6 Server: Stores photos and event information and organizes it as structured data.

[0105] What it does: Store the photos in an image library and add the event information to the associated time series data.

[0106] Step 2: Initiating a conversation

[0107] 2.1 Terminal: Present the initial question to the user.

[0108] What it does: Uses speech synthesis technology to play the question "Tell me about your childhood."

[0109] 2.2 User: Speaks into the device and recounts childhood memories.

[0110] Specific action: "When I was a child, I often went camping with my family in Hokkaido, which is rich in nature."

[0111] 2.3 Terminal: Converts the user's speech into text data.

[0112] What it does: It uses speech recognition software to convert speech into text and send it to a server.

[0113] Step 3: Analyze the data and generate questions

[0114] 3.1 Server: Analyzes the text data and extracts important keywords and phrases.

[0115] Specific operation: Extract keywords such as "Hokkaido, rich in nature" and "camping" using a natural language processing algorithm.

[0116] 3.2 Server: Generates a new question and sends it to the device.

[0117] Specific operation: Generate questions related to "nature" and "camping" and send a text message to the device saying, "What is your most memorable camping experience?"

[0118] 3.3 Terminal: Present the new question to the user.

[0119] What it does: Converts the question text into speech and plays it back to the user.

[0120] Step 4: Editing and creating your autobiography

[0121] 4.1 User: Speaks further details into the terminal in response to the questions presented.

[0122] Specific action: "One summer day, my father, brothers and I went fishing on a big lake and caught a big salmon, and that's a memory I'll never forget."

[0123] 4.2 Terminal: The user's voice is converted back into text data and sent to the server.

[0124] What it does: It uses speech recognition technology to convert speech into text and send it to a server for analysis.

[0125] 4.3 Server: Write your autobiography chapter by chapter based on the data collected.

[0126] What it does: Use natural language generation techniques to write a chapter on "Childhood Memories" and store it in a database.

[0127] 4.4 Server: Formats the autobiography in the style selected by the user.

[0128] Specific operation: Generates text that reflects styles such as "documentary style" or "romantic comedy style."

[0129] Step 5: Export and save

[0130] 5.1 Server: Sends the completed autobiography to the user's device for confirmation.

[0131] Specific behavior: Generates a preview version of the autobiography in PDF format and sends it to the device.

[0132] 5.2 Device: Show preview and ask user for confirmation.

[0133] What it does: Displays a preview of the autobiography in a PDF viewer and provides a "Confirm" button.

[0134] 5.3 User: Review the autobiography and request corrections if necessary.

[0135] Specific actions: Click the "Confirm" button and enter any necessary corrections.

[0136] 5.4 Server: Stores the finalized autobiography and outputs it in a printable or downloadable format.

[0137] Specific behavior: Saves the final version to the database and generates PDF and print files.

[0138] In this way, each step of the system that uses AI to automatically generate an autobiography based on the memories of the elderly person is achieved through collaboration between the user, device, and server.

[0139] Example 1

[0140] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0141] Previous systems that generate autobiographical documents from elderly people's recounting of past events were insufficient in abstracting what was said and in presenting the next question, making it difficult to create a natural dialogue. Furthermore, the style and format of the generated autobiography were limited, making it difficult to flexibly respond to user needs. Furthermore, there was a lack of technology to accurately convert voice input into text, resulting in a decline in the quality of the generated documents.

[0142] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0143] In this invention, the server includes an analysis means for analyzing what has been said and extracting important keywords, a question generation means for generating the next question based on the extracted keywords and presenting it to the elderly person, a database management means for registering and managing the elderly person's profile information and past images, a document generation means for automatically generating an autobiography-style document based on the received information, and a format conversion means for selecting and outputting the autobiography style. This enables a natural flow of dialogue and enables the flexible generation of autobiographies that meet the user's needs.

[0144] The "input means" is a device that allows elderly people to input voice or text recounting past events.

[0145] "Analysis means" refers to a device or program that has the function of analyzing what is said and extracting important keywords and phrases.

[0146] The "question generation means" is a device or program that automatically generates the next question based on the extracted keywords and presents it to the elderly person.

[0147] The "database management means" is a system for registering and managing elderly people's profile information, past images, and data on important events.

[0148] The "document generation means" is a device or program that automatically generates an autobiographical document based on the received information.

[0149] The "format conversion means" is a device or program that selects the style of the generated autobiography, converts it into a specified format, and outputs it.

[0150] The "voice recognition means" is a device or program that inputs the elderly person's speech and converts it into text data.

[0151] The "revision management means" is a device or program for presenting the generated autobiographical writing to the user in a revisable format and for reflecting requests for revisions.

[0152] "Natural language processing technology" is a technology for analyzing, understanding, and generating human language.

[0153] "Elderly people" refers to the subjects who use the system to narrate past events.

[0154] This invention is an information processing system that generates an autobiographical document based on an elderly person's recounting of past events. The system uses speech recognition and natural language processing technologies to convert the elderly person's recounting into text, extract important keywords, and generate the next question. It also includes a function to select the style of the generated autobiography and output it as a document.

[0155] System configuration

[0156] Hardware

[0157] Device: A device equipped with a microphone for voice input and a touchscreen.

[0158] Server: High-performance computer for data analysis and document generation

[0159] Database: A storage system for managing profile information and past images of elderly people

[0160] Display: A screen for displaying questions and generated documentation.

[0161] Speaker: An audio device for presenting questions aloud.

[0162] software

[0163] Speech recognition software: A system that inputs elderly people's speech and converts it into text data

[0164] Natural language processing software: Software that extracts important keywords from text data

[0165] Question generation engine: A program that automatically generates the next question based on the extracted keywords.

[0166] Document generation engine: Software that automatically generates autobiographical documents based on collected data

[0167] Format conversion engine: Software that converts the generated autobiography into a specific style and outputs it.

[0168] Revision management system: Software that presents the generated autobiographical text to the user in a revisable format and reflects revisions.

[0169] System Operation

[0170] Registering user information

[0171] Users log in to their devices and enter basic information about their date of birth and major life events. This information is encrypted on the device and sent to the server, which then stores it in a database as part of the user's profile. The server also receives past images and important events from the user and stores them in the database.

[0172] Starting a conversation

[0173] The device presents the user with an initial question, for example, "Tell me about your childhood." The user speaks into the device, recounting the elderly person's childhood memories. The device converts the speech into text data and sends it to the server.

[0174] Data analysis

[0175] The server analyzes the text data and extracts important keywords. It then generates the next question based on the extracted keywords and sends it to the device. For example, it could ask a specific question such as, "What was your most memorable camping experience?"

[0176] Document Generation

[0177] The user responds to the questions with further details, which are then converted back into text and sent to the server. The server then uses the collected data to compile an autobiographical document, adding chapters to it, and formats the autobiography based on the user's chosen style to produce the final version.

[0178] Output and Modification

[0179] The completed autobiography is sent from the server to the device in PDF format. The device displays a preview version and presents it to the user for confirmation. The user reviews the autobiography and submits a request for any necessary corrections. The server saves the final version after the corrections are applied and outputs it in a format that can be printed or downloaded.

[0180] Examples and prompts

[0181] Example 1: Recounting childhood memories

[0182] When a user says, "I spent my childhood in Hokkaido, a place rich in nature," the device converts this speech into text and sends it to the server. The server extracts the keyword "Hokkaido, a place rich in nature," generates a question, "What was the most memorable event at that time?" and sends it to the device. The user then adds, "My best memory is going camping with my family and fishing on a large lake," and this is added to the autobiography as a "childhood memory."

[0183] Prompt Sentence Examples

[0184] "Tell me an anecdote from your childhood."

[0185] What is your most enjoyable memory from your school days?

[0186] "Tell me some memories you have with your family."

[0187] "Tell us about the most memorable event."

[0188] In this way, it is possible to realize natural and flowing dialogue based on the elderly person's narrative, and generate a high-quality autobiography.

[0189] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0190] Step 1: Initial setup and receiving user information

[0191] User: Log in to the device and enter the required user information (date of birth and data related to major life events).

[0192] Input: Basic information such as date of birth, major life events, etc.

[0193] Output: Initial setting data

[0194] Terminal: The entered information is encrypted and sent to the server.

[0195] Input: Data entered by the user

[0196] Data processing: Encryption processing

[0197] Output: Encrypted user information

[0198] Server: Receives user information, verifies login information, and stores user profiles in a database.

[0199] Input: Encrypted user information

[0200] Data processing: Save to database

[0201] Output: Saved user profile data

[0202] Step 2: Initiating a conversation

[0203] Device: Present the first question to the user aloud, for example, "Tell me about your childhood."

[0204] Input: Fixed question template

[0205] Data processing: Question voice synthesis

[0206] Output: Questions presented by voice

[0207] User: Speaks into device, recounting childhood memories.

[0208] Input: User's voice data

[0209] Output: Recorded audio data

[0210] Device: Converts the user's voice into text data and sends it to the server.

[0211] Input: User's voice data

[0212] Data processing: speech recognition and text conversion

[0213] Output: Text data

[0214] Step 3: Analyze the data and generate questions

[0215] Server: Analyzes the received text data and extracts important keywords using natural language processing technology.

[0216] Input: Text data

[0217] Data processing: Keyword extraction using natural language processing

[0218] Output: Extracted keywords

[0219] Server: Generates the next question based on the extracted keywords and sends it to the device.

[0220] Input: Extracted keywords

[0221] Data processing: Generate new questions using a question generation algorithm

[0222] Output: Generated questions

[0223] Step 4: Editing and creating your autobiography

[0224] User: Provides details in response to questions posed by the device.

[0225] Input: Question from terminal

[0226] Output: User's voice data

[0227] Device: Converts the user's voice into text data and sends it back to the server.

[0228] Input: User's voice data

[0229] Data processing: speech recognition and text conversion

[0230] Output: Text data

[0231] Server: Based on the collected data, compile an autobiographical document and add chapters to it.

[0232] Input: Text data and user profile

[0233] Data processing: text editing and chapter addition

[0234] Output: Updated autobiographical text

[0235] Server: Formats the autobiography based on the style selected by the user and generates the final version.

[0236] Input: Updated autobiographical text

[0237] Data processing: Applying a format conversion engine

[0238] Output: Final formatted autobiography

[0239] Step 5: Export and save

[0240] Server: Generates the completed autobiography in PDF format and sends it to the device.

[0241] Input: Final formatted autobiography

[0242] Data processing: PDF conversion

[0243] Output: Autobiography in PDF format

[0244] On the device: The generated PDF is displayed as a preview and presented to the user for review.

[0245] Input: Autobiography in PDF format

[0246] Output: Preview

[0247] User: Review the autobiography and submit a request for corrections if needed.

[0248] Input: Previewed autobiography

[0249] Output: Correction request

[0250] Server: Save the final version with the corrections and output it in a printable or downloadable format.

[0251] Input: Correction request and autobiography in PDF format

[0252] Data processing: Reflecting corrections and saving the final version

[0253] Output: Final autobiography

[0254] (Application example 1)

[0255] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0256] Elderly people's recounting of their past experiences is an important way to record and pass on personal memories to future generations. However, in today's busy living environment, family members and caregivers often do not have enough time to carefully listen to what the elderly have to say. Furthermore, there is a need to provide experiences related to past memories in order to prevent dementia and help the elderly feel a sense of spiritual fulfillment. The present invention aims to solve these problems by easily generating an autobiography based on the elderly's narrative and providing a customized meal menu based on that autobiography.

[0257] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0258] In this invention, the server includes input means for the elderly person to talk about their past, analysis means for analyzing what has been said and extracting important keywords, question generation means for generating the next question based on the extracted keywords and presenting it to the elderly person, database management means for registering and managing the elderly person's profile information and past photos, text generation means for automatically generating autobiographical text based on the received information, format conversion means for selecting and outputting the autobiographical style, and menu suggestion means for suggesting a customized meal menu based on the elderly person's story. This makes it possible to create an autobiography by having the elderly person talk about events from their past, and to provide a special dining experience based on that story.

[0259] The "elderly" are the subjects who use the system to recount past events and generate autobiographical texts.

[0260] The "input means" is a device used by elderly people to talk about past events, and is a device that allows voice input and text input.

[0261] The "analysis means" is a device or software that analyzes what the elderly say and extracts important keywords.

[0262] The "question generation means" is a device or software that automatically generates the next question based on the extracted keywords and presents it to the elderly person.

[0263] The "database management means" refers to a device or software that registers and manages profile information and past photographs of elderly people.

[0264] The "text generation means" is a device or software that automatically generates autobiographical text based on received information.

[0265] The "format conversion means" is a device or software that selects the style of the generated autobiography, converts it into a specific format, and outputs it.

[0266] The "menu suggestion means" is a device or software that suggests a customized meal menu based on the elderly person's narration.

[0267] This invention is a system that generates autobiographical text by having an elderly person recount events from their past, and proposes a customized meal menu based on that narrative. Specific embodiments of this system will be described in detail below.

[0268] user

[0269] senior citizen

[0270] The subject who uses the system talks about events from his or her past. The system analyzes what he or she says and ultimately compiles it into an autobiography.

[0271] Family members and caregivers

[0272] Provide support to the elderly as needed and help them use the system.

[0273] Terminal

[0274] Input Devices

[0275] This device allows elderly people to input their speech, and includes a microphone and a touchscreen.

[0276] Output Device

[0277] A device for displaying generated questions, autobiographical text, and suggested meal menus, including a display and speaker.

[0278] server

[0279] Database Management Methods

[0280] Stores and manages data such as elderly people's profile information, past photos, and spoken content.

[0281] Analysis means

[0282] It analyzes user speech (audio data) and extracts important keywords and phrases using voice recognition technology (e.g., Google® speech-to-text API) and natural language processing technology (e.g., Hugging Face's Transformers library).

[0283] Question generation means

[0284] The next question is automatically generated based on the extracted keywords, using a generative AI model (e.g., GPT-2).

[0285] Sentence generation means

[0286] Based on the collected data, an autobiographical text is edited and generated, again using natural language generation technology (e.g., GPT-2).

[0287] Format conversion method

[0288] The generated autobiography is converted into a specific style (documentary, romantic comedy, etc.) and output.

[0289] Menu suggestion method

[0290] This system proposes customized meal menus based on the elderly's narratives. It extracts food-related information from keywords in the narratives and proposes related menus.

[0291] Program processing explanation

[0292] The server receives the voice input data of what the elderly person is saying and uses an analysis means to extract important keywords and phrases. Based on the analyzed data, a question generation means generates the next question and sends it to the terminal to present to the user. The user's voice response is sent back to the server, and by repeating the same process, an autobiographical-style text is gradually generated.

[0293] Furthermore, food-related keywords are extracted from the analyzed content, and the menu suggestion means suggests meal menus that are memorable to the elderly.

[0294] For example, if an elderly person says, "When I was a child, I loved the miso soup my grandmother made for me," the analysis method will extract the keywords "childhood," "grandmother," and "miso soup," and generate the next question, "What other dish did you like most at that time?" Based on the answer to that question, the system will ask the next question and eventually suggest related menu items such as "miso soup set" or "grandmother's special miso soup."

[0295] Prompt Sentence Examples

[0296] "Logged in user: {username} said the following: 'When I was a child, I loved the miso soup my grandmother made for me.' Please generate follow-up questions."

[0297] The above is a specific embodiment for carrying out the present invention. This system allows elderly people to create their autobiographies by talking about their past, and furthermore, it makes it possible to provide special dining experiences based on past memories.

[0298] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0299] Step 1:

[0300] A user logs into the system and enters their profile information (date of birth, major life events, etc.) The entered information is sent to the server in the form of data, which is then stored in the database.

[0301] Step 2:

[0302] The device generates an initial question and presents it to the user, for example, "Tell me about your childhood," either by voice or text. The user begins to recount events from their past.

[0303] Step 3:

[0304] The user speaks their answer through a voice input device, and this voice data is input into the terminal.

[0305] Step 4:

[0306] The device converts the voice data into text data using voice recognition technology (e.g., Google speech-to-text API), and the converted text data is sent to the server.

[0307] Step 5:

[0308] The server receives the text data and uses analytical tools to extract important keywords and phrases. Natural language processing techniques (e.g., Hugging Face's Transformers library) are used. The results of this analysis are used to generate the next question.

[0309] Step 6:

[0310] The server generates the next question based on the extracted keywords. It uses a generative AI model (e.g., GPT-2) to create an appropriate next question. For example, a question like, "What was the most memorable event at that time?" is generated. The generated question is sent to the device.

[0311] Step 7:

[0312] The device presents the generated question to the user, who answers the new question and then speaks again.

[0313] Step 8:

[0314] The server receives the voice data again and repeats the same process. Finally, when enough information has been collected, an autobiographical text is automatically generated by the text generation means.

[0315] Step 9:

[0316] The server uses a text generation tool to edit and generate the collected data along an autobiography-like flow. The generated autobiography is formatted based on the style selected by the user (e.g., documentary style, romantic comedy style).

[0317] Step 10:

[0318] The server sends the formatted autobiography to the terminal and displays it as a preview to the user. The user can submit correction requests as needed. If no corrections are required, the final version is generated.

[0319] Step 11:

[0320] The server then saves the final autobiography in a format such as PDF, and outputs it in a printable or downloadable format. At the same time, a menu suggestion tool is used to suggest a customized meal menu based on the elderly person's story. This menu suggestion is also displayed on the device.

[0321] The above is the flow of specific processing steps of the system that realizes the application example.

[0322] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0323] This invention combines an emotion engine with a system that generates autobiographical text based on the elderly's recounting of past events. The system inputs what the elderly person is saying, analyzes it, generates questions to pose next, and ultimately compiles it into an autobiography with emotional depth. The system also recognizes the user's emotions and asks appropriate questions and responds based on the results.

[0324] System configuration

[0325] user

[0326] Elderly people: They are the main users of the system and talk about their own past experiences.

[0327] Family and care staff: Provide support to the older person as needed.

[0328] Terminal

[0329] Input devices: Devices that allow the older adult to input speech, including microphones and touchscreens.

[0330] Output device: A device used to display questions and generated text for confirmation. This can include a display or speaker.

[0331] server

[0332] Database: Stores and manages elderly people's profile information, spoken content, uploaded photos, important events, and emotion recognition results.

[0333] Analysis engine: Analyzes user voice data and extracts important keywords and phrases using natural language processing technology.

[0334] Question generation engine: Automatically generates the next question based on the extracted keywords.

[0335] Emotion engine: Recognizes emotions from the user's voice and facial expressions and tailors questions and responses accordingly, adding depth to sentences depending on the user's emotional state.

[0336] Text generation engine: Automatically generates autobiographical text based on collected data.

[0337] Format conversion engine: Converts the generated autobiography into a specific style (documentary, romantic comedy, etc.) and outputs it.

[0338] Program processing

[0339] Initial setup and receiving user information

[0340] User: Log in to the device and enter basic information (date of birth, major life events, etc.).

[0341] Terminal: The entered information is encrypted and sent to the server.

[0342] Server: Validates login information and stores user profiles in a database. Also receives past photos and important events from users and stores them in a database.

[0343] Starting a conversation

[0344] Device: Present the user with an initial question, for example, "Tell me about your childhood."

[0345] User: Speaks into device, recounting childhood memories.

[0346] Device: Converts the user's voice into text data and sends it to the server.

[0347] Data analysis and question generation

[0348] Server: Analyzes the text data and extracts important keywords.

[0349] Server: The emotion engine recognizes the user's emotional state and reflects it when generating the next question. For example, if the user is telling a happy story, it generates the question, "What was the most memorable happy event at that time?"

[0350] Editing and creating autobiographies

[0351] User: Speaks further details into the device in response to the questions presented.

[0352] Device: The user's voice is converted back into text data and sent to the server.

[0353] Server: Writes an autobiography chapter by chapter based on collected data and emotion recognition results.

[0354] Server: Formats the autobiography based on the style selected by the user and generates the final version.

[0355] Output and Save

[0356] Server: Generates the completed autobiography in PDF format and sends it to the device.

[0357] Device: Shows a preview version and presents it to the user for review.

[0358] User: Review the autobiography and submit a request to make corrections if necessary.

[0359] Server: Save the final version with the changes and make it available for printing or downloading.

[0360] Specific examples

[0361] Example 1: Emotion recognition of childhood memories

[0362] User: "I spent my childhood in Hokkaido, a place rich in nature. I enjoyed every day."

[0363] Device: Converts this speech into text and sends it to the server.

[0364] Server: Extracts keywords such as "Hokkaido, rich in nature" and "fun," and recognizes the user's enjoyment from the tone of their voice. Generates a question such as "What was the most memorable and enjoyable event at that time?" and sends it to the device.

[0365] User: "My favorite memories are going camping with my family and fishing on a big lake."

[0366] Terminal: Converts the voice into text data and sends it back to the server.

[0367] Server: Add this episode to the "Childhood Memories" chapter and generate it as a sentence containing emotions.

[0368] Example 2: Emotion recognition of memories from school days

[0369] User: "I spent my student days in Tokyo. I had lots of adventures with my friends."

[0370] Device: Converts this speech into text and sends it to the server.

[0371] Server: Extracts keywords such as "student days" and "adventure" and recognizes excited emotions from the user's voice. Generates a question such as "What was the most fun adventure you had at that time?" and sends it to the device.

[0372] User: "The most fun I had was climbing Mount Fuji with my friends."

[0373] Device: Converts the voice into text data and sends it back to the server.

[0374] Server: Add this episode to the chapter "Memories of Student Life" and generate a sentence containing emotions.

[0375] In this way, a system in which AI recognizes emotions based on past events recounted by elderly people and automatically generates an autobiography accordingly will be even more effective in preventing dementia among the elderly and promoting communication with their families.

[0376] The processing flow will be explained below.

[0377] Specific processing steps of the program

[0378] Step 1: Initial setup and receiving user information

[0379] 1.1 User: Log in to the device and launch the AI ​​autobiography creation system.

[0380] Specific actions: Enter your username and password on the login screen and click the "Login" button.

[0381] 1.2 Device: Sends login information to server.

[0382] Specific behavior: Encrypts the entered login information and sends it to the server.

[0383] 1.3 Server: Verify login information and retrieve user profile data.

[0384] What it does: Retrieves user profile information (e.g., name, date of birth, major events) from a database.

[0385] 1.4 User: Register past photos and important events (topics and turning points) via the device.

[0386] Specific actions: Click the "Upload Photo" or "Add Event" button to select a file or enter text.

[0387] 1.5 Terminal: Sends uploaded photos and entered event information to the server.

[0388] Specific operation: Transfers the selected photo files and text data to the server.

[0389] 1.6 Server: Stores photos and event information and organizes it as structured data.

[0390] What it does: Store the photos in an image library and add the event information to the associated time series data.

[0391] Step 2: Initiating a conversation

[0392] 2.1 Terminal: Present the initial question to the user.

[0393] What it does: Uses speech synthesis technology to play the question "Tell me about your childhood."

[0394] 2.2 User: Speaks into the device and recounts childhood memories.

[0395] Specific action: "When I was a child, I often went camping with my family in Hokkaido, which is rich in nature."

[0396] 2.3 Terminal: Converts the user's speech into text data.

[0397] What it does: It uses speech recognition software to convert speech into text and send it to a server.

[0398] Step 3: Analyze the data and generate questions

[0399] 3.1 Server: Analyzes the text data and extracts important keywords and phrases.

[0400] Specific operation: Extract keywords such as "Hokkaido, rich in nature" and "camping" using a natural language processing algorithm.

[0401] 3.2 Server: The emotion engine recognizes the user's emotional state and reflects it when generating the next question.

[0402] Specific operation: Analyzes the user's emotions from their tone of voice and facial expressions, generates a question such as "What was the most memorable and enjoyable event at that time?" and sends it to the device.

[0403] 3.3 Terminal: Present the new question to the user.

[0404] What it does: Converts the question text into speech and plays it back to the user.

[0405] Step 4: Editing and creating your autobiography

[0406] 4.1 User: Speaks further details into the terminal in response to the questions presented.

[0407] Specific action: "One summer day, my father, brothers and I went fishing on a big lake and caught a big salmon, and that's a memory I'll never forget."

[0408] 4.2 Terminal: The user's voice is converted back into text data and sent to the server.

[0409] What it does: It uses speech recognition technology to convert speech into text and send it to a server for analysis.

[0410] 4.3 Server: Writes an autobiography chapter by chapter based on the collected data and emotion recognition results.

[0411] What it does: Use natural language generation technology to write a chapter on "Childhood Memories," edit it into emotionally rich text, and store it in a database.

[0412] 4.4 Server: Formats the autobiography in the style selected by the user.

[0413] Specific operation: Generates text that reflects styles such as "documentary style" or "romantic comedy style."

[0414] Step 5: Export and save

[0415] 5.1 Server: Sends the completed autobiography to the user's device for confirmation.

[0416] Specific behavior: Generates a preview version of the autobiography in PDF format and sends it to the device.

[0417] 5.2 Device: Show preview and ask user for confirmation.

[0418] What it does: Displays a preview of the autobiography in a PDF viewer and provides a "Confirm" button.

[0419] 5.3 User: Review the autobiography and request corrections if necessary.

[0420] Specific actions: Click the "Confirm" button and enter any necessary corrections.

[0421] 5.4 Server: Stores the finalized autobiography and outputs it in a printable or downloadable format.

[0422] Specific behavior: Saves the final version to the database and generates PDF and print files.

[0423] In this way, each step of the system, which uses AI to recognize emotions and automatically generate an autobiography based on memories from the past recounted by an elderly person, is achieved through collaboration between the user, device, and server.

[0424] Example 2

[0425] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0426] When elderly people recount past events, organizing them into autobiographical text takes time and effort. Furthermore, generating high-quality text that incorporates emotion requires generating appropriate questions. Furthermore, correcting and formatting the generated text is also time-consuming. Thus, there is a need for an efficient system that can automatically organize and write down past events with emotional depth.

[0427] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0428] In this invention, the server includes emotion recognition means, analysis means, and question generation means, which enable the server to recognize emotions from the elderly person's narrative, analyze the content, and generate appropriate questions, thereby efficiently generating high-quality autobiographies with emotional depth.

[0429] "Input means" refers to devices used by older adults to recount past events, including voice input and touchscreens.

[0430] The "analysis means" is a technology for analyzing what the elderly say and extracting important keywords, and is a device that uses natural language processing technology.

[0431] The "question generation means" is a technology for automatically generating the next question based on the extracted keywords and presenting it to the elderly person.

[0432] "Database management means" refers to technology for registering and managing elderly people's profile information and past photographs.

[0433] "Emotion recognition means" is a technology that recognizes the user's emotions from their voice and facial expressions and asks appropriate questions and responds accordingly.

[0434] The "text generation means" is a technology for automatically generating autobiographical text based on received information.

[0435] The "format conversion means" is a technique for selecting the style of the generated autobiography and outputting it in a specified format.

[0436] "Voice recognition means" is a technology that inputs the elderly person's speech and converts it into text data.

[0437] The "revision management means" is a technique for presenting the generated autobiographical text to the user in a revisable format and for reflecting the user's revisions.

[0438] This invention relates to a system that generates autobiographical text based on the elderly's recounting of past events. This system involves a process of inputting the elderly's speech, analyzing it, generating questions to be presented next, and finally compiling it into an autobiography with emotional depth. The system can also recognize the user's emotions and provide appropriate questions and responses based on the results.

[0439] System configuration

[0440] The system mainly consists of the following elements:

[0441] user

[0442] Elderly people: They are the main users of the system and talk about their own past experiences.

[0443] Family and care staff: Provide support to the older person as needed.

[0444] Terminal

[0445] Input devices: Devices that allow the older adult to input speech, including microphones and touchscreens.

[0446] Output device: A device that displays questions and generated text for confirmation. This includes displays and speakers.

[0447] server

[0448] Database: Stores and manages elderly people's profile information, spoken content, uploaded photos, important events, and emotion recognition results.

[0449] Analysis engine: Analyzes user voice data and extracts important keywords and phrases using natural language processing technology.

[0450] Question generation engine: Automatically generates the next question based on the extracted keywords.

[0451] Emotion engine: Recognizes emotions from the user's voice and facial expressions and tailors questions and responses accordingly, adding depth to sentences depending on the user's emotional state.

[0452] Text generation engine: Automatically generates autobiographical text based on collected data.

[0453] Format conversion engine: Converts the generated autobiography into a specific style (e.g., documentary or romantic comedy) and outputs it.

[0454] System Operation

[0455] Hardware

[0456] Microphone: An input device for receiving a user's voice input.

[0457] Display: An output device used to display questions and generated text.

[0458] Server: A computer on which various engines (analysis engine, question generation engine, emotion engine, sentence generation engine, format conversion engine) run.

[0459] software

[0460] Natural Language Processing: A technique used by analytics engines to extract important keywords and phrases from what users say.

[0461] Emotion Recognition Algorithm: This is the technology used in the emotion engine to recognize emotions from the user's voice and facial expressions.

[0462] AI model: A generative AI model used for question generation and sentence generation.

[0463] Specific examples

[0464] Example 1: Emotion recognition of childhood memories

[0465] User: "I spent my childhood in Hokkaido, a place rich in nature. I enjoyed every day."

[0466] Device: Converts this speech into text and sends it to the server.

[0467] Server: Extracts keywords such as "Hokkaido, rich in nature" and "fun," and recognizes the user's enjoyment from the tone of their voice. Generates a question such as "What was the most memorable and enjoyable event at that time?" and sends it to the device.

[0468] User: "My favorite memories are going camping with my family and fishing on a big lake."

[0469] Terminal: Converts the voice into text data and sends it back to the server.

[0470] Server: Add this episode to the "Childhood Memories" chapter and generate it as a sentence containing emotions.

[0471] Example 2: Emotion recognition of memories from school days

[0472] User: "I spent my student days in Tokyo. I had lots of adventures with my friends."

[0473] Device: Converts this speech into text and sends it to the server.

[0474] Server: Extracts keywords such as "student days" and "adventure" and recognizes excited emotions from the user's voice. Generates a question such as "What was the most fun adventure you had at that time?" and sends it to the device.

[0475] User: "The most fun I had was climbing Mount Fuji with my friends."

[0476] Device: Converts the voice into text data and sends it back to the server.

[0477] Server: Add this episode to the chapter "Memories of Student Life" and generate a sentence containing emotions.

[0478] Prompt Sentence Examples

[0479] 1. A user logs in and enters basic information. Please encrypt this information before sending it.

[0480] 2. Input the user's voice data and convert it into text data.

[0481] 3. Extract specific keywords from text data and recognize the user's emotional state.

[0482] 4. Generate the next appropriate question based on the perceived emotional state and present it to the user.

[0483] 5. Collect the user's responses again and analyze the detailed data on the server.

[0484] 6. Create an autobiography chapter by chapter based on the data you have collected.

[0485] 7. Format your final autobiography in the specified style and print it in PDF format.

[0486] 8. Receive user correction requests, revise and save the final version.

[0487] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0488] Step 1: Enter your user information

[0489] A user logs in to a device and enters basic information such as date of birth and major life events. The device encrypts the information entered and sends it to the server. At this time, the user's basic information is input and the encrypted data is sent to the server as output. The server verifies the login information and stores the user profile in a database. Past photos and important events provided by the user are also stored in the database.

[0490] Step 2: Posting initial questions and receiving answers

[0491] The device presents the user with an initial question. For example, "Tell me about your childhood," which is displayed as voice or text. The user speaks into the device and recounts their childhood memories. The user's voice is the input, and is converted into text data as output, which is sent to the server. The device then converts this voice into text data and sends it to the server.

[0492] Step 3: Data analysis and emotion recognition

[0493] The server analyzes the received text data and extracts important keywords. Specifically, it uses natural language processing technology. At this time, the input is text data and the output is the extracted keywords. Furthermore, an emotion engine analyzes the user's voice data and recognizes their emotional state. The input is the user's voice data and the output is their emotional state.

[0494] Step 4: Question generation and next question presentation

[0495] The server automatically generates the next question based on the extracted keywords and the recognized emotional state. For example, it generates a question such as, "What was the most memorable and enjoyable event at that time?" The input is the keywords and the emotional state, and the output is the generated question. The device presents the next question sent from the server to the user by voice or text.

[0496] Step 5: Collecting user responses and converting them to text

[0497] The user speaks further details to the terminal in response to the presented question. The input is the user's voice and the output is text data. The terminal converts the user's voice back into text data and sends it to the server.

[0498] Step 6: Editing and creating your autobiography

[0499] The server creates the autobiography chapter by chapter based on the collected data and emotion recognition results. The input is the collected data and emotion recognition results, and the output is the autobiography text organized by chapter. It then formats the autobiography based on the user's selected style (documentary, romantic comedy, etc.) and generates the final version. The input is the user's style selection, and the output is the formatted autobiography.

[0500] Step 7: Format and export your autobiography

[0501] The server generates a formatted autobiography in PDF format and sends it to the terminal. The input is the formatted autobiography text and the output is the autobiography in PDF format. The terminal displays a preview version and presents it to the user for confirmation.

[0502] Step 8: User review and correction

[0503] The user reviews the autobiography and submits correction requests as needed via their terminal. The input is the correction request, and the output is the correction. The server saves the final version that reflects the corrections and outputs it in a printable or downloadable format.

[0504] (Application example 2)

[0505] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0506] When elderly people recount past events and generate autobiographical texts, the texts may lack emotional depth or the questions asked may be inconsistent. This problem can be solved by recognizing specific emotions and generating questions and answers based on those emotions. Furthermore, the generated autobiographies must be provided in a digital format to promote a consistent user experience.

[0507] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes an emotion recognition means that analyzes the user's emotions and reflects them in the questions and sentence generation, an output means that outputs the generated autobiographical-style sentences in a digital format and presents them visually and audibly, and an emotion-reflecting question generation means that automatically generates the next question based on the emotion analysis results. This makes it possible to generate an autobiography with greater emotional depth according to the content spoken by the elderly person, enabling a user experience that is visually and audibly consistent.

[0508] "Input methods" refer to the means by which older people recount past events, and include various forms such as voice input and text input.

[0509] "Analysis methods" refer to the means used to analyze what is said and extract important keywords. Natural language processing technology is used, for example.

[0510] The "question generation means" refers to a means for presenting the next question to the elderly person based on the keywords extracted by the analysis means.

[0511] "Database management means" refers to a database system for registering and managing profile information and past photographs of elderly people.

[0512] "Text generation means" refers to a means for automatically generating autobiographical text based on received information.

[0513] "Emotion recognition means" refers to a means for analyzing a user's emotions and reflecting them in questions and sentence generation. This includes technology for reading emotions from voice and text.

[0514] "Output means" refers to a means for outputting the generated autobiographical text in a digital format and presenting it visually and audibly.

[0515] The "format conversion means" refers to a means for converting the generated autobiography into a specific style and outputting it.

[0516] The "means for generating emotion-reflecting questions" refers to a means for automatically generating the next question based on the emotion analysis results.

[0517] "Edit management means" refers to a means for providing an interface that allows the generated autobiographical writing to be displayed in a digital format and for commentary and edits to be made.

[0518] The present invention relates to a system that generates an autobiography by allowing elderly people to talk about past events and combining it with an emotion engine. The system includes the following elements:

[0519] Components

[0520] 1. Input means: Includes voice input devices and text input devices for the elderly to talk about their past events.

[0521] 2. Analysis method: Analyze what is said and extract important keywords and phrases. This is done using natural language processing technology.

[0522] 3. Question generation method: The next question is automatically generated based on the extracted keywords and the results of sentiment analysis and presented to the elderly.

[0523] 4. Database management means: Includes a database that stores and manages profile information and past photos of seniors.

[0524] 5. Text generation method: This is a technology that automatically generates autobiographical-style text based on collected data.

[0525] 6. Emotion recognition: Analyzes emotions based on the user's voice and text and reflects them in questions and sentence generation.

[0526] 7. Output means: The generated autobiography is output in a digital format, including a display and a speaker for visually and audibly presenting it.

[0527] 8. Format conversion method: The generated autobiography is converted into a specific style (documentary, romantic comedy, etc.) and output.

[0528] 9. Emotion-reflecting question generation method: Automatically generate the next question based on the emotion analysis results.

[0529] 10. Editing management tool: The generated autobiographical text is displayed in a digital format, and an interface is provided that allows users to comment and edit.

[0530] Program processing

[0531] Hardware and Software Used

[0532] Hardware: smartphones, tablets, microphones, displays, head-mounted displays (HMDs)

[0533] software:

[0534] Python library (speech_recognition): Used to convert speech to text.

[0535] Transformers Library: Sentiment analysis is performed using the natural language processing model provided by Hugging Face.

[0536] TextBlob: Used for keyword extraction and sentiment analysis of text data.

[0537] Process Description

[0538] The server receives data from a terminal that inputs the elderly person's speech. The speech is converted into text data using a speech recognition means (e.g., Python's speech_recognition library). The converted text is analyzed by an analysis means to extract important keywords. In addition, the user's emotions are analyzed using an emotion recognition means (e.g., Transformers library), and an emotion-reflecting question generation means generates an appropriate next question based on the analysis results. The question is presented to the user via the terminal.

[0539] By repeating this process, the collected data is written as an autobiographical text using the text generation means. The generated text is converted into a specific style using the format conversion means based on the style selected by the user, and is displayed visually and audibly through the output means. The generated autobiographical text is also displayed in a digital format using the revision management means, allowing for commentary and revision.

[0540] Specific examples

[0541] Example 1: Emotion recognition of childhood memories

[0542] User: "I spent my childhood in Hokkaido, a place rich in nature. I enjoyed every day."

[0543] Speech recognition result: "I spent my childhood in Hokkaido, an area rich in nature. Every day was fun."

[0544] Sentiment analysis result: Positive

[0545] Generated question: "What did you enjoy most about that time?"

[0546] Example prompts for generative AI models

[0547] Analyze the text below and identify the sentiment.

[0548] Text: "I spent my childhood in Hokkaido, a place rich in nature. I enjoyed every day."

[0549] Generate the following questions based on the emotions and keywords below.

[0550] Emotion: Positive

[0551] Keywords: ["Hokkaido, rich in nature," fun]

[0552] In this way, the system generates autobiographies with emotional depth based on what the elderly person narrates, providing a consistent user experience.

[0553] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0554] Step 1:

[0555] The user logs in to the device and enters basic information. The device encrypts the information and sends it to the server. The server verifies the login information and saves the user profile in a database. It also receives past photos and important events from the user and saves them in the database.

[0556] Step 2:

[0557] The user speaks into the device's voice input device to recount past events. The device uses a speech recognition method (e.g., Python's speech_recognition library) to convert the speech into text data and sends the text data to the server. Here, the input is the user's voice data, and the output is text data.

[0558] Step 3:

[0559] The server analyzes the received text data using its analysis means and extracts important keywords and phrases. Using natural language processing technology, it processes and calculates the data to extract the necessary information. The input for this step is the voice-recognized text data, and the output is the extracted keywords and phrases.

[0560] Step 4:

[0561] The server uses emotion recognition tools (e.g., the Transformers library) to analyze user emotions from text data. The text is input to an emotion analysis model, which outputs the user's emotional state (positive, negative, or neutral). The input here is the analyzed text data, and the output is the emotion analysis results.

[0562] Step 5:

[0563] The server generates the next question using the emotion-reflecting question generation means based on the emotion analysis results and extracted keywords. For example, if a positive emotion is detected, a question such as "What was the most enjoyable thing at that time?" is automatically generated. The input is the emotion analysis results and extracted keywords, and the output is the generated question.

[0564] Step 6:

[0565] The device presents the generated question to the user. The user then recites what they have said by voice, which is then converted back into text data and sent to the server. By repeating this cycle, more detailed information is collected. The input is the automatically generated question from the server, and the output is the user's voice data and its converted text data.

[0566] Step 7:

[0567] The server uses a text generation tool based on the collected data to automatically generate autobiographical text. It combines the stories told by the elderly with the results of emotion analysis to create text in a natural format. The input is the data told by the elderly and the results of emotion analysis, and the output is an autobiographical text.

[0568] Step 8:

[0569] The server converts the generated autobiographical text into a specific style selected by the user using a format conversion means. For example, it converts into a style that matches the user's preferences, such as a documentary style or a romantic comedy style. The input is the generated autobiographical text, and the output is the format-converted text.

[0570] Step 9:

[0571] The server provides the final generated autobiography to the user in digital format through the output means. The terminal has a display for visual confirmation, allowing the user to check the content. The input is the formatted text, and the output is the autobiography presented in digital format.

[0572] Step 10:

[0573] The user checks the generated autobiographical text and, if any corrections are necessary, sends a correction request using the correction management means. The server receives the correction request, regenerates the final version with the relevant parts corrected, and presents it to the user again. The input is the user's correction request, and the output is the corrected autobiographical text.

[0574] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0575] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0576] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0577] [Second embodiment]

[0578] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0579] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0580] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0581] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0582] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0583] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0584] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0585] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0586] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0587] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0588] In the smart glasses 214, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0589] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0590] The present invention is a system that generates autobiographical text based on the recounting of past events by elderly people. This system involves the process of inputting and analyzing the content spoken by the elderly person, generating questions to be presented next, and finally compiling the autobiography.

[0591] System configuration

[0592] user

[0593] Elderly people: They are the main users of the system and talk about their own past experiences.

[0594] Family and care staff: Provide support to the older person as needed.

[0595] Terminal

[0596] Input devices: Devices that allow the older adult to input speech, including microphones and touchscreens.

[0597] Output device: A device used to display questions and generated text for confirmation. This can include a display or speaker.

[0598] server

[0599] Database: Stores and manages elderly profile information, conversations, uploaded photos and important events.

[0600] Analysis engine: Analyzes user voice data and extracts important keywords and phrases using natural language processing technology.

[0601] Question generation engine: Automatically generates the next question based on the extracted keywords.

[0602] Text generation engine: Automatically generates autobiographical text based on collected data.

[0603] Format conversion engine: Converts the generated autobiography into a specific style (documentary, romantic comedy, etc.) and outputs it.

[0604] Program processing

[0605] Initial setup and receiving user information

[0606] User: Logs in to a device and enters basic information such as date of birth and major life events.

[0607] Terminal: The entered information is encrypted and sent to the server.

[0608] Server: Validates login information and stores user profiles in a database. Also receives past photos and important events from users and stores them in a database.

[0609] Starting a conversation

[0610] Device: Present the user with an initial question, for example, "Tell me about your childhood."

[0611] User: Speaks into the device and recounts memories of the elderly person's childhood.

[0612] Device: Converts the user's voice into text data and sends it to the server.

[0613] Data analysis and question generation

[0614] Server: Analyzes the text data and extracts important keywords.

[0615] Server: Generates the next question based on the extracted keywords and sends it to the device. For example, it might ask, "What was your most memorable camping experience?"

[0616] Editing and creating autobiographies

[0617] User: Speaks further details into the device in response to the questions presented.

[0618] Device: Converts the user's speech into text and sends it back to the server.

[0619] Server: Based on the collected data, compiles autobiographical text and adds it to the autobiographical chapters.

[0620] Server: Formats the autobiography based on the style selected by the user and generates the final version.

[0621] Output and Save

[0622] Server: Generates the completed autobiography in PDF format and sends it to the device.

[0623] Device: Shows a preview version and presents it to the user for review.

[0624] User: Review the autobiography and submit a request to make corrections if necessary.

[0625] Server: Save the final version with the changes and make it available for printing or downloading.

[0626] Specific examples

[0627] Example 1: Recounting childhood memories

[0628] User: "I spent my childhood in Hokkaido, a place rich in nature."

[0629] Device: Converts this speech into text and sends it to the server.

[0630] Server: Extracts the keyword "Hokkaido, rich in nature," generates the question "What was the most memorable event at that time?" and sends it to the device.

[0631] User: "My favorite memories are going camping with my family and fishing on a big lake."

[0632] Terminal: Converts the voice into text data and sends it back to the server.

[0633] Server: Add this episode to the "Childhood Memories" chapter and generate it as text.

[0634] Example 2: Memories from school days

[0635] User: "I spent my student days in Tokyo. I had lots of adventures with my friends."

[0636] Device: Converts this speech into text and sends it to the server.

[0637] Server: Extracts the keywords "student days," "friends," and "adventure," generates a question such as "What was the most fun adventure you had at that time?" and sends it to the device.

[0638] User: "The most fun I had was climbing Mount Fuji with my friends."

[0639] Device: Converts the voice into text data and sends it back to the server.

[0640] Server: Add this episode to the chapter "Memories of Student Life" and generate it as a sentence.

[0641] In this way, a system in which AI generates autobiographical text based on past events recounted by elderly people will be useful in preventing dementia among the elderly and promoting communication with their families.

[0642] The processing flow will be explained below.

[0643] Specific processing steps of the program

[0644] Step 1: Initial setup and receiving user information

[0645] 1.1 User: Log in to the device and launch the AI ​​autobiography creation system.

[0646] Specific actions: Enter your username and password on the login screen displayed on the device and click the "Login" button.

[0647] 1.2 Device: Sends login information to server.

[0648] Specific operation: The entered login information is encrypted and sent to the server.

[0649] 1.3 Server: Verify login information and retrieve user profile data.

[0650] What it does: Retrieves user profile information (e.g., name, date of birth, major events) from a database.

[0651] 1.4 User: Register past photos and important events (topics and turning points) via the device.

[0652] Specific actions: Click the "Upload Photo" or "Add Event" button, select a file, or enter text.

[0653] 1.5 Terminal: Sends uploaded photos and entered event information to the server.

[0654] Specific operation: Transfers the selected photo files and text data to the server.

[0655] 1.6 Server: Stores photos and event information and organizes it as structured data.

[0656] What it does: Store the photos in an image library and add the event information to the associated time series data.

[0657] Step 2: Initiating a conversation

[0658] 2.1 Terminal: Present the initial question to the user.

[0659] What it does: Uses speech synthesis technology to play the question "Tell me about your childhood."

[0660] 2.2 User: Speaks into the device and recounts childhood memories.

[0661] Specific action: "When I was a child, I often went camping with my family in Hokkaido, which is rich in nature."

[0662] 2.3 Terminal: Converts the user's speech into text data.

[0663] What it does: It uses speech recognition software to convert speech into text and send it to a server.

[0664] Step 3: Analyze the data and generate questions

[0665] 3.1 Server: Analyzes the text data and extracts important keywords and phrases.

[0666] Specific operation: Extract keywords such as "Hokkaido, rich in nature" and "camping" using a natural language processing algorithm.

[0667] 3.2 Server: Generates a new question and sends it to the device.

[0668] Specific operation: Generate questions related to "nature" and "camping" and send a text message to the device saying, "What is your most memorable camping experience?"

[0669] 3.3 Terminal: Present the new question to the user.

[0670] What it does: Converts the question text into speech and plays it back to the user.

[0671] Step 4: Editing and creating your autobiography

[0672] 4.1 User: Speaks further details into the terminal in response to the questions presented.

[0673] Specific action: "One summer day, my father, brothers and I went fishing on a big lake and caught a big salmon, and that's a memory I'll never forget."

[0674] 4.2 Terminal: The user's voice is converted back into text data and sent to the server.

[0675] What it does: It uses speech recognition technology to convert speech into text and send it to a server for analysis.

[0676] 4.3 Server: Write your autobiography chapter by chapter based on the data collected.

[0677] What it does: Use natural language generation techniques to write a chapter on "Childhood Memories" and store it in a database.

[0678] 4.4 Server: Formats the autobiography in the style selected by the user.

[0679] Specific operation: Generates text that reflects styles such as "documentary style" or "romantic comedy style."

[0680] Step 5: Export and save

[0681] 5.1 Server: Sends the completed autobiography to the user's device for confirmation.

[0682] Specific behavior: Generates a preview version of the autobiography in PDF format and sends it to the device.

[0683] 5.2 Device: Show preview and ask user for confirmation.

[0684] What it does: Displays a preview of the autobiography in a PDF viewer and provides a "Confirm" button.

[0685] 5.3 User: Review the autobiography and request corrections if necessary.

[0686] Specific actions: Click the "Confirm" button and enter any necessary corrections.

[0687] 5.4 Server: Stores the finalized autobiography and outputs it in a printable or downloadable format.

[0688] Specific behavior: Saves the final version to the database and generates PDF and print files.

[0689] In this way, each step of the system that uses AI to automatically generate an autobiography based on the memories of the elderly person is achieved through collaboration between the user, device, and server.

[0690] Example 1

[0691] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0692] Previous systems that generate autobiographical documents from elderly people's recounting of past events were insufficient in abstracting what was said and in presenting the next question, making it difficult to create a natural dialogue. Furthermore, the style and format of the generated autobiography were limited, making it difficult to flexibly respond to user needs. Furthermore, there was a lack of technology to accurately convert voice input into text, resulting in a decline in the quality of the generated documents.

[0693] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0694] In this invention, the server includes an analysis means for analyzing what has been said and extracting important keywords, a question generation means for generating the next question based on the extracted keywords and presenting it to the elderly person, a database management means for registering and managing the elderly person's profile information and past images, a document generation means for automatically generating an autobiography-style document based on the received information, and a format conversion means for selecting and outputting the autobiography style. This enables a natural flow of dialogue and enables the flexible generation of autobiographies that meet the user's needs.

[0695] The "input means" is a device that allows elderly people to input voice or text recounting past events.

[0696] "Analysis means" refers to a device or program that has the function of analyzing what is said and extracting important keywords and phrases.

[0697] The "question generation means" is a device or program that automatically generates the next question based on the extracted keywords and presents it to the elderly person.

[0698] The "database management means" is a system for registering and managing elderly people's profile information, past images, and data on important events.

[0699] The "document generation means" is a device or program that automatically generates an autobiographical document based on the received information.

[0700] The "format conversion means" is a device or program that selects the style of the generated autobiography, converts it into a specified format, and outputs it.

[0701] The "voice recognition means" is a device or program that inputs the elderly person's speech and converts it into text data.

[0702] The "revision management means" is a device or program for presenting the generated autobiographical writing to the user in a revisable format and for reflecting requests for revisions.

[0703] "Natural language processing technology" is a technology for analyzing, understanding, and generating human language.

[0704] "Elderly people" refers to the subjects who use the system to narrate past events.

[0705] This invention is an information processing system that generates an autobiographical document based on an elderly person's recounting of past events. The system uses speech recognition and natural language processing technologies to convert the elderly person's recounting into text, extract important keywords, and generate the next question. It also includes a function to select the style of the generated autobiography and output it as a document.

[0706] System configuration

[0707] Hardware

[0708] Device: A device equipped with a microphone for voice input and a touchscreen.

[0709] Server: High-performance computer for data analysis and document generation

[0710] Database: A storage system for managing profile information and past images of elderly people

[0711] Display: A screen for displaying questions and generated documentation.

[0712] Speaker: An audio device for presenting questions aloud.

[0713] software

[0714] Speech recognition software: A system that inputs elderly people's speech and converts it into text data

[0715] Natural language processing software: Software that extracts important keywords from text data

[0716] Question generation engine: A program that automatically generates the next question based on the extracted keywords.

[0717] Document generation engine: Software that automatically generates autobiographical documents based on collected data

[0718] Format conversion engine: Software that converts the generated autobiography into a specific style and outputs it.

[0719] Revision management system: Software that presents the generated autobiographical text to the user in a revisable format and reflects revisions.

[0720] System Operation

[0721] Registering user information

[0722] Users log in to their devices and enter basic information about their date of birth and major life events. This information is encrypted on the device and sent to the server, which then stores it in a database as part of the user's profile. The server also receives past images and important events from the user and stores them in the database.

[0723] Starting a conversation

[0724] The device presents the user with an initial question, for example, "Tell me about your childhood." The user speaks into the device, recounting the elderly person's childhood memories. The device converts the speech into text data and sends it to the server.

[0725] Data analysis

[0726] The server analyzes the text data and extracts important keywords. It then generates the next question based on the extracted keywords and sends it to the device. For example, it could ask a specific question such as, "What was your most memorable camping experience?"

[0727] Document Generation

[0728] The user responds to the questions with further details, which are then converted back into text and sent to the server. The server then uses the collected data to compile an autobiographical document, adding chapters to it, and formats the autobiography based on the user's chosen style to produce the final version.

[0729] Output and Modification

[0730] The completed autobiography is sent from the server to the device in PDF format. The device displays a preview version and presents it to the user for confirmation. The user reviews the autobiography and submits a request for any necessary corrections. The server saves the final version after the corrections are applied and outputs it in a format that can be printed or downloaded.

[0731] Examples and prompts

[0732] Example 1: Recounting childhood memories

[0733] When a user says, "I spent my childhood in Hokkaido, a place rich in nature," the device converts this speech into text and sends it to the server. The server extracts the keyword "Hokkaido, a place rich in nature," generates a question, "What was the most memorable event at that time?" and sends it to the device. The user then adds, "My best memory is going camping with my family and fishing on a large lake," and this is added to the autobiography as a "childhood memory."

[0734] Prompt Sentence Examples

[0735] "Tell me an anecdote from your childhood."

[0736] What is your most enjoyable memory from your school days?

[0737] "Tell me some memories you have with your family."

[0738] "Tell us about the most memorable event."

[0739] In this way, it is possible to realize natural and flowing dialogue based on the elderly person's narrative, and generate a high-quality autobiography.

[0740] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0741] Step 1: Initial setup and receiving user information

[0742] User: Log in to the device and enter the required user information (date of birth and data related to major life events).

[0743] Input: Basic information such as date of birth, major life events, etc.

[0744] Output: Initial setting data

[0745] Terminal: The entered information is encrypted and sent to the server.

[0746] Input: Data entered by the user

[0747] Data processing: Encryption processing

[0748] Output: Encrypted user information

[0749] Server: Receives user information, verifies login information, and stores user profiles in a database.

[0750] Input: Encrypted user information

[0751] Data processing: Save to database

[0752] Output: Saved user profile data

[0753] Step 2: Initiating a conversation

[0754] Device: Present the first question to the user aloud, for example, "Tell me about your childhood."

[0755] Input: Fixed question template

[0756] Data processing: Question voice synthesis

[0757] Output: Questions presented by voice

[0758] User: Speaks into device, recounting childhood memories.

[0759] Input: User's voice data

[0760] Output: Recorded audio data

[0761] Device: Converts the user's voice into text data and sends it to the server.

[0762] Input: User's voice data

[0763] Data processing: speech recognition and text conversion

[0764] Output: Text data

[0765] Step 3: Analyze the data and generate questions

[0766] Server: Analyzes the received text data and extracts important keywords using natural language processing technology.

[0767] Input: Text data

[0768] Data processing: Keyword extraction using natural language processing

[0769] Output: Extracted keywords

[0770] Server: Generates the next question based on the extracted keywords and sends it to the device.

[0771] Input: Extracted keywords

[0772] Data processing: Generate new questions using a question generation algorithm

[0773] Output: Generated questions

[0774] Step 4: Editing and creating your autobiography

[0775] User: Provides details in response to questions posed by the device.

[0776] Input: Question from terminal

[0777] Output: User's voice data

[0778] Device: Converts the user's voice into text data and sends it back to the server.

[0779] Input: User's voice data

[0780] Data processing: speech recognition and text conversion

[0781] Output: Text data

[0782] Server: Based on the collected data, compile an autobiographical document and add chapters to it.

[0783] Input: Text data and user profile

[0784] Data processing: text editing and chapter addition

[0785] Output: Updated autobiographical text

[0786] Server: Formats the autobiography based on the style selected by the user and generates the final version.

[0787] Input: Updated autobiographical text

[0788] Data processing: Applying a format conversion engine

[0789] Output: Final formatted autobiography

[0790] Step 5: Export and save

[0791] Server: Generates the completed autobiography in PDF format and sends it to the device.

[0792] Input: Final formatted autobiography

[0793] Data processing: PDF conversion

[0794] Output: Autobiography in PDF format

[0795] On the device: The generated PDF is displayed as a preview and presented to the user for review.

[0796] Input: Autobiography in PDF format

[0797] Output: Preview

[0798] User: Review the autobiography and submit a request for corrections if needed.

[0799] Input: Previewed autobiography

[0800] Output: Correction request

[0801] Server: Save the final version with the corrections and output it in a printable or downloadable format.

[0802] Input: Correction request and autobiography in PDF format

[0803] Data processing: Reflecting corrections and saving the final version

[0804] Output: Final autobiography

[0805] (Application example 1)

[0806] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0807] Elderly people's recounting of their past experiences is an important way to record and pass on personal memories to future generations. However, in today's busy living environment, family members and caregivers often do not have enough time to carefully listen to what the elderly have to say. Furthermore, there is a need to provide experiences related to past memories in order to prevent dementia and help the elderly feel a sense of spiritual fulfillment. The present invention aims to solve these problems by easily generating an autobiography based on the elderly's narrative and providing a customized meal menu based on that autobiography.

[0808] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0809] In this invention, the server includes input means for the elderly person to talk about their past, analysis means for analyzing what has been said and extracting important keywords, question generation means for generating the next question based on the extracted keywords and presenting it to the elderly person, database management means for registering and managing the elderly person's profile information and past photos, text generation means for automatically generating autobiographical text based on the received information, format conversion means for selecting and outputting the autobiographical style, and menu suggestion means for suggesting a customized meal menu based on the elderly person's story. This makes it possible to create an autobiography by having the elderly person talk about events from their past, and to provide a special dining experience based on that story.

[0810] The "elderly" are the subjects who use the system to recount past events and generate autobiographical texts.

[0811] The "input means" is a device used by elderly people to talk about past events, and is a device that allows voice input and text input.

[0812] The "analysis means" is a device or software that analyzes what the elderly say and extracts important keywords.

[0813] The "question generation means" is a device or software that automatically generates the next question based on the extracted keywords and presents it to the elderly person.

[0814] The "database management means" refers to a device or software that registers and manages profile information and past photographs of elderly people.

[0815] The "text generation means" is a device or software that automatically generates autobiographical text based on received information.

[0816] The "format conversion means" is a device or software that selects the style of the generated autobiography, converts it into a specific format, and outputs it.

[0817] The "menu suggestion means" is a device or software that suggests a customized meal menu based on the elderly person's narration.

[0818] This invention is a system that generates autobiographical text by having an elderly person recount events from their past, and proposes a customized meal menu based on that narrative. Specific embodiments of this system will be described in detail below.

[0819] user

[0820] senior citizen

[0821] The subject who uses the system talks about events from his or her past. The system analyzes what he or she says and ultimately compiles it into an autobiography.

[0822] Family members and caregivers

[0823] Provide support to the elderly as needed and help them use the system.

[0824] Terminal

[0825] Input Devices

[0826] This device allows elderly people to input their speech, and includes a microphone and a touchscreen.

[0827] Output Device

[0828] A device for displaying generated questions, autobiographical text, and suggested meal menus, including a display and speaker.

[0829] server

[0830] Database Management Methods

[0831] Stores and manages data such as elderly people's profile information, past photos, and spoken content.

[0832] Analysis means

[0833] Analyze user speech (audio data) and extract important keywords and phrases using voice recognition technology (e.g., Google speech-to-text API) and natural language processing technology (e.g., Hugging Face's Transformers library).

[0834] Question generation means

[0835] The next question is automatically generated based on the extracted keywords, using a generative AI model (e.g., GPT-2).

[0836] Sentence generation means

[0837] Based on the collected data, an autobiographical text is edited and generated, again using natural language generation technology (e.g., GPT-2).

[0838] Format conversion method

[0839] The generated autobiography is converted into a specific style (documentary, romantic comedy, etc.) and output.

[0840] Menu suggestion method

[0841] This system proposes customized meal menus based on the elderly's narratives. It extracts food-related information from keywords in the narratives and proposes related menus.

[0842] Program processing explanation

[0843] The server receives the voice input data of what the elderly person is saying and uses an analysis means to extract important keywords and phrases. Based on the analyzed data, a question generation means generates the next question and sends it to the terminal to present to the user. The user's voice response is sent back to the server, and by repeating the same process, an autobiographical-style text is gradually generated.

[0844] Furthermore, food-related keywords are extracted from the analyzed content, and the menu suggestion means suggests meal menus that are memorable to the elderly.

[0845] For example, if an elderly person says, "When I was a child, I loved the miso soup my grandmother made for me," the analysis method will extract the keywords "childhood," "grandmother," and "miso soup," and generate the next question, "What other dish did you like most at that time?" Based on the answer to that question, the system will ask the next question and eventually suggest related menu items such as "miso soup set" or "grandmother's special miso soup."

[0846] Prompt Sentence Examples

[0847] "Logged in user: {username} said the following: 'When I was a child, I loved the miso soup my grandmother made for me.' Please generate follow-up questions."

[0848] The above is a specific embodiment for carrying out the present invention. This system allows elderly people to create their autobiographies by talking about their past, and furthermore, it makes it possible to provide special dining experiences based on past memories.

[0849] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0850] Step 1:

[0851] A user logs into the system and enters their profile information (date of birth, major life events, etc.) The entered information is sent to the server in the form of data, which is then stored in the database.

[0852] Step 2:

[0853] The device generates an initial question and presents it to the user, for example, "Tell me about your childhood," either by voice or text. The user begins to recount events from their past.

[0854] Step 3:

[0855] The user speaks their answer through a voice input device, and this voice data is input into the terminal.

[0856] Step 4:

[0857] The device converts the voice data into text data using voice recognition technology (e.g., Google speech-to-text API), and the converted text data is sent to the server.

[0858] Step 5:

[0859] The server receives the text data and uses analytical tools to extract important keywords and phrases. Natural language processing techniques (e.g., Hugging Face's Transformers library) are used. The results of this analysis are used to generate the next question.

[0860] Step 6:

[0861] The server generates the next question based on the extracted keywords. It uses a generative AI model (e.g., GPT-2) to create an appropriate next question. For example, a question like, "What was the most memorable event at that time?" is generated. The generated question is sent to the device.

[0862] Step 7:

[0863] The device presents the generated question to the user, who answers the new question and then speaks again.

[0864] Step 8:

[0865] The server receives the voice data again and repeats the same process. Finally, when enough information has been collected, an autobiographical text is automatically generated by the text generation means.

[0866] Step 9:

[0867] The server uses a text generation tool to edit and generate the collected data along an autobiography-like flow. The generated autobiography is formatted based on the style selected by the user (e.g., documentary style, romantic comedy style).

[0868] Step 10:

[0869] The server sends the formatted autobiography to the terminal and displays it as a preview to the user. The user can submit correction requests as needed. If no corrections are required, the final version is generated.

[0870] Step 11:

[0871] The server then saves the final autobiography in a format such as PDF, and outputs it in a printable or downloadable format. At the same time, a menu suggestion tool is used to suggest a customized meal menu based on the elderly person's story. This menu suggestion is also displayed on the device.

[0872] The above is the flow of specific processing steps of the system that realizes the application example.

[0873] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0874] This invention combines an emotion engine with a system that generates autobiographical text based on the elderly's recounting of past events. The system inputs what the elderly person is saying, analyzes it, generates questions to pose next, and ultimately compiles it into an autobiography with emotional depth. The system also recognizes the user's emotions and asks appropriate questions and responds based on the results.

[0875] System configuration

[0876] user

[0877] Elderly people: They are the main users of the system and talk about their own past experiences.

[0878] Family and care staff: Provide support to the older person as needed.

[0879] Terminal

[0880] Input devices: Devices that allow the older adult to input speech, including microphones and touchscreens.

[0881] Output device: A device used to display questions and generated text for confirmation. This can include a display or speaker.

[0882] server

[0883] Database: Stores and manages elderly people's profile information, spoken content, uploaded photos, important events, and emotion recognition results.

[0884] Analysis engine: Analyzes user voice data and extracts important keywords and phrases using natural language processing technology.

[0885] Question generation engine: Automatically generates the next question based on the extracted keywords.

[0886] Emotion engine: Recognizes emotions from the user's voice and facial expressions and tailors questions and responses accordingly, adding depth to sentences depending on the user's emotional state.

[0887] Text generation engine: Automatically generates autobiographical text based on collected data.

[0888] Format conversion engine: Converts the generated autobiography into a specific style (documentary, romantic comedy, etc.) and outputs it.

[0889] Program processing

[0890] Initial setup and receiving user information

[0891] User: Log in to the device and enter basic information (date of birth, major life events, etc.).

[0892] Terminal: The entered information is encrypted and sent to the server.

[0893] Server: Validates login information and stores user profiles in a database. Also receives past photos and important events from users and stores them in a database.

[0894] Starting a conversation

[0895] Device: Present the user with an initial question, for example, "Tell me about your childhood."

[0896] User: Speaks into device, recounting childhood memories.

[0897] Device: Converts the user's voice into text data and sends it to the server.

[0898] Data analysis and question generation

[0899] Server: Analyzes the text data and extracts important keywords.

[0900] Server: The emotion engine recognizes the user's emotional state and reflects it when generating the next question. For example, if the user is telling a happy story, it generates the question, "What was the most memorable happy event at that time?"

[0901] Editing and creating autobiographies

[0902] User: Speaks further details into the device in response to the questions presented.

[0903] Device: The user's voice is converted back into text data and sent to the server.

[0904] Server: Writes an autobiography chapter by chapter based on collected data and emotion recognition results.

[0905] Server: Formats the autobiography based on the style selected by the user and generates the final version.

[0906] Output and Save

[0907] Server: Generates the completed autobiography in PDF format and sends it to the device.

[0908] Device: Shows a preview version and presents it to the user for review.

[0909] User: Review the autobiography and submit a request to make corrections if necessary.

[0910] Server: Save the final version with the changes and make it available for printing or downloading.

[0911] Specific examples

[0912] Example 1: Emotion recognition of childhood memories

[0913] User: "I spent my childhood in Hokkaido, a place rich in nature. I enjoyed every day."

[0914] Device: Converts this speech into text and sends it to the server.

[0915] Server: Extracts keywords such as "Hokkaido, rich in nature" and "fun," and recognizes the user's enjoyment from the tone of their voice. Generates a question such as "What was the most memorable and enjoyable event at that time?" and sends it to the device.

[0916] User: "My favorite memories are going camping with my family and fishing on a big lake."

[0917] Terminal: Converts the voice into text data and sends it back to the server.

[0918] Server: Add this episode to the "Childhood Memories" chapter and generate it as a sentence containing emotions.

[0919] Example 2: Emotion recognition of memories from school days

[0920] User: "I spent my student days in Tokyo. I had lots of adventures with my friends."

[0921] Device: Converts this speech into text and sends it to the server.

[0922] Server: Extracts keywords such as "student days" and "adventure" and recognizes excited emotions from the user's voice. Generates a question such as "What was the most fun adventure you had at that time?" and sends it to the device.

[0923] User: "The most fun I had was climbing Mount Fuji with my friends."

[0924] Device: Converts the voice into text data and sends it back to the server.

[0925] Server: Add this episode to the chapter "Memories of Student Life" and generate a sentence containing emotions.

[0926] In this way, a system in which AI recognizes emotions based on past events recounted by elderly people and automatically generates an autobiography accordingly will be even more effective in preventing dementia among the elderly and promoting communication with their families.

[0927] The processing flow will be explained below.

[0928] Specific processing steps of the program

[0929] Step 1: Initial setup and receiving user information

[0930] 1.1 User: Log in to the device and launch the AI ​​autobiography creation system.

[0931] Specific actions: Enter your username and password on the login screen and click the "Login" button.

[0932] 1.2 Device: Sends login information to server.

[0933] Specific behavior: Encrypts the entered login information and sends it to the server.

[0934] 1.3 Server: Verify login information and retrieve user profile data.

[0935] What it does: Retrieves user profile information (e.g., name, date of birth, major events) from a database.

[0936] 1.4 User: Register past photos and important events (topics and turning points) via the device.

[0937] Specific actions: Click the "Upload Photo" or "Add Event" button to select a file or enter text.

[0938] 1.5 Terminal: Sends uploaded photos and entered event information to the server.

[0939] Specific operation: Transfers the selected photo files and text data to the server.

[0940] 1.6 Server: Stores photos and event information and organizes it as structured data.

[0941] What it does: Store the photos in an image library and add the event information to the associated time series data.

[0942] Step 2: Initiating a conversation

[0943] 2.1 Terminal: Present the initial question to the user.

[0944] What it does: Uses speech synthesis technology to play the question "Tell me about your childhood."

[0945] 2.2 User: Speaks into the device and recounts childhood memories.

[0946] Specific action: "When I was a child, I often went camping with my family in Hokkaido, which is rich in nature."

[0947] 2.3 Terminal: Converts the user's speech into text data.

[0948] What it does: It uses speech recognition software to convert speech into text and send it to a server.

[0949] Step 3: Analyze the data and generate questions

[0950] 3.1 Server: Analyzes the text data and extracts important keywords and phrases.

[0951] Specific operation: Extract keywords such as "Hokkaido, rich in nature" and "camping" using a natural language processing algorithm.

[0952] 3.2 Server: The emotion engine recognizes the user's emotional state and reflects it when generating the next question.

[0953] Specific operation: Analyzes the user's emotions from their tone of voice and facial expressions, generates a question such as "What was the most memorable and enjoyable event at that time?" and sends it to the device.

[0954] 3.3 Terminal: Present the new question to the user.

[0955] What it does: Converts the question text into speech and plays it back to the user.

[0956] Step 4: Editing and creating your autobiography

[0957] 4.1 User: Speaks further details into the terminal in response to the questions presented.

[0958] Specific action: "One summer day, my father, brothers and I went fishing on a big lake and caught a big salmon, and that's a memory I'll never forget."

[0959] 4.2 Terminal: The user's voice is converted back into text data and sent to the server.

[0960] What it does: It uses speech recognition technology to convert speech into text and send it to a server for analysis.

[0961] 4.3 Server: Writes an autobiography chapter by chapter based on the collected data and emotion recognition results.

[0962] What it does: Use natural language generation technology to write a chapter on "Childhood Memories," edit it into emotionally rich text, and store it in a database.

[0963] 4.4 Server: Formats the autobiography in the style selected by the user.

[0964] Specific operation: Generates text that reflects styles such as "documentary style" or "romantic comedy style."

[0965] Step 5: Export and save

[0966] 5.1 Server: Sends the completed autobiography to the user's device for confirmation.

[0967] Specific behavior: Generates a preview version of the autobiography in PDF format and sends it to the device.

[0968] 5.2 Device: Show preview and ask user for confirmation.

[0969] What it does: Displays a preview of the autobiography in a PDF viewer and provides a "Confirm" button.

[0970] 5.3 User: Review the autobiography and request corrections if necessary.

[0971] Specific actions: Click the "Confirm" button and enter any necessary corrections.

[0972] 5.4 Server: Stores the finalized autobiography and outputs it in a printable or downloadable format.

[0973] Specific behavior: Saves the final version to the database and generates PDF and print files.

[0974] In this way, each step of the system, which uses AI to recognize emotions and automatically generate an autobiography based on memories from the past recounted by an elderly person, is achieved through collaboration between the user, device, and server.

[0975] Example 2

[0976] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0977] When elderly people recount past events, organizing them into autobiographical text takes time and effort. Furthermore, generating high-quality text that incorporates emotion requires generating appropriate questions. Furthermore, correcting and formatting the generated text is also time-consuming. Thus, there is a need for an efficient system that can automatically organize and write down past events with emotional depth.

[0978] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0979] In this invention, the server includes emotion recognition means, analysis means, and question generation means, which enable the server to recognize emotions from the elderly person's narrative, analyze the content, and generate appropriate questions, thereby efficiently generating high-quality autobiographies with emotional depth.

[0980] "Input means" refers to devices used by older adults to recount past events, including voice input and touchscreens.

[0981] The "analysis means" is a technology for analyzing what the elderly say and extracting important keywords, and is a device that uses natural language processing technology.

[0982] The "question generation means" is a technology for automatically generating the next question based on the extracted keywords and presenting it to the elderly person.

[0983] "Database management means" refers to technology for registering and managing elderly people's profile information and past photographs.

[0984] "Emotion recognition means" is a technology that recognizes the user's emotions from their voice and facial expressions and asks appropriate questions and responds accordingly.

[0985] The "text generation means" is a technology for automatically generating autobiographical text based on received information.

[0986] The "format conversion means" is a technique for selecting the style of the generated autobiography and outputting it in a specified format.

[0987] "Voice recognition means" is a technology that inputs the elderly person's speech and converts it into text data.

[0988] The "revision management means" is a technique for presenting the generated autobiographical text to the user in a revisable format and for reflecting the user's revisions.

[0989] This invention relates to a system that generates autobiographical text based on the elderly's recounting of past events. This system involves a process of inputting the elderly's speech, analyzing it, generating questions to be presented next, and finally compiling it into an autobiography with emotional depth. The system can also recognize the user's emotions and provide appropriate questions and responses based on the results.

[0990] System configuration

[0991] The system mainly consists of the following elements:

[0992] user

[0993] Elderly people: They are the main users of the system and talk about their own past experiences.

[0994] Family and care staff: Provide support to the older person as needed.

[0995] Terminal

[0996] Input devices: Devices that allow the older adult to input speech, including microphones and touchscreens.

[0997] Output device: A device that displays questions and generated text for confirmation. This includes displays and speakers.

[0998] server

[0999] Database: Stores and manages elderly people's profile information, spoken content, uploaded photos, important events, and emotion recognition results.

[1000] Analysis engine: Analyzes user voice data and extracts important keywords and phrases using natural language processing technology.

[1001] Question generation engine: Automatically generates the next question based on the extracted keywords.

[1002] Emotion engine: Recognizes emotions from the user's voice and facial expressions and tailors questions and responses accordingly, adding depth to sentences depending on the user's emotional state.

[1003] Text generation engine: Automatically generates autobiographical text based on collected data.

[1004] Format conversion engine: Converts the generated autobiography into a specific style (e.g., documentary or romantic comedy) and outputs it.

[1005] System Operation

[1006] Hardware

[1007] Microphone: An input device for receiving a user's voice input.

[1008] Display: An output device used to display questions and generated text.

[1009] Server: A computer on which various engines (analysis engine, question generation engine, emotion engine, sentence generation engine, format conversion engine) run.

[1010] software

[1011] Natural Language Processing: A technique used by analytics engines to extract important keywords and phrases from what users say.

[1012] Emotion Recognition Algorithm: This is the technology used in the emotion engine to recognize emotions from the user's voice and facial expressions.

[1013] AI model: A generative AI model used for question generation and sentence generation.

[1014] Specific examples

[1015] Example 1: Emotion recognition of childhood memories

[1016] User: "I spent my childhood in Hokkaido, a place rich in nature. I enjoyed every day."

[1017] Device: Converts this speech into text and sends it to the server.

[1018] Server: Extracts keywords such as "Hokkaido, rich in nature" and "fun," and recognizes the user's enjoyment from the tone of their voice. Generates a question such as "What was the most memorable and enjoyable event at that time?" and sends it to the device.

[1019] User: "My favorite memories are going camping with my family and fishing on a big lake."

[1020] Terminal: Converts the voice into text data and sends it back to the server.

[1021] Server: Add this episode to the "Childhood Memories" chapter and generate it as a sentence containing emotions.

[1022] Example 2: Emotion recognition of memories from school days

[1023] User: "I spent my student days in Tokyo. I had lots of adventures with my friends."

[1024] Device: Converts this speech into text and sends it to the server.

[1025] Server: Extracts keywords such as "student days" and "adventure" and recognizes excited emotions from the user's voice. Generates a question such as "What was the most fun adventure you had at that time?" and sends it to the device.

[1026] User: "The most fun I had was climbing Mount Fuji with my friends."

[1027] Device: Converts the voice into text data and sends it back to the server.

[1028] Server: Add this episode to the chapter "Memories of Student Life" and generate a sentence containing emotions.

[1029] Prompt Sentence Examples

[1030] 1. A user logs in and enters basic information. Please encrypt this information before sending it.

[1031] 2. Input the user's voice data and convert it into text data.

[1032] 3. Extract specific keywords from text data and recognize the user's emotional state.

[1033] 4. Generate the next appropriate question based on the perceived emotional state and present it to the user.

[1034] 5. Collect the user's responses again and analyze the detailed data on the server.

[1035] 6. Create an autobiography chapter by chapter based on the data you have collected.

[1036] 7. Format your final autobiography in the specified style and print it in PDF format.

[1037] 8. Receive user correction requests, revise and save the final version.

[1038] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1039] Step 1: Enter your user information

[1040] A user logs in to a device and enters basic information such as date of birth and major life events. The device encrypts the information entered and sends it to the server. At this time, the user's basic information is input and the encrypted data is sent to the server as output. The server verifies the login information and stores the user profile in a database. Past photos and important events provided by the user are also stored in the database.

[1041] Step 2: Posting initial questions and receiving answers

[1042] The device presents the user with an initial question. For example, "Tell me about your childhood," which is displayed as voice or text. The user speaks into the device and recounts their childhood memories. The user's voice is the input, and is converted into text data as output, which is sent to the server. The device then converts this voice into text data and sends it to the server.

[1043] Step 3: Data analysis and emotion recognition

[1044] The server analyzes the received text data and extracts important keywords. Specifically, it uses natural language processing technology. At this time, the input is text data and the output is the extracted keywords. Furthermore, an emotion engine analyzes the user's voice data and recognizes their emotional state. The input is the user's voice data and the output is their emotional state.

[1045] Step 4: Question generation and next question presentation

[1046] The server automatically generates the next question based on the extracted keywords and the recognized emotional state. For example, it generates a question such as, "What was the most memorable and enjoyable event at that time?" The input is the keywords and the emotional state, and the output is the generated question. The device presents the next question sent from the server to the user by voice or text.

[1047] Step 5: Collecting user responses and converting them to text

[1048] The user speaks further details to the terminal in response to the presented question. The input is the user's voice and the output is text data. The terminal converts the user's voice back into text data and sends it to the server.

[1049] Step 6: Editing and creating your autobiography

[1050] The server creates the autobiography chapter by chapter based on the collected data and emotion recognition results. The input is the collected data and emotion recognition results, and the output is the autobiography text organized by chapter. It then formats the autobiography based on the user's selected style (documentary, romantic comedy, etc.) and generates the final version. The input is the user's style selection, and the output is the formatted autobiography.

[1051] Step 7: Format and export your autobiography

[1052] The server generates a formatted autobiography in PDF format and sends it to the terminal. The input is the formatted autobiography text and the output is the autobiography in PDF format. The terminal displays a preview version and presents it to the user for confirmation.

[1053] Step 8: User review and correction

[1054] The user reviews the autobiography and submits correction requests as needed via their terminal. The input is the correction request, and the output is the correction. The server saves the final version that reflects the corrections and outputs it in a printable or downloadable format.

[1055] (Application example 2)

[1056] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1057] When elderly people recount past events and generate autobiographical texts, the texts may lack emotional depth or the questions asked may be inconsistent. This problem can be solved by recognizing specific emotions and generating questions and answers based on those emotions. Furthermore, the generated autobiographies must be provided in a digital format to promote a consistent user experience.

[1058] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes an emotion recognition means that analyzes the user's emotions and reflects them in the questions and sentence generation, an output means that outputs the generated autobiographical-style sentences in a digital format and presents them visually and audibly, and an emotion-reflecting question generation means that automatically generates the next question based on the emotion analysis results. This makes it possible to generate an autobiography with greater emotional depth according to the content spoken by the elderly person, enabling a user experience that is visually and audibly consistent.

[1059] "Input methods" refer to the means by which older people recount past events, and include various forms such as voice input and text input.

[1060] "Analysis methods" refer to the means used to analyze what is said and extract important keywords. Natural language processing technology is used, for example.

[1061] The "question generation means" refers to a means for presenting the next question to the elderly person based on the keywords extracted by the analysis means.

[1062] "Database management means" refers to a database system for registering and managing profile information and past photographs of elderly people.

[1063] "Text generation means" refers to a means for automatically generating autobiographical text based on received information.

[1064] "Emotion recognition means" refers to a means for analyzing a user's emotions and reflecting them in questions and sentence generation. This includes technology for reading emotions from voice and text.

[1065] "Output means" refers to a means for outputting the generated autobiographical text in a digital format and presenting it visually and audibly.

[1066] The "format conversion means" refers to a means for converting the generated autobiography into a specific style and outputting it.

[1067] The "means for generating emotion-reflecting questions" refers to a means for automatically generating the next question based on the emotion analysis results.

[1068] "Edit management means" refers to a means for providing an interface that allows the generated autobiographical writing to be displayed in a digital format and for commentary and edits to be made.

[1069] The present invention relates to a system that generates an autobiography by allowing elderly people to talk about past events and combining it with an emotion engine. The system includes the following elements:

[1070] Components

[1071] 1. Input means: Includes voice input devices and text input devices for the elderly to talk about their past events.

[1072] 2. Analysis method: Analyze what is said and extract important keywords and phrases. This is done using natural language processing technology.

[1073] 3. Question generation method: The next question is automatically generated based on the extracted keywords and the results of sentiment analysis and presented to the elderly.

[1074] 4. Database management means: Includes a database that stores and manages profile information and past photos of seniors.

[1075] 5. Text generation method: This is a technology that automatically generates autobiographical-style text based on collected data.

[1076] 6. Emotion recognition: Analyzes emotions based on the user's voice and text and reflects them in questions and sentence generation.

[1077] 7. Output means: The generated autobiography is output in a digital format, including a display and a speaker for visually and audibly presenting it.

[1078] 8. Format conversion method: The generated autobiography is converted into a specific style (documentary, romantic comedy, etc.) and output.

[1079] 9. Emotion-reflecting question generation method: Automatically generate the next question based on the emotion analysis results.

[1080] 10. Editing management tool: The generated autobiographical text is displayed in a digital format, and an interface is provided that allows users to comment and edit.

[1081] Program processing

[1082] Hardware and Software Used

[1083] Hardware: smartphones, tablets, microphones, displays, head-mounted displays (HMDs)

[1084] software:

[1085] Python library (speech_recognition): Used to convert speech to text.

[1086] Transformers Library: Sentiment analysis is performed using the natural language processing model provided by Hugging Face.

[1087] TextBlob: Used for keyword extraction and sentiment analysis of text data.

[1088] Process Description

[1089] The server receives data from a terminal that inputs the elderly person's speech. The speech is converted into text data using a speech recognition means (e.g., Python's speech_recognition library). The converted text is analyzed by an analysis means to extract important keywords. In addition, the user's emotions are analyzed using an emotion recognition means (e.g., Transformers library), and an emotion-reflecting question generation means generates an appropriate next question based on the analysis results. The question is presented to the user via the terminal.

[1090] By repeating this process, the collected data is written as an autobiographical text using the text generation means. The generated text is converted into a specific style using the format conversion means based on the style selected by the user, and is displayed visually and audibly through the output means. The generated autobiographical text is also displayed in a digital format using the revision management means, allowing for commentary and revision.

[1091] Specific examples

[1092] Example 1: Emotion recognition of childhood memories

[1093] User: "I spent my childhood in Hokkaido, a place rich in nature. I enjoyed every day."

[1094] Speech recognition result: "I spent my childhood in Hokkaido, an area rich in nature. Every day was fun."

[1095] Sentiment analysis result: Positive

[1096] Generated question: "What did you enjoy most about that time?"

[1097] Example prompts for generative AI models

[1098] Analyze the text below and identify the sentiment.

[1099] Text: "I spent my childhood in Hokkaido, a place rich in nature. I enjoyed every day."

[1100] Generate the following questions based on the emotions and keywords below.

[1101] Emotion: Positive

[1102] Keywords: ["Hokkaido, rich in nature," fun]

[1103] In this way, the system generates autobiographies with emotional depth based on what the elderly person narrates, providing a consistent user experience.

[1104] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1105] Step 1:

[1106] The user logs in to the device and enters basic information. The device encrypts the information and sends it to the server. The server verifies the login information and saves the user profile in a database. It also receives past photos and important events from the user and saves them in the database.

[1107] Step 2:

[1108] The user speaks into the device's voice input device to recount past events. The device uses a speech recognition method (e.g., Python's speech_recognition library) to convert the speech into text data and sends the text data to the server. Here, the input is the user's voice data, and the output is text data.

[1109] Step 3:

[1110] The server analyzes the received text data using its analysis means and extracts important keywords and phrases. Using natural language processing technology, it processes and calculates the data to extract the necessary information. The input for this step is the voice-recognized text data, and the output is the extracted keywords and phrases.

[1111] Step 4:

[1112] The server uses emotion recognition tools (e.g., the Transformers library) to analyze user emotions from text data. The text is input to an emotion analysis model, which outputs the user's emotional state (positive, negative, or neutral). The input here is the analyzed text data, and the output is the emotion analysis results.

[1113] Step 5:

[1114] The server generates the next question using the emotion-reflecting question generation means based on the emotion analysis results and extracted keywords. For example, if a positive emotion is detected, a question such as "What was the most enjoyable thing at that time?" is automatically generated. The input is the emotion analysis results and extracted keywords, and the output is the generated question.

[1115] Step 6:

[1116] The device presents the generated question to the user. The user then recites what they have said by voice, which is then converted back into text data and sent to the server. By repeating this cycle, more detailed information is collected. The input is the automatically generated question from the server, and the output is the user's voice data and its converted text data.

[1117] Step 7:

[1118] The server uses a text generation tool based on the collected data to automatically generate autobiographical text. It combines the stories told by the elderly with the results of emotion analysis to create text in a natural format. The input is the data told by the elderly and the results of emotion analysis, and the output is an autobiographical text.

[1119] Step 8:

[1120] The server converts the generated autobiographical text into a specific style selected by the user using a format conversion means. For example, it converts into a style that matches the user's preferences, such as a documentary style or a romantic comedy style. The input is the generated autobiographical text, and the output is the format-converted text.

[1121] Step 9:

[1122] The server provides the final generated autobiography to the user in digital format through the output means. The terminal has a display for visual confirmation, allowing the user to check the content. The input is the formatted text, and the output is the autobiography presented in digital format.

[1123] Step 10:

[1124] The user checks the generated autobiographical text and, if any corrections are necessary, sends a correction request using the correction management means. The server receives the correction request, regenerates the final version with the relevant parts corrected, and presents it to the user again. The input is the user's correction request, and the output is the corrected autobiographical text.

[1125] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1126] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1127] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1128] [Third embodiment]

[1129] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1130] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1131] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1132] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1133] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1134] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1135] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1136] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1137] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1138] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1139] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1140] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1141] The present invention is a system that generates autobiographical text based on the recounting of past events by elderly people. This system involves the process of inputting and analyzing the content spoken by the elderly person, generating questions to be presented next, and finally compiling the autobiography.

[1142] System configuration

[1143] user

[1144] Elderly people: They are the main users of the system and talk about their own past experiences.

[1145] Family and care staff: Provide support to the older person as needed.

[1146] Terminal

[1147] Input devices: Devices that allow the older adult to input speech, including microphones and touchscreens.

[1148] Output device: A device used to display questions and generated text for confirmation. This can include a display or speaker.

[1149] server

[1150] Database: Stores and manages elderly profile information, conversations, uploaded photos and important events.

[1151] Analysis engine: Analyzes user voice data and extracts important keywords and phrases using natural language processing technology.

[1152] Question generation engine: Automatically generates the next question based on the extracted keywords.

[1153] Text generation engine: Automatically generates autobiographical text based on collected data.

[1154] Format conversion engine: Converts the generated autobiography into a specific style (documentary, romantic comedy, etc.) and outputs it.

[1155] Program processing

[1156] Initial setup and receiving user information

[1157] User: Logs in to a device and enters basic information such as date of birth and major life events.

[1158] Terminal: The entered information is encrypted and sent to the server.

[1159] Server: Validates login information and stores user profiles in a database. Also receives past photos and important events from users and stores them in a database.

[1160] Starting a conversation

[1161] Device: Present the user with an initial question, for example, "Tell me about your childhood."

[1162] User: Speaks into the device and recounts memories of the elderly person's childhood.

[1163] Device: Converts the user's voice into text data and sends it to the server.

[1164] Data analysis and question generation

[1165] Server: Analyzes the text data and extracts important keywords.

[1166] Server: Generates the next question based on the extracted keywords and sends it to the device. For example, it might ask, "What was your most memorable camping experience?"

[1167] Editing and creating autobiographies

[1168] User: Speaks further details into the device in response to the questions presented.

[1169] Device: Converts the user's speech into text and sends it back to the server.

[1170] Server: Based on the collected data, compiles autobiographical text and adds it to the autobiographical chapters.

[1171] Server: Formats the autobiography based on the style selected by the user and generates the final version.

[1172] Output and Save

[1173] Server: Generates the completed autobiography in PDF format and sends it to the device.

[1174] Device: Shows a preview version and presents it to the user for review.

[1175] User: Review the autobiography and submit a request to make corrections if necessary.

[1176] Server: Save the final version with the changes and make it available for printing or downloading.

[1177] Specific examples

[1178] Example 1: Recounting childhood memories

[1179] User: "I spent my childhood in Hokkaido, a place rich in nature."

[1180] Device: Converts this speech into text and sends it to the server.

[1181] Server: Extracts the keyword "Hokkaido, rich in nature," generates the question "What was the most memorable event at that time?" and sends it to the device.

[1182] User: "My favorite memories are going camping with my family and fishing on a big lake."

[1183] Terminal: Converts the voice into text data and sends it back to the server.

[1184] Server: Add this episode to the "Childhood Memories" chapter and generate it as text.

[1185] Example 2: Memories from school days

[1186] User: "I spent my student days in Tokyo. I had lots of adventures with my friends."

[1187] Device: Converts this speech into text and sends it to the server.

[1188] Server: Extracts the keywords "student days," "friends," and "adventure," generates a question such as "What was the most fun adventure you had at that time?" and sends it to the device.

[1189] User: "The most fun I had was climbing Mount Fuji with my friends."

[1190] Device: Converts the voice into text data and sends it back to the server.

[1191] Server: Add this episode to the chapter "Memories of Student Life" and generate it as a sentence.

[1192] In this way, a system in which AI generates autobiographical text based on past events recounted by elderly people will be useful in preventing dementia among the elderly and promoting communication with their families.

[1193] The processing flow will be explained below.

[1194] Specific processing steps of the program

[1195] Step 1: Initial setup and receiving user information

[1196] 1.1 User: Log in to the device and launch the AI ​​autobiography creation system.

[1197] Specific actions: Enter your username and password on the login screen displayed on the device and click the "Login" button.

[1198] 1.2 Device: Sends login information to server.

[1199] Specific operation: The entered login information is encrypted and sent to the server.

[1200] 1.3 Server: Verify login information and retrieve user profile data.

[1201] What it does: Retrieves user profile information (e.g., name, date of birth, major events) from a database.

[1202] 1.4 User: Register past photos and important events (topics and turning points) via the device.

[1203] Specific actions: Click the "Upload Photo" or "Add Event" button, select a file, or enter text.

[1204] 1.5 Terminal: Sends uploaded photos and entered event information to the server.

[1205] Specific operation: Transfers the selected photo files and text data to the server.

[1206] 1.6 Server: Stores photos and event information and organizes it as structured data.

[1207] What it does: Store the photos in an image library and add the event information to the associated time series data.

[1208] Step 2: Initiating a conversation

[1209] 2.1 Terminal: Present the initial question to the user.

[1210] What it does: Uses speech synthesis technology to play the question "Tell me about your childhood."

[1211] 2.2 User: Speaks into the device and recounts childhood memories.

[1212] Specific action: "When I was a child, I often went camping with my family in Hokkaido, which is rich in nature."

[1213] 2.3 Terminal: Converts the user's speech into text data.

[1214] What it does: It uses speech recognition software to convert speech into text and send it to a server.

[1215] Step 3: Analyze the data and generate questions

[1216] 3.1 Server: Analyzes the text data and extracts important keywords and phrases.

[1217] Specific operation: Extract keywords such as "Hokkaido, rich in nature" and "camping" using a natural language processing algorithm.

[1218] 3.2 Server: Generates a new question and sends it to the device.

[1219] Specific operation: Generate questions related to "nature" and "camping" and send a text message to the device saying, "What is your most memorable camping experience?"

[1220] 3.3 Terminal: Present the new question to the user.

[1221] What it does: Converts the question text into speech and plays it back to the user.

[1222] Step 4: Editing and creating your autobiography

[1223] 4.1 User: Speaks further details into the terminal in response to the questions presented.

[1224] Specific action: "One summer day, my father, brothers and I went fishing on a big lake and caught a big salmon, and that's a memory I'll never forget."

[1225] 4.2 Terminal: The user's voice is converted back into text data and sent to the server.

[1226] What it does: It uses speech recognition technology to convert speech into text and send it to a server for analysis.

[1227] 4.3 Server: Write your autobiography chapter by chapter based on the data collected.

[1228] What it does: Use natural language generation techniques to write a chapter on "Childhood Memories" and store it in a database.

[1229] 4.4 Server: Formats the autobiography in the style selected by the user.

[1230] Specific operation: Generates text that reflects styles such as "documentary style" or "romantic comedy style."

[1231] Step 5: Export and save

[1232] 5.1 Server: Sends the completed autobiography to the user's device for confirmation.

[1233] Specific behavior: Generates a preview version of the autobiography in PDF format and sends it to the device.

[1234] 5.2 Device: Show preview and ask user for confirmation.

[1235] What it does: Displays a preview of the autobiography in a PDF viewer and provides a "Confirm" button.

[1236] 5.3 User: Review the autobiography and request corrections if necessary.

[1237] Specific actions: Click the "Confirm" button and enter any necessary corrections.

[1238] 5.4 Server: Stores the finalized autobiography and outputs it in a printable or downloadable format.

[1239] Specific behavior: Saves the final version to the database and generates PDF and print files.

[1240] In this way, each step of the system that uses AI to automatically generate an autobiography based on the memories of the elderly person is achieved through collaboration between the user, device, and server.

[1241] Example 1

[1242] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1243] Previous systems that generate autobiographical documents from elderly people's recounting of past events were insufficient in abstracting what was said and in presenting the next question, making it difficult to create a natural dialogue. Furthermore, the style and format of the generated autobiography were limited, making it difficult to flexibly respond to user needs. Furthermore, there was a lack of technology to accurately convert voice input into text, resulting in a decline in the quality of the generated documents.

[1244] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1245] In this invention, the server includes an analysis means for analyzing what has been said and extracting important keywords, a question generation means for generating the next question based on the extracted keywords and presenting it to the elderly person, a database management means for registering and managing the elderly person's profile information and past images, a document generation means for automatically generating an autobiography-style document based on the received information, and a format conversion means for selecting and outputting the autobiography style. This enables a natural flow of dialogue and enables the flexible generation of autobiographies that meet the user's needs.

[1246] The "input means" is a device that allows elderly people to input voice or text recounting past events.

[1247] "Analysis means" refers to a device or program that has the function of analyzing what is said and extracting important keywords and phrases.

[1248] The "question generation means" is a device or program that automatically generates the next question based on the extracted keywords and presents it to the elderly person.

[1249] The "database management means" is a system for registering and managing elderly people's profile information, past images, and data on important events.

[1250] The "document generation means" is a device or program that automatically generates an autobiographical document based on the received information.

[1251] The "format conversion means" is a device or program that selects the style of the generated autobiography, converts it into a specified format, and outputs it.

[1252] The "voice recognition means" is a device or program that inputs the elderly person's speech and converts it into text data.

[1253] The "revision management means" is a device or program for presenting the generated autobiographical writing to the user in a revisable format and for reflecting requests for revisions.

[1254] "Natural language processing technology" is a technology for analyzing, understanding, and generating human language.

[1255] "Elderly people" refers to the subjects who use the system to narrate past events.

[1256] This invention is an information processing system that generates an autobiographical document based on an elderly person's recounting of past events. The system uses speech recognition and natural language processing technologies to convert the elderly person's recounting into text, extract important keywords, and generate the next question. It also includes a function to select the style of the generated autobiography and output it as a document.

[1257] System configuration

[1258] Hardware

[1259] Device: A device equipped with a microphone for voice input and a touchscreen.

[1260] Server: High-performance computer for data analysis and document generation

[1261] Database: A storage system for managing profile information and past images of elderly people

[1262] Display: A screen for displaying questions and generated documentation.

[1263] Speaker: An audio device for presenting questions aloud.

[1264] software

[1265] Speech recognition software: A system that inputs elderly people's speech and converts it into text data

[1266] Natural language processing software: Software that extracts important keywords from text data

[1267] Question generation engine: A program that automatically generates the next question based on the extracted keywords.

[1268] Document generation engine: Software that automatically generates autobiographical documents based on collected data

[1269] Format conversion engine: Software that converts the generated autobiography into a specific style and outputs it.

[1270] Revision management system: Software that presents the generated autobiographical text to the user in a revisable format and reflects revisions.

[1271] System Operation

[1272] Registering user information

[1273] Users log in to their devices and enter basic information about their date of birth and major life events. This information is encrypted on the device and sent to the server, which then stores it in a database as part of the user's profile. The server also receives past images and important events from the user and stores them in the database.

[1274] Starting a conversation

[1275] The device presents the user with an initial question, for example, "Tell me about your childhood." The user speaks into the device, recounting the elderly person's childhood memories. The device converts the speech into text data and sends it to the server.

[1276] Data analysis

[1277] The server analyzes the text data and extracts important keywords. It then generates the next question based on the extracted keywords and sends it to the device. For example, it could ask a specific question such as, "What was your most memorable camping experience?"

[1278] Document Generation

[1279] The user responds to the questions with further details, which are then converted back into text and sent to the server. The server then uses the collected data to compile an autobiographical document, adding chapters to it, and formats the autobiography based on the user's chosen style to produce the final version.

[1280] Output and Modification

[1281] The completed autobiography is sent from the server to the device in PDF format. The device displays a preview version and presents it to the user for confirmation. The user reviews the autobiography and submits a request for any necessary corrections. The server saves the final version after the corrections are applied and outputs it in a format that can be printed or downloaded.

[1282] Examples and prompts

[1283] Example 1: Recounting childhood memories

[1284] When a user says, "I spent my childhood in Hokkaido, a place rich in nature," the device converts this speech into text and sends it to the server. The server extracts the keyword "Hokkaido, a place rich in nature," generates a question, "What was the most memorable event at that time?" and sends it to the device. The user then adds, "My best memory is going camping with my family and fishing on a large lake," and this is added to the autobiography as a "childhood memory."

[1285] Prompt Sentence Examples

[1286] "Tell me an anecdote from your childhood."

[1287] What is your most enjoyable memory from your school days?

[1288] "Tell me some memories you have with your family."

[1289] "Tell us about the most memorable event."

[1290] In this way, it is possible to realize natural and flowing dialogue based on the elderly person's narrative, and generate a high-quality autobiography.

[1291] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1292] Step 1: Initial setup and receiving user information

[1293] User: Log in to the device and enter the required user information (date of birth and data related to major life events).

[1294] Input: Basic information such as date of birth, major life events, etc.

[1295] Output: Initial setting data

[1296] Terminal: The entered information is encrypted and sent to the server.

[1297] Input: Data entered by the user

[1298] Data processing: Encryption processing

[1299] Output: Encrypted user information

[1300] Server: Receives user information, verifies login information, and stores user profiles in a database.

[1301] Input: Encrypted user information

[1302] Data processing: Save to database

[1303] Output: Saved user profile data

[1304] Step 2: Initiating a conversation

[1305] Device: Present the first question to the user aloud, for example, "Tell me about your childhood."

[1306] Input: Fixed question template

[1307] Data processing: Question voice synthesis

[1308] Output: Questions presented by voice

[1309] User: Speaks into device, recounting childhood memories.

[1310] Input: User's voice data

[1311] Output: Recorded audio data

[1312] Device: Converts the user's voice into text data and sends it to the server.

[1313] Input: User's voice data

[1314] Data processing: speech recognition and text conversion

[1315] Output: Text data

[1316] Step 3: Analyze the data and generate questions

[1317] Server: Analyzes the received text data and extracts important keywords using natural language processing technology.

[1318] Input: Text data

[1319] Data processing: Keyword extraction using natural language processing

[1320] Output: Extracted keywords

[1321] Server: Generates the next question based on the extracted keywords and sends it to the device.

[1322] Input: Extracted keywords

[1323] Data processing: Generate new questions using a question generation algorithm

[1324] Output: Generated questions

[1325] Step 4: Editing and creating your autobiography

[1326] User: Provides details in response to questions posed by the device.

[1327] Input: Question from terminal

[1328] Output: User's voice data

[1329] Device: Converts the user's voice into text data and sends it back to the server.

[1330] Input: User's voice data

[1331] Data processing: speech recognition and text conversion

[1332] Output: Text data

[1333] Server: Based on the collected data, compile an autobiographical document and add chapters to it.

[1334] Input: Text data and user profile

[1335] Data processing: text editing and chapter addition

[1336] Output: Updated autobiographical text

[1337] Server: Formats the autobiography based on the style selected by the user and generates the final version.

[1338] Input: Updated autobiographical text

[1339] Data processing: Applying a format conversion engine

[1340] Output: Final formatted autobiography

[1341] Step 5: Export and save

[1342] Server: Generates the completed autobiography in PDF format and sends it to the device.

[1343] Input: Final formatted autobiography

[1344] Data processing: PDF conversion

[1345] Output: Autobiography in PDF format

[1346] On the device: The generated PDF is displayed as a preview and presented to the user for review.

[1347] Input: Autobiography in PDF format

[1348] Output: Preview

[1349] User: Review the autobiography and submit a request for corrections if needed.

[1350] Input: Previewed autobiography

[1351] Output: Correction request

[1352] Server: Save the final version with the corrections and output it in a printable or downloadable format.

[1353] Input: Correction request and autobiography in PDF format

[1354] Data processing: Reflecting corrections and saving the final version

[1355] Output: Final autobiography

[1356] (Application example 1)

[1357] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1358] Elderly people's recounting of their past experiences is an important way to record and pass on personal memories to future generations. However, in today's busy living environment, family members and caregivers often do not have enough time to carefully listen to what the elderly have to say. Furthermore, there is a need to provide experiences related to past memories in order to prevent dementia and help the elderly feel a sense of spiritual fulfillment. The present invention aims to solve these problems by easily generating an autobiography based on the elderly's narrative and providing a customized meal menu based on that autobiography.

[1359] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1360] In this invention, the server includes input means for the elderly person to talk about their past, analysis means for analyzing what has been said and extracting important keywords, question generation means for generating the next question based on the extracted keywords and presenting it to the elderly person, database management means for registering and managing the elderly person's profile information and past photos, text generation means for automatically generating autobiographical text based on the received information, format conversion means for selecting and outputting the autobiographical style, and menu suggestion means for suggesting a customized meal menu based on the elderly person's story. This makes it possible to create an autobiography by having the elderly person talk about events from their past, and to provide a special dining experience based on that story.

[1361] The "elderly" are the subjects who use the system to recount past events and generate autobiographical texts.

[1362] The "input means" is a device used by elderly people to talk about past events, and is a device that allows voice input and text input.

[1363] The "analysis means" is a device or software that analyzes what the elderly say and extracts important keywords.

[1364] The "question generation means" is a device or software that automatically generates the next question based on the extracted keywords and presents it to the elderly person.

[1365] The "database management means" refers to a device or software that registers and manages profile information and past photographs of elderly people.

[1366] The "text generation means" is a device or software that automatically generates autobiographical text based on received information.

[1367] The "format conversion means" is a device or software that selects the style of the generated autobiography, converts it into a specific format, and outputs it.

[1368] The "menu suggestion means" is a device or software that suggests a customized meal menu based on the elderly person's narration.

[1369] This invention is a system that generates autobiographical text by having an elderly person recount events from their past, and proposes a customized meal menu based on that narrative. Specific embodiments of this system will be described in detail below.

[1370] user

[1371] senior citizen

[1372] The subject who uses the system talks about events from his or her past. The system analyzes what he or she says and ultimately compiles it into an autobiography.

[1373] Family members and caregivers

[1374] Provide support to the elderly as needed and help them use the system.

[1375] Terminal

[1376] Input Devices

[1377] This device allows elderly people to input their speech, and includes a microphone and a touchscreen.

[1378] Output Device

[1379] A device for displaying generated questions, autobiographical text, and suggested meal menus, including a display and speaker.

[1380] server

[1381] Database Management Methods

[1382] Stores and manages data such as elderly people's profile information, past photos, and spoken content.

[1383] Analysis means

[1384] Analyze user speech (audio data) and extract important keywords and phrases using voice recognition technology (e.g., Google speech-to-text API) and natural language processing technology (e.g., Hugging Face's Transformers library).

[1385] Question generation means

[1386] The next question is automatically generated based on the extracted keywords, using a generative AI model (e.g., GPT-2).

[1387] Sentence generation means

[1388] Based on the collected data, an autobiographical text is edited and generated, again using natural language generation technology (e.g., GPT-2).

[1389] Format conversion method

[1390] The generated autobiography is converted into a specific style (documentary, romantic comedy, etc.) and output.

[1391] Menu suggestion method

[1392] This system proposes customized meal menus based on the elderly's narratives. It extracts food-related information from keywords in the narratives and proposes related menus.

[1393] Program processing explanation

[1394] The server receives the voice input data of what the elderly person is saying and uses an analysis means to extract important keywords and phrases. Based on the analyzed data, a question generation means generates the next question and sends it to the terminal to present to the user. The user's voice response is sent back to the server, and by repeating the same process, an autobiographical-style text is gradually generated.

[1395] Furthermore, food-related keywords are extracted from the analyzed content, and the menu suggestion means suggests meal menus that are memorable to the elderly.

[1396] For example, if an elderly person says, "When I was a child, I loved the miso soup my grandmother made for me," the analysis method will extract the keywords "childhood," "grandmother," and "miso soup," and generate the next question, "What other dish did you like most at that time?" Based on the answer to that question, the system will ask the next question and eventually suggest related menu items such as "miso soup set" or "grandmother's special miso soup."

[1397] Prompt Sentence Examples

[1398] "Logged in user: {username} said the following: 'When I was a child, I loved the miso soup my grandmother made for me.' Please generate follow-up questions."

[1399] The above is a specific embodiment for carrying out the present invention. This system allows elderly people to create their autobiographies by talking about their past, and furthermore, it makes it possible to provide special dining experiences based on past memories.

[1400] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1401] Step 1:

[1402] A user logs into the system and enters their profile information (date of birth, major life events, etc.) The entered information is sent to the server in the form of data, which is then stored in the database.

[1403] Step 2:

[1404] The device generates an initial question and presents it to the user, for example, "Tell me about your childhood," either by voice or text. The user begins to recount events from their past.

[1405] Step 3:

[1406] The user speaks their answer through a voice input device, and this voice data is input into the terminal.

[1407] Step 4:

[1408] The device converts the voice data into text data using voice recognition technology (e.g., Google speech-to-text API), and the converted text data is sent to the server.

[1409] Step 5:

[1410] The server receives the text data and uses analytical tools to extract important keywords and phrases. Natural language processing techniques (e.g., Hugging Face's Transformers library) are used. The results of this analysis are used to generate the next question.

[1411] Step 6:

[1412] The server generates the next question based on the extracted keywords. It uses a generative AI model (e.g., GPT-2) to create an appropriate next question. For example, a question like, "What was the most memorable event at that time?" is generated. The generated question is sent to the device.

[1413] Step 7:

[1414] The device presents the generated question to the user, who answers the new question and then speaks again.

[1415] Step 8:

[1416] The server receives the voice data again and repeats the same process. Finally, when enough information has been collected, an autobiographical text is automatically generated by the text generation means.

[1417] Step 9:

[1418] The server uses a text generation tool to edit and generate the collected data along an autobiography-like flow. The generated autobiography is formatted based on the style selected by the user (e.g., documentary style, romantic comedy style).

[1419] Step 10:

[1420] The server sends the formatted autobiography to the terminal and displays it as a preview to the user. The user can submit correction requests as needed. If no corrections are required, the final version is generated.

[1421] Step 11:

[1422] The server then saves the final autobiography in a format such as PDF, and outputs it in a printable or downloadable format. At the same time, a menu suggestion tool is used to suggest a customized meal menu based on the elderly person's story. This menu suggestion is also displayed on the device.

[1423] The above is the flow of specific processing steps of the system that realizes the application example.

[1424] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1425] This invention combines an emotion engine with a system that generates autobiographical text based on the elderly's recounting of past events. The system inputs what the elderly person is saying, analyzes it, generates questions to pose next, and ultimately compiles it into an autobiography with emotional depth. The system also recognizes the user's emotions and asks appropriate questions and responds based on the results.

[1426] System configuration

[1427] user

[1428] Elderly people: They are the main users of the system and talk about their own past experiences.

[1429] Family and care staff: Provide support to the older person as needed.

[1430] Terminal

[1431] Input devices: Devices that allow the older adult to input speech, including microphones and touchscreens.

[1432] Output device: A device used to display questions and generated text for confirmation. This can include a display or speaker.

[1433] server

[1434] Database: Stores and manages elderly people's profile information, spoken content, uploaded photos, important events, and emotion recognition results.

[1435] Analysis engine: Analyzes user voice data and extracts important keywords and phrases using natural language processing technology.

[1436] Question generation engine: Automatically generates the next question based on the extracted keywords.

[1437] Emotion engine: Recognizes emotions from the user's voice and facial expressions and tailors questions and responses accordingly, adding depth to sentences depending on the user's emotional state.

[1438] Text generation engine: Automatically generates autobiographical text based on collected data.

[1439] Format conversion engine: Converts the generated autobiography into a specific style (documentary, romantic comedy, etc.) and outputs it.

[1440] Program processing

[1441] Initial setup and receiving user information

[1442] User: Log in to the device and enter basic information (date of birth, major life events, etc.).

[1443] Terminal: The entered information is encrypted and sent to the server.

[1444] Server: Validates login information and stores user profiles in a database. Also receives past photos and important events from users and stores them in a database.

[1445] Starting a conversation

[1446] Device: Present the user with an initial question, for example, "Tell me about your childhood."

[1447] User: Speaks into device, recounting childhood memories.

[1448] Device: Converts the user's voice into text data and sends it to the server.

[1449] Data analysis and question generation

[1450] Server: Analyzes the text data and extracts important keywords.

[1451] Server: The emotion engine recognizes the user's emotional state and reflects it when generating the next question. For example, if the user is telling a happy story, it generates the question, "What was the most memorable happy event at that time?"

[1452] Editing and creating autobiographies

[1453] User: Speaks further details into the device in response to the questions presented.

[1454] Device: The user's voice is converted back into text data and sent to the server.

[1455] Server: Writes an autobiography chapter by chapter based on collected data and emotion recognition results.

[1456] Server: Formats the autobiography based on the style selected by the user and generates the final version.

[1457] Output and Save

[1458] Server: Generates the completed autobiography in PDF format and sends it to the device.

[1459] Device: Shows a preview version and presents it to the user for review.

[1460] User: Review the autobiography and submit a request to make corrections if necessary.

[1461] Server: Save the final version with the changes and make it available for printing or downloading.

[1462] Specific examples

[1463] Example 1: Emotion recognition of childhood memories

[1464] User: "I spent my childhood in Hokkaido, a place rich in nature. I enjoyed every day."

[1465] Device: Converts this speech into text and sends it to the server.

[1466] Server: Extracts keywords such as "Hokkaido, rich in nature" and "fun," and recognizes the user's enjoyment from the tone of their voice. Generates a question such as "What was the most memorable and enjoyable event at that time?" and sends it to the device.

[1467] User: "My favorite memories are going camping with my family and fishing on a big lake."

[1468] Terminal: Converts the voice into text data and sends it back to the server.

[1469] Server: Add this episode to the "Childhood Memories" chapter and generate it as a sentence containing emotions.

[1470] Example 2: Emotion recognition of memories from school days

[1471] User: "I spent my student days in Tokyo. I had lots of adventures with my friends."

[1472] Device: Converts this speech into text and sends it to the server.

[1473] Server: Extracts keywords such as "student days" and "adventure" and recognizes excited emotions from the user's voice. Generates a question such as "What was the most fun adventure you had at that time?" and sends it to the device.

[1474] User: "The most fun I had was climbing Mount Fuji with my friends."

[1475] Device: Converts the voice into text data and sends it back to the server.

[1476] Server: Add this episode to the chapter "Memories of Student Life" and generate a sentence containing emotions.

[1477] In this way, a system in which AI recognizes emotions based on past events recounted by elderly people and automatically generates an autobiography accordingly will be even more effective in preventing dementia among the elderly and promoting communication with their families.

[1478] The processing flow will be explained below.

[1479] Specific processing steps of the program

[1480] Step 1: Initial setup and receiving user information

[1481] 1.1 User: Log in to the device and launch the AI ​​autobiography creation system.

[1482] Specific actions: Enter your username and password on the login screen and click the "Login" button.

[1483] 1.2 Device: Sends login information to server.

[1484] Specific behavior: Encrypts the entered login information and sends it to the server.

[1485] 1.3 Server: Verify login information and retrieve user profile data.

[1486] What it does: Retrieves user profile information (e.g., name, date of birth, major events) from a database.

[1487] 1.4 User: Register past photos and important events (topics and turning points) via the device.

[1488] Specific actions: Click the "Upload Photo" or "Add Event" button to select a file or enter text.

[1489] 1.5 Terminal: Sends uploaded photos and entered event information to the server.

[1490] Specific operation: Transfers the selected photo files and text data to the server.

[1491] 1.6 Server: Stores photos and event information and organizes it as structured data.

[1492] What it does: Store the photos in an image library and add the event information to the associated time series data.

[1493] Step 2: Initiating a conversation

[1494] 2.1 Terminal: Present the initial question to the user.

[1495] What it does: Uses speech synthesis technology to play the question "Tell me about your childhood."

[1496] 2.2 User: Speaks into the device and recounts childhood memories.

[1497] Specific action: "When I was a child, I often went camping with my family in Hokkaido, which is rich in nature."

[1498] 2.3 Terminal: Converts the user's speech into text data.

[1499] What it does: It uses speech recognition software to convert speech into text and send it to a server.

[1500] Step 3: Analyze the data and generate questions

[1501] 3.1 Server: Analyzes the text data and extracts important keywords and phrases.

[1502] Specific operation: Extract keywords such as "Hokkaido, rich in nature" and "camping" using a natural language processing algorithm.

[1503] 3.2 Server: The emotion engine recognizes the user's emotional state and reflects it when generating the next question.

[1504] Specific operation: Analyzes the user's emotions from their tone of voice and facial expressions, generates a question such as "What was the most memorable and enjoyable event at that time?" and sends it to the device.

[1505] 3.3 Terminal: Present the new question to the user.

[1506] What it does: Converts the question text into speech and plays it back to the user.

[1507] Step 4: Editing and creating your autobiography

[1508] 4.1 User: Speaks further details into the terminal in response to the questions presented.

[1509] Specific action: "One summer day, my father, brothers and I went fishing on a big lake and caught a big salmon, and that's a memory I'll never forget."

[1510] 4.2 Terminal: The user's voice is converted back into text data and sent to the server.

[1511] What it does: It uses speech recognition technology to convert speech into text and send it to a server for analysis.

[1512] 4.3 Server: Writes an autobiography chapter by chapter based on the collected data and emotion recognition results.

[1513] What it does: Use natural language generation technology to write a chapter on "Childhood Memories," edit it into emotionally rich text, and store it in a database.

[1514] 4.4 Server: Formats the autobiography in the style selected by the user.

[1515] Specific operation: Generates text that reflects styles such as "documentary style" or "romantic comedy style."

[1516] Step 5: Export and save

[1517] 5.1 Server: Sends the completed autobiography to the user's device for confirmation.

[1518] Specific behavior: Generates a preview version of the autobiography in PDF format and sends it to the device.

[1519] 5.2 Device: Show preview and ask user for confirmation.

[1520] What it does: Displays a preview of the autobiography in a PDF viewer and provides a "Confirm" button.

[1521] 5.3 User: Review the autobiography and request corrections if necessary.

[1522] Specific actions: Click the "Confirm" button and enter any necessary corrections.

[1523] 5.4 Server: Stores the finalized autobiography and outputs it in a printable or downloadable format.

[1524] Specific behavior: Saves the final version to the database and generates PDF and print files.

[1525] In this way, each step of the system, which uses AI to recognize emotions and automatically generate an autobiography based on memories from the past recounted by an elderly person, is achieved through collaboration between the user, device, and server.

[1526] Example 2

[1527] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1528] When elderly people recount past events, organizing them into autobiographical text takes time and effort. Furthermore, generating high-quality text that incorporates emotion requires generating appropriate questions. Furthermore, correcting and formatting the generated text is also time-consuming. Thus, there is a need for an efficient system that can automatically organize and write down past events with emotional depth.

[1529] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1530] In this invention, the server includes emotion recognition means, analysis means, and question generation means, which enable the server to recognize emotions from the elderly person's narrative, analyze the content, and generate appropriate questions, thereby efficiently generating high-quality autobiographies with emotional depth.

[1531] "Input means" refers to devices used by older adults to recount past events, including voice input and touchscreens.

[1532] The "analysis means" is a technology for analyzing what the elderly say and extracting important keywords, and is a device that uses natural language processing technology.

[1533] The "question generation means" is a technology for automatically generating the next question based on the extracted keywords and presenting it to the elderly person.

[1534] "Database management means" refers to technology for registering and managing elderly people's profile information and past photographs.

[1535] "Emotion recognition means" is a technology that recognizes the user's emotions from their voice and facial expressions and asks appropriate questions and responds accordingly.

[1536] The "text generation means" is a technology for automatically generating autobiographical text based on received information.

[1537] The "format conversion means" is a technique for selecting the style of the generated autobiography and outputting it in a specified format.

[1538] "Voice recognition means" is a technology that inputs the elderly person's speech and converts it into text data.

[1539] The "revision management means" is a technique for presenting the generated autobiographical text to the user in a revisable format and for reflecting the user's revisions.

[1540] This invention relates to a system that generates autobiographical text based on the elderly's recounting of past events. This system involves a process of inputting the elderly's speech, analyzing it, generating questions to be presented next, and finally compiling it into an autobiography with emotional depth. The system can also recognize the user's emotions and provide appropriate questions and responses based on the results.

[1541] System configuration

[1542] The system mainly consists of the following elements:

[1543] user

[1544] Elderly people: They are the main users of the system and talk about their own past experiences.

[1545] Family and care staff: Provide support to the older person as needed.

[1546] Terminal

[1547] Input devices: Devices that allow the older adult to input speech, including microphones and touchscreens.

[1548] Output device: A device that displays questions and generated text for confirmation. This includes displays and speakers.

[1549] server

[1550] Database: Stores and manages elderly people's profile information, spoken content, uploaded photos, important events, and emotion recognition results.

[1551] Analysis engine: Analyzes user voice data and extracts important keywords and phrases using natural language processing technology.

[1552] Question generation engine: Automatically generates the next question based on the extracted keywords.

[1553] Emotion engine: Recognizes emotions from the user's voice and facial expressions and tailors questions and responses accordingly, adding depth to sentences depending on the user's emotional state.

[1554] Text generation engine: Automatically generates autobiographical text based on collected data.

[1555] Format conversion engine: Converts the generated autobiography into a specific style (e.g., documentary or romantic comedy) and outputs it.

[1556] System Operation

[1557] Hardware

[1558] Microphone: An input device for receiving a user's voice input.

[1559] Display: An output device used to display questions and generated text.

[1560] Server: A computer on which various engines (analysis engine, question generation engine, emotion engine, sentence generation engine, format conversion engine) run.

[1561] software

[1562] Natural Language Processing: A technique used by analytics engines to extract important keywords and phrases from what users say.

[1563] Emotion Recognition Algorithm: This is the technology used in the emotion engine to recognize emotions from the user's voice and facial expressions.

[1564] AI model: A generative AI model used for question generation and sentence generation.

[1565] Specific examples

[1566] Example 1: Emotion recognition of childhood memories

[1567] User: "I spent my childhood in Hokkaido, a place rich in nature. I enjoyed every day."

[1568] Device: Converts this speech into text and sends it to the server.

[1569] Server: Extracts keywords such as "Hokkaido, rich in nature" and "fun," and recognizes the user's enjoyment from the tone of their voice. Generates a question such as "What was the most memorable and enjoyable event at that time?" and sends it to the device.

[1570] User: "My favorite memories are going camping with my family and fishing on a big lake."

[1571] Terminal: Converts the voice into text data and sends it back to the server.

[1572] Server: Add this episode to the "Childhood Memories" chapter and generate it as a sentence containing emotions.

[1573] Example 2: Emotion recognition of memories from school days

[1574] User: "I spent my student days in Tokyo. I had lots of adventures with my friends."

[1575] Device: Converts this speech into text and sends it to the server.

[1576] Server: Extracts keywords such as "student days" and "adventure" and recognizes excited emotions from the user's voice. Generates a question such as "What was the most fun adventure you had at that time?" and sends it to the device.

[1577] User: "The most fun I had was climbing Mount Fuji with my friends."

[1578] Device: Converts the voice into text data and sends it back to the server.

[1579] Server: Add this episode to the chapter "Memories of Student Life" and generate a sentence containing emotions.

[1580] Prompt Sentence Examples

[1581] 1. A user logs in and enters basic information. Please encrypt this information before sending it.

[1582] 2. Input the user's voice data and convert it into text data.

[1583] 3. Extract specific keywords from text data and recognize the user's emotional state.

[1584] 4. Generate the next appropriate question based on the perceived emotional state and present it to the user.

[1585] 5. Collect the user's responses again and analyze the detailed data on the server.

[1586] 6. Create an autobiography chapter by chapter based on the data you have collected.

[1587] 7. Format your final autobiography in the specified style and print it in PDF format.

[1588] 8. Receive user correction requests, revise and save the final version.

[1589] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1590] Step 1: Enter your user information

[1591] A user logs in to a device and enters basic information such as date of birth and major life events. The device encrypts the information entered and sends it to the server. At this time, the user's basic information is input and the encrypted data is sent to the server as output. The server verifies the login information and stores the user profile in a database. Past photos and important events provided by the user are also stored in the database.

[1592] Step 2: Posting initial questions and receiving answers

[1593] The device presents the user with an initial question. For example, "Tell me about your childhood," which is displayed as voice or text. The user speaks into the device and recounts their childhood memories. The user's voice is the input, and is converted into text data as output, which is sent to the server. The device then converts this voice into text data and sends it to the server.

[1594] Step 3: Data analysis and emotion recognition

[1595] The server analyzes the received text data and extracts important keywords. Specifically, it uses natural language processing technology. At this time, the input is text data and the output is the extracted keywords. Furthermore, an emotion engine analyzes the user's voice data and recognizes their emotional state. The input is the user's voice data and the output is their emotional state.

[1596] Step 4: Question generation and next question presentation

[1597] The server automatically generates the next question based on the extracted keywords and the recognized emotional state. For example, it generates a question such as, "What was the most memorable and enjoyable event at that time?" The input is the keywords and the emotional state, and the output is the generated question. The device presents the next question sent from the server to the user by voice or text.

[1598] Step 5: Collecting user responses and converting them to text

[1599] The user speaks further details to the terminal in response to the presented question. The input is the user's voice and the output is text data. The terminal converts the user's voice back into text data and sends it to the server.

[1600] Step 6: Editing and creating your autobiography

[1601] The server creates the autobiography chapter by chapter based on the collected data and emotion recognition results. The input is the collected data and emotion recognition results, and the output is the autobiography text organized by chapter. It then formats the autobiography based on the user's selected style (documentary, romantic comedy, etc.) and generates the final version. The input is the user's style selection, and the output is the formatted autobiography.

[1602] Step 7: Format and export your autobiography

[1603] The server generates a formatted autobiography in PDF format and sends it to the terminal. The input is the formatted autobiography text and the output is the autobiography in PDF format. The terminal displays a preview version and presents it to the user for confirmation.

[1604] Step 8: User review and correction

[1605] The user reviews the autobiography and submits correction requests as needed via their terminal. The input is the correction request, and the output is the correction. The server saves the final version that reflects the corrections and outputs it in a printable or downloadable format.

[1606] (Application example 2)

[1607] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1608] When elderly people recount past events and generate autobiographical texts, the texts may lack emotional depth or the questions asked may be inconsistent. This problem can be solved by recognizing specific emotions and generating questions and answers based on those emotions. Furthermore, the generated autobiographies must be provided in a digital format to promote a consistent user experience.

[1609] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes an emotion recognition means that analyzes the user's emotions and reflects them in the questions and sentence generation, an output means that outputs the generated autobiographical-style sentences in a digital format and presents them visually and audibly, and an emotion-reflecting question generation means that automatically generates the next question based on the emotion analysis results. This makes it possible to generate an autobiography with greater emotional depth according to the content spoken by the elderly person, enabling a user experience that is visually and audibly consistent.

[1610] "Input methods" refer to the means by which older people recount past events, and include various forms such as voice input and text input.

[1611] "Analysis methods" refer to the means used to analyze what is said and extract important keywords. Natural language processing technology is used, for example.

[1612] The "question generation means" refers to a means for presenting the next question to the elderly person based on the keywords extracted by the analysis means.

[1613] "Database management means" refers to a database system for registering and managing profile information and past photographs of elderly people.

[1614] "Text generation means" refers to a means for automatically generating autobiographical text based on received information.

[1615] "Emotion recognition means" refers to a means for analyzing a user's emotions and reflecting them in questions and sentence generation. This includes technology for reading emotions from voice and text.

[1616] "Output means" refers to a means for outputting the generated autobiographical text in a digital format and presenting it visually and audibly.

[1617] The "format conversion means" refers to a means for converting the generated autobiography into a specific style and outputting it.

[1618] The "means for generating emotion-reflecting questions" refers to a means for automatically generating the next question based on the emotion analysis results.

[1619] "Edit management means" refers to a means for providing an interface that allows the generated autobiographical writing to be displayed in a digital format and for commentary and edits to be made.

[1620] The present invention relates to a system that generates an autobiography by allowing elderly people to talk about past events and combining it with an emotion engine. The system includes the following elements:

[1621] Components

[1622] 1. Input means: Includes voice input devices and text input devices for the elderly to talk about their past events.

[1623] 2. Analysis method: Analyze what is said and extract important keywords and phrases. This is done using natural language processing technology.

[1624] 3. Question generation method: The next question is automatically generated based on the extracted keywords and the results of sentiment analysis and presented to the elderly.

[1625] 4. Database management means: Includes a database that stores and manages profile information and past photos of seniors.

[1626] 5. Text generation method: This is a technology that automatically generates autobiographical-style text based on collected data.

[1627] 6. Emotion recognition: Analyzes emotions based on the user's voice and text and reflects them in questions and sentence generation.

[1628] 7. Output means: The generated autobiography is output in a digital format, including a display and a speaker for visually and audibly presenting it.

[1629] 8. Format conversion method: The generated autobiography is converted into a specific style (documentary, romantic comedy, etc.) and output.

[1630] 9. Emotion-reflecting question generation method: Automatically generate the next question based on the emotion analysis results.

[1631] 10. Editing management tool: The generated autobiographical text is displayed in a digital format, and an interface is provided that allows users to comment and edit.

[1632] Program processing

[1633] Hardware and Software Used

[1634] Hardware: smartphones, tablets, microphones, displays, head-mounted displays (HMDs)

[1635] software:

[1636] Python library (speech_recognition): Used to convert speech to text.

[1637] Transformers Library: Sentiment analysis is performed using the natural language processing model provided by Hugging Face.

[1638] TextBlob: Used for keyword extraction and sentiment analysis of text data.

[1639] Process Description

[1640] The server receives data from a terminal that inputs the elderly person's speech. The speech is converted into text data using a speech recognition means (e.g., Python's speech_recognition library). The converted text is analyzed by an analysis means to extract important keywords. In addition, the user's emotions are analyzed using an emotion recognition means (e.g., Transformers library), and an emotion-reflecting question generation means generates an appropriate next question based on the analysis results. The question is presented to the user via the terminal.

[1641] By repeating this process, the collected data is written as an autobiographical text using the text generation means. The generated text is converted into a specific style using the format conversion means based on the style selected by the user, and is displayed visually and audibly through the output means. The generated autobiographical text is also displayed in a digital format using the revision management means, allowing for commentary and revision.

[1642] Specific examples

[1643] Example 1: Emotion recognition of childhood memories

[1644] User: "I spent my childhood in Hokkaido, a place rich in nature. I enjoyed every day."

[1645] Speech recognition result: "I spent my childhood in Hokkaido, an area rich in nature. Every day was fun."

[1646] Sentiment analysis result: Positive

[1647] Generated question: "What did you enjoy most about that time?"

[1648] Example prompts for generative AI models

[1649] Analyze the text below and identify the sentiment.

[1650] Text: "I spent my childhood in Hokkaido, a place rich in nature. I enjoyed every day."

[1651] Generate the following questions based on the emotions and keywords below.

[1652] Emotion: Positive

[1653] Keywords: ["Hokkaido, rich in nature," fun]

[1654] In this way, the system generates autobiographies with emotional depth based on what the elderly person narrates, providing a consistent user experience.

[1655] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1656] Step 1:

[1657] The user logs in to the device and enters basic information. The device encrypts the information and sends it to the server. The server verifies the login information and saves the user profile in a database. It also receives past photos and important events from the user and saves them in the database.

[1658] Step 2:

[1659] The user speaks into the device's voice input device to recount past events. The device uses a speech recognition method (e.g., Python's speech_recognition library) to convert the speech into text data and sends the text data to the server. Here, the input is the user's voice data, and the output is text data.

[1660] Step 3:

[1661] The server analyzes the received text data using its analysis means and extracts important keywords and phrases. Using natural language processing technology, it processes and calculates the data to extract the necessary information. The input for this step is the voice-recognized text data, and the output is the extracted keywords and phrases.

[1662] Step 4:

[1663] The server uses emotion recognition tools (e.g., the Transformers library) to analyze user emotions from text data. The text is input to an emotion analysis model, which outputs the user's emotional state (positive, negative, or neutral). The input here is the analyzed text data, and the output is the emotion analysis results.

[1664] Step 5:

[1665] The server generates the next question using the emotion-reflecting question generation means based on the emotion analysis results and extracted keywords. For example, if a positive emotion is detected, a question such as "What was the most enjoyable thing at that time?" is automatically generated. The input is the emotion analysis results and extracted keywords, and the output is the generated question.

[1666] Step 6:

[1667] The device presents the generated question to the user. The user then recites what they have said by voice, which is then converted back into text data and sent to the server. By repeating this cycle, more detailed information is collected. The input is the automatically generated question from the server, and the output is the user's voice data and its converted text data.

[1668] Step 7:

[1669] The server uses a text generation tool based on the collected data to automatically generate autobiographical text. It combines the stories told by the elderly with the results of emotion analysis to create text in a natural format. The input is the data told by the elderly and the results of emotion analysis, and the output is an autobiographical text.

[1670] Step 8:

[1671] The server converts the generated autobiographical text into a specific style selected by the user using a format conversion means. For example, it converts into a style that matches the user's preferences, such as a documentary style or a romantic comedy style. The input is the generated autobiographical text, and the output is the format-converted text.

[1672] Step 9:

[1673] The server provides the final generated autobiography to the user in digital format through the output means. The terminal has a display for visual confirmation, allowing the user to check the content. The input is the formatted text, and the output is the autobiography presented in digital format.

[1674] Step 10:

[1675] The user checks the generated autobiographical text and, if any corrections are necessary, sends a correction request using the correction management means. The server receives the correction request, regenerates the final version with the relevant parts corrected, and presents it to the user again. The input is the user's correction request, and the output is the corrected autobiographical text.

[1676] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1677] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1678] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1679] [Fourth embodiment]

[1680] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1681] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1682] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1683] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1684] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1685] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1686] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1687] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1688] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1689] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1690] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1691] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1692] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1693] The present invention is a system that generates autobiographical text based on the recounting of past events by elderly people. This system involves the process of inputting and analyzing the content spoken by the elderly person, generating questions to be presented next, and finally compiling the autobiography.

[1694] System configuration

[1695] user

[1696] Elderly people: They are the main users of the system and talk about their own past experiences.

[1697] Family and care staff: Provide support to the older person as needed.

[1698] Terminal

[1699] Input devices: Devices that allow the older adult to input speech, including microphones and touchscreens.

[1700] Output device: A device used to display questions and generated text for confirmation. This can include a display or speaker.

[1701] server

[1702] Database: Stores and manages elderly profile information, conversations, uploaded photos and important events.

[1703] Analysis engine: Analyzes user voice data and extracts important keywords and phrases using natural language processing technology.

[1704] Question generation engine: Automatically generates the next question based on the extracted keywords.

[1705] Text generation engine: Automatically generates autobiographical text based on collected data.

[1706] Format conversion engine: Converts the generated autobiography into a specific style (documentary, romantic comedy, etc.) and outputs it.

[1707] Program processing

[1708] Initial setup and receiving user information

[1709] User: Logs in to a device and enters basic information such as date of birth and major life events.

[1710] Terminal: The entered information is encrypted and sent to the server.

[1711] Server: Validates login information and stores user profiles in a database. Also receives past photos and important events from users and stores them in a database.

[1712] Starting a conversation

[1713] Device: Present the user with an initial question, for example, "Tell me about your childhood."

[1714] User: Speaks into the device and recounts memories of the elderly person's childhood.

[1715] Device: Converts the user's voice into text data and sends it to the server.

[1716] Data analysis and question generation

[1717] Server: Analyzes the text data and extracts important keywords.

[1718] Server: Generates the next question based on the extracted keywords and sends it to the device. For example, it might ask, "What was your most memorable camping experience?"

[1719] Editing and creating autobiographies

[1720] User: Speaks further details into the device in response to the questions presented.

[1721] Device: Converts the user's speech into text and sends it back to the server.

[1722] Server: Based on the collected data, compiles autobiographical text and adds it to the autobiographical chapters.

[1723] Server: Formats the autobiography based on the style selected by the user and generates the final version.

[1724] Output and Save

[1725] Server: Generates the completed autobiography in PDF format and sends it to the device.

[1726] Device: Shows a preview version and presents it to the user for review.

[1727] User: Review the autobiography and submit a request to make corrections if necessary.

[1728] Server: Save the final version with the changes and make it available for printing or downloading.

[1729] Specific examples

[1730] Example 1: Recounting childhood memories

[1731] User: "I spent my childhood in Hokkaido, a place rich in nature."

[1732] Device: Converts this speech into text and sends it to the server.

[1733] Server: Extracts the keyword "Hokkaido, rich in nature," generates the question "What was the most memorable event at that time?" and sends it to the device.

[1734] User: "My favorite memories are going camping with my family and fishing on a big lake."

[1735] Terminal: Converts the voice into text data and sends it back to the server.

[1736] Server: Add this episode to the "Childhood Memories" chapter and generate it as text.

[1737] Example 2: Memories from school days

[1738] User: "I spent my student days in Tokyo. I had lots of adventures with my friends."

[1739] Device: Converts this speech into text and sends it to the server.

[1740] Server: Extracts the keywords "student days," "friends," and "adventure," generates a question such as "What was the most fun adventure you had at that time?" and sends it to the device.

[1741] User: "The most fun I had was climbing Mount Fuji with my friends."

[1742] Device: Converts the voice into text data and sends it back to the server.

[1743] Server: Add this episode to the chapter "Memories of Student Life" and generate it as a sentence.

[1744] In this way, a system in which AI generates autobiographical text based on past events recounted by elderly people will be useful in preventing dementia among the elderly and promoting communication with their families.

[1745] The processing flow will be explained below.

[1746] Specific processing steps of the program

[1747] Step 1: Initial setup and receiving user information

[1748] 1.1 User: Log in to the device and launch the AI ​​autobiography creation system.

[1749] Specific actions: Enter your username and password on the login screen displayed on the device and click the "Login" button.

[1750] 1.2 Device: Sends login information to server.

[1751] Specific operation: The entered login information is encrypted and sent to the server.

[1752] 1.3 Server: Verify login information and retrieve user profile data.

[1753] What it does: Retrieves user profile information (e.g., name, date of birth, major events) from a database.

[1754] 1.4 User: Register past photos and important events (topics and turning points) via the device.

[1755] Specific actions: Click the "Upload Photo" or "Add Event" button, select a file, or enter text.

[1756] 1.5 Terminal: Sends uploaded photos and entered event information to the server.

[1757] Specific operation: Transfers the selected photo files and text data to the server.

[1758] 1.6 Server: Stores photos and event information and organizes it as structured data.

[1759] What it does: Store the photos in an image library and add the event information to the associated time series data.

[1760] Step 2: Initiating a conversation

[1761] 2.1 Terminal: Present the initial question to the user.

[1762] What it does: Uses speech synthesis technology to play the question "Tell me about your childhood."

[1763] 2.2 User: Speaks into the device and recounts childhood memories.

[1764] Specific action: "When I was a child, I often went camping with my family in Hokkaido, which is rich in nature."

[1765] 2.3 Terminal: Converts the user's speech into text data.

[1766] What it does: It uses speech recognition software to convert speech into text and send it to a server.

[1767] Step 3: Analyze the data and generate questions

[1768] 3.1 Server: Analyzes the text data and extracts important keywords and phrases.

[1769] Specific operation: Extract keywords such as "Hokkaido, rich in nature" and "camping" using a natural language processing algorithm.

[1770] 3.2 Server: Generates a new question and sends it to the device.

[1771] Specific operation: Generate questions related to "nature" and "camping" and send a text message to the device saying, "What is your most memorable camping experience?"

[1772] 3.3 Terminal: Present the new question to the user.

[1773] What it does: Converts the question text into speech and plays it back to the user.

[1774] Step 4: Editing and creating your autobiography

[1775] 4.1 User: Speaks further details into the terminal in response to the questions presented.

[1776] Specific action: "One summer day, my father, brothers and I went fishing on a big lake and caught a big salmon, and that's a memory I'll never forget."

[1777] 4.2 Terminal: The user's voice is converted back into text data and sent to the server.

[1778] What it does: It uses speech recognition technology to convert speech into text and send it to a server for analysis.

[1779] 4.3 Server: Write your autobiography chapter by chapter based on the data collected.

[1780] What it does: Use natural language generation techniques to write a chapter on "Childhood Memories" and store it in a database.

[1781] 4.4 Server: Formats the autobiography in the style selected by the user.

[1782] Specific operation: Generates text that reflects styles such as "documentary style" or "romantic comedy style."

[1783] Step 5: Export and save

[1784] 5.1 Server: Sends the completed autobiography to the user's device for confirmation.

[1785] Specific behavior: Generates a preview version of the autobiography in PDF format and sends it to the device.

[1786] 5.2 Device: Show preview and ask user for confirmation.

[1787] What it does: Displays a preview of the autobiography in a PDF viewer and provides a "Confirm" button.

[1788] 5.3 User: Review the autobiography and request corrections if necessary.

[1789] Specific actions: Click the "Confirm" button and enter any necessary corrections.

[1790] 5.4 Server: Stores the finalized autobiography and outputs it in a printable or downloadable format.

[1791] Specific behavior: Saves the final version to the database and generates PDF and print files.

[1792] In this way, each step of the system that uses AI to automatically generate an autobiography based on the memories of the elderly person is achieved through collaboration between the user, device, and server.

[1793] Example 1

[1794] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1795] Previous systems that generate autobiographical documents from elderly people's recounting of past events were insufficient in abstracting what was said and in presenting the next question, making it difficult to create a natural dialogue. Furthermore, the style and format of the generated autobiography were limited, making it difficult to flexibly respond to user needs. Furthermore, there was a lack of technology to accurately convert voice input into text, resulting in a decline in the quality of the generated documents.

[1796] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1797] In this invention, the server includes an analysis means for analyzing what has been said and extracting important keywords, a question generation means for generating the next question based on the extracted keywords and presenting it to the elderly person, a database management means for registering and managing the elderly person's profile information and past images, a document generation means for automatically generating an autobiography-style document based on the received information, and a format conversion means for selecting and outputting the autobiography style. This enables a natural flow of dialogue and enables the flexible generation of autobiographies that meet the user's needs.

[1798] The "input means" is a device that allows elderly people to input voice or text recounting past events.

[1799] "Analysis means" refers to a device or program that has the function of analyzing what is said and extracting important keywords and phrases.

[1800] The "question generation means" is a device or program that automatically generates the next question based on the extracted keywords and presents it to the elderly person.

[1801] The "database management means" is a system for registering and managing elderly people's profile information, past images, and data on important events.

[1802] The "document generation means" is a device or program that automatically generates an autobiographical document based on the received information.

[1803] The "format conversion means" is a device or program that selects the style of the generated autobiography, converts it into a specified format, and outputs it.

[1804] The "voice recognition means" is a device or program that inputs the elderly person's speech and converts it into text data.

[1805] The "revision management means" is a device or program for presenting the generated autobiographical writing to the user in a revisable format and for reflecting requests for revisions.

[1806] "Natural language processing technology" is a technology for analyzing, understanding, and generating human language.

[1807] "Elderly people" refers to the subjects who use the system to narrate past events.

[1808] This invention is an information processing system that generates an autobiographical document based on an elderly person's recounting of past events. The system uses speech recognition and natural language processing technologies to convert the elderly person's recounting into text, extract important keywords, and generate the next question. It also includes a function to select the style of the generated autobiography and output it as a document.

[1809] System configuration

[1810] Hardware

[1811] Device: A device equipped with a microphone for voice input and a touchscreen.

[1812] Server: High-performance computer for data analysis and document generation

[1813] Database: A storage system for managing profile information and past images of elderly people

[1814] Display: A screen for displaying questions and generated documentation.

[1815] Speaker: An audio device for presenting questions aloud.

[1816] software

[1817] Speech recognition software: A system that inputs elderly people's speech and converts it into text data

[1818] Natural language processing software: Software that extracts important keywords from text data

[1819] Question generation engine: A program that automatically generates the next question based on the extracted keywords.

[1820] Document generation engine: Software that automatically generates autobiographical documents based on collected data

[1821] Format conversion engine: Software that converts the generated autobiography into a specific style and outputs it.

[1822] Revision management system: Software that presents the generated autobiographical text to the user in a revisable format and reflects revisions.

[1823] System Operation

[1824] Registering user information

[1825] Users log in to their devices and enter basic information about their date of birth and major life events. This information is encrypted on the device and sent to the server, which then stores it in a database as part of the user's profile. The server also receives past images and important events from the user and stores them in the database.

[1826] Starting a conversation

[1827] The device presents the user with an initial question, for example, "Tell me about your childhood." The user speaks into the device, recounting the elderly person's childhood memories. The device converts the speech into text data and sends it to the server.

[1828] Data analysis

[1829] The server analyzes the text data and extracts important keywords. It then generates the next question based on the extracted keywords and sends it to the device. For example, it could ask a specific question such as, "What was your most memorable camping experience?"

[1830] Document Generation

[1831] The user responds to the questions with further details, which are then converted back into text and sent to the server. The server then uses the collected data to compile an autobiographical document, adding chapters to it, and formats the autobiography based on the user's chosen style to produce the final version.

[1832] Output and Modification

[1833] The completed autobiography is sent from the server to the device in PDF format. The device displays a preview version and presents it to the user for confirmation. The user reviews the autobiography and submits a request for any necessary corrections. The server saves the final version after the corrections are applied and outputs it in a format that can be printed or downloaded.

[1834] Examples and prompts

[1835] Example 1: Recounting childhood memories

[1836] When a user says, "I spent my childhood in Hokkaido, a place rich in nature," the device converts this speech into text and sends it to the server. The server extracts the keyword "Hokkaido, a place rich in nature," generates a question, "What was the most memorable event at that time?" and sends it to the device. The user then adds, "My best memory is going camping with my family and fishing on a large lake," and this is added to the autobiography as a "childhood memory."

[1837] Prompt Sentence Examples

[1838] "Tell me an anecdote from your childhood."

[1839] What is your most enjoyable memory from your school days?

[1840] "Tell me some memories you have with your family."

[1841] "Tell us about the most memorable event."

[1842] In this way, it is possible to realize natural and flowing dialogue based on the elderly person's narrative, and generate a high-quality autobiography.

[1843] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1844] Step 1: Initial setup and receiving user information

[1845] User: Log in to the device and enter the required user information (date of birth and data related to major life events).

[1846] Input: Basic information such as date of birth, major life events, etc.

[1847] Output: Initial setting data

[1848] Terminal: The entered information is encrypted and sent to the server.

[1849] Input: Data entered by the user

[1850] Data processing: Encryption processing

[1851] Output: Encrypted user information

[1852] Server: Receives user information, verifies login information, and stores user profiles in a database.

[1853] Input: Encrypted user information

[1854] Data processing: Save to database

[1855] Output: Saved user profile data

[1856] Step 2: Initiating a conversation

[1857] Device: Present the first question to the user aloud, for example, "Tell me about your childhood."

[1858] Input: Fixed question template

[1859] Data processing: Question voice synthesis

[1860] Output: Questions presented by voice

[1861] User: Speaks into device, recounting childhood memories.

[1862] Input: User's voice data

[1863] Output: Recorded audio data

[1864] Device: Converts the user's voice into text data and sends it to the server.

[1865] Input: User's voice data

[1866] Data processing: speech recognition and text conversion

[1867] Output: Text data

[1868] Step 3: Analyze the data and generate questions

[1869] Server: Analyzes the received text data and extracts important keywords using natural language processing technology.

[1870] Input: Text data

[1871] Data processing: Keyword extraction using natural language processing

[1872] Output: Extracted keywords

[1873] Server: Generates the next question based on the extracted keywords and sends it to the device.

[1874] Input: Extracted keywords

[1875] Data processing: Generate new questions using a question generation algorithm

[1876] Output: Generated questions

[1877] Step 4: Editing and creating your autobiography

[1878] User: Provides details in response to questions posed by the device.

[1879] Input: Question from terminal

[1880] Output: User's voice data

[1881] Device: Converts the user's voice into text data and sends it back to the server.

[1882] Input: User's voice data

[1883] Data processing: speech recognition and text conversion

[1884] Output: Text data

[1885] Server: Based on the collected data, compile an autobiographical document and add chapters to it.

[1886] Input: Text data and user profile

[1887] Data processing: text editing and chapter addition

[1888] Output: Updated autobiographical text

[1889] Server: Formats the autobiography based on the style selected by the user and generates the final version.

[1890] Input: Updated autobiographical text

[1891] Data processing: Applying a format conversion engine

[1892] Output: Final formatted autobiography

[1893] Step 5: Export and save

[1894] Server: Generates the completed autobiography in PDF format and sends it to the device.

[1895] Input: Final formatted autobiography

[1896] Data processing: PDF conversion

[1897] Output: Autobiography in PDF format

[1898] On the device: The generated PDF is displayed as a preview and presented to the user for review.

[1899] Input: Autobiography in PDF format

[1900] Output: Preview

[1901] User: Review the autobiography and submit a request for corrections if needed.

[1902] Input: Previewed autobiography

[1903] Output: Correction request

[1904] Server: Save the final version with the corrections and output it in a printable or downloadable format.

[1905] Input: Correction request and autobiography in PDF format

[1906] Data processing: Reflecting corrections and saving the final version

[1907] Output: Final autobiography

[1908] (Application example 1)

[1909] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1910] Elderly people's recounting of their past experiences is an important way to record and pass on personal memories to future generations. However, in today's busy living environment, family members and caregivers often do not have enough time to carefully listen to what the elderly have to say. Furthermore, there is a need to provide experiences related to past memories in order to prevent dementia and help the elderly feel a sense of spiritual fulfillment. The present invention aims to solve these problems by easily generating an autobiography based on the elderly's narrative and providing a customized meal menu based on that autobiography.

[1911] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1912] In this invention, the server includes input means for the elderly person to talk about their past, analysis means for analyzing what has been said and extracting important keywords, question generation means for generating the next question based on the extracted keywords and presenting it to the elderly person, database management means for registering and managing the elderly person's profile information and past photos, text generation means for automatically generating autobiographical text based on the received information, format conversion means for selecting and outputting the autobiographical style, and menu suggestion means for suggesting a customized meal menu based on the elderly person's story. This makes it possible to create an autobiography by having the elderly person talk about events from their past, and to provide a special dining experience based on that story.

[1913] The "elderly" are the subjects who use the system to recount past events and generate autobiographical texts.

[1914] The "input means" is a device used by elderly people to talk about past events, and is a device that allows voice input and text input.

[1915] The "analysis means" is a device or software that analyzes what the elderly say and extracts important keywords.

[1916] The "question generation means" is a device or software that automatically generates the next question based on the extracted keywords and presents it to the elderly person.

[1917] The "database management means" refers to a device or software that registers and manages profile information and past photographs of elderly people.

[1918] The "text generation means" is a device or software that automatically generates autobiographical text based on received information.

[1919] The "format conversion means" is a device or software that selects the style of the generated autobiography, converts it into a specific format, and outputs it.

[1920] The "menu suggestion means" is a device or software that suggests a customized meal menu based on the elderly person's narration.

[1921] This invention is a system that generates autobiographical text by having an elderly person recount events from their past, and proposes a customized meal menu based on that narrative. Specific embodiments of this system will be described in detail below.

[1922] user

[1923] senior citizen

[1924] The subject who uses the system talks about events from his or her past. The system analyzes what he or she says and ultimately compiles it into an autobiography.

[1925] Family members and caregivers

[1926] Provide support to the elderly as needed and help them use the system.

[1927] Terminal

[1928] Input Devices

[1929] This device allows elderly people to input their speech, and includes a microphone and a touchscreen.

[1930] Output Device

[1931] A device for displaying generated questions, autobiographical text, and suggested meal menus, including a display and speaker.

[1932] server

[1933] Database Management Methods

[1934] Stores and manages data such as elderly people's profile information, past photos, and spoken content.

[1935] Analysis means

[1936] Analyze user speech (audio data) and extract important keywords and phrases using voice recognition technology (e.g., Google speech-to-text API) and natural language processing technology (e.g., Hugging Face's Transformers library).

[1937] Question generation means

[1938] The next question is automatically generated based on the extracted keywords, using a generative AI model (e.g., GPT-2).

[1939] Sentence generation means

[1940] Based on the collected data, an autobiographical text is edited and generated, again using natural language generation technology (e.g., GPT-2).

[1941] Format conversion method

[1942] The generated autobiography is converted into a specific style (documentary, romantic comedy, etc.) and output.

[1943] Menu suggestion method

[1944] This system proposes customized meal menus based on the elderly's narratives. It extracts food-related information from keywords in the narratives and proposes related menus.

[1945] Program processing explanation

[1946] The server receives the voice input data of what the elderly person is saying and uses an analysis means to extract important keywords and phrases. Based on the analyzed data, a question generation means generates the next question and sends it to the terminal to present to the user. The user's voice response is sent back to the server, and by repeating the same process, an autobiographical-style text is gradually generated.

[1947] Furthermore, food-related keywords are extracted from the analyzed content, and the menu suggestion means suggests meal menus that are memorable to the elderly.

[1948] For example, if an elderly person says, "When I was a child, I loved the miso soup my grandmother made for me," the analysis method will extract the keywords "childhood," "grandmother," and "miso soup," and generate the next question, "What other dish did you like most at that time?" Based on the answer to that question, the system will ask the next question and eventually suggest related menu items such as "miso soup set" or "grandmother's special miso soup."

[1949] Prompt Sentence Examples

[1950] "Logged in user: {username} said the following: 'When I was a child, I loved the miso soup my grandmother made for me.' Please generate follow-up questions."

[1951] The above is a specific embodiment for carrying out the present invention. This system allows elderly people to create their autobiographies by talking about their past, and furthermore, it makes it possible to provide special dining experiences based on past memories.

[1952] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1953] Step 1:

[1954] A user logs into the system and enters their profile information (date of birth, major life events, etc.) The entered information is sent to the server in the form of data, which is then stored in the database.

[1955] Step 2:

[1956] The device generates an initial question and presents it to the user, for example, "Tell me about your childhood," either by voice or text. The user begins to recount events from their past.

[1957] Step 3:

[1958] The user speaks their answer through a voice input device, and this voice data is input into the terminal.

[1959] Step 4:

[1960] The device converts the voice data into text data using voice recognition technology (e.g., Google speech-to-text API), and the converted text data is sent to the server.

[1961] Step 5:

[1962] The server receives the text data and uses analytical tools to extract important keywords and phrases. Natural language processing techniques (e.g., Hugging Face's Transformers library) are used. The results of this analysis are used to generate the next question.

[1963] Step 6:

[1964] The server generates the next question based on the extracted keywords. It uses a generative AI model (e.g., GPT-2) to create an appropriate next question. For example, a question like, "What was the most memorable event at that time?" is generated. The generated question is sent to the device.

[1965] Step 7:

[1966] The device presents the generated question to the user, who answers the new question and then speaks again.

[1967] Step 8:

[1968] The server receives the voice data again and repeats the same process. Finally, when enough information has been collected, an autobiographical text is automatically generated by the text generation means.

[1969] Step 9:

[1970] The server uses a text generation tool to edit and generate the collected data along an autobiography-like flow. The generated autobiography is formatted based on the style selected by the user (e.g., documentary style, romantic comedy style).

[1971] Step 10:

[1972] The server sends the formatted autobiography to the terminal and displays it as a preview to the user. The user can submit correction requests as needed. If no corrections are required, the final version is generated.

[1973] Step 11:

[1974] The server then saves the final autobiography in a format such as PDF, and outputs it in a printable or downloadable format. At the same time, a menu suggestion tool is used to suggest a customized meal menu based on the elderly person's story. This menu suggestion is also displayed on the device.

[1975] The above is the flow of specific processing steps of the system that realizes the application example.

[1976] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1977] This invention combines an emotion engine with a system that generates autobiographical text based on the elderly's recounting of past events. The system inputs what the elderly person is saying, analyzes it, generates questions to pose next, and ultimately compiles it into an autobiography with emotional depth. The system also recognizes the user's emotions and asks appropriate questions and responds based on the results.

[1978] System configuration

[1979] user

[1980] Elderly people: They are the main users of the system and talk about their own past experiences.

[1981] Family and care staff: Provide support to the older person as needed.

[1982] Terminal

[1983] Input devices: Devices that allow the older adult to input speech, including microphones and touchscreens.

[1984] Output device: A device used to display questions and generated text for confirmation. This can include a display or speaker.

[1985] server

[1986] Database: Stores and manages elderly people's profile information, spoken content, uploaded photos, important events, and emotion recognition results.

[1987] Analysis engine: Analyzes user voice data and extracts important keywords and phrases using natural language processing technology.

[1988] Question generation engine: Automatically generates the next question based on the extracted keywords.

[1989] Emotion engine: Recognizes emotions from the user's voice and facial expressions and tailors questions and responses accordingly, adding depth to sentences depending on the user's emotional state.

[1990] Text generation engine: Automatically generates autobiographical text based on collected data.

[1991] Format conversion engine: Converts the generated autobiography into a specific style (documentary, romantic comedy, etc.) and outputs it.

[1992] Program processing

[1993] Initial setup and receiving user information

[1994] User: Log in to the device and enter basic information (date of birth, major life events, etc.).

[1995] Terminal: The entered information is encrypted and sent to the server.

[1996] Server: Validates login information and stores user profiles in a database. Also receives past photos and important events from users and stores them in a database.

[1997] Starting a conversation

[1998] Device: Present the user with an initial question, for example, "Tell me about your childhood."

[1999] User: Speaks into device, recounting childhood memories.

[2000] Device: Converts the user's voice into text data and sends it to the server.

[2001] Data analysis and question generation

[2002] Server: Analyzes the text data and extracts important keywords.

[2003] Server: The emotion engine recognizes the user's emotional state and reflects it when generating the next question. For example, if the user is telling a happy story, it generates the question, "What was the most memorable happy event at that time?"

[2004] Editing and creating autobiographies

[2005] User: Speaks further details into the device in response to the questions presented.

[2006] Device: The user's voice is converted back into text data and sent to the server.

[2007] Server: Writes an autobiography chapter by chapter based on collected data and emotion recognition results.

[2008] Server: Formats the autobiography based on the style selected by the user and generates the final version.

[2009] Output and Save

[2010] Server: Generates the completed autobiography in PDF format and sends it to the device.

[2011] Device: Shows a preview version and presents it to the user for review.

[2012] User: Review the autobiography and submit a request to make corrections if necessary.

[2013] Server: Save the final version with the changes and make it available for printing or downloading.

[2014] Specific examples

[2015] Example 1: Emotion recognition of childhood memories

[2016] User: "I spent my childhood in Hokkaido, a place rich in nature. I enjoyed every day."

[2017] Device: Converts this speech into text and sends it to the server.

[2018] Server: Extracts keywords such as "Hokkaido, rich in nature" and "fun," and recognizes the user's enjoyment from the tone of their voice. Generates a question such as "What was the most memorable and enjoyable event at that time?" and sends it to the device.

[2019] User: "My favorite memories are going camping with my family and fishing on a big lake."

[2020] Terminal: Converts the voice into text data and sends it back to the server.

[2021] Server: Add this episode to the "Childhood Memories" chapter and generate it as a sentence containing emotions.

[2022] Example 2: Emotion recognition of memories from school days

[2023] User: "I spent my student days in Tokyo. I had lots of adventures with my friends."

[2024] Device: Converts this speech into text and sends it to the server.

[2025] Server: Extracts keywords such as "student days" and "adventure" and recognizes excited emotions from the user's voice. Generates a question such as "What was the most fun adventure you had at that time?" and sends it to the device.

[2026] User: "The most fun I had was climbing Mount Fuji with my friends."

[2027] Device: Converts the voice into text data and sends it back to the server.

[2028] Server: Add this episode to the chapter "Memories of Student Life" and generate a sentence containing emotions.

[2029] In this way, a system in which AI recognizes emotions based on past events recounted by elderly people and automatically generates an autobiography accordingly will be even more effective in preventing dementia among the elderly and promoting communication with their families.

[2030] The processing flow will be explained below.

[2031] Specific processing steps of the program

[2032] Step 1: Initial setup and receiving user information

[2033] 1.1 User: Log in to the device and launch the AI ​​autobiography creation system.

[2034] Specific actions: Enter your username and password on the login screen and click the "Login" button.

[2035] 1.2 Device: Sends login information to server.

[2036] Specific behavior: Encrypts the entered login information and sends it to the server.

[2037] 1.3 Server: Verify login information and retrieve user profile data.

[2038] What it does: Retrieves user profile information (e.g., name, date of birth, major events) from a database.

[2039] 1.4 User: Register past photos and important events (topics and turning points) via the device.

[2040] Specific actions: Click the "Upload Photo" or "Add Event" button to select a file or enter text.

[2041] 1.5 Terminal: Sends uploaded photos and entered event information to the server.

[2042] Specific operation: Transfers the selected photo files and text data to the server.

[2043] 1.6 Server: Stores photos and event information and organizes it as structured data.

[2044] What it does: Store the photos in an image library and add the event information to the associated time series data.

[2045] Step 2: Initiating a conversation

[2046] 2.1 Terminal: Present the initial question to the user.

[2047] What it does: Uses speech synthesis technology to play the question "Tell me about your childhood."

[2048] 2.2 User: Speaks into the device and recounts childhood memories.

[2049] Specific action: "When I was a child, I often went camping with my family in Hokkaido, which is rich in nature."

[2050] 2.3 Terminal: Converts the user's speech into text data.

[2051] What it does: It uses speech recognition software to convert speech into text and send it to a server.

[2052] Step 3: Analyze the data and generate questions

[2053] 3.1 Server: Analyzes the text data and extracts important keywords and phrases.

[2054] Specific operation: Extract keywords such as "Hokkaido, rich in nature" and "camping" using a natural language processing algorithm.

[2055] 3.2 Server: The emotion engine recognizes the user's emotional state and reflects it when generating the next question.

[2056] Specific operation: Analyzes the user's emotions from their tone of voice and facial expressions, generates a question such as "What was the most memorable and enjoyable event at that time?" and sends it to the device.

[2057] 3.3 Terminal: Present the new question to the user.

[2058] What it does: Converts the question text into speech and plays it back to the user.

[2059] Step 4: Editing and creating your autobiography

[2060] 4.1 User: Speaks further details into the terminal in response to the questions presented.

[2061] Specific action: "One summer day, my father, brothers and I went fishing on a big lake and caught a big salmon, and that's a memory I'll never forget."

[2062] 4.2 Terminal: The user's voice is converted back into text data and sent to the server.

[2063] What it does: It uses speech recognition technology to convert speech into text and send it to a server for analysis.

[2064] 4.3 Server: Writes an autobiography chapter by chapter based on the collected data and emotion recognition results.

[2065] What it does: Use natural language generation technology to write a chapter on "Childhood Memories," edit it into emotionally rich text, and store it in a database.

[2066] 4.4 Server: Formats the autobiography in the style selected by the user.

[2067] Specific operation: Generates text that reflects styles such as "documentary style" or "romantic comedy style."

[2068] Step 5: Export and save

[2069] 5.1 Server: Sends the completed autobiography to the user's device for confirmation.

[2070] Specific behavior: Generates a preview version of the autobiography in PDF format and sends it to the device.

[2071] 5.2 Device: Show preview and ask user for confirmation.

[2072] What it does: Displays a preview of the autobiography in a PDF viewer and provides a "Confirm" button.

[2073] 5.3 User: Review the autobiography and request corrections if necessary.

[2074] Specific actions: Click the "Confirm" button and enter any necessary corrections.

[2075] 5.4 Server: Stores the finalized autobiography and outputs it in a printable or downloadable format.

[2076] Specific behavior: Saves the final version to the database and generates PDF and print files.

[2077] In this way, each step of the system, which uses AI to recognize emotions and automatically generate an autobiography based on memories from the past recounted by an elderly person, is achieved through collaboration between the user, device, and server.

[2078] Example 2

[2079] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2080] When elderly people recount past events, organizing them into autobiographical text takes time and effort. Furthermore, generating high-quality text that incorporates emotion requires generating appropriate questions. Furthermore, correcting and formatting the generated text is also time-consuming. Thus, there is a need for an efficient system that can automatically organize and write down past events with emotional depth.

[2081] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[2082] In this invention, the server includes emotion recognition means, analysis means, and question generation means, which enable the server to recognize emotions from the elderly person's narrative, analyze the content, and generate appropriate questions, thereby efficiently generating high-quality autobiographies with emotional depth.

[2083] "Input means" refers to devices used by older adults to recount past events, including voice input and touchscreens.

[2084] The "analysis means" is a technology for analyzing what the elderly say and extracting important keywords, and is a device that uses natural language processing technology.

[2085] The "question generation means" is a technology for automatically generating the next question based on the extracted keywords and presenting it to the elderly person.

[2086] "Database management means" refers to technology for registering and managing elderly people's profile information and past photographs.

[2087] "Emotion recognition means" is a technology that recognizes the user's emotions from their voice and facial expressions and asks appropriate questions and responds accordingly.

[2088] The "text generation means" is a technology for automatically generating autobiographical text based on received information.

[2089] The "format conversion means" is a technique for selecting the style of the generated autobiography and outputting it in a specified format.

[2090] "Voice recognition means" is a technology that inputs the elderly person's speech and converts it into text data.

[2091] The "revision management means" is a technique for presenting the generated autobiographical text to the user in a revisable format and for reflecting the user's revisions.

[2092] This invention relates to a system that generates autobiographical text based on the elderly's recounting of past events. This system involves a process of inputting the elderly's speech, analyzing it, generating questions to be presented next, and finally compiling it into an autobiography with emotional depth. The system can also recognize the user's emotions and provide appropriate questions and responses based on the results.

[2093] System configuration

[2094] The system mainly consists of the following elements:

[2095] user

[2096] Elderly people: They are the main users of the system and talk about their own past experiences.

[2097] Family and care staff: Provide support to the older person as needed.

[2098] Terminal

[2099] Input devices: Devices that allow the older adult to input speech, including microphones and touchscreens.

[2100] Output device: A device that displays questions and generated text for confirmation. This includes displays and speakers.

[2101] server

[2102] Database: Stores and manages elderly people's profile information, spoken content, uploaded photos, important events, and emotion recognition results.

[2103] Analysis engine: Analyzes user voice data and extracts important keywords and phrases using natural language processing technology.

[2104] Question generation engine: Automatically generates the next question based on the extracted keywords.

[2105] Emotion engine: Recognizes emotions from the user's voice and facial expressions and tailors questions and responses accordingly, adding depth to sentences depending on the user's emotional state.

[2106] Text generation engine: Automatically generates autobiographical text based on collected data.

[2107] Format conversion engine: Converts the generated autobiography into a specific style (e.g., documentary or romantic comedy) and outputs it.

[2108] System Operation

[2109] Hardware

[2110] Microphone: An input device for receiving a user's voice input.

[2111] Display: An output device used to display questions and generated text.

[2112] Server: A computer on which various engines (analysis engine, question generation engine, emotion engine, sentence generation engine, format conversion engine) run.

[2113] software

[2114] Natural Language Processing: A technique used by analytics engines to extract important keywords and phrases from what users say.

[2115] Emotion Recognition Algorithm: This is the technology used in the emotion engine to recognize emotions from the user's voice and facial expressions.

[2116] AI model: A generative AI model used for question generation and sentence generation.

[2117] Specific examples

[2118] Example 1: Emotion recognition of childhood memories

[2119] User: "I spent my childhood in Hokkaido, a place rich in nature. I enjoyed every day."

[2120] Device: Converts this speech into text and sends it to the server.

[2121] Server: Extracts keywords such as "Hokkaido, rich in nature" and "fun," and recognizes the user's enjoyment from the tone of their voice. Generates a question such as "What was the most memorable and enjoyable event at that time?" and sends it to the device.

[2122] User: "My favorite memories are going camping with my family and fishing on a big lake."

[2123] Terminal: Converts the voice into text data and sends it back to the server.

[2124] Server: Add this episode to the "Childhood Memories" chapter and generate it as a sentence containing emotions.

[2125] Example 2: Emotion recognition of memories from school days

[2126] User: "I spent my student days in Tokyo. I had lots of adventures with my friends."

[2127] Device: Converts this speech into text and sends it to the server.

[2128] Server: Extracts keywords such as "student days" and "adventure" and recognizes excited emotions from the user's voice. Generates a question such as "What was the most fun adventure you had at that time?" and sends it to the device.

[2129] User: "The most fun I had was climbing Mount Fuji with my friends."

[2130] Device: Converts the voice into text data and sends it back to the server.

[2131] Server: Add this episode to the chapter "Memories of Student Life" and generate a sentence containing emotions.

[2132] Prompt Sentence Examples

[2133] 1. A user logs in and enters basic information. Please encrypt this information before sending it.

[2134] 2. Input the user's voice data and convert it into text data.

[2135] 3. Extract specific keywords from text data and recognize the user's emotional state.

[2136] 4. Generate the next appropriate question based on the perceived emotional state and present it to the user.

[2137] 5. Collect the user's responses again and analyze the detailed data on the server.

[2138] 6. Create an autobiography chapter by chapter based on the data you have collected.

[2139] 7. Format your final autobiography in the specified style and print it in PDF format.

[2140] 8. Receive user correction requests, revise and save the final version.

[2141] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2142] Step 1: Enter your user information

[2143] A user logs in to a device and enters basic information such as date of birth and major life events. The device encrypts the information entered and sends it to the server. At this time, the user's basic information is input and the encrypted data is sent to the server as output. The server verifies the login information and stores the user profile in a database. Past photos and important events provided by the user are also stored in the database.

[2144] Step 2: Posting initial questions and receiving answers

[2145] The device presents the user with an initial question. For example, "Tell me about your childhood," which is displayed as voice or text. The user speaks into the device and recounts their childhood memories. The user's voice is the input, and is converted into text data as output, which is sent to the server. The device then converts this voice into text data and sends it to the server.

[2146] Step 3: Data analysis and emotion recognition

[2147] The server analyzes the received text data and extracts important keywords. Specifically, it uses natural language processing technology. At this time, the input is text data and the output is the extracted keywords. Furthermore, an emotion engine analyzes the user's voice data and recognizes their emotional state. The input is the user's voice data and the output is their emotional state.

[2148] Step 4: Question generation and next question presentation

[2149] The server automatically generates the next question based on the extracted keywords and the recognized emotional state. For example, it generates a question such as, "What was the most memorable and enjoyable event at that time?" The input is the keywords and the emotional state, and the output is the generated question. The device presents the next question sent from the server to the user by voice or text.

[2150] Step 5: Collecting user responses and converting them to text

[2151] The user speaks further details to the terminal in response to the presented question. The input is the user's voice and the output is text data. The terminal converts the user's voice back into text data and sends it to the server.

[2152] Step 6: Editing and creating your autobiography

[2153] The server creates the autobiography chapter by chapter based on the collected data and emotion recognition results. The input is the collected data and emotion recognition results, and the output is the autobiography text organized by chapter. It then formats the autobiography based on the user's selected style (documentary, romantic comedy, etc.) and generates the final version. The input is the user's style selection, and the output is the formatted autobiography.

[2154] Step 7: Format and export your autobiography

[2155] The server generates a formatted autobiography in PDF format and sends it to the terminal. The input is the formatted autobiography text and the output is the autobiography in PDF format. The terminal displays a preview version and presents it to the user for confirmation.

[2156] Step 8: User review and correction

[2157] The user reviews the autobiography and submits correction requests as needed via their terminal. The input is the correction request, and the output is the correction. The server saves the final version that reflects the corrections and outputs it in a printable or downloadable format.

[2158] (Application example 2)

[2159] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2160] When elderly people recount past events and generate autobiographical texts, the texts may lack emotional depth or the questions asked may be inconsistent. This problem can be solved by recognizing specific emotions and generating questions and answers based on those emotions. Furthermore, the generated autobiographies must be provided in a digital format to promote a consistent user experience.

[2161] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes an emotion recognition means that analyzes the user's emotions and reflects them in the questions and sentence generation, an output means that outputs the generated autobiographical-style sentences in a digital format and presents them visually and audibly, and an emotion-reflecting question generation means that automatically generates the next question based on the emotion analysis results. This makes it possible to generate an autobiography with greater emotional depth according to the content spoken by the elderly person, enabling a user experience that is visually and audibly consistent.

[2162] "Input methods" refer to the means by which older people recount past events, and include various forms such as voice input and text input.

[2163] "Analysis methods" refer to the means used to analyze what is said and extract important keywords. Natural language processing technology is used, for example.

[2164] The "question generation means" refers to a means for presenting the next question to the elderly person based on the keywords extracted by the analysis means.

[2165] "Database management means" refers to a database system for registering and managing profile information and past photographs of elderly people.

[2166] "Text generation means" refers to a means for automatically generating autobiographical text based on received information.

[2167] "Emotion recognition means" refers to a means for analyzing a user's emotions and reflecting them in questions and sentence generation. This includes technology for reading emotions from voice and text.

[2168] "Output means" refers to a means for outputting the generated autobiographical text in a digital format and presenting it visually and audibly.

[2169] The "format conversion means" refers to a means for converting the generated autobiography into a specific style and outputting it.

[2170] The "means for generating emotion-reflecting questions" refers to a means for automatically generating the next question based on the emotion analysis results.

[2171] "Edit management means" refers to a means for providing an interface that allows the generated autobiographical writing to be displayed in a digital format and for commentary and edits to be made.

[2172] The present invention relates to a system that generates an autobiography by allowing elderly people to talk about past events and combining it with an emotion engine. The system includes the following elements:

[2173] Components

[2174] 1. Input means: Includes voice input devices and text input devices for the elderly to talk about their past events.

[2175] 2. Analysis method: Analyze what is said and extract important keywords and phrases. This is done using natural language processing technology.

[2176] 3. Question generation method: The next question is automatically generated based on the extracted keywords and the results of sentiment analysis and presented to the elderly.

[2177] 4. Database management means: Includes a database that stores and manages profile information and past photos of seniors.

[2178] 5. Text generation method: This is a technology that automatically generates autobiographical-style text based on collected data.

[2179] 6. Emotion recognition: Analyzes emotions based on the user's voice and text and reflects them in questions and sentence generation.

[2180] 7. Output means: The generated autobiography is output in a digital format, including a display and a speaker for visually and audibly presenting it.

[2181] 8. Format conversion method: The generated autobiography is converted into a specific style (documentary, romantic comedy, etc.) and output.

[2182] 9. Emotion-reflecting question generation method: Automatically generate the next question based on the emotion analysis results.

[2183] 10. Editing management tool: The generated autobiographical text is displayed in a digital format, and an interface is provided that allows users to comment and edit.

[2184] Program processing

[2185] Hardware and Software Used

[2186] Hardware: smartphones, tablets, microphones, displays, head-mounted displays (HMDs)

[2187] software:

[2188] Python library (speech_recognition): Used to convert speech to text.

[2189] Transformers Library: Sentiment analysis is performed using the natural language processing model provided by Hugging Face.

[2190] TextBlob: Used for keyword extraction and sentiment analysis of text data.

[2191] Process Description

[2192] The server receives data from a terminal that inputs the elderly person's speech. The speech is converted into text data using a speech recognition means (e.g., Python's speech_recognition library). The converted text is analyzed by an analysis means to extract important keywords. In addition, the user's emotions are analyzed using an emotion recognition means (e.g., Transformers library), and an emotion-reflecting question generation means generates an appropriate next question based on the analysis results. The question is presented to the user via the terminal.

[2193] By repeating this process, the collected data is written as an autobiographical text using the text generation means. The generated text is converted into a specific style using the format conversion means based on the style selected by the user, and is displayed visually and audibly through the output means. The generated autobiographical text is also displayed in a digital format using the revision management means, allowing for commentary and revision.

[2194] Specific examples

[2195] Example 1: Emotion recognition of childhood memories

[2196] User: "I spent my childhood in Hokkaido, a place rich in nature. I enjoyed every day."

[2197] Speech recognition result: "I spent my childhood in Hokkaido, an area rich in nature. Every day was fun."

[2198] Sentiment analysis result: Positive

[2199] Generated question: "What did you enjoy most about that time?"

[2200] Example prompts for generative AI models

[2201] Analyze the text below and identify the sentiment.

[2202] Text: "I spent my childhood in Hokkaido, a place rich in nature. I enjoyed every day."

[2203] Generate the following questions based on the emotions and keywords below.

[2204] Emotion: Positive

[2205] Keywords: ["Hokkaido, rich in nature," fun]

[2206] In this way, the system generates autobiographies with emotional depth based on what the elderly person narrates, providing a consistent user experience.

[2207] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2208] Step 1:

[2209] The user logs in to the device and enters basic information. The device encrypts the information and sends it to the server. The server verifies the login information and saves the user profile in a database. It also receives past photos and important events from the user and saves them in the database.

[2210] Step 2:

[2211] The user speaks into the device's voice input device to recount past events. The device uses a speech recognition method (e.g., Python's speech_recognition library) to convert the speech into text data and sends the text data to the server. Here, the input is the user's voice data, and the output is text data.

[2212] Step 3:

[2213] The server analyzes the received text data using its analysis means and extracts important keywords and phrases. Using natural language processing technology, it processes and calculates the data to extract the necessary information. The input for this step is the voice-recognized text data, and the output is the extracted keywords and phrases.

[2214] Step 4:

[2215] The server uses emotion recognition tools (e.g., the Transformers library) to analyze user emotions from text data. The text is input to an emotion analysis model, which outputs the user's emotional state (positive, negative, or neutral). The input here is the analyzed text data, and the output is the emotion analysis results.

[2216] Step 5:

[2217] The server generates the next question using the emotion-reflecting question generation means based on the emotion analysis results and extracted keywords. For example, if a positive emotion is detected, a question such as "What was the most enjoyable thing at that time?" is automatically generated. The input is the emotion analysis results and extracted keywords, and the output is the generated question.

[2218] Step 6:

[2219] The device presents the generated question to the user. The user then recites what they have said by voice, which is then converted back into text data and sent to the server. By repeating this cycle, more detailed information is collected. The input is the automatically generated question from the server, and the output is the user's voice data and its converted text data.

[2220] Step 7:

[2221] The server uses a text generation tool based on the collected data to automatically generate autobiographical text. It combines the stories told by the elderly with the results of emotion analysis to create text in a natural format. The input is the data told by the elderly and the results of emotion analysis, and the output is an autobiographical text.

[2222] Step 8:

[2223] The server converts the generated autobiographical text into a specific style selected by the user using a format conversion means. For example, it converts into a style that matches the user's preferences, such as a documentary style or a romantic comedy style. The input is the generated autobiographical text, and the output is the format-converted text.

[2224] Step 9:

[2225] The server provides the final generated autobiography to the user in digital format through the output means. The terminal has a display for visual confirmation, allowing the user to check the content. The input is the formatted text, and the output is the autobiography presented in digital format.

[2226] Step 10:

[2227] The user checks the generated autobiographical text and, if any corrections are necessary, sends a correction request using the correction management means. The server receives the correction request, regenerates the final version with the relevant parts corrected, and presents it to the user again. The input is the user's correction request, and the output is the corrected autobiographical text.

[2228] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2229] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemin...

Claims

1. This system allows elderly people to talk about past events and generates autobiographical text based on this. An input method for elderly people to talk about their past, An analytical method to analyze what was said and extract important keywords, a question generation means for generating a next question based on the extracted keywords and presenting the next question to the elderly person; A database management method for registering and managing profile information and past photos of elderly people; A text generation means for automatically generating an autobiographical text based on the received information; a format conversion means for selecting and outputting the style of the autobiography; A system including:

2. 2. The system according to claim 1, further comprising a voice recognition means for inputting the elderly person's speech and converting it into text data.

3. 2. The system according to claim 1, further comprising a revision management means for presenting the generated autobiographical writing to the user in a revisable form and reflecting revisions.

Images