System

Abstract

A system is provided.SOLUTION: The system includes means for converting a question from a parent or a child into character data by voice recognition, means for analyzing the character data by using a natural language processing technology, and means for acquiring an appropriate answer from a database on the basis of an analysis result and outputting the answer by voice.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In today's child-rearing environment, parents are often faced with anxiety and loneliness, confusion due to information overload, and a lack of psychological support. Furthermore, despite the importance of managing the health and stress levels of parents and children, there is a lack of appropriate methods. In addition, there is a lack of environments where it is easy to keep records of growth or share complaints. To address these issues, a comprehensive child-rearing support system that supports parents and children is needed. [Means for solving the problem]

[0005] The present invention provides a system that includes a means for converting questions from parents or children into text data using speech recognition and analyzing them using natural language processing technology, a means for retrieving appropriate answers from a database based on the analysis results, and a means for outputting the answers audibly. The system also includes a means for converting parent or child growth records into text data and storing them in a database, and a means for managing and displaying the stored growth records. The system also includes a means for collecting physical condition data, analyzing them using data analysis technology, generating recommendations for daily rhythms and activities based on the analysis results, and outputting them audibly; a means for converting everyday conversations into text data, analyzing them using stress analysis technology, assessing stress levels, and generating advice based on the assessment results; and a means for analyzing complaints using emotion analysis technology and generating empathetic messages. This helps parents maintain mental stability and improves the quality of their child-rearing.

[0006] "Speech recognition" is a technology that converts speech into text data.

[0007] "Natural language processing" is an artificial intelligence technology for analyzing and understanding human language.

[0008] A "database" is a system for organizing, managing, and quickly searching data.

[0009] A "growth record" is data that records important events and progress during a child's development.

[0010] "Data analysis" is the process of analyzing collected data and extracting useful information and trends.

[0011] "Stress analysis" is a technology that evaluates stress levels and conditions based on conversational and behavioral data.

[0012] An "empathy message" is a message that shows understanding and sympathy for the user's feelings and situation.

[0013] "Advice" provides suggestions and instructions that are tailored to the user's situation.

[0014] "Physical condition data" is data that includes information about the physical condition of the user.

[0015] "Audio output" is a technology that converts text data into audio and lets the user hear it. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] The present invention is a system that combines speech recognition technology, natural language processing technology, database management technology, and data analysis technology. The purpose of this system is to alleviate the concerns of parents and children and support child-rearing. The following describes in detail the implementation of each function.

[0038] Implementation of QA support functions

[0039] In this system, when a parent or child asks a question, the device uses voice recognition to convert the speech into text data. The device then sends the text data to a server. The server uses natural language processing technology to analyze the text data and retrieves an appropriate answer from a database based on the analysis results. The answer is then sent from the server to the device, which then verbally communicates the answer to the user.

[0040] Example: A child asks, "Why is the sky blue?" The device converts the speech into text and sends it to a server. The server responds, "The sky is blue because sunlight scatters molecules in the atmosphere," and the device then relays the answer.

[0041] Embodiment of growth record saving function

[0042] The system allows parents to record their child's growth by voice. The device uses voice recognition to convert the record into text data, which is then sent to a server. The server stores the text data in a database. Parents can then review the saved growth records later.

[0043] Example: When a parent says, "Today was the first time I rode a bike," the device converts the speech into text and sends it to a server. The server then stores the record that "my child rode a bike for the first time" in a database.

[0044] An embodiment of the health check and data analysis function

[0045] This system periodically collects and analyzes the physical condition data of parents and children. The device collects the data and sends it to a server. The server uses data analysis technology to analyze the data and generates recommendations regarding daily rhythms and next actions based on the analysis results. The recommendations are sent to the device, which then communicates them to the user via audio.

[0046] Example: When a parent types, "I have a headache today," the device sends that data to the server. The server then sends the analysis result, "I recommend you drink plenty of fluids and get plenty of rest," and the device then announces this in voice.

[0047] Stress check and advice function implementation example

[0048] This system records everyday conversations between parents and children and performs stress analysis. The device converts the conversations into text data using voice recognition and sends it to a server. The server then analyzes the text data using stress analysis technology to evaluate the stress level. Based on the evaluation results, the server generates appropriate advice and sends it to the device. The device then communicates the advice to the user via voice.

[0049] Example: If a parent says, "I was really busy at work today," the device converts the conversation into text and sends it to a server. The server then sends the analysis result, "You seem to be stressed. Take a moment to relax and take a deep breath," and the device then conveys this information in voice.

[0050] An embodiment of the complaint listening function

[0051] This system can listen to a parent's complaints and return a message of empathy. The device uses voice recognition to convert the parent's complaints into text data and send it to the server. The server then uses emotion analysis technology to analyze the text data and generate a message of empathy. The message of empathy is then sent to the device, which then relays it to the user via voice.

[0052] Example: When a parent says, "Nothing went well today," the device converts the complaint into text and sends it to a server. The server generates a sympathetic message, saying, "That must have been tough. Take a break and refresh yourself," and the device then relays this message in voice.

[0053] This allows users (parents and children) to easily ask questions or share their frustrations, and also allows them to easily check their health and keep a record of their child's growth. Throughout the system, users receive emotional support, enabling them to raise their children more effectively and with peace of mind.

[0054] The processing flow will be explained below.

[0055] Processing steps for QA-enabled features

[0056] Step 1:

[0057] User: Asks the AI ​​robot aloud, "Why is the sky blue?"

[0058] Step 2:

[0059] Terminal: A voice recognition system is used to convert the user's question from voice data to text data.

[0060] Step 3:

[0061] Terminal: Sends the converted question to the server.

[0062] Step 4:

[0063] Server: Analyzes the received text data using natural language processing technology.

[0064] Step 5:

[0065] Server: Retrieves the appropriate answer from the database based on the analysis results.

[0066] Step 6:

[0067] Server: Sends the obtained answer to the device.

[0068] Step 7:

[0069] Terminal: Converts the received answer into speech and conveys it to the user.

[0070] Growth record saving function processing steps

[0071] Step 1:

[0072] User: Says through the microphone, "Today was my first time riding a bike."

[0073] Step 2:

[0074] Terminal: A speech recognition system is used to convert the user's speech into text data.

[0075] Step 3:

[0076] Terminal: Sends the converted text data to the server.

[0077] Step 4:

[0078] Server: Saves the received text data in a database.

[0079] Step 5:

[0080] Server: Sends a message to the terminal indicating that the save is complete.

[0081] Step 6:

[0082] Terminal: A message will be output to the user indicating that the save is complete.

[0083] Health check and data analysis function processing steps

[0084] Step 1:

[0085] User: Enter "I have a headache today" as physical condition data.

[0086] Step 2:

[0087] Terminal: Sends collected physical condition data to the server.

[0088] Step 3:

[0089] Server: Analyzes the received physical condition data using data analysis technology.

[0090] Step 4:

[0091] Server: Generates recommendations for daily rhythms and activities based on the analysis results.

[0092] Step 5:

[0093] Server: Sends the generated recommendations to the device.

[0094] Step 6:

[0095] Device: Recommendations are spoken to the user.

[0096] Stress check and advice function processing steps

[0097] Step 1:

[0098] User: In everyday conversation, say, "I was really busy at work today."

[0099] Step 2:

[0100] Terminal: Uses a voice recognition system to convert speech into text.

[0101] Step 3:

[0102] Terminal: Sends the converted text data to the server.

[0103] Step 4:

[0104] Server: Analyzes the received text data using stress analysis technology.

[0105] Step 5:

[0106] Server: Evaluates stress levels and generates appropriate advice.

[0107] Step 6:

[0108] Server: Sends the generated advice to the device.

[0109] Step 7:

[0110] Terminal: Advice is given to the user by voice.

[0111] Processing steps of the Complaint Listening feature

[0112] Step 1:

[0113] User: Complains to the AI ​​robot, "Nothing went well today."

[0114] Step 2:

[0115] Terminal: Uses a voice recognition system to convert the complaint from voice data to text data.

[0116] Step 3:

[0117] Terminal: Sends the converted text data to the server.

[0118] Step 4:

[0119] Server: Analyzes the received text data using emotion analysis technology.

[0120] Step 5:

[0121] Server: Generates empathy messages.

[0122] Step 6:

[0123] Server: Sends the generated empathy message to the device.

[0124] Step 7:

[0125] Terminal: Conveys empathetic messages to the user via voice.

[0126] Example 1

[0127] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0128] There is a need for support to quickly and efficiently resolve the questions and concerns that parents and children have while raising children. In particular, there is a need for a wide range of support, such as appropriate answers to questions, management of growth records, and checks on health and stress levels. However, it has been difficult to comprehensively manage all of these in a single system and integrate voice input and output. Furthermore, while it is important to have the ability to show empathy and support when parents complain, existing systems have not been able to adequately address this need.

[0129] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0130] In this invention, the server includes: means for converting questions from a parent or child into text data using voice recognition; means for analyzing the text data using natural language processing technology; and means for retrieving appropriate answers from an information storage device based on the analysis results and outputting the answers audibly. This enables prompt and appropriate answers to user questions. The server also includes means for converting parent or child growth records into text data using voice recognition and storing the text data in an information storage device; and means for managing and displaying the stored growth records. This enables easy recording and management of growth records. The server also includes means for collecting parent or child health data, analyzing the health data using data analysis technology, and generating recommendations regarding lifestyle rhythms and next actions based on the analysis results and outputting the recommendations audibly. This enables appropriate advice regarding improving lifestyle rhythms and next actions to be provided to users. The server also includes means for converting everyday conversations between a parent or child into text data using voice recognition, analyzing the text data using emotion analysis technology, and generating empathy messages based on the analysis results and outputting the empathy messages audibly. This allows for a response that is sensitive to the user's emotions. The system also includes a means for converting the complaints of parents or children into text data using voice recognition and analyzing the text data using emotion analysis technology, and a means for generating a message of sympathy based on the analysis results and outputting the message of sympathy by voice. This makes it possible to provide a sympathetic response even when the user vents their frustrations.

[0131] "Speech recognition" is a technology that converts speech into text data.

[0132] "Character data" is text-format data generated by speech recognition.

[0133] "Natural language processing technology" is a computer programming technology for analyzing and understanding human language.

[0134] "Analysis results" refers to information obtained after analyzing character data using natural language processing technology.

[0135] An "information storage device" is a device for storing and managing data, and specifically refers to a database.

[0136] "Audio output" refers to transmitting text data to the user as audio using speech synthesis technology.

[0137] A "growth record" is data that records information about important events and developments during a child's growth process.

[0138] "Management and display" refers to properly organizing the stored data and presenting it to the user as needed.

[0139] "Physical condition data" refers to information about the health status of parents and children, and is primarily data about physical conditions.

[0140] "Data analysis technology" is the technology for analyzing collected data and deriving useful information.

[0141] "Life rhythm" refers to the patterns of activity and rest in daily life.

[0142] "Recommendations for next steps" are specific guidelines for action suggested based on your current health condition and lifestyle.

[0143] "Daily conversation" refers to the general conversations that parents and children have every day.

[0144] "Emotion analysis technology" is a technology for analyzing the emotions contained in words and sentences and understanding their content.

[0145] An "empathetic message" is a message that is generated in a way that understands and sympathizes with the user's emotions.

[0146] "Complaining" is the act of parents and children verbally expressing the dissatisfaction and stress they feel in their daily lives.

[0147] The present invention is a system that combines technologies of speech recognition, natural language processing, database management, and data analysis to provide support and resolve various concerns and questions that parents and children have while raising children. Specific embodiments of the present invention will be described in detail below.

[0148] Implementation of QA support functions

[0149] In this system, when a user asks a question, the device converts the question into text data using voice recognition technology. Specifically, it uses Google's Cloud Speech-to-Text API. The converted text data is then sent from the device to the server.

[0150] The server analyzes the text data using Google's Natural Language API. Based on the analysis results, the server retrieves an appropriate answer from an information storage device (a database, such as MySQL®). The retrieved answer is then sent from the server to the terminal.

[0151] The device uses Google's Text-to-Speech API to convert the answer into audio and convey it to the user.

[0152] Examples:

[0153] When a user asks, "Why is the sky blue?", the device converts the speech into text using Google's Cloud Speech-to-Text API and sends it to the server. The server then analyzes it using Google's Natural Language API and obtains the answer, "The sky is blue because sunlight scatters molecules in the atmosphere." The device then uses Google's Text-to-Speech API to synthesize the answer into voice and convey it to the user.

[0154] Example prompt sentence:

[0155] When a user asks "Why is the sky blue?", the voice is converted to text using Google's Cloud Speech-to-Text API, the question is analyzed using Google's Natural Language API, and the answer is synthesized using Google's Text-to-Speech API.

[0156] Embodiment of growth record saving function

[0157] When a user enters their child's growth record by voice, the device converts the voice into text using Google's Cloud Speech-to-Text API, which then sends the text data to the server.

[0158] The server stores the received text data in an information storage device (database, for example, MySQL), which the user can later check and manage.

[0159] Examples:

[0160] When a parent says, "Today was my first time riding a bike," the device converts the speech into text using Google's Cloud Speech-to-Text API and sends it to the server, which then stores the record of "my child rode a bike for the first time" in a database.

[0161] Example prompt sentence:

[0162] When a parent says, "Today was my first time riding my bike," Google's Cloud Speech-to-Text API converts the speech to text and stores that text in a MySQL database.

[0163] An embodiment of the health check and data analysis function

[0164] When a user enters their health data, the device sends the data to a server, which then analyzes the data using Python's Pandas library and Scikit-learn and generates appropriate recommendations based on the analysis results.

[0165] The generated recommendations are sent from the server to the device, which then uses Google's Text-to-Speech API to convert the recommendations into audio and convey them to the user.

[0166] Examples:

[0167] When a parent types, "I have a headache today," the device sends that data to a server. The server analyzes the data using Python's Pandas and Scikit-learn, and generates recommendations such as, "I recommend drinking plenty of water and getting plenty of rest." The device then uses Google's Text-to-Speech API to synthesize the recommendations into voice and deliver them to the user.

[0168] Example prompt sentence:

[0169] When a parent types "I have a headache today," the data is sent to the server and analyzed using Python's Pandas and Scikit-learn. The results are then synthesized into speech using Google's Text-to-Speech API.

[0170] Stress check and advice function implementation example

[0171] The system converts the user's everyday conversations into text data using voice recognition technology and sends the text data to a server, which then analyzes the text data using emotion analysis technology to evaluate the user's stress level.

[0172] Based on the evaluation results, the server generates appropriate advice and sends it to the device, which then converts the advice into audio using Google's Text-to-Speech API and conveys it to the user.

[0173] Examples:

[0174] When a parent says, "I was really busy at work today," the device converts the conversation into text and sends it to the server. The server then generates advice, such as, "You seem stressed. Take a moment to relax and take a deep breath." The device then uses Google's Text-to-Speech API to synthesize the advice into voice and relay it to the user.

[0175] Example prompt sentence:

[0176] When a parent says, "I was really busy at work today," emotion analysis technology is used to analyze their stress level, and appropriate advice is synthesized using Google's Text-to-Speech API.

[0177] An embodiment of the complaint listening function

[0178] When a user complains, the device uses voice recognition technology to convert the complaint into text data and sends it to the server. The server then uses emotion analysis technology to analyze the text data and generate a message of empathy.

[0179] The generated empathy message is sent from the server to the device, which then uses Google's Text-to-Speech API to convert the message into audio and convey it to the user.

[0180] Examples:

[0181] When a parent says, "Nothing went well today," the device converts the complaint into text and sends it to the server. The server then generates a message of sympathy, saying, "That must have been tough. Take a break and refresh yourself." The device then uses Google's Text-to-Speech API to synthesize the message of sympathy into voice and deliver it to the user.

[0182] Example prompt sentence:

[0183] When a parent says, "Nothing went well today," emotion analysis technology generates an empathetic message, which is then synthesized into voice using Google's Text-to-Speech API.

[0184] In this way, by combining various cutting-edge technologies, this system is able to provide comprehensive support for the various problems and concerns that users face on a daily basis.

[0185] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0186] Processing steps for QA-enabled features

[0187] Step 1:

[0188] The user asks a question, for example, "Why is the sky blue?"

[0189] Input: Audio data

[0190] Output: None

[0191] Step 2:

[0192] The device converts the voice data into text data using Google's Cloud Speech-to-Text API.

[0193] Input: Audio data

[0194] Output: Character data

[0195] Step 3:

[0196] The character data generated by the terminal is sent to the server using an HTTP request.

[0197] Input: Character data

[0198] Output: HTTP request to the server

[0199] Step 4:

[0200] The server analyzes the received text data using Google's Natural Language API.

[0201] Input: Character data

[0202] Output: Analysis results

[0203] Step 5:

[0204] Based on the analysis results, the server queries the information storage device (database) to obtain the appropriate answer.

[0205] Input: Analysis results

[0206] Output: Response data

[0207] Step 6:

[0208] The server sends the response data to the terminal.

[0209] Input: Answer data

[0210] Output: HTTP response to the device

[0211] Step 7:

[0212] The response data received by the device is converted into audio data using Google's Text-to-Speech API.

[0213] Input: Answer data

[0214] Output: Audio data

[0215] Step 8:

[0216] The terminal plays the audio data to the user.

[0217] Input: Audio data

[0218] Output: Audio output to the user

[0219] Growth record saving function processing steps

[0220] Step 1:

[0221] The user enters their child's growth record by voice, for example, "Today was the first time I rode a bike."

[0222] Input: Audio data

[0223] Output: None

[0224] Step 2:

[0225] The device converts the voice data into text data using Google's Cloud Speech-to-Text API.

[0226] Input: Audio data

[0227] Output: Character data

[0228] Step 3:

[0229] The character data generated by the terminal is sent to the server.

[0230] Input: Character data

[0231] Output: HTTP request to the server

[0232] Step 4:

[0233] The server stores the received character data in an information storage device (database).

[0234] Input: Character data

[0235] Output: Save to database

[0236] Health check and data analysis function processing steps

[0237] Step 1:

[0238] The user inputs data about his / her physical condition. For example, he / she inputs "I have a headache today."

[0239] Input: Audio data (or text data)

[0240] Output: None

[0241] Step 2:

[0242] The device converts the voice data into text data and sends it to the server. If the data is text, it is sent as is.

[0243] Input: Audio or text data

[0244] Output: HTTP request to the server

[0245] Step 3:

[0246] The health data received by the server is analyzed using Python's Pandas library and Scikit-learn.

[0247] Input: Physical condition data

[0248] Output: Analysis results

[0249] Step 4:

[0250] The server generates recommendations regarding daily routines and next actions based on the analysis results.

[0251] Input: Analysis results

[0252] Output: Recommendations

[0253] Step 5:

[0254] The server sends the recommendations to the device.

[0255] Input: Recommendations

[0256] Output: HTTP response to the device

[0257] Step 6:

[0258] The device converts the recommendations it receives into audio data using Google's Text-to-Speech API.

[0259] Input: Recommendations

[0260] Output: Audio data

[0261] Step 7:

[0262] The terminal plays the audio data to the user.

[0263] Input: Audio data

[0264] Output: Audio output to the user

[0265] Stress check and advice function processing steps

[0266] Step 1:

[0267] The user engages in everyday conversation, for example, saying, "I was really busy at work today."

[0268] Input: Audio data

[0269] Output: None

[0270] Step 2:

[0271] The device converts the voice data into text data using Google's Cloud Speech-to-Text API.

[0272] Input: Audio data

[0273] Output: Character data

[0274] Step 3:

[0275] The character data generated by the terminal is sent to the server.

[0276] Input: Character data

[0277] Output: HTTP request to the server

[0278] Step 4:

[0279] The server analyzes the received text data using emotion analysis technology.

[0280] Input: Character data

[0281] Output: Analysis results (stress level)

[0282] Step 5:

[0283] The server generates appropriate advice based on the analysis results.

[0284] Input: Analysis results

[0285] Output: Advice data

[0286] Step 6:

[0287] The server transmits the advice data to the terminal.

[0288] Input: Advice data

[0289] Output: HTTP response to the device

[0290] Step 7:

[0291] The advice data received by the device is converted into audio data using Google's Text-to-Speech API.

[0292] Input: Advice data

[0293] Output: Audio data

[0294] Step 8:

[0295] The terminal plays the audio data to the user.

[0296] Input: Audio data

[0297] Output: Audio output to the user

[0298] Processing steps of the Complaint Listening feature

[0299] Step 1:

[0300] The user complains, for example, "Nothing went well today."

[0301] Input: Audio data

[0302] Output: None

[0303] Step 2:

[0304] The device converts the voice data into text data using Google's Cloud Speech-to-Text API.

[0305] Input: Audio data

[0306] Output: Character data

[0307] Step 3:

[0308] The character data generated by the terminal is sent to the server.

[0309] Input: Character data

[0310] Output: HTTP request to the server

[0311] Step 4:

[0312] The server analyzes the received text data using emotion analysis technology.

[0313] Input: Character data

[0314] Output: Analysis results (emotion data)

[0315] Step 5:

[0316] The server generates a message of sympathy based on the analysis results.

[0317] Input: Analysis results

[0318] Output: Empathy message

[0319] Step 6:

[0320] The server sends a sympathy message to the terminal.

[0321] Input: Sympathy message

[0322] Output: HTTP response to the device

[0323] Step 7:

[0324] The device converts the empathy message received into audio data using Google's Text-to-Speech API.

[0325] Input: Sympathy message

[0326] Output: Audio data

[0327] Step 8:

[0328] The terminal plays the audio data to the user.

[0329] Input: Audio data

[0330] Output: Audio output to the user

[0331] (Application example 1)

[0332] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0333] In modern society, parents face the challenge of a lack of support when going out with their children and shopping in brick-and-mortar stores. In particular, parents have limited means to easily gather information about their children's health, growth records, and daily concerns, and to receive appropriate advice. Furthermore, there are insufficient systems in place to alleviate parents' own stress and worries, making it an urgent task to address these issues.

[0334] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0335] In this invention, the server includes: means for converting questions from a parent or child into text data using voice recognition; means for analyzing the text data using natural language processing technology; and means for retrieving appropriate answers from a database based on the analysis results and outputting the answers audibly. This allows parents to ask questions or seek advice about child-rearing in real time while shopping at a physical store and receive appropriate answers. The server also includes means for collecting physical condition data of the parent or child and analyzing the physical condition data using data analysis technology; and means for generating recommendations regarding daily rhythms and next actions based on the analysis results and outputting the recommendations audibly. This allows parents to understand their child's health status and take appropriate measures even while shopping at a physical store. The server also includes means for recording daily conversations between the parent or child and analyzing the conversation data using stress analysis technology; means for generating appropriate advice based on the analysis results and outputting the advice audibly; and means for allowing the parent or child to input complaints audibly, generating empathetic messages using emotion analysis technology, and outputting the messages audibly. This reduces stress and worries for parents and improves overall child-rearing support.

[0336] definition statement

[0337] "Speech recognition" is a technology that receives speech as input and automatically converts that speech into text data.

[0338] "Character data" refers to data in text format obtained by speech recognition.

[0339] "Natural language processing technology" is a technology that enables computers to understand, analyze, and generate natural human language.

[0340] The "analysis results" are the results of analyzing character data using natural language processing technology.

[0341] A "database" is a system that allows data to be stored, searched, and managed.

[0342] "Audio output means" refers to technology that provides analysis results and answers to users as audio.

[0343] "Physical condition data" refers to data indicating the health condition of a parent or child.

[0344] "Data analysis technology" is a technology for analyzing collected data and extracting useful information.

[0345] "Life rhythm" refers to an individual's activity patterns and time allocation in their daily lives.

[0346] "Recommendations for next actions" are specific actions or advice suggested based on the analysis results.

[0347] "Daily conversation" refers to the verbal exchanges that parents or children have on a daily basis.

[0348] "Stress analysis technology" is a technology that evaluates an individual's stress level based on conversation and behavioral data.

[0349] "Advice" refers to advice or suggestions provided based on the results of the analysis.

[0350] "Guchi" is a word that expresses dissatisfaction or anxiety felt by parents or children.

[0351] "Emotion analysis technology" is a technology that evaluates the emotional state of text data and generates emotional responses based on that data.

[0352] An "empathetic message" is a message that shows understanding and empathy for the user's feelings.

[0353] MODE FOR CARRYING OUT THE INVENTION

[0354] The present invention is a child-rearing support system that uses voice recognition technology, natural language processing technology, database management technology, and data analysis technology. This system provides various child-rearing-related services when parents and children enjoy shopping at physical stores. Specific embodiments are described below.

[0355] System Configuration and Hardware

[0356] This system consists of a smartphone, a cloud server, a voice recognition engine, a natural language processing engine, a database, a text-to-speech engine, etc.

[0357] The smartphone is a device on which a dedicated app is installed and which directly interacts with parents or children. The cloud server processes various data and provides appropriate information to parents or children.

[0358] Specific roles and operations of each technology

[0359] Speech recognition: When a user (parent or child) speaks into a smartphone, a speech recognition engine (such as the Google Speech-to-Text API) converts the speech into text data.

[0360] Natural language processing: Text data is sent to a cloud server and analyzed using a natural language processing engine (such as Google Cloud Natural Language API). For example, if a user says, "It looks like my child has caught a cold," the cloud server analyzes this text data and understands its meaning.

[0361] Database management: Based on the results analyzed by the natural language processing engine, the cloud server retrieves appropriate answers and support information from a database (such as MongoDB). For example, advice such as "Give the child plenty of warm drinks, and if symptoms persist, consult a pediatrician" may be obtained.

[0362] Voice output: The obtained answer is converted into voice by a text-to-speech engine (such as Google Text-to-Speech API) and is transmitted to the user through the smartphone speaker.

[0363] Example of childcare support function

[0364] 1. Question and Answering

[0365] When a user speaks to their smartphone, "My child has a fever. What should I do?", the speech recognition engine converts the speech into text data and sends it to a cloud server. The cloud server then uses natural language processing technology to analyze the text and retrieves the appropriate answer from a database. The answer, "Ensure adequate hydration and consult a doctor if necessary," is then played back as audio by the text-to-speech engine.

[0366] 2. Keeping a record of growth

[0367] When a user says, "Today was my first time riding a bicycle," the speech recognition engine converts this into text data and sends it to the cloud server, which then stores this data in a database and manages it for later review.

[0368] 3. Health check and advice

[0369] When a user says "I have a headache," the speech recognition engine converts this into text data and sends it to the cloud server. The cloud server analyzes this data and generates advice such as "We recommend you drink plenty of fluids and get some rest," which is then conveyed to the user via voice.

[0370] 4. Stress check and emotion analysis

[0371] The system continuously records the user's conversations and periodically analyzes their stress levels. For example, if a user says, "Work was very hard today," the cloud server will evaluate their stress level and provide advice such as, "Take a break to relax."

[0372] Prompt Sentence Examples

[0373] A user might ask, "My child has a cough. What should I do?" and use the following prompt:

[0374] I would like to answer questions about children's health. For example, "Why is the sky blue?" I would like to answer something like, "The sky is blue because sunlight scatters molecules in the atmosphere."

[0375] This provides an environment where parents and children can focus on raising their children with peace of mind while enjoying shopping in a physical store.

[0376] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0377] Program processing steps

[0378] Step 1:

[0379] The user (parent or child) enters questions or information into the smartphone by voice. For example, the user might say, "My child seems to have caught a cold." This voice data is then entered into the system.

[0380] Step 2:

[0381] The device uses a speech recognition engine (Google Speech-to-Text API) to convert the voice data from step 1 into text data, which then becomes the input for the next step.

[0382] Step 3:

[0383] The device formats the text data as a preprocessing step and sends it to the cloud server. The cloud server receives this text data and analyzes it using a natural language processing engine (Google Cloud Natural Language API). As a result of the analysis, the meaning and intent of the text data are extracted.

[0384] Step 4:

[0385] Based on the analysis results, the server retrieves the appropriate answer corresponding to the question from a database (MongoDB). This database contains a wide variety of questions and their answers in advance. For example, the answer retrieved might be, "Have the child drink plenty of warm fluids, and if symptoms persist, consult a pediatrician."

[0386] Step 5:

[0387] The server sends the answer retrieved from the database to a text-to-speech engine (Google Text-to-Speech API), which converts the answer into audio data, which becomes the input for the next step.

[0388] Step 6:

[0389] The device receives the voice data sent from the text-to-speech engine and conveys it to the user. Specifically, the smartphone speaker plays a voice message saying, "Give your child plenty of warm drinks, and if symptoms persist, consult a pediatrician."

[0390] Step 7:

[0391] The user receives advice and answers from the app and takes appropriate action. At this step, the user gets a solution to their problem.

[0392] Step 8:

[0393] If necessary, the user can again input their daily conversations, physical condition, and growth records by voice, which the system will record and analyze. This step is important so that the system can continuously monitor the user's condition and provide appropriate advice.

[0394] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0395] This invention is based on an AI system that interacts with parents or children, and incorporates an emotion engine to recognize and respond to the user's emotions. This system combines voice recognition technology, natural language processing technology, database management technology, data analysis technology, and emotion engine technology to address various issues faced by parents and children.

[0396] An embodiment of emotion recognition QA function

[0397] In this system, when a parent or child asks a question, the device uses a voice recognition system to convert the speech into text data. The text data is then sent to a server. The server uses natural language processing technology and an emotion engine to analyze the text and recognize the user's emotional state. Based on this emotional state, the server generates an appropriate response or a tone-adjusted empathetic message, which is then sent from the server to the device. The device then communicates the generated response or empathetic message to the user via voice.

[0398] For example, if a child asks, "Why do I have to go to school?", the device converts the speech into text and sends it to a server. The server recognizes the child's anxiety and stress contained in the question and generates a tailored response: "School is a place where you can make new friends and have fun learning. Don't worry, just relax and have fun," which the device then relays.

[0399] An embodiment of a growth record keeping function using emotion recognition

[0400] The system also uses the emotion engine when parents enter their child's growth record by voice. The device converts the voice into text data, running the emotion engine to send the entered content and emotional state to the server. The server stores the received data in a database and, if necessary, tags the emotional state. Parents can later review the saved growth record and easily search for important events in which a specific emotional state was recorded.

[0401] Example: When a parent says, "Today I was so happy because I rode my bike for the first time," the device converts the speech into text, recognizes the "happy" emotion using an emotion engine, and sends this information to the server. The server then saves the record of "my child rode a bike for the first time" in a database with a tag of "happy."

[0402] An embodiment of a health check and data analysis function using emotion recognition

[0403] This system periodically collects and analyzes the physical condition data of parents and children, while also analyzing the user's emotional state using an emotion engine. The device collects emotional data along with the physical condition data and sends it to a server. The server analyzes the data using data analysis technology and the emotion engine, and based on the analysis results, generates recommendations for daily rhythms and activities, as well as advice for psychological support. The recommendations and advice are sent to the device, which then communicates them to the user via voice.

[0404] Example: If a parent types, "I have a headache today and I'm feeling a bit down," the device will send that data to a server. The server will then send an analysis result, such as, "Drink plenty of fluids, get some rest, and listen to your favorite music to change your mood," and the device will then announce this in voice.

[0405] An embodiment of a stress check and advice function using emotion recognition

[0406] This system records everyday conversations between parents and children and uses an emotion engine to perform stress analysis. The device uses voice recognition to convert the conversation into text data, analyzes the emotional state using the emotion engine, and sends this to a server. The server uses both stress analysis technology and the emotion engine to evaluate stress levels and generate appropriate advice. The evaluation results and advice are sent to the device, which then communicates them to the user via voice.

[0407] Example: If a parent says, "I'm really busy at work today and I'm tired," the device converts the conversation into text, recognizes the "tired" emotion using an emotion engine, and sends it to the server. The server then sends the analysis results, saying, "You seem to be stressed. Take some time to relax and take a deep breath," and the device then conveys this in voice.

[0408] Embodiment of a function for listening to complaints using emotion recognition

[0409] This system also uses the emotion engine when a parent complains. The device uses voice recognition to convert the complaint into text data, analyzes the emotion using the emotion engine, and sends this to the server. The server then uses emotion analysis technology to generate an empathy message, which is tailored based on the parent's emotional state. The empathy message is then sent to the device, which then relays it to the user via voice.

[0410] Example: When a parent complains, "Nothing went well today," the device converts the complaint into text, recognizes the "disappointment" in its emotion engine, and sends it to the server. The server then generates a sympathetic message, saying, "That must have been really tough. Get some rest and start tomorrow with a fresh attitude," and the device then relays this message in voice.

[0411] This allows users (parents and children) to easily ask questions or express their frustrations, and also allows for easy health checks and growth records to be kept. Furthermore, emotion recognition technology allows for more sympathetic responses, providing greater emotional support. Throughout the system, users can raise their children in a comfortable environment.

[0412] The processing flow will be explained below.

[0413] Processing steps for emotion recognition QA function

[0414] Step 1:

[0415] User: Asks the AI ​​robot aloud, "Why do I have to go to school?"

[0416] Step 2:

[0417] Terminal: A voice recognition system is used to convert the user's question from voice data to text data.

[0418] Step 3:

[0419] Terminal: Sends the converted text data to the server.

[0420] Step 4:

[0421] Server: Analyzes the received text data using natural language processing technology and an emotion engine to recognize the user's emotions.

[0422] Step 5:

[0423] Server: Based on the user's emotions, retrieves appropriate answers from the database and generates empathetic messages with adjusted tone.

[0424] Step 6:

[0425] Server: Sends the generated answer and sympathy message to the device.

[0426] Step 7:

[0427] Terminal: Converts the received answers and sympathy messages into audio and conveys them to the user.

[0428] Processing steps for growth record keeping function using emotion recognition

[0429] Step 1:

[0430] User: Through the microphone, says, "Today was my first time riding a bike and I was so happy."

[0431] Step 2:

[0432] Terminal: A speech recognition system is used to convert the user's speech into text data.

[0433] Step 3:

[0434] Terminal: Sends the converted text data to the server.

[0435] Step 4:

[0436] Server: The received text data is analyzed using an emotion engine to recognize the emotion "happy."

[0437] Step 5:

[0438] Server: Stores the text data along with the recognized emotions in a database.

[0439] Step 6:

[0440] Server: Sends a message to the terminal indicating that the save is complete.

[0441] Step 7:

[0442] Terminal: A message will be output to the user indicating that the save is complete.

[0443] Processing steps for health check and data analysis functions using emotion recognition

[0444] Step 1:

[0445] User: Enters "I have a headache today and feel a little depressed" as health data.

[0446] Step 2:

[0447] Terminal: Sends collected physical condition data to the server.

[0448] Step 3:

[0449] Server: Analyzes the received physical condition data and emotion data using data analysis technology and an emotion engine.

[0450] Step 4:

[0451] Server: Generates recommendations for daily routines and behaviors based on the analysis results, and also generates advice for psychological support.

[0452] Step 5:

[0453] Server: Sends the generated recommendations and advice to the device.

[0454] Step 6:

[0455] Device: Provides recommendations and advice to the user via voice.

[0456] Processing steps for stress check and advice function using emotion recognition

[0457] Step 1:

[0458] User: In everyday conversation, say, "I'm really busy at work today and I'm tired."

[0459] Step 2:

[0460] Terminal: Uses a voice recognition system to convert speech into text.

[0461] Step 3:

[0462] Terminal: Sends the converted text data to the server.

[0463] Step 4:

[0464] Server: Analyzes the received text data using stress analysis technology and an emotion engine to evaluate stress levels and emotional states.

[0465] Step 5:

[0466] Server: Generates appropriate advice based on the evaluation results.

[0467] Step 6:

[0468] Server: Sends the generated advice to the device.

[0469] Step 7:

[0470] Terminal: Advice is given to the user by voice.

[0471] Processing steps for listening to complaints using emotion recognition

[0472] Step 1:

[0473] User: Complains to the AI ​​robot, "Nothing went well today."

[0474] Step 2:

[0475] Terminal: Uses a voice recognition system to convert the complaint from voice data to text data.

[0476] Step 3:

[0477] Terminal: Sends the converted text data to the server.

[0478] Step 4:

[0479] Server: Analyzes the received text data using an emotion engine and recognizes "disappointment."

[0480] Step 5:

[0481] Server: Generates empathetic messages based on emotional states.

[0482] Step 6:

[0483] Server: Sends the generated empathy message to the device.

[0484] Step 7:

[0485] Terminal: Conveys empathetic messages to the user via voice.

[0486] Example 2

[0487] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0488] Conventional conversational AI systems simply convert user questions and comments about their physical condition into text data and generate answers. This makes it difficult to respond in a way that takes into account the user's emotional state and psychological aspects, resulting in insufficient psychological support for the user. Furthermore, when storing and managing growth records and physical condition data, data that includes the user's emotional state is not recorded, making it difficult to fully grasp important events or changes in the user's psychological state.

[0489] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0490] In this invention, the server includes: a means for converting questions from a parent or child into text data using voice recognition; a means for analyzing the text data using natural language processing; an emotion analysis means for recognizing the user's emotional state based on the analysis results; a means for generating an appropriately adjusted answer or empathetic message based on the recognized emotional state; and a means for outputting the generated answer or empathetic message by voice. This enables more friendly responses and psychological support tailored to the user's emotional state. Furthermore, growth records and physical condition data, including emotional states, can be saved and managed, making it easy to understand important events and psychological changes.

[0491] "Speech recognition" is a technology that analyzes voice signals and converts them into text data.

[0492] "Text data" refers to data that includes text information converted by voice recognition.

[0493] "Natural language processing" is a technology that allows computers to understand, analyze, and generate human language.

[0494] "Emotion analysis means" is a technology that recognizes the emotional state of a user from the content of their comments.

[0495] A "database" is a system for efficiently managing, storing, and searching large amounts of data.

[0496] "Data analysis technology" is a technology for analyzing collected data and extracting useful information.

[0497] "Recommendations" are suggestions about behaviors and lifestyle habits provided to users based on the analysis results.

[0498] A "sympathetic message" is a message that expresses sympathy and is generated in accordance with the user's emotional state.

[0499] "Audio output means" refers to a technology that converts text data into audio and conveys it to the user.

[0500] A "growth record" is a record that stores data about a parent's or child's growth through voice recognition and emotion analysis.

[0501] "Physical condition data" is data that includes information related to the user's health condition.

[0502] "Emotional support" refers to providing advice and messages to help users achieve emotional stability and comfort.

[0503] MODE FOR CARRYING OUT THE INVENTION

[0504] This invention is based on an AI system that interacts with parents or children, and incorporates an emotion analysis engine to recognize and respond to the user's emotions. This system combines voice recognition technology, natural language processing technology, database management technology, data analysis technology, and emotion analysis technology to address various issues faced by parents and children.

[0505] Components

[0506] Speech recognition technology: Converts speech into text data. For example, speech recognition software (e.g., Google Cloud Speech-to-Text) is used.

[0507] Natural language processing technology: Analyzes text data and understands its meaning. For example, natural language processing software (e.g., NLP API) is used.

[0508] Database management technology: Technology for effectively storing and managing converted data and analysis results. For example, a database management system (e.g., MySQL) is used.

[0509] Data analysis technology: Technology to analyze collected physical condition data and growth records and extract information useful to users. For example, data analysis libraries (e.g., Pandas, NumPy) are used.

[0510] Emotion analysis technology: Technology that recognizes the user's emotional state and generates appropriate responses based on that information. As a specific example, we will use an emotion analysis engine (e.g., Affectiva API).

[0511] Program processing

[0512] 1. Speech recognition process: The device converts the user's questions or speech into text data using speech recognition software, such as Google Cloud Speech-to-Text.

[0513] 2. Data transmission process: The converted text data is sent to the server, which uses HTTP requests and responses to analyze the received data.

[0514] 3. Text analysis process: The server uses natural language processing technology to analyze the text data and understand its contents. Natural language processing APIs (e.g., Google Cloud Natural Language API) are used to understand the meaning of the text.

[0515] 4. Sentiment Analysis Process: Based on the analysis results, the server uses sentiment analysis technology to recognize the user's emotional state. It uses a sentiment analysis engine (such as Affectiva API) to specifically identify the user's emotions.

[0516] 5. Answer generation process: Based on the recognized emotional state, the server uses a generative AI model (e.g., OpenAI's GPT model) to generate appropriate answers and empathetic messages, adjusting the tone and content according to the emotional state.

[0517] 6. Database storage process: The generated answers and emotional state data are stored in a database with appropriate tagging. A database management system such as MySQL is used for this purpose.

[0518] 7. Voice output process: Finally, the generated answers and messages are sent to the device and audibly conveyed to the user using speech synthesis technology (e.g., Amazon Polly).

[0519] Examples of concrete examples and prompts

[0520] Example 1:

[0521] When a user asks, "Why do I have to go to school?", the device converts the speech into text and sends it to the server. The server recognizes the user's anxiety and stress contained in the question and generates a tailored response: "School is a place where you can make new friends and have fun learning. Don't worry, just relax and have fun." The device then delivers the response via voice.

[0522] Example prompt sentence:

[0523] "Why do I have to go to school?"

[0524] Example 2:

[0525] When a parent says, "Today I was so happy to ride a bike for the first time," the device converts the speech into text, recognizes the "happy" emotion using an emotion engine, and sends this information to the server, which then stores the record of "my child's first bike ride" in a database with a tag of "happy."

[0526] Example prompt sentence:

[0527] "Today was the first time I rode a bike."

[0528] This invention allows users to receive friendly responses and support according to their emotions, and also makes it possible to store and manage growth records and physical condition data together with their emotional state.

[0529] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0530] Specific flow of program processing

[0531] Emotion-aware QA function

[0532] Step 1:

[0533] User utterance input

[0534] The user asks a question to the device. Specifically, they say, "Why do I have to go to school?" This input is treated as voice data.

[0535] Step 2:

[0536] Speech-to-text

[0537] The device uses speech recognition software (e.g., Google Cloud Speech-to-Text) to convert the user's voice into text data. An example of the converted output is the text data "Why do I have to go to school?"

[0538] Step 3:

[0539] Sending text to the server

[0540] The terminal sends the converted text data to the server via an HTTP request. The input is the converted text data, and the output is a notification to the server that transmission has been completed.

[0541] Step 4:

[0542] Text data analysis

[0543] The text data received by the server is analyzed using natural language processing technology (e.g., Google Cloud Natural Language API). The input is text data, and the output is the semantic and structural information of the analyzed text. Specifically, the question "Why do I have to go to school?" is analyzed, and its meaning and structure are understood.

[0544] Step 5:

[0545] Recognition of emotional states

[0546] The server uses emotion analysis technology (e.g., Affectiva's Emotion Analysis API) to recognize the user's emotional state based on the analysis results. The input is the analyzed text data, and the output is the identified emotional state (e.g., anxiety or stress).

[0547] Step 6:

[0548] Emotion-based answer generation

[0549] The server uses a generative AI model (e.g., OpenAI's GPT model) to generate an appropriate answer or empathetic message based on the emotional state. The input is the identified emotional state and text data, and the output is a generated answer or empathetic message. For example, a generated answer might be, "School is a place where you can make new friends and have fun learning. Don't worry, just relax and have fun."

[0550] Step 7:

[0551] Sending answers to the device

[0552] The server sends the generated answer to the terminal as an HTTP response. The input is the generated answer data, and the output is a notification of completion of transmission to the terminal.

[0553] Step 8:

[0554] Voice output of answers

[0555] The device uses speech synthesis technology (e.g., Amazon Polly) to convert the text-based answer into speech and convey it to the user. The input is the generated answer text, and the output is audio data. Specifically, the answer "School is a place where you can make new friends and have fun learning. Don't worry, just take your time and have fun" is provided to the user via voice.

[0556] ---

[0557] Emotion recognition-based growth record keeping function

[0558] Step 1:

[0559] User utterance input

[0560] A parent talks to the device about their child's growth. For example, they might say, "Today was my first time riding a bike, and I was so happy." The input is treated as voice data.

[0561] Step 2:

[0562] Speech-to-text

[0563] The device uses voice recognition software to convert the parent's speech into text data, and the output is the converted text data.

[0564] Step 3:

[0565] Server transmission of text and emotions

[0566] The device sends the converted text data and the emotion analysis results from the emotion analysis engine to the server. The input is the text data and emotion data, and the output is a notification of completion of transmission to the server.

[0567] Step 4:

[0568] Text and sentiment storage

[0569] The server temporarily stores the received text data and emotional state in a database. The input is the received data, and the output is a notification that the data has been saved to the database.

[0570] Step 5:

[0571] Data tagging

[0572] The server assigns a tag of "happy" as an emotional state. The input is text data and emotional data, and the output is tagged data.

[0573] Step 6:

[0574] Saving to a database

[0575] The server saves the tagged text data to a database. The input is the tagged text data, and the output is a notification that the data has been permanently saved to the database.

[0576] Step 7:

[0577] Checking the data

[0578] When a parent wants to check the growth record later, they can search the saved data through the device's user interface, filter it by a specific emotional state (e.g., "happy"), and display it. The input is the search criteria, and the output is the search results. Specifically, the record of "my child riding a bicycle for the first time" is displayed with the tag "happy."

[0579] ---

[0580] Health check and data analysis function using emotion recognition

[0581] Step 1:

[0582] User utterance input

[0583] The user speaks to the device about their physical condition. Specifically, they say, "I have a headache today, and I'm feeling a little depressed." The input is treated as voice data.

[0584] Step 2:

[0585] Speech-to-text and sentiment analysis

[0586] The device uses voice recognition software to convert speech into text data, and then uses an emotion analysis engine to analyze the emotional state, with the output being the converted text data and emotion data.

[0587] Step 3:

[0588] Sending data to the server

[0589] The terminal sends the converted text data and emotion data to the server. The input is the text data and emotion data, and the output is a notification of completion of transmission to the server.

[0590] Step 4:

[0591] Data integration

[0592] The server integrates the physical condition data and emotional data. The input is the collected data, and the output is the integrated data.

[0593] Step 5:

[0594] Data analysis

[0595] The server analyzes the integrated data using data analysis techniques (Python's Pandas and NumPy). The input is the integrated data, and the output is the analysis results. Specifically, the server evaluates the user's health status and emotional state.

[0596] Step 6:

[0597] Generating Advice

[0598] Based on the analysis results, the server generates recommendations for daily routines and behaviors, as well as advice for psychological support. The input is the analysis results, and the output is the generated advice. For example, the generated advice might be, "Drink plenty of water, get plenty of rest, and listen to your favorite music to change your mood."

[0599] Step 7:

[0600] Sending advice to the device

[0601] The server sends the generated advice to the terminal. The input is the generated advice, and the output is a notification of the completion of the transmission to the terminal.

[0602] Step 8:

[0603] Audio output of advice

[0604] The device uses voice synthesis technology such as Amazon Polly to convert the advice into speech and convey it to the user. The input is the generated advice text, and the output is audio data. Specifically, the advice provided to the user is "Drink plenty of water, get plenty of rest, and listen to your favorite music to change your mood."

[0605] (Application example 2)

[0606] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0607] Conventional food delivery systems face challenges in achieving sufficient customer satisfaction in customer service and complaint handling. Furthermore, there are also issues with the lack of effective ways to utilize customer feedback and measures to reduce the stress of customer support staff. This can lead to delays in improving service quality and customer satisfaction.

[0608] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0609] In this invention, the server includes means for converting questions from a parent or child into text data using voice recognition, means for analyzing the text data using natural language processing technology, means for retrieving an appropriate answer from a database based on the analysis result and outputting the answer by voice, means for recognizing emotions and generating an empathy message in an appropriate tone based on the analysis result, means for outputting the empathy message by voice, means for saving feedback from customers with emotion tags, and means for analyzing the feedback and providing proactive support. This improves the quality of customer service, effectively utilizes feedback, reduces the burden on customer support staff through stress checks, and enables overall improvement of service quality.

[0610] "Speech recognition" is a technology that analyzes speech and converts it into text data.

[0611] "Natural language processing" is a technology that uses computers to analyze and generate natural language.

[0612] "Emotion recognition" is a technology that detects a user's emotional state from voice or text.

[0613] A "database" is a system for systematically storing and managing data.

[0614] "Data analysis technology" is the technology for extracting useful information and patterns from large amounts of data.

[0615] A "sympathetic message" is a message with sympathetic content that is generated in accordance with the user's emotional state.

[0616] "Feedback" refers to opinions and ratings provided by users.

[0617] An "emotion tag" is tag information that indicates an emotional state that is assigned to feedback or data.

[0618] "Proactive support" is a support method that takes preventative measures before a problem occurs.

[0619] "Stress check" is a technique for assessing the stress level of users and staff.

[0620] System program generation

[0621] The system embodying this invention converts questions and feedback from parents or children into text data using voice recognition and incorporates an AI engine that recognizes emotions. This improves the quality of customer service and makes it easier to utilize feedback. It also provides a means to reduce stress for customer support representatives.

[0622] Program processing explanation

[0623] First, we use the SpeechRecognition library for speech recognition, which converts the voice input from the parent or child into text data. Then, we use the HuggingFace transformers library to analyze emotions from the text data using the bhadresh-savani / bert-base-uncased-emotion emotion recognition model.

[0624] The server uses natural language processing technology to generate empathetic messages based on the analyzed emotional data. The generated messages are then delivered to the user via voice synthesis technology. Customer feedback is tagged with emotion and stored in a database.

[0625] Furthermore, the server periodically analyzes the conversations of customer support representatives and performs stress checks. Based on the results, it generates appropriate advice and provides it to the representative.

[0626] This system will improve the quality of customer service and provide proactive support through stress checks.

[0627] Specific examples

[0628] For example, suppose a customer complains that "today's delivery was too late." At this time, the voice recognition system converts the voice into text, and the emotion recognition AI analyzes "anger." The server generates an empathetic message and replies, "We apologize for keeping you waiting. We will check it right away, so please wait a little longer." This allows the customer to receive an appropriate response, improving their satisfaction.

[0629] Customer interaction history is saved in a database with an "anger" tag attached. At a later date, a stress check is conducted and the person in charge is given advice such as "Take time to relax today."

[0630] Prompt Sentence Examples

[0631] text

[0632] Customer enquiry: "Delivery is late" --> Analyse their sentiment.

[0633] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0634] Step 1:

[0635] The user speaks a question or complaint. The device receives this speech and sends it to the speech recognition process. Input: User's speech data. Output: Input data to the speech recognition model.

[0636] Step 2:

[0637] The device uses the SpeechRecognition library to convert the speech data to text, which is then sent to the next processing step: Input: Input data to the speech recognition model. Output: Text data of the speech.

[0638] Step 3:

[0639] The server uses the HuggingFace transformers library to perform emotion recognition on the text data. It uses the bhadresh-savani / bert-base-uncased-emotion model to analyze the emotions contained in the text. Input: Speech text data. Output: Emotion analysis results.

[0640] Step 4:

[0641] The server generates a message of empathy based on the results of the sentiment analysis using natural language processing technology. The generated message is used in the next step. Input: Sentiment analysis result. Output: Message of empathy.

[0642] Step 5:

[0643] The server converts the empathy message into voice data using voice synthesis technology. This voice data is sent to the terminal and output to the user. Input: Empathy message. Output: Voice message data.

[0644] Step 6:

[0645] The terminal plays back the voice message and outputs it to the user, who receives the message.

[0646] Input: Voice message data

[0647] Output: Playing audio to the user

[0648] Step 7:

[0649] The server stores the user feedback in a database with emotion tags. Input: User feedback and emotion data. Output: Emotion-tagged feedback database entries.

[0650] Step 8:

[0651] The server periodically analyzes the feedback data and generates proactive support. Input: Emotion-tagged feedback data in a database. Output: Support notifications and advice.

[0652] Step 9:

[0653] The server performs stress checks on customer support representatives and generates appropriate advice based on the results. Input: Representative conversation content and emotional data. Output: Stress assessment and advice.

[0654] Step 10:

[0655] The server converts the generated advice into voice data using voice synthesis technology and sends it to the person in charge. Input: Stress assessment and advice. Output: Voice message data for the person in charge.

[0656] Step 11:

[0657] The terminal outputs advice to the person in charge by voice, aiming to reduce stress. Input: Voice message data to the person in charge. Output: Voice playback to the person in charge.

[0658] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0659] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0660] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0661] [Second embodiment]

[0662] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0663] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0664] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0665] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0666] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0667] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0668] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0669] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0670] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0671] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0672] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0673] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0674] The present invention is a system that combines speech recognition technology, natural language processing technology, database management technology, and data analysis technology. The purpose of this system is to alleviate the concerns of parents and children and support child-rearing. The following describes in detail the implementation of each function.

[0675] Implementation of QA support functions

[0676] In this system, when a parent or child asks a question, the device uses voice recognition to convert the speech into text data. The device then sends the text data to a server. The server uses natural language processing technology to analyze the text data and retrieves an appropriate answer from a database based on the analysis results. The answer is then sent from the server to the device, which then verbally communicates the answer to the user.

[0677] Example: A child asks, "Why is the sky blue?" The device converts the speech into text and sends it to a server. The server responds, "The sky is blue because sunlight scatters molecules in the atmosphere," and the device then relays the answer.

[0678] Embodiment of growth record saving function

[0679] The system allows parents to record their child's growth by voice. The device uses voice recognition to convert the record into text data, which is then sent to a server. The server stores the text data in a database. Parents can then review the saved growth records later.

[0680] Example: When a parent says, "Today was the first time I rode a bike," the device converts the speech into text and sends it to a server. The server then stores the record that "my child rode a bike for the first time" in a database.

[0681] An embodiment of the health check and data analysis function

[0682] This system periodically collects and analyzes the physical condition data of parents and children. The device collects the data and sends it to a server. The server uses data analysis technology to analyze the data and generates recommendations regarding daily rhythms and next actions based on the analysis results. The recommendations are sent to the device, which then communicates them to the user via audio.

[0683] Example: When a parent types, "I have a headache today," the device sends that data to the server. The server then sends the analysis result, "I recommend you drink plenty of fluids and get plenty of rest," and the device then announces this in voice.

[0684] Stress check and advice function implementation example

[0685] This system records everyday conversations between parents and children and performs stress analysis. The device converts the conversations into text data using voice recognition and sends it to a server. The server then analyzes the text data using stress analysis technology to evaluate the stress level. Based on the evaluation results, the server generates appropriate advice and sends it to the device. The device then communicates the advice to the user via voice.

[0686] Example: If a parent says, "I was really busy at work today," the device converts the conversation into text and sends it to a server. The server then sends the analysis result, "You seem to be stressed. Take a moment to relax and take a deep breath," and the device then conveys this information in voice.

[0687] An embodiment of the complaint listening function

[0688] This system can listen to a parent's complaints and return a message of empathy. The device uses voice recognition to convert the parent's complaints into text data and send it to the server. The server then uses emotion analysis technology to analyze the text data and generate a message of empathy. The message of empathy is then sent to the device, which then relays it to the user via voice.

[0689] Example: When a parent says, "Nothing went well today," the device converts the complaint into text and sends it to a server. The server generates a sympathetic message, saying, "That must have been tough. Take a break and refresh yourself," and the device then relays this message in voice.

[0690] This allows users (parents and children) to easily ask questions or share their frustrations, and also allows them to easily check their health and keep a record of their child's growth. Throughout the system, users receive emotional support, enabling them to raise their children more effectively and with peace of mind.

[0691] The processing flow will be explained below.

[0692] Processing steps for QA-enabled features

[0693] Step 1:

[0694] User: Asks the AI ​​robot aloud, "Why is the sky blue?"

[0695] Step 2:

[0696] Terminal: A voice recognition system is used to convert the user's question from voice data to text data.

[0697] Step 3:

[0698] Terminal: Sends the converted question to the server.

[0699] Step 4:

[0700] Server: Analyzes the received text data using natural language processing technology.

[0701] Step 5:

[0702] Server: Retrieves the appropriate answer from the database based on the analysis results.

[0703] Step 6:

[0704] Server: Sends the obtained answer to the device.

[0705] Step 7:

[0706] Terminal: Converts the received answer into speech and conveys it to the user.

[0707] Growth record saving function processing steps

[0708] Step 1:

[0709] User: Says through the microphone, "Today was my first time riding a bike."

[0710] Step 2:

[0711] Terminal: A speech recognition system is used to convert the user's speech into text data.

[0712] Step 3:

[0713] Terminal: Sends the converted text data to the server.

[0714] Step 4:

[0715] Server: Saves the received text data in a database.

[0716] Step 5:

[0717] Server: Sends a message to the terminal indicating that the save is complete.

[0718] Step 6:

[0719] Terminal: A message will be output to the user indicating that the save is complete.

[0720] Health check and data analysis function processing steps

[0721] Step 1:

[0722] User: Enter "I have a headache today" as physical condition data.

[0723] Step 2:

[0724] Terminal: Sends collected physical condition data to the server.

[0725] Step 3:

[0726] Server: Analyzes the received physical condition data using data analysis technology.

[0727] Step 4:

[0728] Server: Generates recommendations for daily rhythms and activities based on the analysis results.

[0729] Step 5:

[0730] Server: Sends the generated recommendations to the device.

[0731] Step 6:

[0732] Device: Recommendations are spoken to the user.

[0733] Stress check and advice function processing steps

[0734] Step 1:

[0735] User: In everyday conversation, say, "I was really busy at work today."

[0736] Step 2:

[0737] Terminal: Uses a voice recognition system to convert speech into text.

[0738] Step 3:

[0739] Terminal: Sends the converted text data to the server.

[0740] Step 4:

[0741] Server: Analyzes the received text data using stress analysis technology.

[0742] Step 5:

[0743] Server: Evaluates stress levels and generates appropriate advice.

[0744] Step 6:

[0745] Server: Sends the generated advice to the device.

[0746] Step 7:

[0747] Terminal: Advice is given to the user by voice.

[0748] Processing steps of the Complaint Listening feature

[0749] Step 1:

[0750] User: Complains to the AI ​​robot, "Nothing went well today."

[0751] Step 2:

[0752] Terminal: Uses a voice recognition system to convert the complaint from voice data to text data.

[0753] Step 3:

[0754] Terminal: Sends the converted text data to the server.

[0755] Step 4:

[0756] Server: Analyzes the received text data using emotion analysis technology.

[0757] Step 5:

[0758] Server: Generates empathy messages.

[0759] Step 6:

[0760] Server: Sends the generated empathy message to the device.

[0761] Step 7:

[0762] Terminal: Conveys empathetic messages to the user via voice.

[0763] Example 1

[0764] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0765] There is a need for support to quickly and efficiently resolve the questions and concerns that parents and children have while raising children. In particular, there is a need for a wide range of support, such as appropriate answers to questions, management of growth records, and checks on health and stress levels. However, it has been difficult to comprehensively manage all of these in a single system and integrate voice input and output. Furthermore, while it is important to have the ability to show empathy and support when parents complain, existing systems have not been able to adequately address this need.

[0766] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0767] In this invention, the server includes: means for converting questions from a parent or child into text data using voice recognition; means for analyzing the text data using natural language processing technology; and means for retrieving appropriate answers from an information storage device based on the analysis results and outputting the answers audibly. This enables prompt and appropriate answers to user questions. The server also includes means for converting parent or child growth records into text data using voice recognition and storing the text data in an information storage device; and means for managing and displaying the stored growth records. This enables easy recording and management of growth records. The server also includes means for collecting parent or child health data, analyzing the health data using data analysis technology, and generating recommendations regarding lifestyle rhythms and next actions based on the analysis results and outputting the recommendations audibly. This enables appropriate advice regarding improving lifestyle rhythms and next actions to be provided to users. The server also includes means for converting everyday conversations between a parent or child into text data using voice recognition, analyzing the text data using emotion analysis technology, and generating empathy messages based on the analysis results and outputting the empathy messages audibly. This allows for a response that is sensitive to the user's emotions. The system also includes a means for converting the complaints of parents or children into text data using voice recognition and analyzing the text data using emotion analysis technology, and a means for generating a message of sympathy based on the analysis results and outputting the message of sympathy by voice. This makes it possible to provide a sympathetic response even when the user vents their frustrations.

[0768] "Speech recognition" is a technology that converts speech into text data.

[0769] "Character data" is text-format data generated by speech recognition.

[0770] "Natural language processing technology" is a computer programming technology for analyzing and understanding human language.

[0771] "Analysis results" refers to information obtained after analyzing character data using natural language processing technology.

[0772] An "information storage device" is a device for storing and managing data, and specifically refers to a database.

[0773] "Audio output" refers to transmitting text data to the user as audio using speech synthesis technology.

[0774] A "growth record" is data that records information about important events and developments during a child's growth process.

[0775] "Management and display" refers to properly organizing the stored data and presenting it to the user as needed.

[0776] "Physical condition data" refers to information about the health status of parents and children, and is primarily data about physical conditions.

[0777] "Data analysis technology" is the technology for analyzing collected data and deriving useful information.

[0778] "Life rhythm" refers to the patterns of activity and rest in daily life.

[0779] "Recommendations for next steps" are specific guidelines for action suggested based on your current health condition and lifestyle.

[0780] "Daily conversation" refers to the general conversations that parents and children have every day.

[0781] "Emotion analysis technology" is a technology for analyzing the emotions contained in words and sentences and understanding their content.

[0782] An "empathetic message" is a message that is generated in a way that understands and sympathizes with the user's emotions.

[0783] "Complaining" is the act of parents and children verbally expressing the dissatisfaction and stress they feel in their daily lives.

[0784] The present invention is a system that combines technologies of speech recognition, natural language processing, database management, and data analysis to provide support and resolve various concerns and questions that parents and children have while raising children. Specific embodiments of the present invention will be described in detail below.

[0785] Implementation of QA support functions

[0786] In this system, when a user asks a question, the device converts the question into text data using voice recognition technology. Specifically, it uses Google's Cloud Speech-to-Text API. The converted text data is then sent from the device to the server.

[0787] The server analyzes the text data using Google's Natural Language API. Based on the analysis results, the server retrieves the appropriate answer from an information storage device (a database, such as MySQL). The retrieved answer is then sent from the server to the device.

[0788] The device uses Google's Text-to-Speech API to convert the answer into audio and convey it to the user.

[0789] Examples:

[0790] When a user asks, "Why is the sky blue?", the device converts the speech into text using Google's Cloud Speech-to-Text API and sends it to the server. The server then analyzes it using Google's Natural Language API and obtains the answer, "The sky is blue because sunlight scatters molecules in the atmosphere." The device then uses Google's Text-to-Speech API to synthesize the answer into voice and convey it to the user.

[0791] Example prompt sentence:

[0792] When a user asks "Why is the sky blue?", the voice is converted to text using Google's Cloud Speech-to-Text API, the question is analyzed using Google's Natural Language API, and the answer is synthesized using Google's Text-to-Speech API.

[0793] Embodiment of growth record saving function

[0794] When a user enters their child's growth record by voice, the device converts the voice into text using Google's Cloud Speech-to-Text API, which then sends the text data to the server.

[0795] The server stores the received text data in an information storage device (database, for example, MySQL), which the user can later check and manage.

[0796] Examples:

[0797] When a parent says, "Today was my first time riding a bike," the device converts the speech into text using Google's Cloud Speech-to-Text API and sends it to the server, which then stores the record of "my child rode a bike for the first time" in a database.

[0798] Example prompt sentence:

[0799] When a parent says, "Today was my first time riding my bike," Google's Cloud Speech-to-Text API converts the speech to text and stores that text in a MySQL database.

[0800] An embodiment of the health check and data analysis function

[0801] When a user enters their health data, the device sends the data to a server, which then analyzes the data using Python's Pandas library and Scikit-learn and generates appropriate recommendations based on the analysis results.

[0802] The generated recommendations are sent from the server to the device, which then uses Google's Text-to-Speech API to convert the recommendations into audio and convey them to the user.

[0803] Examples:

[0804] When a parent types, "I have a headache today," the device sends that data to a server. The server analyzes the data using Python's Pandas and Scikit-learn, and generates recommendations such as, "I recommend drinking plenty of water and getting plenty of rest." The device then uses Google's Text-to-Speech API to synthesize the recommendations into voice and deliver them to the user.

[0805] Example prompt sentence:

[0806] When a parent types "I have a headache today," the data is sent to the server and analyzed using Python's Pandas and Scikit-learn. The results are then synthesized into speech using Google's Text-to-Speech API.

[0807] Stress check and advice function implementation example

[0808] The system converts the user's everyday conversations into text data using voice recognition technology and sends the text data to a server, which then analyzes the text data using emotion analysis technology to evaluate the user's stress level.

[0809] Based on the evaluation results, the server generates appropriate advice and sends it to the device, which then converts the advice into audio using Google's Text-to-Speech API and conveys it to the user.

[0810] Examples:

[0811] When a parent says, "I was really busy at work today," the device converts the conversation into text and sends it to the server. The server then generates advice, such as, "You seem stressed. Take a moment to relax and take a deep breath." The device then uses Google's Text-to-Speech API to synthesize the advice into voice and relay it to the user.

[0812] Example prompt sentence:

[0813] When a parent says, "I was really busy at work today," emotion analysis technology is used to analyze their stress level, and appropriate advice is synthesized using Google's Text-to-Speech API.

[0814] An embodiment of the complaint listening function

[0815] When a user complains, the device uses voice recognition technology to convert the complaint into text data and sends it to the server. The server then uses emotion analysis technology to analyze the text data and generate a message of empathy.

[0816] The generated empathy message is sent from the server to the device, which then uses Google's Text-to-Speech API to convert the message into audio and convey it to the user.

[0817] Examples:

[0818] When a parent says, "Nothing went well today," the device converts the complaint into text and sends it to the server. The server then generates a message of sympathy, saying, "That must have been tough. Take a break and refresh yourself." The device then uses Google's Text-to-Speech API to synthesize the message of sympathy into voice and deliver it to the user.

[0819] Example prompt sentence:

[0820] When a parent says, "Nothing went well today," emotion analysis technology generates an empathetic message, which is then synthesized into voice using Google's Text-to-Speech API.

[0821] In this way, by combining various cutting-edge technologies, this system is able to provide comprehensive support for the various problems and concerns that users face on a daily basis.

[0822] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0823] Processing steps for QA-enabled features

[0824] Step 1:

[0825] The user asks a question, for example, "Why is the sky blue?"

[0826] Input: Audio data

[0827] Output: None

[0828] Step 2:

[0829] The device converts the voice data into text data using Google's Cloud Speech-to-Text API.

[0830] Input: Audio data

[0831] Output: Character data

[0832] Step 3:

[0833] The character data generated by the terminal is sent to the server using an HTTP request.

[0834] Input: Character data

[0835] Output: HTTP request to the server

[0836] Step 4:

[0837] The server analyzes the received text data using Google's Natural Language API.

[0838] Input: Character data

[0839] Output: Analysis results

[0840] Step 5:

[0841] Based on the analysis results, the server queries the information storage device (database) to obtain the appropriate answer.

[0842] Input: Analysis results

[0843] Output: Response data

[0844] Step 6:

[0845] The server sends the response data to the terminal.

[0846] Input: Answer data

[0847] Output: HTTP response to the device

[0848] Step 7:

[0849] The response data received by the device is converted into audio data using Google's Text-to-Speech API.

[0850] Input: Answer data

[0851] Output: Audio data

[0852] Step 8:

[0853] The terminal plays the audio data to the user.

[0854] Input: Audio data

[0855] Output: Audio output to the user

[0856] Growth record saving function processing steps

[0857] Step 1:

[0858] The user enters their child's growth record by voice, for example, "Today was the first time I rode a bike."

[0859] Input: Audio data

[0860] Output: None

[0861] Step 2:

[0862] The device converts the voice data into text data using Google's Cloud Speech-to-Text API.

[0863] Input: Audio data

[0864] Output: Character data

[0865] Step 3:

[0866] The character data generated by the terminal is sent to the server.

[0867] Input: Character data

[0868] Output: HTTP request to the server

[0869] Step 4:

[0870] The server stores the received character data in an information storage device (database).

[0871] Input: Character data

[0872] Output: Save to database

[0873] Health check and data analysis function processing steps

[0874] Step 1:

[0875] The user inputs data about his / her physical condition. For example, he / she inputs "I have a headache today."

[0876] Input: Audio data (or text data)

[0877] Output: None

[0878] Step 2:

[0879] The device converts the voice data into text data and sends it to the server. If the data is text, it is sent as is.

[0880] Input: Audio or text data

[0881] Output: HTTP request to the server

[0882] Step 3:

[0883] The health data received by the server is analyzed using Python's Pandas library and Scikit-learn.

[0884] Input: Physical condition data

[0885] Output: Analysis results

[0886] Step 4:

[0887] The server generates recommendations regarding daily routines and next actions based on the analysis results.

[0888] Input: Analysis results

[0889] Output: Recommendations

[0890] Step 5:

[0891] The server sends the recommendations to the device.

[0892] Input: Recommendations

[0893] Output: HTTP response to the device

[0894] Step 6:

[0895] The device converts the recommendations it receives into audio data using Google's Text-to-Speech API.

[0896] Input: Recommendations

[0897] Output: Audio data

[0898] Step 7:

[0899] The terminal plays the audio data to the user.

[0900] Input: Audio data

[0901] Output: Audio output to the user

[0902] Stress check and advice function processing steps

[0903] Step 1:

[0904] The user engages in everyday conversation, for example, saying, "I was really busy at work today."

[0905] Input: Audio data

[0906] Output: None

[0907] Step 2:

[0908] The device converts the voice data into text data using Google's Cloud Speech-to-Text API.

[0909] Input: Audio data

[0910] Output: Character data

[0911] Step 3:

[0912] The character data generated by the terminal is sent to the server.

[0913] Input: Character data

[0914] Output: HTTP request to the server

[0915] Step 4:

[0916] The server analyzes the received text data using emotion analysis technology.

[0917] Input: Character data

[0918] Output: Analysis results (stress level)

[0919] Step 5:

[0920] The server generates appropriate advice based on the analysis results.

[0921] Input: Analysis results

[0922] Output: Advice data

[0923] Step 6:

[0924] The server transmits the advice data to the terminal.

[0925] Input: Advice data

[0926] Output: HTTP response to the device

[0927] Step 7:

[0928] The advice data received by the device is converted into audio data using Google's Text-to-Speech API.

[0929] Input: Advice data

[0930] Output: Audio data

[0931] Step 8:

[0932] The terminal plays the audio data to the user.

[0933] Input: Audio data

[0934] Output: Audio output to the user

[0935] Processing steps of the Complaint Listening feature

[0936] Step 1:

[0937] The user complains, for example, "Nothing went well today."

[0938] Input: Audio data

[0939] Output: None

[0940] Step 2:

[0941] The device converts the voice data into text data using Google's Cloud Speech-to-Text API.

[0942] Input: Audio data

[0943] Output: Character data

[0944] Step 3:

[0945] The character data generated by the terminal is sent to the server.

[0946] Input: Character data

[0947] Output: HTTP request to the server

[0948] Step 4:

[0949] The server analyzes the received text data using emotion analysis technology.

[0950] Input: Character data

[0951] Output: Analysis results (emotion data)

[0952] Step 5:

[0953] The server generates a message of sympathy based on the analysis results.

[0954] Input: Analysis results

[0955] Output: Empathy message

[0956] Step 6:

[0957] The server sends a sympathy message to the terminal.

[0958] Input: Sympathy message

[0959] Output: HTTP response to the device

[0960] Step 7:

[0961] The device converts the empathy message received into audio data using Google's Text-to-Speech API.

[0962] Input: Sympathy message

[0963] Output: Audio data

[0964] Step 8:

[0965] The terminal plays the audio data to the user.

[0966] Input: Audio data

[0967] Output: Audio output to the user

[0968] (Application example 1)

[0969] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0970] In modern society, parents face the challenge of a lack of support when going out with their children and shopping in brick-and-mortar stores. In particular, parents have limited means to easily gather information about their children's health, growth records, and daily concerns, and to receive appropriate advice. Furthermore, there are insufficient systems in place to alleviate parents' own stress and worries, making it an urgent task to address these issues.

[0971] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0972] In this invention, the server includes: means for converting questions from a parent or child into text data using voice recognition; means for analyzing the text data using natural language processing technology; and means for retrieving appropriate answers from a database based on the analysis results and outputting the answers audibly. This allows parents to ask questions or seek advice about child-rearing in real time while shopping at a physical store and receive appropriate answers. The server also includes means for collecting physical condition data of the parent or child and analyzing the physical condition data using data analysis technology; and means for generating recommendations regarding daily rhythms and next actions based on the analysis results and outputting the recommendations audibly. This allows parents to understand their child's health status and take appropriate measures even while shopping at a physical store. The server also includes means for recording daily conversations between the parent or child and analyzing the conversation data using stress analysis technology; means for generating appropriate advice based on the analysis results and outputting the advice audibly; and means for allowing the parent or child to input complaints audibly, generating empathetic messages using emotion analysis technology, and outputting the messages audibly. This reduces stress and worries for parents and improves overall child-rearing support.

[0973] definition statement

[0974] "Speech recognition" is a technology that receives speech as input and automatically converts that speech into text data.

[0975] "Character data" refers to data in text format obtained by speech recognition.

[0976] "Natural language processing technology" is a technology that enables computers to understand, analyze, and generate natural human language.

[0977] The "analysis results" are the results of analyzing character data using natural language processing technology.

[0978] A "database" is a system that allows data to be stored, searched, and managed.

[0979] "Audio output means" refers to technology that provides analysis results and answers to users as audio.

[0980] "Physical condition data" refers to data indicating the health condition of a parent or child.

[0981] "Data analysis technology" is a technology for analyzing collected data and extracting useful information.

[0982] "Life rhythm" refers to an individual's activity patterns and time allocation in their daily lives.

[0983] "Recommendations for next actions" are specific actions or advice suggested based on the analysis results.

[0984] "Daily conversation" refers to the verbal exchanges that parents or children have on a daily basis.

[0985] "Stress analysis technology" is a technology that evaluates an individual's stress level based on conversation and behavioral data.

[0986] "Advice" refers to advice or suggestions provided based on the results of the analysis.

[0987] "Guchi" is a word that expresses dissatisfaction or anxiety felt by parents or children.

[0988] "Emotion analysis technology" is a technology that evaluates the emotional state of text data and generates emotional responses based on that data.

[0989] An "empathetic message" is a message that shows understanding and empathy for the user's feelings.

[0990] MODE FOR CARRYING OUT THE INVENTION

[0991] The present invention is a child-rearing support system that uses voice recognition technology, natural language processing technology, database management technology, and data analysis technology. This system provides various child-rearing-related services when parents and children enjoy shopping at physical stores. Specific embodiments are described below.

[0992] System Configuration and Hardware

[0993] This system consists of a smartphone, a cloud server, a voice recognition engine, a natural language processing engine, a database, a text-to-speech engine, etc.

[0994] The smartphone is a device on which a dedicated app is installed and which directly interacts with parents or children. The cloud server processes various data and provides appropriate information to parents or children.

[0995] Specific roles and operations of each technology

[0996] Speech recognition: When a user (parent or child) speaks into a smartphone, a speech recognition engine (such as the Google Speech-to-Text API) converts the speech into text data.

[0997] Natural language processing: Text data is sent to a cloud server and analyzed using a natural language processing engine (such as Google Cloud Natural Language API). For example, if a user says, "It looks like my child has caught a cold," the cloud server analyzes this text data and understands its meaning.

[0998] Database management: Based on the results analyzed by the natural language processing engine, the cloud server retrieves appropriate answers and support information from a database (such as MongoDB). For example, advice such as "Give the child plenty of warm drinks, and if symptoms persist, consult a pediatrician" may be obtained.

[0999] Voice output: The obtained answer is converted into voice by a text-to-speech engine (such as Google Text-to-Speech API) and is transmitted to the user through the smartphone speaker.

[1000] Example of childcare support function

[1001] 1. Question and Answering

[1002] When a user speaks to their smartphone, "My child has a fever. What should I do?", the speech recognition engine converts the speech into text data and sends it to a cloud server. The cloud server then uses natural language processing technology to analyze the text and retrieves the appropriate answer from a database. The answer, "Ensure adequate hydration and consult a doctor if necessary," is then played back as audio by the text-to-speech engine.

[1003] 2. Keeping a record of growth

[1004] When a user says, "Today was my first time riding a bicycle," the speech recognition engine converts this into text data and sends it to the cloud server, which then stores this data in a database and manages it for later review.

[1005] 3. Health check and advice

[1006] When a user says "I have a headache," the speech recognition engine converts this into text data and sends it to the cloud server. The cloud server analyzes this data and generates advice such as "We recommend you drink plenty of fluids and get some rest," which is then conveyed to the user via voice.

[1007] 4. Stress check and emotion analysis

[1008] The system continuously records the user's conversations and periodically analyzes their stress levels. For example, if a user says, "Work was very hard today," the cloud server will evaluate their stress level and provide advice such as, "Take a break to relax."

[1009] Prompt Sentence Examples

[1010] A user might ask, "My child has a cough. What should I do?" and use the following prompt:

[1011] I would like to answer questions about children's health. For example, "Why is the sky blue?" I would like to answer something like, "The sky is blue because sunlight scatters molecules in the atmosphere."

[1012] This provides an environment where parents and children can focus on raising their children with peace of mind while enjoying shopping in a physical store.

[1013] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1014] Program processing steps

[1015] Step 1:

[1016] The user (parent or child) enters questions or information into the smartphone by voice. For example, the user might say, "My child seems to have caught a cold." This voice data is then entered into the system.

[1017] Step 2:

[1018] The device uses a speech recognition engine (Google Speech-to-Text API) to convert the voice data from step 1 into text data, which then becomes the input for the next step.

[1019] Step 3:

[1020] The device formats the text data as a preprocessing step and sends it to the cloud server. The cloud server receives this text data and analyzes it using a natural language processing engine (Google Cloud Natural Language API). As a result of the analysis, the meaning and intent of the text data are extracted.

[1021] Step 4:

[1022] Based on the analysis results, the server retrieves the appropriate answer corresponding to the question from a database (MongoDB). This database contains a wide variety of questions and their answers in advance. For example, the answer retrieved might be, "Have the child drink plenty of warm fluids, and if symptoms persist, consult a pediatrician."

[1023] Step 5:

[1024] The server sends the answer retrieved from the database to a text-to-speech engine (Google Text-to-Speech API), which converts the answer into audio data, which becomes the input for the next step.

[1025] Step 6:

[1026] The device receives the voice data sent from the text-to-speech engine and conveys it to the user. Specifically, the smartphone speaker plays a voice message saying, "Give your child plenty of warm drinks, and if symptoms persist, consult a pediatrician."

[1027] Step 7:

[1028] The user receives advice and answers from the app and takes appropriate action. At this step, the user gets a solution to their problem.

[1029] Step 8:

[1030] If necessary, the user can again input their daily conversations, physical condition, and growth records by voice, which the system will record and analyze. This step is important so that the system can continuously monitor the user's condition and provide appropriate advice.

[1031] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1032] This invention is based on an AI system that interacts with parents or children, and incorporates an emotion engine to recognize and respond to the user's emotions. This system combines voice recognition technology, natural language processing technology, database management technology, data analysis technology, and emotion engine technology to address various issues faced by parents and children.

[1033] An embodiment of emotion recognition QA function

[1034] In this system, when a parent or child asks a question, the device uses a voice recognition system to convert the speech into text data. The text data is then sent to a server. The server uses natural language processing technology and an emotion engine to analyze the text and recognize the user's emotional state. Based on this emotional state, the server generates an appropriate response or a tone-adjusted empathetic message, which is then sent from the server to the device. The device then communicates the generated response or empathetic message to the user via voice.

[1035] For example, if a child asks, "Why do I have to go to school?", the device converts the speech into text and sends it to a server. The server recognizes the child's anxiety and stress contained in the question and generates a tailored response: "School is a place where you can make new friends and have fun learning. Don't worry, just relax and have fun," which the device then relays.

[1036] An embodiment of a growth record keeping function using emotion recognition

[1037] The system also uses the emotion engine when parents enter their child's growth record by voice. The device converts the voice into text data, running the emotion engine to send the entered content and emotional state to the server. The server stores the received data in a database and, if necessary, tags the emotional state. Parents can later review the saved growth record and easily search for important events in which a specific emotional state was recorded.

[1038] Example: When a parent says, "Today I was so happy because I rode my bike for the first time," the device converts the speech into text, recognizes the "happy" emotion using an emotion engine, and sends this information to the server. The server then saves the record of "my child rode a bike for the first time" in a database with a tag of "happy."

[1039] An embodiment of a health check and data analysis function using emotion recognition

[1040] This system periodically collects and analyzes the physical condition data of parents and children, while also analyzing the user's emotional state using an emotion engine. The device collects emotional data along with the physical condition data and sends it to a server. The server analyzes the data using data analysis technology and the emotion engine, and based on the analysis results, generates recommendations for daily rhythms and activities, as well as advice for psychological support. The recommendations and advice are sent to the device, which then communicates them to the user via voice.

[1041] Example: If a parent types, "I have a headache today and I'm feeling a bit down," the device will send that data to a server. The server will then send an analysis result, such as, "Drink plenty of fluids, get some rest, and listen to your favorite music to change your mood," and the device will then announce this in voice.

[1042] An embodiment of a stress check and advice function using emotion recognition

[1043] This system records everyday conversations between parents and children and uses an emotion engine to perform stress analysis. The device uses voice recognition to convert the conversation into text data, analyzes the emotional state using the emotion engine, and sends this to a server. The server uses both stress analysis technology and the emotion engine to evaluate stress levels and generate appropriate advice. The evaluation results and advice are sent to the device, which then communicates them to the user via voice.

[1044] Example: If a parent says, "I'm really busy at work today and I'm tired," the device converts the conversation into text, recognizes the "tired" emotion using an emotion engine, and sends it to the server. The server then sends the analysis results, saying, "You seem to be stressed. Take some time to relax and take a deep breath," and the device then conveys this in voice.

[1045] Embodiment of a function for listening to complaints using emotion recognition

[1046] This system also uses the emotion engine when a parent complains. The device uses voice recognition to convert the complaint into text data, analyzes the emotion using the emotion engine, and sends this to the server. The server then uses emotion analysis technology to generate an empathy message, which is tailored based on the parent's emotional state. The empathy message is then sent to the device, which then relays it to the user via voice.

[1047] Example: When a parent complains, "Nothing went well today," the device converts the complaint into text, recognizes the "disappointment" in its emotion engine, and sends it to the server. The server then generates a sympathetic message, saying, "That must have been really tough. Get some rest and start tomorrow with a fresh attitude," and the device then relays this message in voice.

[1048] This allows users (parents and children) to easily ask questions or express their frustrations, and also allows for easy health checks and growth records to be kept. Furthermore, emotion recognition technology allows for more sympathetic responses, providing greater emotional support. Throughout the system, users can raise their children in a comfortable environment.

[1049] The processing flow will be explained below.

[1050] Processing steps for emotion recognition QA function

[1051] Step 1:

[1052] User: Asks the AI ​​robot aloud, "Why do I have to go to school?"

[1053] Step 2:

[1054] Terminal: A voice recognition system is used to convert the user's question from voice data to text data.

[1055] Step 3:

[1056] Terminal: Sends the converted text data to the server.

[1057] Step 4:

[1058] Server: Analyzes the received text data using natural language processing technology and an emotion engine to recognize the user's emotions.

[1059] Step 5:

[1060] Server: Based on the user's emotions, retrieves appropriate answers from the database and generates empathetic messages with adjusted tone.

[1061] Step 6:

[1062] Server: Sends the generated answer and sympathy message to the device.

[1063] Step 7:

[1064] Terminal: Converts the received answers and sympathy messages into audio and conveys them to the user.

[1065] Processing steps for growth record keeping function using emotion recognition

[1066] Step 1:

[1067] User: Through the microphone, says, "Today was my first time riding a bike and I was so happy."

[1068] Step 2:

[1069] Terminal: A speech recognition system is used to convert the user's speech into text data.

[1070] Step 3:

[1071] Terminal: Sends the converted text data to the server.

[1072] Step 4:

[1073] Server: The received text data is analyzed using an emotion engine to recognize the emotion "happy."

[1074] Step 5:

[1075] Server: Stores the text data along with the recognized emotions in a database.

[1076] Step 6:

[1077] Server: Sends a message to the terminal indicating that the save is complete.

[1078] Step 7:

[1079] Terminal: A message will be output to the user indicating that the save is complete.

[1080] Processing steps for health check and data analysis functions using emotion recognition

[1081] Step 1:

[1082] User: Enters "I have a headache today and feel a little depressed" as health data.

[1083] Step 2:

[1084] Terminal: Sends collected physical condition data to the server.

[1085] Step 3:

[1086] Server: Analyzes the received physical condition data and emotion data using data analysis technology and an emotion engine.

[1087] Step 4:

[1088] Server: Generates recommendations for daily routines and behaviors based on the analysis results, and also generates advice for psychological support.

[1089] Step 5:

[1090] Server: Sends the generated recommendations and advice to the device.

[1091] Step 6:

[1092] Device: Provides recommendations and advice to the user via voice.

[1093] Processing steps for stress check and advice function using emotion recognition

[1094] Step 1:

[1095] User: In everyday conversation, say, "I'm really busy at work today and I'm tired."

[1096] Step 2:

[1097] Terminal: Uses a voice recognition system to convert speech into text.

[1098] Step 3:

[1099] Terminal: Sends the converted text data to the server.

[1100] Step 4:

[1101] Server: Analyzes the received text data using stress analysis technology and an emotion engine to evaluate stress levels and emotional states.

[1102] Step 5:

[1103] Server: Generates appropriate advice based on the evaluation results.

[1104] Step 6:

[1105] Server: Sends the generated advice to the device.

[1106] Step 7:

[1107] Terminal: Advice is given to the user by voice.

[1108] Processing steps for listening to complaints using emotion recognition

[1109] Step 1:

[1110] User: Complains to the AI ​​robot, "Nothing went well today."

[1111] Step 2:

[1112] Terminal: Uses a voice recognition system to convert the complaint from voice data to text data.

[1113] Step 3:

[1114] Terminal: Sends the converted text data to the server.

[1115] Step 4:

[1116] Server: Analyzes the received text data using an emotion engine and recognizes "disappointment."

[1117] Step 5:

[1118] Server: Generates empathetic messages based on emotional states.

[1119] Step 6:

[1120] Server: Sends the generated empathy message to the device.

[1121] Step 7:

[1122] Terminal: Conveys empathetic messages to the user via voice.

[1123] Example 2

[1124] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1125] Conventional conversational AI systems simply convert user questions and comments about their physical condition into text data and generate answers. This makes it difficult to respond in a way that takes into account the user's emotional state and psychological aspects, resulting in insufficient psychological support for the user. Furthermore, when storing and managing growth records and physical condition data, data that includes the user's emotional state is not recorded, making it difficult to fully grasp important events or changes in the user's psychological state.

[1126] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1127] In this invention, the server includes: a means for converting questions from a parent or child into text data using voice recognition; a means for analyzing the text data using natural language processing; an emotion analysis means for recognizing the user's emotional state based on the analysis results; a means for generating an appropriately adjusted answer or empathetic message based on the recognized emotional state; and a means for outputting the generated answer or empathetic message by voice. This enables more friendly responses and psychological support tailored to the user's emotional state. Furthermore, growth records and physical condition data, including emotional states, can be saved and managed, making it easy to understand important events and psychological changes.

[1128] "Speech recognition" is a technology that analyzes voice signals and converts them into text data.

[1129] "Text data" refers to data that includes text information converted by voice recognition.

[1130] "Natural language processing" is a technology that allows computers to understand, analyze, and generate human language.

[1131] "Emotion analysis means" is a technology that recognizes the emotional state of a user from the content of their comments.

[1132] A "database" is a system for efficiently managing, storing, and searching large amounts of data.

[1133] "Data analysis technology" is a technology for analyzing collected data and extracting useful information.

[1134] "Recommendations" are suggestions about behaviors and lifestyle habits provided to users based on the analysis results.

[1135] A "sympathetic message" is a message that expresses sympathy and is generated in accordance with the user's emotional state.

[1136] "Audio output means" refers to a technology that converts text data into audio and conveys it to the user.

[1137] A "growth record" is a record that stores data about a parent's or child's growth through voice recognition and emotion analysis.

[1138] "Physical condition data" is data that includes information related to the user's health condition.

[1139] "Emotional support" refers to providing advice and messages to help users achieve emotional stability and comfort.

[1140] MODE FOR CARRYING OUT THE INVENTION

[1141] This invention is based on an AI system that interacts with parents or children, and incorporates an emotion analysis engine to recognize and respond to the user's emotions. This system combines voice recognition technology, natural language processing technology, database management technology, data analysis technology, and emotion analysis technology to address various issues faced by parents and children.

[1142] Components

[1143] Speech recognition technology: Converts speech into text data. For example, speech recognition software (e.g., Google Cloud Speech-to-Text) is used.

[1144] Natural language processing technology: Analyzes text data and understands its meaning. For example, natural language processing software (e.g., NLP API) is used.

[1145] Database management technology: Technology for effectively storing and managing converted data and analysis results. For example, a database management system (e.g., MySQL) is used.

[1146] Data analysis technology: Technology to analyze collected physical condition data and growth records and extract information useful to users. For example, data analysis libraries (e.g., Pandas, NumPy) are used.

[1147] Emotion analysis technology: Technology that recognizes the user's emotional state and generates appropriate responses based on that information. As a specific example, we will use an emotion analysis engine (e.g., Affectiva API).

[1148] Program processing

[1149] 1. Speech recognition process: The device converts the user's questions or speech into text data using speech recognition software, such as Google Cloud Speech-to-Text.

[1150] 2. Data transmission process: The converted text data is sent to the server, which uses HTTP requests and responses to analyze the received data.

[1151] 3. Text analysis process: The server uses natural language processing technology to analyze the text data and understand its contents. Natural language processing APIs (e.g., Google Cloud Natural Language API) are used to understand the meaning of the text.

[1152] 4. Sentiment Analysis Process: Based on the analysis results, the server uses sentiment analysis technology to recognize the user's emotional state. It uses a sentiment analysis engine (such as Affectiva API) to specifically identify the user's emotions.

[1153] 5. Answer generation process: Based on the recognized emotional state, the server uses a generative AI model (e.g., OpenAI's GPT model) to generate appropriate answers and empathetic messages, adjusting the tone and content according to the emotional state.

[1154] 6. Database storage process: The generated answers and emotional state data are stored in a database with appropriate tagging. A database management system such as MySQL is used for this purpose.

[1155] 7. Voice output process: Finally, the generated answers and messages are sent to the device and audibly conveyed to the user using speech synthesis technology (e.g., Amazon Polly).

[1156] Examples of concrete examples and prompts

[1157] Example 1:

[1158] When a user asks, "Why do I have to go to school?", the device converts the speech into text and sends it to the server. The server recognizes the user's anxiety and stress contained in the question and generates a tailored response: "School is a place where you can make new friends and have fun learning. Don't worry, just relax and have fun." The device then delivers the response via voice.

[1159] Example prompt sentence:

[1160] "Why do I have to go to school?"

[1161] Example 2:

[1162] When a parent says, "Today I was so happy to ride a bike for the first time," the device converts the speech into text, recognizes the "happy" emotion using an emotion engine, and sends this information to the server, which then stores the record of "my child's first bike ride" in a database with a tag of "happy."

[1163] Example prompt sentence:

[1164] "Today was the first time I rode a bike."

[1165] This invention allows users to receive friendly responses and support according to their emotions, and also makes it possible to store and manage growth records and physical condition data together with their emotional state.

[1166] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1167] Specific flow of program processing

[1168] Emotion-aware QA function

[1169] Step 1:

[1170] User utterance input

[1171] The user asks a question to the device. Specifically, they say, "Why do I have to go to school?" This input is treated as voice data.

[1172] Step 2:

[1173] Speech-to-text

[1174] The device uses speech recognition software (e.g., Google Cloud Speech-to-Text) to convert the user's voice into text data. An example of the converted output is the text data "Why do I have to go to school?"

[1175] Step 3:

[1176] Sending text to the server

[1177] The terminal sends the converted text data to the server via an HTTP request. The input is the converted text data, and the output is a notification to the server that transmission has been completed.

[1178] Step 4:

[1179] Text data analysis

[1180] The text data received by the server is analyzed using natural language processing technology (e.g., Google Cloud Natural Language API). The input is text data, and the output is the semantic and structural information of the analyzed text. Specifically, the question "Why do I have to go to school?" is analyzed, and its meaning and structure are understood.

[1181] Step 5:

[1182] Recognition of emotional states

[1183] The server uses emotion analysis technology (e.g., Affectiva's Emotion Analysis API) to recognize the user's emotional state based on the analysis results. The input is the analyzed text data, and the output is the identified emotional state (e.g., anxiety or stress).

[1184] Step 6:

[1185] Emotion-based answer generation

[1186] The server uses a generative AI model (e.g., OpenAI's GPT model) to generate an appropriate answer or empathetic message based on the emotional state. The input is the identified emotional state and text data, and the output is a generated answer or empathetic message. For example, a generated answer might be, "School is a place where you can make new friends and have fun learning. Don't worry, just relax and have fun."

[1187] Step 7:

[1188] Sending answers to the device

[1189] The server sends the generated answer to the terminal as an HTTP response. The input is the generated answer data, and the output is a notification of completion of transmission to the terminal.

[1190] Step 8:

[1191] Voice output of answers

[1192] The device uses speech synthesis technology (e.g., Amazon Polly) to convert the text-based answer into speech and convey it to the user. The input is the generated answer text, and the output is audio data. Specifically, the answer "School is a place where you can make new friends and have fun learning. Don't worry, just take your time and have fun" is provided to the user via voice.

[1193] ---

[1194] Emotion recognition-based growth record keeping function

[1195] Step 1:

[1196] User utterance input

[1197] A parent talks to the device about their child's growth. For example, they might say, "Today was my first time riding a bike, and I was so happy." The input is treated as voice data.

[1198] Step 2:

[1199] Speech-to-text

[1200] The device uses voice recognition software to convert the parent's speech into text data, and the output is the converted text data.

[1201] Step 3:

[1202] Server transmission of text and emotions

[1203] The device sends the converted text data and the emotion analysis results from the emotion analysis engine to the server. The input is the text data and emotion data, and the output is a notification of completion of transmission to the server.

[1204] Step 4:

[1205] Text and sentiment storage

[1206] The server temporarily stores the received text data and emotional state in a database. The input is the received data, and the output is a notification that the data has been saved to the database.

[1207] Step 5:

[1208] Data tagging

[1209] The server assigns a tag of "happy" as an emotional state. The input is text data and emotional data, and the output is tagged data.

[1210] Step 6:

[1211] Saving to a database

[1212] The server saves the tagged text data to a database. The input is the tagged text data, and the output is a notification that the data has been permanently saved to the database.

[1213] Step 7:

[1214] Checking the data

[1215] When a parent wants to check the growth record later, they can search the saved data through the device's user interface, filter it by a specific emotional state (e.g., "happy"), and display it. The input is the search criteria, and the output is the search results. Specifically, the record of "my child riding a bicycle for the first time" is displayed with the tag "happy."

[1216] ---

[1217] Health check and data analysis function using emotion recognition

[1218] Step 1:

[1219] User utterance input

[1220] The user speaks to the device about their physical condition. Specifically, they say, "I have a headache today, and I'm feeling a little depressed." The input is treated as voice data.

[1221] Step 2:

[1222] Speech-to-text and sentiment analysis

[1223] The device uses voice recognition software to convert speech into text data, and then uses an emotion analysis engine to analyze the emotional state, with the output being the converted text data and emotion data.

[1224] Step 3:

[1225] Sending data to the server

[1226] The terminal sends the converted text data and emotion data to the server. The input is the text data and emotion data, and the output is a notification of completion of transmission to the server.

[1227] Step 4:

[1228] Data integration

[1229] The server integrates the physical condition data and emotional data. The input is the collected data, and the output is the integrated data.

[1230] Step 5:

[1231] Data analysis

[1232] The server analyzes the integrated data using data analysis techniques (Python's Pandas and NumPy). The input is the integrated data, and the output is the analysis results. Specifically, the server evaluates the user's health status and emotional state.

[1233] Step 6:

[1234] Generating Advice

[1235] Based on the analysis results, the server generates recommendations for daily routines and behaviors, as well as advice for psychological support. The input is the analysis results, and the output is the generated advice. For example, the generated advice might be, "Drink plenty of water, get plenty of rest, and listen to your favorite music to change your mood."

[1236] Step 7:

[1237] Sending advice to the device

[1238] The server sends the generated advice to the terminal. The input is the generated advice, and the output is a notification of the completion of the transmission to the terminal.

[1239] Step 8:

[1240] Audio output of advice

[1241] The device uses voice synthesis technology such as Amazon Polly to convert the advice into speech and convey it to the user. The input is the generated advice text, and the output is audio data. Specifically, the advice provided to the user is "Drink plenty of water, get plenty of rest, and listen to your favorite music to change your mood."

[1242] (Application example 2)

[1243] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1244] Conventional food delivery systems face challenges in achieving sufficient customer satisfaction in customer service and complaint handling. Furthermore, there are also issues with the lack of effective ways to utilize customer feedback and measures to reduce the stress of customer support staff. This can lead to delays in improving service quality and customer satisfaction.

[1245] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1246] In this invention, the server includes means for converting questions from a parent or child into text data using voice recognition, means for analyzing the text data using natural language processing technology, means for retrieving an appropriate answer from a database based on the analysis result and outputting the answer by voice, means for recognizing emotions and generating an empathy message in an appropriate tone based on the analysis result, means for outputting the empathy message by voice, means for saving feedback from customers with emotion tags, and means for analyzing the feedback and providing proactive support. This improves the quality of customer service, effectively utilizes feedback, reduces the burden on customer support staff through stress checks, and enables overall improvement of service quality.

[1247] "Speech recognition" is a technology that analyzes speech and converts it into text data.

[1248] "Natural language processing" is a technology that uses computers to analyze and generate natural language.

[1249] "Emotion recognition" is a technology that detects a user's emotional state from voice or text.

[1250] A "database" is a system for systematically storing and managing data.

[1251] "Data analysis technology" is the technology for extracting useful information and patterns from large amounts of data.

[1252] A "sympathetic message" is a message with sympathetic content that is generated in accordance with the user's emotional state.

[1253] "Feedback" refers to opinions and ratings provided by users.

[1254] An "emotion tag" is tag information that indicates an emotional state that is assigned to feedback or data.

[1255] "Proactive support" is a support method that takes preventative measures before a problem occurs.

[1256] "Stress check" is a technique for assessing the stress level of users and staff.

[1257] System program generation

[1258] The system embodying this invention converts questions and feedback from parents or children into text data using voice recognition and incorporates an AI engine that recognizes emotions. This improves the quality of customer service and makes it easier to utilize feedback. It also provides a means to reduce stress for customer support representatives.

[1259] Program processing explanation

[1260] First, we use the SpeechRecognition library for speech recognition, which converts the voice input from the parent or child into text data. Then, we use the HuggingFace transformers library to analyze emotions from the text data using the bhadresh-savani / bert-base-uncased-emotion emotion recognition model.

[1261] The server uses natural language processing technology to generate empathetic messages based on the analyzed emotional data. The generated messages are then delivered to the user via voice synthesis technology. Customer feedback is tagged with emotion and stored in a database.

[1262] Furthermore, the server periodically analyzes the conversations of customer support representatives and performs stress checks. Based on the results, it generates appropriate advice and provides it to the representative.

[1263] This system will improve the quality of customer service and provide proactive support through stress checks.

[1264] Specific examples

[1265] For example, suppose a customer complains that "today's delivery was too late." At this time, the voice recognition system converts the voice into text, and the emotion recognition AI analyzes "anger." The server generates an empathetic message and replies, "We apologize for keeping you waiting. We will check it right away, so please wait a little longer." This allows the customer to receive an appropriate response, improving their satisfaction.

[1266] Customer interaction history is saved in a database with an "anger" tag attached. At a later date, a stress check is conducted and the person in charge is given advice such as "Take time to relax today."

[1267] Prompt Sentence Examples

[1268] text

[1269] Customer enquiry: "Delivery is late" --> Analyse their sentiment.

[1270] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1271] Step 1:

[1272] The user speaks a question or complaint. The device receives this speech and sends it to the speech recognition process. Input: User's speech data. Output: Input data to the speech recognition model.

[1273] Step 2:

[1274] The device uses the SpeechRecognition library to convert the speech data to text, which is then sent to the next processing step: Input: Input data to the speech recognition model. Output: Text data of the speech.

[1275] Step 3:

[1276] The server uses the HuggingFace transformers library to perform emotion recognition on the text data. It uses the bhadresh-savani / bert-base-uncased-emotion model to analyze the emotions contained in the text. Input: Speech text data. Output: Emotion analysis results.

[1277] Step 4:

[1278] The server generates a message of empathy based on the results of the sentiment analysis using natural language processing technology. The generated message is used in the next step. Input: Sentiment analysis result. Output: Message of empathy.

[1279] Step 5:

[1280] The server converts the empathy message into voice data using voice synthesis technology. This voice data is sent to the terminal and output to the user. Input: Empathy message. Output: Voice message data.

[1281] Step 6:

[1282] The terminal plays back the voice message and outputs it to the user, who receives the message.

[1283] Input: Voice message data

[1284] Output: Playing audio to the user

[1285] Step 7:

[1286] The server stores the user feedback in a database with emotion tags. Input: User feedback and emotion data. Output: Emotion-tagged feedback database entries.

[1287] Step 8:

[1288] The server periodically analyzes the feedback data and generates proactive support. Input: Emotion-tagged feedback data in a database. Output: Support notifications and advice.

[1289] Step 9:

[1290] The server performs stress checks on customer support representatives and generates appropriate advice based on the results. Input: Representative conversation content and emotional data. Output: Stress assessment and advice.

[1291] Step 10:

[1292] The server converts the generated advice into voice data using voice synthesis technology and sends it to the person in charge. Input: Stress assessment and advice. Output: Voice message data for the person in charge.

[1293] Step 11:

[1294] The terminal outputs advice to the person in charge by voice, aiming to reduce stress. Input: Voice message data to the person in charge. Output: Voice playback to the person in charge.

[1295] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1296] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1297] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1298] [Third embodiment]

[1299] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1300] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1301] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1302] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1303] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1304] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1305] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1306] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1307] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1308] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1309] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1310] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1311] The present invention is a system that combines speech recognition technology, natural language processing technology, database management technology, and data analysis technology. The purpose of this system is to alleviate the concerns of parents and children and support child-rearing. The following describes in detail the implementation of each function.

[1312] Implementation of QA support functions

[1313] In this system, when a parent or child asks a question, the device uses voice recognition to convert the speech into text data. The device then sends the text data to a server. The server uses natural language processing technology to analyze the text data and retrieves an appropriate answer from a database based on the analysis results. The answer is then sent from the server to the device, which then verbally communicates the answer to the user.

[1314] Example: A child asks, "Why is the sky blue?" The device converts the speech into text and sends it to a server. The server responds, "The sky is blue because sunlight scatters molecules in the atmosphere," and the device then relays the answer.

[1315] Embodiment of growth record saving function

[1316] The system allows parents to record their child's growth by voice. The device uses voice recognition to convert the record into text data, which is then sent to a server. The server stores the text data in a database. Parents can then review the saved growth records later.

[1317] Example: When a parent says, "Today was the first time I rode a bike," the device converts the speech into text and sends it to a server. The server then stores the record that "my child rode a bike for the first time" in a database.

[1318] An embodiment of the health check and data analysis function

[1319] This system periodically collects and analyzes the physical condition data of parents and children. The device collects the data and sends it to a server. The server uses data analysis technology to analyze the data and generates recommendations regarding daily rhythms and next actions based on the analysis results. The recommendations are sent to the device, which then communicates them to the user via audio.

[1320] Example: When a parent types, "I have a headache today," the device sends that data to the server. The server then sends the analysis result, "I recommend you drink plenty of fluids and get plenty of rest," and the device then announces this in voice.

[1321] Stress check and advice function implementation example

[1322] This system records everyday conversations between parents and children and performs stress analysis. The device converts the conversations into text data using voice recognition and sends it to a server. The server then analyzes the text data using stress analysis technology to evaluate the stress level. Based on the evaluation results, the server generates appropriate advice and sends it to the device. The device then communicates the advice to the user via voice.

[1323] Example: If a parent says, "I was really busy at work today," the device converts the conversation into text and sends it to a server. The server then sends the analysis result, "You seem to be stressed. Take a moment to relax and take a deep breath," and the device then conveys this information in voice.

[1324] An embodiment of the complaint listening function

[1325] This system can listen to a parent's complaints and return a message of empathy. The device uses voice recognition to convert the parent's complaints into text data and send it to the server. The server then uses emotion analysis technology to analyze the text data and generate a message of empathy. The message of empathy is then sent to the device, which then relays it to the user via voice.

[1326] Example: When a parent says, "Nothing went well today," the device converts the complaint into text and sends it to a server. The server generates a sympathetic message, saying, "That must have been tough. Take a break and refresh yourself," and the device then relays this message in voice.

[1327] This allows users (parents and children) to easily ask questions or share their frustrations, and also allows them to easily check their health and keep a record of their child's growth. Throughout the system, users receive emotional support, enabling them to raise their children more effectively and with peace of mind.

[1328] The processing flow will be explained below.

[1329] Processing steps for QA-enabled features

[1330] Step 1:

[1331] User: Asks the AI ​​robot aloud, "Why is the sky blue?"

[1332] Step 2:

[1333] Terminal: A voice recognition system is used to convert the user's question from voice data to text data.

[1334] Step 3:

[1335] Terminal: Sends the converted question to the server.

[1336] Step 4:

[1337] Server: Analyzes the received text data using natural language processing technology.

[1338] Step 5:

[1339] Server: Retrieves the appropriate answer from the database based on the analysis results.

[1340] Step 6:

[1341] Server: Sends the obtained answer to the device.

[1342] Step 7:

[1343] Terminal: Converts the received answer into speech and conveys it to the user.

[1344] Growth record saving function processing steps

[1345] Step 1:

[1346] User: Says through the microphone, "Today was my first time riding a bike."

[1347] Step 2:

[1348] Terminal: A speech recognition system is used to convert the user's speech into text data.

[1349] Step 3:

[1350] Terminal: Sends the converted text data to the server.

[1351] Step 4:

[1352] Server: Saves the received text data in a database.

[1353] Step 5:

[1354] Server: Sends a message to the terminal indicating that the save is complete.

[1355] Step 6:

[1356] Terminal: A message will be output to the user indicating that the save is complete.

[1357] Health check and data analysis function processing steps

[1358] Step 1:

[1359] User: Enter "I have a headache today" as physical condition data.

[1360] Step 2:

[1361] Terminal: Sends collected physical condition data to the server.

[1362] Step 3:

[1363] Server: Analyzes the received physical condition data using data analysis technology.

[1364] Step 4:

[1365] Server: Generates recommendations for daily rhythms and activities based on the analysis results.

[1366] Step 5:

[1367] Server: Sends the generated recommendations to the device.

[1368] Step 6:

[1369] Device: Recommendations are spoken to the user.

[1370] Stress check and advice function processing steps

[1371] Step 1:

[1372] User: In everyday conversation, say, "I was really busy at work today."

[1373] Step 2:

[1374] Terminal: Uses a voice recognition system to convert speech into text.

[1375] Step 3:

[1376] Terminal: Sends the converted text data to the server.

[1377] Step 4:

[1378] Server: Analyzes the received text data using stress analysis technology.

[1379] Step 5:

[1380] Server: Evaluates stress levels and generates appropriate advice.

[1381] Step 6:

[1382] Server: Sends the generated advice to the device.

[1383] Step 7:

[1384] Terminal: Advice is given to the user by voice.

[1385] Processing steps of the Complaint Listening feature

[1386] Step 1:

[1387] User: Complains to the AI ​​robot, "Nothing went well today."

[1388] Step 2:

[1389] Terminal: Uses a voice recognition system to convert the complaint from voice data to text data.

[1390] Step 3:

[1391] Terminal: Sends the converted text data to the server.

[1392] Step 4:

[1393] Server: Analyzes the received text data using emotion analysis technology.

[1394] Step 5:

[1395] Server: Generates empathy messages.

[1396] Step 6:

[1397] Server: Sends the generated empathy message to the device.

[1398] Step 7:

[1399] Terminal: Conveys empathetic messages to the user via voice.

[1400] Example 1

[1401] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1402] There is a need for support to quickly and efficiently resolve the questions and concerns that parents and children have while raising children. In particular, there is a need for a wide range of support, such as appropriate answers to questions, management of growth records, and checks on health and stress levels. However, it has been difficult to comprehensively manage all of these in a single system and integrate voice input and output. Furthermore, while it is important to have the ability to show empathy and support when parents complain, existing systems have not been able to adequately address this need.

[1403] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1404] In this invention, the server includes: means for converting questions from a parent or child into text data using voice recognition; means for analyzing the text data using natural language processing technology; and means for retrieving appropriate answers from an information storage device based on the analysis results and outputting the answers audibly. This enables prompt and appropriate answers to user questions. The server also includes means for converting parent or child growth records into text data using voice recognition and storing the text data in an information storage device; and means for managing and displaying the stored growth records. This enables easy recording and management of growth records. The server also includes means for collecting parent or child health data, analyzing the health data using data analysis technology, and generating recommendations regarding lifestyle rhythms and next actions based on the analysis results and outputting the recommendations audibly. This enables appropriate advice regarding improving lifestyle rhythms and next actions to be provided to users. The server also includes means for converting everyday conversations between a parent or child into text data using voice recognition, analyzing the text data using emotion analysis technology, and generating empathy messages based on the analysis results and outputting the empathy messages audibly. This allows for a response that is sensitive to the user's emotions. The system also includes a means for converting the complaints of parents or children into text data using voice recognition and analyzing the text data using emotion analysis technology, and a means for generating a message of sympathy based on the analysis results and outputting the message of sympathy by voice. This makes it possible to provide a sympathetic response even when the user vents their frustrations.

[1405] "Speech recognition" is a technology that converts speech into text data.

[1406] "Character data" is text-format data generated by speech recognition.

[1407] "Natural language processing technology" is a computer programming technology for analyzing and understanding human language.

[1408] "Analysis results" refers to information obtained after analyzing character data using natural language processing technology.

[1409] An "information storage device" is a device for storing and managing data, and specifically refers to a database.

[1410] "Audio output" refers to transmitting text data to the user as audio using speech synthesis technology.

[1411] A "growth record" is data that records information about important events and developments during a child's growth process.

[1412] "Management and display" refers to properly organizing the stored data and presenting it to the user as needed.

[1413] "Physical condition data" refers to information about the health status of parents and children, and is primarily data about physical conditions.

[1414] "Data analysis technology" is the technology for analyzing collected data and deriving useful information.

[1415] "Life rhythm" refers to the patterns of activity and rest in daily life.

[1416] "Recommendations for next steps" are specific guidelines for action suggested based on your current health condition and lifestyle.

[1417] "Daily conversation" refers to the general conversations that parents and children have every day.

[1418] "Emotion analysis technology" is a technology for analyzing the emotions contained in words and sentences and understanding their content.

[1419] An "empathetic message" is a message that is generated in a way that understands and sympathizes with the user's emotions.

[1420] "Complaining" is the act of parents and children verbally expressing the dissatisfaction and stress they feel in their daily lives.

[1421] The present invention is a system that combines technologies of speech recognition, natural language processing, database management, and data analysis to provide support and resolve various concerns and questions that parents and children have while raising children. Specific embodiments of the present invention will be described in detail below.

[1422] Implementation of QA support functions

[1423] In this system, when a user asks a question, the device converts the question into text data using voice recognition technology. Specifically, it uses Google's Cloud Speech-to-Text API. The converted text data is then sent from the device to the server.

[1424] The server analyzes the text data using Google's Natural Language API. Based on the analysis results, the server retrieves the appropriate answer from an information storage device (a database, such as MySQL). The retrieved answer is then sent from the server to the device.

[1425] The device uses Google's Text-to-Speech API to convert the answer into audio and convey it to the user.

[1426] Examples:

[1427] When a user asks, "Why is the sky blue?", the device converts the speech into text using Google's Cloud Speech-to-Text API and sends it to the server. The server then analyzes it using Google's Natural Language API and obtains the answer, "The sky is blue because sunlight scatters molecules in the atmosphere." The device then uses Google's Text-to-Speech API to synthesize the answer into voice and convey it to the user.

[1428] Example prompt sentence:

[1429] When a user asks "Why is the sky blue?", the voice is converted to text using Google's Cloud Speech-to-Text API, the question is analyzed using Google's Natural Language API, and the answer is synthesized using Google's Text-to-Speech API.

[1430] Embodiment of growth record saving function

[1431] When a user enters their child's growth record by voice, the device converts the voice into text using Google's Cloud Speech-to-Text API, which then sends the text data to the server.

[1432] The server stores the received text data in an information storage device (database, for example, MySQL), which the user can later check and manage.

[1433] Examples:

[1434] When a parent says, "Today was my first time riding a bike," the device converts the speech into text using Google's Cloud Speech-to-Text API and sends it to the server, which then stores the record of "my child rode a bike for the first time" in a database.

[1435] Example prompt sentence:

[1436] When a parent says, "Today was my first time riding my bike," Google's Cloud Speech-to-Text API converts the speech to text and stores that text in a MySQL database.

[1437] An embodiment of the health check and data analysis function

[1438] When a user enters their health data, the device sends the data to a server, which then analyzes the data using Python's Pandas library and Scikit-learn and generates appropriate recommendations based on the analysis results.

[1439] The generated recommendations are sent from the server to the device, which then uses Google's Text-to-Speech API to convert the recommendations into audio and convey them to the user.

[1440] Examples:

[1441] When a parent types, "I have a headache today," the device sends that data to a server. The server analyzes the data using Python's Pandas and Scikit-learn, and generates recommendations such as, "I recommend drinking plenty of water and getting plenty of rest." The device then uses Google's Text-to-Speech API to synthesize the recommendations into voice and deliver them to the user.

[1442] Example prompt sentence:

[1443] When a parent types "I have a headache today," the data is sent to the server and analyzed using Python's Pandas and Scikit-learn. The results are then synthesized into speech using Google's Text-to-Speech API.

[1444] Stress check and advice function implementation example

[1445] The system converts the user's everyday conversations into text data using voice recognition technology and sends the text data to a server, which then analyzes the text data using emotion analysis technology to evaluate the user's stress level.

[1446] Based on the evaluation results, the server generates appropriate advice and sends it to the device, which then converts the advice into audio using Google's Text-to-Speech API and conveys it to the user.

[1447] Examples:

[1448] When a parent says, "I was really busy at work today," the device converts the conversation into text and sends it to the server. The server then generates advice, such as, "You seem stressed. Take a moment to relax and take a deep breath." The device then uses Google's Text-to-Speech API to synthesize the advice into voice and relay it to the user.

[1449] Example prompt sentence:

[1450] When a parent says, "I was really busy at work today," emotion analysis technology is used to analyze their stress level, and appropriate advice is synthesized using Google's Text-to-Speech API.

[1451] An embodiment of the complaint listening function

[1452] When a user complains, the device uses voice recognition technology to convert the complaint into text data and sends it to the server. The server then uses emotion analysis technology to analyze the text data and generate a message of empathy.

[1453] The generated empathy message is sent from the server to the device, which then uses Google's Text-to-Speech API to convert the message into audio and convey it to the user.

[1454] Examples:

[1455] When a parent says, "Nothing went well today," the device converts the complaint into text and sends it to the server. The server then generates a message of sympathy, saying, "That must have been tough. Take a break and refresh yourself." The device then uses Google's Text-to-Speech API to synthesize the message of sympathy into voice and deliver it to the user.

[1456] Example prompt sentence:

[1457] When a parent says, "Nothing went well today," emotion analysis technology generates an empathetic message, which is then synthesized into voice using Google's Text-to-Speech API.

[1458] In this way, by combining various cutting-edge technologies, this system is able to provide comprehensive support for the various problems and concerns that users face on a daily basis.

[1459] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1460] Processing steps for QA-enabled features

[1461] Step 1:

[1462] The user asks a question, for example, "Why is the sky blue?"

[1463] Input: Audio data

[1464] Output: None

[1465] Step 2:

[1466] The device converts the voice data into text data using Google's Cloud Speech-to-Text API.

[1467] Input: Audio data

[1468] Output: Character data

[1469] Step 3:

[1470] The character data generated by the terminal is sent to the server using an HTTP request.

[1471] Input: Character data

[1472] Output: HTTP request to the server

[1473] Step 4:

[1474] The server analyzes the received text data using Google's Natural Language API.

[1475] Input: Character data

[1476] Output: Analysis results

[1477] Step 5:

[1478] Based on the analysis results, the server queries the information storage device (database) to obtain the appropriate answer.

[1479] Input: Analysis results

[1480] Output: Response data

[1481] Step 6:

[1482] The server sends the response data to the terminal.

[1483] Input: Answer data

[1484] Output: HTTP response to the device

[1485] Step 7:

[1486] The response data received by the device is converted into audio data using Google's Text-to-Speech API.

[1487] Input: Answer data

[1488] Output: Audio data

[1489] Step 8:

[1490] The terminal plays the audio data to the user.

[1491] Input: Audio data

[1492] Output: Audio output to the user

[1493] Growth record saving function processing steps

[1494] Step 1:

[1495] The user enters their child's growth record by voice, for example, "Today was the first time I rode a bike."

[1496] Input: Audio data

[1497] Output: None

[1498] Step 2:

[1499] The device converts the voice data into text data using Google's Cloud Speech-to-Text API.

[1500] Input: Audio data

[1501] Output: Character data

[1502] Step 3:

[1503] The character data generated by the terminal is sent to the server.

[1504] Input: Character data

[1505] Output: HTTP request to the server

[1506] Step 4:

[1507] The server stores the received character data in an information storage device (database).

[1508] Input: Character data

[1509] Output: Save to database

[1510] Health check and data analysis function processing steps

[1511] Step 1:

[1512] The user inputs data about his / her physical condition. For example, he / she inputs "I have a headache today."

[1513] Input: Audio data (or text data)

[1514] Output: None

[1515] Step 2:

[1516] The device converts the voice data into text data and sends it to the server. If the data is text, it is sent as is.

[1517] Input: Audio or text data

[1518] Output: HTTP request to the server

[1519] Step 3:

[1520] The health data received by the server is analyzed using Python's Pandas library and Scikit-learn.

[1521] Input: Physical condition data

[1522] Output: Analysis results

[1523] Step 4:

[1524] The server generates recommendations regarding daily routines and next actions based on the analysis results.

[1525] Input: Analysis results

[1526] Output: Recommendations

[1527] Step 5:

[1528] The server sends the recommendations to the device.

[1529] Input: Recommendations

[1530] Output: HTTP response to the device

[1531] Step 6:

[1532] The device converts the recommendations it receives into audio data using Google's Text-to-Speech API.

[1533] Input: Recommendations

[1534] Output: Audio data

[1535] Step 7:

[1536] The terminal plays the audio data to the user.

[1537] Input: Audio data

[1538] Output: Audio output to the user

[1539] Stress check and advice function processing steps

[1540] Step 1:

[1541] The user engages in everyday conversation, for example, saying, "I was really busy at work today."

[1542] Input: Audio data

[1543] Output: None

[1544] Step 2:

[1545] The device converts the voice data into text data using Google's Cloud Speech-to-Text API.

[1546] Input: Audio data

[1547] Output: Character data

[1548] Step 3:

[1549] The character data generated by the terminal is sent to the server.

[1550] Input: Character data

[1551] Output: HTTP request to the server

[1552] Step 4:

[1553] The server analyzes the received text data using emotion analysis technology.

[1554] Input: Character data

[1555] Output: Analysis results (stress level)

[1556] Step 5:

[1557] The server generates appropriate advice based on the analysis results.

[1558] Input: Analysis results

[1559] Output: Advice data

[1560] Step 6:

[1561] The server transmits the advice data to the terminal.

[1562] Input: Advice data

[1563] Output: HTTP response to the device

[1564] Step 7:

[1565] The advice data received by the device is converted into audio data using Google's Text-to-Speech API.

[1566] Input: Advice data

[1567] Output: Audio data

[1568] Step 8:

[1569] The terminal plays the audio data to the user.

[1570] Input: Audio data

[1571] Output: Audio output to the user

[1572] Processing steps of the Complaint Listening feature

[1573] Step 1:

[1574] The user complains, for example, "Nothing went well today."

[1575] Input: Audio data

[1576] Output: None

[1577] Step 2:

[1578] The device converts the voice data into text data using Google's Cloud Speech-to-Text API.

[1579] Input: Audio data

[1580] Output: Character data

[1581] Step 3:

[1582] The character data generated by the terminal is sent to the server.

[1583] Input: Character data

[1584] Output: HTTP request to the server

[1585] Step 4:

[1586] The server analyzes the received text data using emotion analysis technology.

[1587] Input: Character data

[1588] Output: Analysis results (emotion data)

[1589] Step 5:

[1590] The server generates a message of sympathy based on the analysis results.

[1591] Input: Analysis results

[1592] Output: Empathy message

[1593] Step 6:

[1594] The server sends a sympathy message to the terminal.

[1595] Input: Sympathy message

[1596] Output: HTTP response to the device

[1597] Step 7:

[1598] The device converts the empathy message received into audio data using Google's Text-to-Speech API.

[1599] Input: Sympathy message

[1600] Output: Audio data

[1601] Step 8:

[1602] The terminal plays the audio data to the user.

[1603] Input: Audio data

[1604] Output: Audio output to the user

[1605] (Application example 1)

[1606] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1607] In modern society, parents face the challenge of a lack of support when going out with their children and shopping in brick-and-mortar stores. In particular, parents have limited means to easily gather information about their children's health, growth records, and daily concerns, and to receive appropriate advice. Furthermore, there are insufficient systems in place to alleviate parents' own stress and worries, making it an urgent task to address these issues.

[1608] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1609] In this invention, the server includes: means for converting questions from a parent or child into text data using voice recognition; means for analyzing the text data using natural language processing technology; and means for retrieving appropriate answers from a database based on the analysis results and outputting the answers audibly. This allows parents to ask questions or seek advice about child-rearing in real time while shopping at a physical store and receive appropriate answers. The server also includes means for collecting physical condition data of the parent or child and analyzing the physical condition data using data analysis technology; and means for generating recommendations regarding daily rhythms and next actions based on the analysis results and outputting the recommendations audibly. This allows parents to understand their child's health status and take appropriate measures even while shopping at a physical store. The server also includes means for recording daily conversations between the parent or child and analyzing the conversation data using stress analysis technology; means for generating appropriate advice based on the analysis results and outputting the advice audibly; and means for allowing the parent or child to input complaints audibly, generating empathetic messages using emotion analysis technology, and outputting the messages audibly. This reduces stress and worries for parents and improves overall child-rearing support.

[1610] definition statement

[1611] "Speech recognition" is a technology that receives speech as input and automatically converts that speech into text data.

[1612] "Character data" refers to data in text format obtained by speech recognition.

[1613] "Natural language processing technology" is a technology that enables computers to understand, analyze, and generate natural human language.

[1614] The "analysis results" are the results of analyzing character data using natural language processing technology.

[1615] A "database" is a system that allows data to be stored, searched, and managed.

[1616] "Audio output means" refers to technology that provides analysis results and answers to users as audio.

[1617] "Physical condition data" refers to data indicating the health condition of a parent or child.

[1618] "Data analysis technology" is a technology for analyzing collected data and extracting useful information.

[1619] "Life rhythm" refers to an individual's activity patterns and time allocation in their daily lives.

[1620] "Recommendations for next actions" are specific actions or advice suggested based on the analysis results.

[1621] "Daily conversation" refers to the verbal exchanges that parents or children have on a daily basis.

[1622] "Stress analysis technology" is a technology that evaluates an individual's stress level based on conversation and behavioral data.

[1623] "Advice" refers to advice or suggestions provided based on the results of the analysis.

[1624] "Guchi" is a word that expresses dissatisfaction or anxiety felt by parents or children.

[1625] "Emotion analysis technology" is a technology that evaluates the emotional state of text data and generates emotional responses based on that data.

[1626] An "empathetic message" is a message that shows understanding and empathy for the user's feelings.

[1627] MODE FOR CARRYING OUT THE INVENTION

[1628] The present invention is a child-rearing support system that uses voice recognition technology, natural language processing technology, database management technology, and data analysis technology. This system provides various child-rearing-related services when parents and children enjoy shopping at physical stores. Specific embodiments are described below.

[1629] System Configuration and Hardware

[1630] This system consists of a smartphone, a cloud server, a voice recognition engine, a natural language processing engine, a database, a text-to-speech engine, etc.

[1631] The smartphone is a device on which a dedicated app is installed and which directly interacts with parents or children. The cloud server processes various data and provides appropriate information to parents or children.

[1632] Specific roles and operations of each technology

[1633] Speech recognition: When a user (parent or child) speaks into a smartphone, a speech recognition engine (such as the Google Speech-to-Text API) converts the speech into text data.

[1634] Natural language processing: Text data is sent to a cloud server and analyzed using a natural language processing engine (such as Google Cloud Natural Language API). For example, if a user says, "It looks like my child has caught a cold," the cloud server analyzes this text data and understands its meaning.

[1635] Database management: Based on the results analyzed by the natural language processing engine, the cloud server retrieves appropriate answers and support information from a database (such as MongoDB). For example, advice such as "Give the child plenty of warm drinks, and if symptoms persist, consult a pediatrician" may be obtained.

[1636] Voice output: The obtained answer is converted into voice by a text-to-speech engine (such as Google Text-to-Speech API) and is transmitted to the user through the smartphone speaker.

[1637] Example of childcare support function

[1638] 1. Question and Answering

[1639] When a user speaks to their smartphone, "My child has a fever. What should I do?", the speech recognition engine converts the speech into text data and sends it to a cloud server. The cloud server then uses natural language processing technology to analyze the text and retrieves the appropriate answer from a database. The answer, "Ensure adequate hydration and consult a doctor if necessary," is then played back as audio by the text-to-speech engine.

[1640] 2. Keeping a record of growth

[1641] When a user says, "Today was my first time riding a bicycle," the speech recognition engine converts this into text data and sends it to the cloud server, which then stores this data in a database and manages it for later review.

[1642] 3. Health check and advice

[1643] When a user says "I have a headache," the speech recognition engine converts this into text data and sends it to the cloud server. The cloud server analyzes this data and generates advice such as "We recommend you drink plenty of fluids and get some rest," which is then conveyed to the user via voice.

[1644] 4. Stress check and emotion analysis

[1645] The system continuously records the user's conversations and periodically analyzes their stress levels. For example, if a user says, "Work was very hard today," the cloud server will evaluate their stress level and provide advice such as, "Take a break to relax."

[1646] Prompt Sentence Examples

[1647] A user might ask, "My child has a cough. What should I do?" and use the following prompt:

[1648] I would like to answer questions about children's health. For example, "Why is the sky blue?" I would like to answer something like, "The sky is blue because sunlight scatters molecules in the atmosphere."

[1649] This provides an environment where parents and children can focus on raising their children with peace of mind while enjoying shopping in a physical store.

[1650] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1651] Program processing steps

[1652] Step 1:

[1653] The user (parent or child) enters questions or information into the smartphone by voice. For example, the user might say, "My child seems to have caught a cold." This voice data is then entered into the system.

[1654] Step 2:

[1655] The device uses a speech recognition engine (Google Speech-to-Text API) to convert the voice data from step 1 into text data, which then becomes the input for the next step.

[1656] Step 3:

[1657] The device formats the text data as a preprocessing step and sends it to the cloud server. The cloud server receives this text data and analyzes it using a natural language processing engine (Google Cloud Natural Language API). As a result of the analysis, the meaning and intent of the text data are extracted.

[1658] Step 4:

[1659] Based on the analysis results, the server retrieves the appropriate answer corresponding to the question from a database (MongoDB). This database contains a wide variety of questions and their answers in advance. For example, the answer retrieved might be, "Have the child drink plenty of warm fluids, and if symptoms persist, consult a pediatrician."

[1660] Step 5:

[1661] The server sends the answer retrieved from the database to a text-to-speech engine (Google Text-to-Speech API), which converts the answer into audio data, which becomes the input for the next step.

[1662] Step 6:

[1663] The device receives the voice data sent from the text-to-speech engine and conveys it to the user. Specifically, the smartphone speaker plays a voice message saying, "Give your child plenty of warm drinks, and if symptoms persist, consult a pediatrician."

[1664] Step 7:

[1665] The user receives advice and answers from the app and takes appropriate action. At this step, the user gets a solution to their problem.

[1666] Step 8:

[1667] If necessary, the user can again input their daily conversations, physical condition, and growth records by voice, which the system will record and analyze. This step is important so that the system can continuously monitor the user's condition and provide appropriate advice.

[1668] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1669] This invention is based on an AI system that interacts with parents or children, and incorporates an emotion engine to recognize and respond to the user's emotions. This system combines voice recognition technology, natural language processing technology, database management technology, data analysis technology, and emotion engine technology to address various issues faced by parents and children.

[1670] An embodiment of emotion recognition QA function

[1671] In this system, when a parent or child asks a question, the device uses a voice recognition system to convert the speech into text data. The text data is then sent to a server. The server uses natural language processing technology and an emotion engine to analyze the text and recognize the user's emotional state. Based on this emotional state, the server generates an appropriate response or a tone-adjusted empathetic message, which is then sent from the server to the device. The device then communicates the generated response or empathetic message to the user via voice.

[1672] For example, if a child asks, "Why do I have to go to school?", the device converts the speech into text and sends it to a server. The server recognizes the child's anxiety and stress contained in the question and generates a tailored response: "School is a place where you can make new friends and have fun learning. Don't worry, just relax and have fun," which the device then relays.

[1673] An embodiment of a growth record keeping function using emotion recognition

[1674] The system also uses the emotion engine when parents enter their child's growth record by voice. The device converts the voice into text data, running the emotion engine to send the entered content and emotional state to the server. The server stores the received data in a database and, if necessary, tags the emotional state. Parents can later review the saved growth record and easily search for important events in which a specific emotional state was recorded.

[1675] Example: When a parent says, "Today I was so happy because I rode my bike for the first time," the device converts the speech into text, recognizes the "happy" emotion using an emotion engine, and sends this information to the server. The server then saves the record of "my child rode a bike for the first time" in a database with a tag of "happy."

[1676] An embodiment of a health check and data analysis function using emotion recognition

[1677] This system periodically collects and analyzes the physical condition data of parents and children, while also analyzing the user's emotional state using an emotion engine. The device collects emotional data along with the physical condition data and sends it to a server. The server analyzes the data using data analysis technology and the emotion engine, and based on the analysis results, generates recommendations for daily rhythms and activities, as well as advice for psychological support. The recommendations and advice are sent to the device, which then communicates them to the user via voice.

[1678] Example: If a parent types, "I have a headache today and I'm feeling a bit down," the device will send that data to a server. The server will then send an analysis result, such as, "Drink plenty of fluids, get some rest, and listen to your favorite music to change your mood," and the device will then announce this in voice.

[1679] An embodiment of a stress check and advice function using emotion recognition

[1680] This system records everyday conversations between parents and children and uses an emotion engine to perform stress analysis. The device uses voice recognition to convert the conversation into text data, analyzes the emotional state using the emotion engine, and sends this to a server. The server uses both stress analysis technology and the emotion engine to evaluate stress levels and generate appropriate advice. The evaluation results and advice are sent to the device, which then communicates them to the user via voice.

[1681] Example: If a parent says, "I'm really busy at work today and I'm tired," the device converts the conversation into text, recognizes the "tired" emotion using an emotion engine, and sends it to the server. The server then sends the analysis results, saying, "You seem to be stressed. Take some time to relax and take a deep breath," and the device then conveys this in voice.

[1682] Embodiment of a function for listening to complaints using emotion recognition

[1683] This system also uses the emotion engine when a parent complains. The device uses voice recognition to convert the complaint into text data, analyzes the emotion using the emotion engine, and sends this to the server. The server then uses emotion analysis technology to generate an empathy message, which is tailored based on the parent's emotional state. The empathy message is then sent to the device, which then relays it to the user via voice.

[1684] Example: When a parent complains, "Nothing went well today," the device converts the complaint into text, recognizes the "disappointment" in its emotion engine, and sends it to the server. The server then generates a sympathetic message, saying, "That must have been really tough. Get some rest and start tomorrow with a fresh attitude," and the device then relays this message in voice.

[1685] This allows users (parents and children) to easily ask questions or express their frustrations, and also allows for easy health checks and growth records to be kept. Furthermore, emotion recognition technology allows for more sympathetic responses, providing greater emotional support. Throughout the system, users can raise their children in a comfortable environment.

[1686] The processing flow will be explained below.

[1687] Processing steps for emotion recognition QA function

[1688] Step 1:

[1689] User: Asks the AI ​​robot aloud, "Why do I have to go to school?"

[1690] Step 2:

[1691] Terminal: A voice recognition system is used to convert the user's question from voice data to text data.

[1692] Step 3:

[1693] Terminal: Sends the converted text data to the server.

[1694] Step 4:

[1695] Server: Analyzes the received text data using natural language processing technology and an emotion engine to recognize the user's emotions.

[1696] Step 5:

[1697] Server: Based on the user's emotions, retrieves appropriate answers from the database and generates empathetic messages with adjusted tone.

[1698] Step 6:

[1699] Server: Sends the generated answer and sympathy message to the device.

[1700] Step 7:

[1701] Terminal: Converts the received answers and sympathy messages into audio and conveys them to the user.

[1702] Processing steps for growth record keeping function using emotion recognition

[1703] Step 1:

[1704] User: Through the microphone, says, "Today was my first time riding a bike and I was so happy."

[1705] Step 2:

[1706] Terminal: A speech recognition system is used to convert the user's speech into text data.

[1707] Step 3:

[1708] Terminal: Sends the converted text data to the server.

[1709] Step 4:

[1710] Server: The received text data is analyzed using an emotion engine to recognize the emotion "happy."

[1711] Step 5:

[1712] Server: Stores the text data along with the recognized emotions in a database.

[1713] Step 6:

[1714] Server: Sends a message to the terminal indicating that the save is complete.

[1715] Step 7:

[1716] Terminal: A message will be output to the user indicating that the save is complete.

[1717] Processing steps for health check and data analysis functions using emotion recognition

[1718] Step 1:

[1719] User: Enters "I have a headache today and feel a little depressed" as health data.

[1720] Step 2:

[1721] Terminal: Sends collected physical condition data to the server.

[1722] Step 3:

[1723] Server: Analyzes the received physical condition data and emotion data using data analysis technology and an emotion engine.

[1724] Step 4:

[1725] Server: Generates recommendations for daily routines and behaviors based on the analysis results, and also generates advice for psychological support.

[1726] Step 5:

[1727] Server: Sends the generated recommendations and advice to the device.

[1728] Step 6:

[1729] Device: Provides recommendations and advice to the user via voice.

[1730] Processing steps for stress check and advice function using emotion recognition

[1731] Step 1:

[1732] User: In everyday conversation, say, "I'm really busy at work today and I'm tired."

[1733] Step 2:

[1734] Terminal: Uses a voice recognition system to convert speech into text.

[1735] Step 3:

[1736] Terminal: Sends the converted text data to the server.

[1737] Step 4:

[1738] Server: Analyzes the received text data using stress analysis technology and an emotion engine to evaluate stress levels and emotional states.

[1739] Step 5:

[1740] Server: Generates appropriate advice based on the evaluation results.

[1741] Step 6:

[1742] Server: Sends the generated advice to the device.

[1743] Step 7:

[1744] Terminal: Advice is given to the user by voice.

[1745] Processing steps for listening to complaints using emotion recognition

[1746] Step 1:

[1747] User: Complains to the AI ​​robot, "Nothing went well today."

[1748] Step 2:

[1749] Terminal: Uses a voice recognition system to convert the complaint from voice data to text data.

[1750] Step 3:

[1751] Terminal: Sends the converted text data to the server.

[1752] Step 4:

[1753] Server: Analyzes the received text data using an emotion engine and recognizes "disappointment."

[1754] Step 5:

[1755] Server: Generates empathetic messages based on emotional states.

[1756] Step 6:

[1757] Server: Sends the generated empathy message to the device.

[1758] Step 7:

[1759] Terminal: Conveys empathetic messages to the user via voice.

[1760] Example 2

[1761] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1762] Conventional conversational AI systems simply convert user questions and comments about their physical condition into text data and generate answers. This makes it difficult to respond in a way that takes into account the user's emotional state and psychological aspects, resulting in insufficient psychological support for the user. Furthermore, when storing and managing growth records and physical condition data, data that includes the user's emotional state is not recorded, making it difficult to fully grasp important events or changes in the user's psychological state.

[1763] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1764] In this invention, the server includes: a means for converting questions from a parent or child into text data using voice recognition; a means for analyzing the text data using natural language processing; an emotion analysis means for recognizing the user's emotional state based on the analysis results; a means for generating an appropriately adjusted answer or empathetic message based on the recognized emotional state; and a means for outputting the generated answer or empathetic message by voice. This enables more friendly responses and psychological support tailored to the user's emotional state. Furthermore, growth records and physical condition data, including emotional states, can be saved and managed, making it easy to understand important events and psychological changes.

[1765] "Speech recognition" is a technology that analyzes voice signals and converts them into text data.

[1766] "Text data" refers to data that includes text information converted by voice recognition.

[1767] "Natural language processing" is a technology that allows computers to understand, analyze, and generate human language.

[1768] "Emotion analysis means" is a technology that recognizes the emotional state of a user from the content of their comments.

[1769] A "database" is a system for efficiently managing, storing, and searching large amounts of data.

[1770] "Data analysis technology" is a technology for analyzing collected data and extracting useful information.

[1771] "Recommendations" are suggestions about behaviors and lifestyle habits provided to users based on the analysis results.

[1772] A "sympathetic message" is a message that expresses sympathy and is generated in accordance with the user's emotional state.

[1773] "Audio output means" refers to a technology that converts text data into audio and conveys it to the user.

[1774] A "growth record" is a record that stores data about a parent's or child's growth through voice recognition and emotion analysis.

[1775] "Physical condition data" is data that includes information related to the user's health condition.

[1776] "Emotional support" refers to providing advice and messages to help users achieve emotional stability and comfort.

[1777] MODE FOR CARRYING OUT THE INVENTION

[1778] This invention is based on an AI system that interacts with parents or children, and incorporates an emotion analysis engine to recognize and respond to the user's emotions. This system combines voice recognition technology, natural language processing technology, database management technology, data analysis technology, and emotion analysis technology to address various issues faced by parents and children.

[1779] Components

[1780] Speech recognition technology: Converts speech into text data. For example, speech recognition software (e.g., Google Cloud Speech-to-Text) is used.

[1781] Natural language processing technology: Analyzes text data and understands its meaning. For example, natural language processing software (e.g., NLP API) is used.

[1782] Database management technology: Technology for effectively storing and managing converted data and analysis results. For example, a database management system (e.g., MySQL) is used.

[1783] Data analysis technology: Technology to analyze collected physical condition data and growth records and extract information useful to users. For example, data analysis libraries (e.g., Pandas, NumPy) are used.

[1784] Emotion analysis technology: Technology that recognizes the user's emotional state and generates appropriate responses based on that information. As a specific example, we will use an emotion analysis engine (e.g., Affectiva API).

[1785] Program processing

[1786] 1. Speech recognition process: The device converts the user's questions or speech into text data using speech recognition software, such as Google Cloud Speech-to-Text.

[1787] 2. Data transmission process: The converted text data is sent to the server, which uses HTTP requests and responses to analyze the received data.

[1788] 3. Text analysis process: The server uses natural language processing technology to analyze the text data and understand its contents. Natural language processing APIs (e.g., Google Cloud Natural Language API) are used to understand the meaning of the text.

[1789] 4. Sentiment Analysis Process: Based on the analysis results, the server uses sentiment analysis technology to recognize the user's emotional state. It uses a sentiment analysis engine (such as Affectiva API) to specifically identify the user's emotions.

[1790] 5. Answer generation process: Based on the recognized emotional state, the server uses a generative AI model (e.g., OpenAI's GPT model) to generate appropriate answers and empathetic messages, adjusting the tone and content according to the emotional state.

[1791] 6. Database storage process: The generated answers and emotional state data are stored in a database with appropriate tagging. A database management system such as MySQL is used for this purpose.

[1792] 7. Voice output process: Finally, the generated answers and messages are sent to the device and audibly conveyed to the user using speech synthesis technology (e.g., Amazon Polly).

[1793] Examples of concrete examples and prompts

[1794] Example 1:

[1795] When a user asks, "Why do I have to go to school?", the device converts the speech into text and sends it to the server. The server recognizes the user's anxiety and stress contained in the question and generates a tailored response: "School is a place where you can make new friends and have fun learning. Don't worry, just relax and have fun." The device then delivers the response via voice.

[1796] Example prompt sentence:

[1797] "Why do I have to go to school?"

[1798] Example 2:

[1799] When a parent says, "Today I was so happy to ride a bike for the first time," the device converts the speech into text, recognizes the "happy" emotion using an emotion engine, and sends this information to the server, which then stores the record of "my child's first bike ride" in a database with a tag of "happy."

[1800] Example prompt sentence:

[1801] "Today was the first time I rode a bike."

[1802] This invention allows users to receive friendly responses and support according to their emotions, and also makes it possible to store and manage growth records and physical condition data together with their emotional state.

[1803] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1804] Specific flow of program processing

[1805] Emotion-aware QA function

[1806] Step 1:

[1807] User utterance input

[1808] The user asks a question to the device. Specifically, they say, "Why do I have to go to school?" This input is treated as voice data.

[1809] Step 2:

[1810] Speech-to-text

[1811] The device uses speech recognition software (e.g., Google Cloud Speech-to-Text) to convert the user's voice into text data. An example of the converted output is the text data "Why do I have to go to school?"

[1812] Step 3:

[1813] Sending text to the server

[1814] The terminal sends the converted text data to the server via an HTTP request. The input is the converted text data, and the output is a notification to the server that transmission has been completed.

[1815] Step 4:

[1816] Text data analysis

[1817] The text data received by the server is analyzed using natural language processing technology (e.g., Google Cloud Natural Language API). The input is text data, and the output is the semantic and structural information of the analyzed text. Specifically, the question "Why do I have to go to school?" is analyzed, and its meaning and structure are understood.

[1818] Step 5:

[1819] Recognition of emotional states

[1820] The server uses emotion analysis technology (e.g., Affectiva's Emotion Analysis API) to recognize the user's emotional state based on the analysis results. The input is the analyzed text data, and the output is the identified emotional state (e.g., anxiety or stress).

[1821] Step 6:

[1822] Emotion-based answer generation

[1823] The server uses a generative AI model (e.g., OpenAI's GPT model) to generate an appropriate answer or empathetic message based on the emotional state. The input is the identified emotional state and text data, and the output is a generated answer or empathetic message. For example, a generated answer might be, "School is a place where you can make new friends and have fun learning. Don't worry, just relax and have fun."

[1824] Step 7:

[1825] Sending answers to the device

[1826] The server sends the generated answer to the terminal as an HTTP response. The input is the generated answer data, and the output is a notification of completion of transmission to the terminal.

[1827] Step 8:

[1828] Voice output of answers

[1829] The device uses speech synthesis technology (e.g., Amazon Polly) to convert the text-based answer into speech and convey it to the user. The input is the generated answer text, and the output is audio data. Specifically, the answer "School is a place where you can make new friends and have fun learning. Don't worry, just take your time and have fun" is provided to the user via voice.

[1830] ---

[1831] Emotion recognition-based growth record keeping function

[1832] Step 1:

[1833] User utterance input

[1834] A parent talks to the device about their child's growth. For example, they might say, "Today was my first time riding a bike, and I was so happy." The input is treated as voice data.

[1835] Step 2:

[1836] Speech-to-text

[1837] The device uses voice recognition software to convert the parent's speech into text data, and the output is the converted text data.

[1838] Step 3:

[1839] Server transmission of text and emotions

[1840] The device sends the converted text data and the emotion analysis results from the emotion analysis engine to the server. The input is the text data and emotion data, and the output is a notification of completion of transmission to the server.

[1841] Step 4:

[1842] Text and sentiment storage

[1843] The server temporarily stores the received text data and emotional state in a database. The input is the received data, and the output is a notification that the data has been saved to the database.

[1844] Step 5:

[1845] Data tagging

[1846] The server assigns a tag of "happy" as an emotional state. The input is text data and emotional data, and the output is tagged data.

[1847] Step 6:

[1848] Saving to a database

[1849] The server saves the tagged text data to a database. The input is the tagged text data, and the output is a notification that the data has been permanently saved to the database.

[1850] Step 7:

[1851] Checking the data

[1852] When a parent wants to check the growth record later, they can search the saved data through the device's user interface, filter it by a specific emotional state (e.g., "happy"), and display it. The input is the search criteria, and the output is the search results. Specifically, the record of "my child riding a bicycle for the first time" is displayed with the tag "happy."

[1853] ---

[1854] Health check and data analysis function using emotion recognition

[1855] Step 1:

[1856] User utterance input

[1857] The user speaks to the device about their physical condition. Specifically, they say, "I have a headache today, and I'm feeling a little depressed." The input is treated as voice data.

[1858] Step 2:

[1859] Speech-to-text and sentiment analysis

[1860] The device uses voice recognition software to convert speech into text data, and then uses an emotion analysis engine to analyze the emotional state, with the output being the converted text data and emotion data.

[1861] Step 3:

[1862] Sending data to the server

[1863] The terminal sends the converted text data and emotion data to the server. The input is the text data and emotion data, and the output is a notification of completion of transmission to the server.

[1864] Step 4:

[1865] Data integration

[1866] The server integrates the physical condition data and emotional data. The input is the collected data, and the output is the integrated data.

[1867] Step 5:

[1868] Data analysis

[1869] The server analyzes the integrated data using data analysis techniques (Python's Pandas and NumPy). The input is the integrated data, and the output is the analysis results. Specifically, the server evaluates the user's health status and emotional state.

[1870] Step 6:

[1871] Generating Advice

[1872] Based on the analysis results, the server generates recommendations for daily routines and behaviors, as well as advice for psychological support. The input is the analysis results, and the output is the generated advice. For example, the generated advice might be, "Drink plenty of water, get plenty of rest, and listen to your favorite music to change your mood."

[1873] Step 7:

[1874] Sending advice to the device

[1875] The server sends the generated advice to the terminal. The input is the generated advice, and the output is a notification of the completion of the transmission to the terminal.

[1876] Step 8:

[1877] Audio output of advice

[1878] The device uses voice synthesis technology such as Amazon Polly to convert the advice into speech and convey it to the user. The input is the generated advice text, and the output is audio data. Specifically, the advice provided to the user is "Drink plenty of water, get plenty of rest, and listen to your favorite music to change your mood."

[1879] (Application example 2)

[1880] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1881] Conventional food delivery systems face challenges in achieving sufficient customer satisfaction in customer service and complaint handling. Furthermore, there are also issues with the lack of effective ways to utilize customer feedback and measures to reduce the stress of customer support staff. This can lead to delays in improving service quality and customer satisfaction.

[1882] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1883] In this invention, the server includes means for converting questions from a parent or child into text data using voice recognition, means for analyzing the text data using natural language processing technology, means for retrieving an appropriate answer from a database based on the analysis result and outputting the answer by voice, means for recognizing emotions and generating an empathy message in an appropriate tone based on the analysis result, means for outputting the empathy message by voice, means for saving feedback from customers with emotion tags, and means for analyzing the feedback and providing proactive support. This improves the quality of customer service, effectively utilizes feedback, reduces the burden on customer support staff through stress checks, and enables overall improvement of service quality.

[1884] "Speech recognition" is a technology that analyzes speech and converts it into text data.

[1885] "Natural language processing" is a technology that uses computers to analyze and generate natural language.

[1886] "Emotion recognition" is a technology that detects a user's emotional state from voice or text.

[1887] A "database" is a system for systematically storing and managing data.

[1888] "Data analysis technology" is the technology for extracting useful information and patterns from large amounts of data.

[1889] A "sympathetic message" is a message with sympathetic content that is generated in accordance with the user's emotional state.

[1890] "Feedback" refers to opinions and ratings provided by users.

[1891] An "emotion tag" is tag information that indicates an emotional state that is assigned to feedback or data.

[1892] "Proactive support" is a support method that takes preventative measures before a problem occurs.

[1893] "Stress check" is a technique for assessing the stress level of users and staff.

[1894] System program generation

[1895] The system embodying this invention converts questions and feedback from parents or children into text data using voice recognition and incorporates an AI engine that recognizes emotions. This improves the quality of customer service and makes it easier to utilize feedback. It also provides a means to reduce stress for customer support representatives.

[1896] Program processing explanation

[1897] First, we use the SpeechRecognition library for speech recognition, which converts the voice input from the parent or child into text data. Then, we use the HuggingFace transformers library to analyze emotions from the text data using the bhadresh-savani / bert-base-uncased-emotion emotion recognition model.

[1898] The server uses natural language processing technology to generate empathetic messages based on the analyzed emotional data. The generated messages are then delivered to the user via voice synthesis technology. Customer feedback is tagged with emotion and stored in a database.

[1899] Furthermore, the server periodically analyzes the conversations of customer support representatives and performs stress checks. Based on the results, it generates appropriate advice and provides it to the representative.

[1900] This system will improve the quality of customer service and provide proactive support through stress checks.

[1901] Specific examples

[1902] For example, suppose a customer complains that "today's delivery was too late." At this time, the voice recognition system converts the voice into text, and the emotion recognition AI analyzes "anger." The server generates an empathetic message and replies, "We apologize for keeping you waiting. We will check it right away, so please wait a little longer." This allows the customer to receive an appropriate response, improving their satisfaction.

[1903] Customer interaction history is saved in a database with an "anger" tag attached. At a later date, a stress check is conducted and the person in charge is given advice such as "Take time to relax today."

[1904] Prompt Sentence Examples

[1905] text

[1906] Customer enquiry: "Delivery is late" --> Analyse their sentiment.

[1907] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1908] Step 1:

[1909] The user speaks a question or complaint. The device receives this speech and sends it to the speech recognition process. Input: User's speech data. Output: Input data to the speech recognition model.

[1910] Step 2:

[1911] The device uses the SpeechRecognition library to convert the speech data to text, which is then sent to the next processing step: Input: Input data to the speech recognition model. Output: Text data of the speech.

[1912] Step 3:

[1913] The server uses the HuggingFace transformers library to perform emotion recognition on the text data. It uses the bhadresh-savani / bert-base-uncased-emotion model to analyze the emotions contained in the text. Input: Speech text data. Output: Emotion analysis results.

[1914] Step 4:

[1915] The server generates a message of empathy based on the results of the sentiment analysis using natural language processing technology. The generated message is used in the next step. Input: Sentiment analysis result. Output: Message of empathy.

[1916] Step 5:

[1917] The server converts the empathy message into voice data using voice synthesis technology. This voice data is sent to the terminal and output to the user. Input: Empathy message. Output: Voice message data.

[1918] Step 6:

[1919] The terminal plays back the voice message and outputs it to the user, who receives the message.

[1920] Input: Voice message data

[1921] Output: Playing audio to the user

[1922] Step 7:

[1923] The server stores the user feedback in a database with emotion tags. Input: User feedback and emotion data. Output: Emotion-tagged feedback database entries.

[1924] Step 8:

[1925] The server periodically analyzes the feedback data and generates proactive support. Input: Emotion-tagged feedback data in a database. Output: Support notifications and advice.

[1926] Step 9:

[1927] The server performs stress checks on customer support representatives and generates appropriate advice based on the results. Input: Representative conversation content and emotional data. Output: Stress assessment and advice.

[1928] Step 10:

[1929] The server converts the generated advice into voice data using voice synthesis technology and sends it to the person in charge. Input: Stress assessment and advice. Output: Voice message data for the person in charge.

[1930] Step 11:

[1931] The terminal outputs advice to the person in charge by voice, aiming to reduce stress. Input: Voice message data to the person in charge. Output: Voice playback to the person in charge.

[1932] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1933] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1934] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1935] [Fourth embodiment]

[1936] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1937] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1938] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1939] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1940] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1941] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1942] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1943] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1944] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1945] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1946] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1947] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1948] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1949] The present invention is a system that combines speech recognition technology, natural language processing technology, database management technology, and data analysis technology. The purpose of this system is to alleviate the concerns of parents and children and support child-rearing. The following describes in detail the implementation of each function.

[1950] Implementation of QA support functions

[1951] In this system, when a parent or child asks a question, the device uses voice recognition to convert the speech into text data. The device then sends the text data to a server. The server uses natural language processing technology to analyze the text data and retrieves an appropriate answer from a database based on the analysis results. The answer is then sent from the server to the device, which then verbally communicates the answer to the user.

[1952] Example: A child asks, "Why is the sky blue?" The device converts the speech into text and sends it to a server. The server responds, "The sky is blue because sunlight scatters molecules in the atmosphere," and the device then relays the answer.

[1953] Embodiment of growth record saving function

[1954] The system allows parents to record their child's growth by voice. The device uses voice recognition to convert the record into text data, which is then sent to a server. The server stores the text data in a database. Parents can then review the saved growth records later.

[1955] Example: When a parent says, "Today was the first time I rode a bike," the device converts the speech into text and sends it to a server. The server then stores the record that "my child rode a bike for the first time" in a database.

[1956] An embodiment of the health check and data analysis function

[1957] This system periodically collects and analyzes the physical condition data of parents and children. The device collects the data and sends it to a server. The server uses data analysis technology to analyze the data and generates recommendations regarding daily rhythms and next actions based on the analysis results. The recommendations are sent to the device, which then communicates them to the user via audio.

[1958] Example: When a parent types, "I have a headache today," the device sends that data to the server. The server then sends the analysis result, "I recommend you drink plenty of fluids and get plenty of rest," and the device then announces this in voice.

[1959] Stress check and advice function implementation example

[1960] This system records everyday conversations between parents and children and performs stress analysis. The device converts the conversations into text data using voice recognition and sends it to a server. The server then analyzes the text data using stress analysis technology to evaluate the stress level. Based on the evaluation results, the server generates appropriate advice and sends it to the device. The device then communicates the advice to the user via voice.

[1961] Example: If a parent says, "I was really busy at work today," the device converts the conversation into text and sends it to a server. The server then sends the analysis result, "You seem to be stressed. Take a moment to relax and take a deep breath," and the device then conveys this information in voice.

[1962] An embodiment of the complaint listening function

[1963] This system can listen to a parent's complaints and return a message of empathy. The device uses voice recognition to convert the parent's complaints into text data and send it to the server. The server then uses emotion analysis technology to analyze the text data and generate a message of empathy. The message of empathy is then sent to the device, which then relays it to the user via voice.

[1964] Example: When a parent says, "Nothing went well today," the device converts the complaint into text and sends it to a server. The server generates a sympathetic message, saying, "That must have been tough. Take a break and refresh yourself," and the device then relays this message in voice.

[1965] This allows users (parents and children) to easily ask questions or share their frustrations, and also allows them to easily check their health and keep a record of their child's growth. Throughout the system, users receive emotional support, enabling them to raise their children more effectively and with peace of mind.

[1966] The processing flow will be explained below.

[1967] Processing steps for QA-enabled features

[1968] Step 1:

[1969] User: Asks the AI ​​robot aloud, "Why is the sky blue?"

[1970] Step 2:

[1971] Terminal: A voice recognition system is used to convert the user's question from voice data to text data.

[1972] Step 3:

[1973] Terminal: Sends the converted question to the server.

[1974] Step 4:

[1975] Server: Analyzes the received text data using natural language processing technology.

[1976] Step 5:

[1977] Server: Retrieves the appropriate answer from the database based on the analysis results.

[1978] Step 6:

[1979] Server: Sends the obtained answer to the device.

[1980] Step 7:

[1981] Terminal: Converts the received answer into speech and conveys it to the user.

[1982] Growth record saving function processing steps

[1983] Step 1:

[1984] User: Says through the microphone, "Today was my first time riding a bike."

[1985] Step 2:

[1986] Terminal: A speech recognition system is used to convert the user's speech into text data.

[1987] Step 3:

[1988] Terminal: Sends the converted text data to the server.

[1989] Step 4:

[1990] Server: Saves the received text data in a database.

[1991] Step 5:

[1992] Server: Sends a message to the terminal indicating that the save is complete.

[1993] Step 6:

[1994] Terminal: A message will be output to the user indicating that the save is complete.

[1995] Health check and data analysis function processing steps

[1996] Step 1:

[1997] User: Enter "I have a headache today" as physical condition data.

[1998] Step 2:

[1999] Terminal: Sends collected physical condition data to the server.

[2000] Step 3:

[2001] Server: Analyzes the received physical condition data using data analysis technology.

[2002] Step 4:

[2003] Server: Generates recommendations for daily rhythms and activities based on the analysis results.

[2004] Step 5:

[2005] Server: Sends the generated recommendations to the device.

[2006] Step 6:

[2007] Device: Recommendations are spoken to the user.

[2008] Stress check and advice function processing steps

[2009] Step 1:

[2010] User: In everyday conversation, say, "I was really busy at work today."

[2011] Step 2:

[2012] Terminal: Uses a voice recognition system to convert speech into text.

[2013] Step 3:

[2014] Terminal: Sends the converted text data to the server.

[2015] Step 4:

[2016] Server: Analyzes the received text data using stress analysis technology.

[2017] Step 5:

[2018] Server: Evaluates stress levels and generates appropriate advice.

[2019] Step 6:

[2020] Server: Sends the generated advice to the device.

[2021] Step 7:

[2022] Terminal: Advice is given to the user by voice.

[2023] Processing steps of the Complaint Listening feature

[2024] Step 1:

[2025] User: Complains to the AI ​​robot, "Nothing went well today."

[2026] Step 2:

[2027] Terminal: Uses a voice recognition system to convert the complaint from voice data to text data.

[2028] Step 3:

[2029] Terminal: Sends the converted text data to the server.

[2030] Step 4:

[2031] Server: Analyzes the received text data using emotion analysis technology.

[2032] Step 5:

[2033] Server: Generates empathy messages.

[2034] Step 6:

[2035] Server: Sends the generated empathy message to the device.

[2036] Step 7:

[2037] Terminal: Conveys empathetic messages to the user via voice.

[2038] Example 1

[2039] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2040] There is a need for support to quickly and efficiently resolve the questions and concerns that parents and children have while raising children. In particular, there is a need for a wide range of support, such as appropriate answers to questions, management of growth records, and checks on health and stress levels. However, it has been difficult to comprehensively manage all of these in a single system and integrate voice input and output. Furthermore, while it is important to have the ability to show empathy and support when parents complain, existing systems have not been able to adequately address this need.

[2041] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[2042] In this invention, the server includes: means for converting questions from a parent or child into text data using voice recognition; means for analyzing the text data using natural language processing technology; and means for retrieving appropriate answers from an information storage device based on the analysis results and outputting the answers audibly. This enables prompt and appropriate answers to user questions. The server also includes means for converting parent or child growth records into text data using voice recognition and storing the text data in an information storage device; and means for managing and displaying the stored growth records. This enables easy recording and management of growth records. The server also includes means for collecting parent or child health data, analyzing the health data using data analysis technology, and generating recommendations regarding lifestyle rhythms and next actions based on the analysis results and outputting the recommendations audibly. This enables appropriate advice regarding improving lifestyle rhythms and next actions to be provided to users. The server also includes means for converting everyday conversations between a parent or child into text data using voice recognition, analyzing the text data using emotion analysis technology, and generating empathy messages based on the analysis results and outputting the empathy messages audibly. This allows for a response that is sensitive to the user's emotions. The system also includes a means for converting the complaints of parents or children into text data using voice recognition and analyzing the text data using emotion analysis technology, and a means for generating a message of sympathy based on the analysis results and outputting the message of sympathy by voice. This makes it possible to provide a sympathetic response even when the user vents their frustrations.

[2043] "Speech recognition" is a technology that converts speech into text data.

[2044] "Character data" is text-format data generated by speech recognition.

[2045] "Natural language processing technology" is a computer programming technology for analyzing and understanding human language.

[2046] "Analysis results" refers to information obtained after analyzing character data using natural language processing technology.

[2047] An "information storage device" is a device for storing and managing data, and specifically refers to a database.

[2048] "Audio output" refers to transmitting text data to the user as audio using speech synthesis technology.

[2049] A "growth record" is data that records information about important events and developments during a child's growth process.

[2050] "Management and display" refers to properly organizing the stored data and presenting it to the user as needed.

[2051] "Physical condition data" refers to information about the health status of parents and children, and is primarily data about physical conditions.

[2052] "Data analysis technology" is the technology for analyzing collected data and deriving useful information.

[2053] "Life rhythm" refers to the patterns of activity and rest in daily life.

[2054] "Recommendations for next steps" are specific guidelines for action suggested based on your current health condition and lifestyle.

[2055] "Daily conversation" refers to the general conversations that parents and children have every day.

[2056] "Emotion analysis technology" is a technology for analyzing the emotions contained in words and sentences and understanding their content.

[2057] An "empathetic message" is a message that is generated in a way that understands and sympathizes with the user's emotions.

[2058] "Complaining" is the act of parents and children verbally expressing the dissatisfaction and stress they feel in their daily lives.

[2059] The present invention is a system that combines technologies of speech recognition, natural language processing, database management, and data analysis to provide support and resolve various concerns and questions that parents and children have while raising children. Specific embodiments of the present invention will be described in detail below.

[2060] Implementation of QA support functions

[2061] In this system, when a user asks a question, the device converts the question into text data using voice recognition technology. Specifically, it uses Google's Cloud Speech-to-Text API. The converted text data is then sent from the device to the server.

[2062] The server analyzes the text data using Google's Natural Language API. Based on the analysis results, the server retrieves the appropriate answer from an information storage device (a database, such as MySQL). The retrieved answer is then sent from the server to the device.

[2063] The device uses Google's Text-to-Speech API to convert the answer into audio and convey it to the user.

[2064] Examples:

[2065] When a user asks, "Why is the sky blue?", the device converts the speech into text using Google's Cloud Speech-to-Text API and sends it to the server. The server then analyzes it using Google's Natural Language API and obtains the answer, "The sky is blue because sunlight scatters molecules in the atmosphere." The device then uses Google's Text-to-Speech API to synthesize the answer into voice and convey it to the user.

[2066] Example prompt sentence:

[2067] When a user asks "Why is the sky blue?", the voice is converted to text using Google's Cloud Speech-to-Text API, the question is analyzed using Google's Natural Language API, and the answer is synthesized using Google's Text-to-Speech API.

[2068] Embodiment of growth record saving function

[2069] When a user enters their child's growth record by voice, the device converts the voice into text using Google's Cloud Speech-to-Text API, which then sends the text data to the server.

[2070] The server stores the received text data in an information storage device (database, for example, MySQL), which the user can later check and manage.

[2071] Examples:

[2072] When a parent says, "Today was my first time riding a bike," the device converts the speech into text using Google's Cloud Speech-to-Text API and sends it to the server, which then stores the record of "my child rode a bike for the first time" in a database.

[2073] Example prompt sentence:

[2074] When a parent says, "Today was my first time riding my bike," Google's Cloud Speech-to-Text API converts the speech to text and stores that text in a MySQL database.

[2075] An embodiment of the health check and data analysis function

[2076] When a user enters their health data, the device sends the data to a server, which then analyzes the data using Python's Pandas library and Scikit-learn and generates appropriate recommendations based on the analysis results.

[2077] The generated recommendations are sent from the server to the device, which then uses Google's Text-to-Speech API to convert the recommendations into audio and convey them to the user.

[2078] Examples:

[2079] When a parent types, "I have a headache today," the device sends that data to a server. The server analyzes the data using Python's Pandas and Scikit-learn, and generates recommendations such as, "I recommend drinking plenty of water and getting plenty of rest." The device then uses Google's Text-to-Speech API to synthesize the recommendations into voice and deliver them to the user.

[2080] Example prompt sentence:

[2081] When a parent types "I have a headache today," the data is sent to the server and analyzed using Python's Pandas and Scikit-learn. The results are then synthesized into speech using Google's Text-to-Speech API.

[2082] Stress check and advice function implementation example

[2083] The system converts the user's everyday conversations into text data using voice recognition technology and sends the text data to a server, which then analyzes the text data using emotion analysis technology to evaluate the user's stress level.

[2084] Based on the evaluation results, the server generates appropriate advice and sends it to the device, which then converts the advice into audio using Google's Text-to-Speech API and conveys it to the user.

[2085] Examples:

[2086] When a parent says, "I was really busy at work today," the device converts the conversation into text and sends it to the server. The server then generates advice, such as, "You seem stressed. Take a moment to relax and take a deep breath." The device then uses Google's Text-to-Speech API to synthesize the advice into voice and relay it to the user.

[2087] Example prompt sentence:

[2088] When a parent says, "I was really busy at work today," emotion analysis technology is used to analyze their stress level, and appropriate advice is synthesized using Google's Text-to-Speech API.

[2089] An embodiment of the complaint listening function

[2090] When a user complains, the device uses voice recognition technology to convert the complaint into text data and sends it to the server. The server then uses emotion analysis technology to analyze the text data and generate a message of empathy.

[2091] The generated empathy message is sent from the server to the device, which then uses Google's Text-to-Speech API to convert the message into audio and convey it to the user.

[2092] Examples:

[2093] When a parent says, "Nothing went well today," the device converts the complaint into text and sends it to the server. The server then generates a message of sympathy, saying, "That must have been tough. Take a break and refresh yourself." The device then uses Google's Text-to-Speech API to synthesize the message of sympathy into voice and deliver it to the user.

[2094] Example prompt sentence:

[2095] When a parent says, "Nothing went well today," emotion analysis technology generates an empathetic message, which is then synthesized into voice using Google's Text-to-Speech API.

[2096] In this way, by combining various cutting-edge technologies, this system is able to provide comprehensive support for the various problems and concerns that users face on a daily basis.

[2097] The flow of the identification process in the first embodiment will be described with reference to FIG.

[2098] Processing steps for QA-enabled features

[2099] Step 1:

[2100] The user asks a question, for example, "Why is the sky blue?"

[2101] Input: Audio data

[2102] Output: None

[2103] Step 2:

[2104] The device converts the voice data into text data using Google's Cloud Speech-to-Text API.

[2105] Input: Audio data

[2106] Output: Character data

[2107] Step 3:

[2108] The character data generated by the terminal is sent to the server using an HTTP request.

[2109] Input: Character data

[2110] Output: HTTP request to the server

[2111] Step 4:

[2112] The server analyzes the received text data using Google's Natural Language API.

[2113] Input: Character data

[2114] Output: Analysis results

[2115] Step 5:

[2116] Based on the analysis results, the server queries the information storage device (database) to obtain the appropriate answer.

[2117] Input: Analysis results

[2118] Output: Response data

[2119] Step 6:

[2120] The server sends the response data to the terminal.

[2121] Input: Answer data

[2122] Output: HTTP response to the device

[2123] Step 7:

[2124] The response data received by the device is converted into audio data using Google's Text-to-Speech API.

[2125] Input: Answer data

[2126] Output: Audio data

[2127] Step 8:

[2128] The terminal plays the audio data to the user.

[2129] Input: Audio data

[2130] Output: Audio output to the user

[2131] Growth record saving function processing steps

[2132] Step 1:

[2133] The user enters their child's growth record by voice, for example, "Today was the first time I rode a bike."

[2134] Input: Audio data

[2135] Output: None

[2136] Step 2:

[2137] The device converts the voice data into text data using Google's Cloud Speech-to-Text API.

[2138] Input: Audio data

[2139] Output: Character data

[2140] Step 3:

[2141] The character data generated by the terminal is sent to the server.

[2142] Input: Character data

[2143] Output: HTTP request to the server

[2144] Step 4:

[2145] The server stores the received character data in an information storage device (database).

[2146] Input: Character data

[2147] Output: Save to database

[2148] Health check and data analysis function processing steps

[2149] Step 1:

[2150] The user inputs data about his / her physical condition. For example, he / she inputs "I have a headache today."

[2151] Input: Audio data (or text data)

[2152] Output: None

[2153] Step 2:

[2154] The device converts the voice data into text data and sends it to the server. If the data is text, it is sent as is.

[2155] Input: Audio or text data

[2156] Output: HTTP request to the server

[2157] Step 3:

[2158] The health data received by the server is analyzed using Python's Pandas library and Scikit-learn.

[2159] Input: Physical condition data

[2160] Output: Analysis results

[2161] Step 4:

[2162] The server generates recommendations regarding daily routines and next actions based on the analysis results.

[2163] Input: Analysis results

[2164] Output: Recommendations

[2165] Step 5:

[2166] The server sends the recommendations to the device.

[2167] Input: Recommendations

[2168] Output: HTTP response to the device

[2169] Step 6:

[2170] The device converts the recommendations it receives into audio data using Google's Text-to-Speech API.

[2171] Input: Recommendations

[2172] Output: Audio data

[2173] Step 7:

[2174] The terminal plays the audio data to the user.

[2175] Input: Audio data

[2176] Output: Audio output to the user

[2177] Stress check and advice function processing steps

[2178] Step 1:

[2179] The user engages in everyday conversation, for example, saying, "I was really busy at work today."

[2180] Input: Audio data

[2181] Output: None

[2182] Step 2:

[2183] The device converts the voice data into text data using Google's Cloud Speech-to-Text API.

[2184] Input: Audio data

[2185] Output: Character data

[2186] Step 3:

[2187] The character data generated by the terminal is sent to the server.

[2188] Input: Character data

[2189] Output: HTTP request to the server

[2190] Step 4:

[2191] The server analyzes the received text data using emotion analysis technology.

[2192] Input: Character data

[2193] Output: Analysis results (stress level)

[2194] Step 5:

[2195] The server generates appropriate advice based on the analysis results.

[2196] Input: Analysis results

[2197] Output: Advice data

[2198] Step 6:

[2199] The server transmits the advice data to the terminal.

[2200] Input: Advice data

[2201] Output: HTTP response to the device

[2202] Step 7:

[2203] The advice data received by the device is converted into audio data using Google's Text-to-Speech API.

[2204] Input: Advice data

[2205] Output: Audio data

[2206] Step 8:

[2207] The terminal plays the audio data to the user.

[2208] Input: Audio data

[2209] Output: Audio output to the user

[2210] Processing steps of the Complaint Listening feature

[2211] Step 1:

[2212] The user complains, for example, "Nothing went well today."

[2213] Input: Audio data

[2214] Output: None

[2215] Step 2:

[2216] The device converts the voice data into text data using Google's Cloud Speech-to-Text API.

[2217] Input: Audio data

[2218] Output: Character data

[2219] Step 3:

[2220] The character data generated by the terminal is sent to the server.

[2221] Input: Character data

[2222] Output: HTTP request to the server

[2223] Step 4:

[2224] The server analyzes the received text data using emotion analysis technology.

[2225] Input: Character data

[2226] Output: Analysis results (emotion data)

[2227] Step 5:

[2228] The server generates a message of sympathy based on the analysis results.

[2229] Input: Analysis results

[2230] Output: Empathy message

[2231] Step 6:

[2232] The server sends a sympathy message to the terminal.

[2233] Input: Sympathy message

[2234] Output: HTTP response to the device

[2235] Step 7:

[2236] The device converts the empathy message received into audio data using Google's Text-to-Speech API.

[2237] Input: Sympathy message

[2238] Output: Audio data

[2239] Step 8:

[2240] The terminal plays the audio data to the user.

[2241] Input: Audio data

[2242] Output: Audio output to the user

[2243] (Application example 1)

[2244] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2245] In modern society, parents face the challenge of a lack of support when going out with their children and shopping in brick-and-mortar stores. In particular, parents have limited means to easily gather information about their children's health, growth records, and daily concerns, and to receive appropriate advice. Furthermore, there are insufficient systems in place to alleviate parents' own stress and worries, making it an urgent task to address these issues.

[2246] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[2247] In this invention, the server includes: means for converting questions from a parent or child into text data using voice recognition; means for analyzing the text data using natural language processing technology; and means for retrieving appropriate answers from a database based on the analysis results and outputting the answers audibly. This allows parents to ask questions or seek advice about child-rearing in real time while shopping at a physical store and receive appropriate answers. The server also includes means for collecting physical condition data of the parent or child and analyzing the physical condition data using data analysis technology; and means for generating recommendations regarding daily rhythms and next actions based on the analysis results and outputting the recommendations audibly. This allows parents to understand their child's health status and take appropriate measures even while shopping at a physical store. The server also includes means for recording daily conversations between the parent or child and analyzing the conversation data using stress analysis technology; means for generating appropriate advice based on the analysis results and outputting the advice audibly; and means for allowing the parent or child to input complaints audibly, generating empathetic messages using emotion analysis technology, and outputting the messages audibly. This reduces stress and worries for parents and improves overall child-rearing support.

[2248] definition statement

[2249] "Speech recognition" is a technology that receives speech as input and automatically converts that speech into text data.

[2250] "Character data" refers to data in text format obtained by speech recognition.

[2251] "Natural language processing technology" is a technology that enables computers to understand, analyze, and generate natural human language.

[2252] The "analysis results" are the results of analyzing character data using natural language processing technology.

[2253] A "database" is a system that allows data to be stored, searched, and managed.

[2254] "Audio output means" refers to technology that provides analysis results and answers to users as audio.

[2255] "Physical condition data" refers to data indicating the health condition of a parent or child.

[2256] "Data analysis technology" is a technology for analyzing collected data and extracting useful information.

[2257] "Life rhythm" refers to an individual's activity patterns and time allocation in their daily lives.

[2258] "Recommendations for next actions" are specific actions or advice suggested based on the analysis results.

[2259] "Daily conversation" refers to the verbal exchanges that parents or children have on a daily basis.

[2260] "Stress analysis technology" is a technology that evaluates an individual's stress level based on conversation and behavioral data.

[2261] "Advice" refers to advice or suggestions provided based on the results of the analysis.

[2262] "Guchi" is a word that expresses dissatisfaction or anxiety felt by parents or children.

[2263] "Emotion analysis technology" is a technology that evaluates the emotional state of text data and generates emotional responses based on that data.

[2264] An "empathetic message" is a message that shows understanding and empathy for the user's feelings.

[2265] MODE FOR CARRYING OUT THE INVENTION

[2266] The present invention is a child-rearing support system that uses voice recognition technology, natural language processing technology, database management technology, and data analysis technology. This system provides various child-rearing-related services when parents and children enjoy shopping at physical stores. Specific embodiments are described below.

[2267] System Configuration and Hardware

[2268] This system consists of a smartphone, a cloud server, a voice recognition engine, a natural language processing engine, a database, a text-to-speech engine, etc.

[2269] The smartphone is a device on which a dedicated app is installed and which directly interacts with parents or children. The cloud server processes various data and provides appropriate information to parents or children.

[2270] Specific roles and operations of each technology

[2271] Speech recognition: When a user (parent or child) speaks into a smartphone, a speech recognition engine (such as the Google Speech-to-Text API) converts the speech into text data.

[2272] Natural language processing: Text data is sent to a cloud server and analyzed using a natural language processing engine (such as Google Cloud Natural Language API). For example, if a user says, "It looks like my child has caught a cold," the cloud server analyzes this text data and understands its meaning.

[2273] Database management: Based on the results analyzed by the natural language processing engine, the cloud server retrieves appropriate answers and support information from a database (such as MongoDB). For example, advice such as "Give the child plenty of warm drinks, and if symptoms persist, consult a pediatrician" may be obtained.

[2274] Voice output: The obtained answer is converted into voice by a text-to-speech engine (such as Google Text-to-Speech API) and is transmitted to the user through the smartphone speaker.

[2275] Example of childcare support function

[2276] 1. Question and Answering

[2277] When a user speaks to their smartphone, "My child has a fever. What should I do?", the speech recognition engine converts the speech into text data and sends it to a cloud server. The cloud server then uses natural language processing technology to analyze the text and retrieves the appropriate answer from a database. The answer, "Ensure adequate hydration and consult a doctor if necessary," is then played back as audio by the text-to-speech engine.

[2278] 2. Keeping a record of growth

[2279] When a user says, "Today was my first time riding a bicycle," the speech recognition engine converts this into text data and sends it to the cloud server, which then stores this data in a database and manages it for later review.

[2280] 3. Health check and advice

[2281] When a user says "I have a headache," the speech recognition engine converts this into text data and sends it to the cloud server. The cloud server analyzes this data and generates advice such as "We recommend you drink plenty of fluids and get some rest," which is then conveyed to the user via voice.

[2282] 4. Stress check and emotion analysis

[2283] The system continuously records the user's conversations and periodically analyzes their stress levels. For example, if a user says, "Work was very hard today," the cloud server will evaluate their stress level and provide advice such as, "Take a break to relax."

[2284] Prompt Sentence Examples

[2285] A user might ask, "My child has a cough. What should I do?" and use the following prompt:

[2286] I would like to answer questions about children's health. For example, "Why is the sky blue?" I would like to answer something like, "The sky is blue because sunlight scatters molecules in the atmosphere."

[2287] This provides an environment where parents and children can focus on raising their children with peace of mind while enjoying shopping in a physical store.

[2288] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[2289] Program processing steps

[2290] Step 1:

[2291] The user (parent or child) enters questions or information into the smartphone by voice. For example, the user might say, "My child seems to have caught a cold." This voice data is then entered into the system.

[2292] Step 2:

[2293] The device uses a speech recognition engine (Google Speech-to-Text API) to convert the voice data from step 1 into text data, which then becomes the input for the next step.

[2294] Step 3:

[2295] The device formats the text data as a preprocessing step and sends it to the cloud server. The cloud server receives this text data and analyzes it using a natural language processing engine (Google Cloud Natural Language API). As a result of the analysis, the meaning and intent of the text data are extracted.

[2296] Step 4:

[2297] Based on the analysis results, the server retrieves the appropriate answer corresponding to the question from a database (MongoDB). This database contains a wide variety of questions and their answers in advance. For example, the answer retrieved might be, "Have the child drink plenty of warm fluids, and if symptoms persist, consult a pediatrician."

[2298] Step 5:

[2299] The server sends the answer retrieved from the database to a text-to-speech engine (Google Text-to-Speech API), which converts the answer into audio data, which becomes the input for the next step.

[2300] Step 6:

[2301] The device receives the voice data sent from the text-to-speech engine and conveys it to the user. Specifically, the smartphone speaker plays a voice message saying, "Give your child plenty of warm drinks, and if symptoms persist, consult a pediatrician."

[2302] Step 7:

[2303] The user receives advice and answers from the app and takes appropriate action. At this step, the user gets a solution to their problem.

[2304] Step 8:

[2305] If necessary, the user can again input their daily conversations, physical condition, and growth records by voice, which the system will record and analyze. This step is important so that the system can continuously monitor the user's condition and provide appropriate advice.

[2306] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[2307] This invention is based on an AI system that interacts with parents or children, and incorporates an emotion engine to recognize and respond to the user's emotions. This system combines voice recognition technology, natural language processing technology, database management technology, data analysis technology, and emotion engine technology to address various issues faced by parents and children.

[2308] An embodiment of emotion recognition QA function

[2309] In this system, when a parent or child asks a question, the device uses a voice recognition system to convert the speech into text data. The text data is then sent to a server. The server uses natural language processing technology and an emotion engine to analyze the text and recognize the user's emotional state. Based on this emotional state, the server generates an appropriate response or a tone-adjusted empathetic message, which is then sent from the server to the device. The device then communicates the generated response or empathetic message to the user via voice.

[2310] For example, if a child asks, "Why do I have to go to school?", the device converts the speech into text and sends it to a server. The server recognizes the child's anxiety and stress contained in the question and generates a tailored response: "School is a place where you can make new friends and have fun learning. Don't worry, just relax and have fun," which the device then relays.

[2311] An embodiment of a growth record keeping function using emotion recognition

[2312] The system also uses the emotion engine when parents enter their child's growth record by voice. The device converts the voice into text data, running the emotion engine to send the entered content and emotional state to the server. The server stores the received data in a database and, if necessary, tags the emotional state. Parents can later review the saved growth record and easily search for important events in which a specific emotional state was recorded.

[2313] Example: When a parent says, "Today I was so happy because I rode my bike for the first time," the device converts the speech into text, recognizes the "happy" emotion using an emotion engine, and sends this information to the server. The server then saves the record of "my child rode a bike for the first time" in a database with a tag of "happy."

[2314] An embodiment of a health check and data analysis function using emotion recognition

[2315] This system periodically collects and analyzes the physical condition data of parents and children, while also analyzing the user's emotional state using an emotion engine. The device collects emotional data along with the physical condition data and sends it to a server. The server analyzes the data using data analysis technology and the emotion engine, and based on the analysis results, generates recommendations for daily rhythms and activities, as well as advice for psychological support. The recommendations and advice are sent to the device, which then communicates them to the user via voice.

[2316] Example: If a parent types, "I have a headache today and I'm feeling a bit down," the device will send that data to a server. The server will then send an analysis result, such as, "Drink plenty of fluids, get some rest, and listen to your favorite music to change your mood," and the device will then announce this in voice.

[2317] An embodiment of a stress check and advice function using emotion recognition

[2318] This system records everyday conversations between parents and children and uses an emotion engine to perform stress analysis. The device uses voice recognition to convert the conversation into text data, analyzes the emotional state using the emotion engine, and sends this to a server. The server uses both stress analysis technology and the emotion engine to evaluate stress levels and generate appropriate advice. The evaluation results and advice are sent to the device, which then communicates them to the user via voice.

[2319] Example: If a parent says, "I'm really busy at work today and I'm tired," the device converts the conversation into text, recognizes the "tired" emotion using an emotion engine, and sends it to the server. The server then sends the analysis results, saying, "You seem to be stressed. Take some time to relax and take a deep breath," and the device then conveys this in voice.

[2320] Embodiment of a function for listening to complaints using emotion recognition

[2321] This system also uses the emotion engine when a parent complains. The device uses voice recognition to convert the complaint into text data, analyzes the emotion using the emotion engine, and sends this to the server. The server then uses emotion analysis technology to generate an empathy message, which is tailored based on the parent's emotional state. The empathy message is then sent to the device, which then relays it to the user via voice.

[2322] Example: When a parent complains, "Nothing went well today," the device converts the complaint into text, recognizes the "disappointment" in its emotion engine, and sends it to the server. The server then generates a sympathetic message, saying, "That must have been really tough. Get some rest and start tomorrow with a fresh attitude," and the device then relays this message in voice.

[2323] This allows users (parents and children) to easily ask questions or express their frustrations, and also allows for easy health checks and growth records to be kept. Furthermore, emotion recognition technology allows for more sympathetic responses, providing greater emotional support. Throughout the system, users can raise their children in a comfortable environment.

[2324] The processing flow will be explained below.

[2325] Processing steps for emotion recognition QA function

[2326] Step 1:

[2327] User: Asks the AI ​​robot aloud, "Why do I have to go to school?"

[2328] Step 2:

[2329] Terminal: A voice recognition system is used to convert the user's question from voice data to text data.

[2330] Step 3:

[2331] Terminal: Sends the converted text data to the server.

[2332] Step 4:

[2333] Server: Analyzes the received text data using natural language processing technology and an emotion engine to recognize the user's emotions.

[2334] Step 5:

[2335] Server: Based on the user's emotions, retrieves appropriate answers from the database and generates empathetic messages with adjusted tone.

[2336] Step 6:

[2337] Server: Sends the generated answer and sympathy message to the device.

[2338] Step 7:

[2339] Terminal: Converts the received answers and sympathy messages into audio and conveys them to the user.

[2340] Processing steps for growth record keeping function using emotion recognition

[2341] Step 1:

[2342] User: Through the microphone, says, "Today was my first time riding a bike and I was so happy."

[2343] Step 2:

[2344] Terminal: A speech recognition system is used to convert the user's speech into text data.

[2345] Step 3:

[2346] Terminal: Sends the converted text data to the server.

[2347] Step 4:

[2348] Server: The received text data is analyzed using an emotion engine to recognize the emotion "happy."

[2349] Step 5:

[2350] Server: Stores the text data along with the recognized emotions in a database.

[2351] Step 6:

[2352] Server: Sends a message to the terminal indicating that the save is complete.

[2353] Step 7:

[2354] Terminal: A message will be output to the user indicating that the save is complete.

[2355] Processing steps for health check and data analysis functions using emotion recognition

[2356] Step 1:

[2357] User: Enters "I have a headache today and feel a little depressed" as health data.

[2358] Step 2:

[2359] Terminal: Sends collected physical condition data to the server.

[2360] Step 3:

[2361] Server: Analyzes the received physical condition data and emotion data using data analysis technology and an emotion engine.

[2362] Step 4:

[2363] Server: Generates recommendations for daily routines and behaviors based on the analysis results, and also generates advice for psychological support.

[2364] Step 5:

[2365] Server: Sends the generated recommendations and advice to the device.

[2366] Step 6:

[2367] Device: Provides recommendations and advice to the user via voice.

[2368] Processing steps for stress check and advice function using emotion recognition

[2369] Step 1:

[2370] User: In everyday conversation, say, "I'm really busy at work today and I'm tired."

[2371] Step 2:

[2372] Terminal: Uses a voice recognition system to convert speech into text.

[2373] Step 3:

[2374] Terminal: Sends the converted text data to the server.

[2375] Step 4:

[2376] Server: Analyzes the received text data using stress analysis technology and an emotion engine to evaluate stress levels and emotional states.

[2377] Step 5:

[2378] Server: Generates appropriate advice based on the evaluation results.

[2379] Step 6:

[2380] Server: Sends the generated advice to the device.

[2381] Step 7:

[2382] Terminal: Advice is given to the user by voice.

[2383] Processing steps for listening to complaints using emotion recognition

[2384] Step 1:

[2385] User: Complains to the AI ​​robot, "Nothing went well today."

[2386] Step 2:

[2387] Terminal: Uses a voice recognition system to convert the complaint from voice data to text data.

[2388] Step 3:

[2389] Terminal: Sends the converted text data to the server.

[2390] Step 4:

[2391] Server: Analyzes the received text data using an emotion engine and recognizes "disappointment."

[2392] Step 5:

[2393] Server: Generates empathetic messages based on emotional states.

[2394] Step 6:

[2395] Server: Sends the generated empathy message to the device.

[2396] Step 7:

[2397] Terminal: Conveys empathetic messages to the user via voice.

[2398] Example 2

[2399] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2400] Conventional conversational AI systems simply convert user questions and comments about their physical condition into text data and generate answers. This makes it difficult to respond in a way that takes into account the user's emotional state and psychological aspects, resulting in insufficient psychological support for the user. Furthermore, when storing and managing growth records and physical condition data, data that includes the user's emotional state is not recorded, making it difficult to fully grasp important events or changes in the user's psychological state.

[2401] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[2402] In this invention, the server includes: a means for converting questions from a parent or child into text data using voice recognition; a means for analyzing the text data using natural language processing; an emotion analysis means for recognizing the user's emotional state based on the analysis results; a means for generating an appropriately adjusted answer or empathetic message based on the recognized emotional state; and a means for outputting the generated answer or empathetic message by voice. This enables more friendly responses and psychological support tailored to the user's emotional state. Furthermore, growth records and physical condition data, including emotional states, can be saved and managed, making it easy to understand important events and psychological changes.

[2403] "Speech recognition" is a technology that analyzes voice signals and converts them into text data.

[2404] "Text data" refers to data that includes text information converted by voice recognition.

[2405] "Natural language processing" is a technology that allows computers to understand, analyze, and generate human language.

[2406] "Emotion analysis means" is a technology that recognizes the emotional state of a user from the content of their comments.

[2407] A "database" is a system for efficiently managing, storing, and searching large amounts of data.

[2408] "Data analysis technology" is a technology for analyzing collected data and extracting useful information.

[2409] "Recommendations" are suggestions about behaviors and lifestyle habits provided to users based on the analysis results.

[2410] A "sympathetic message" is a message that expresses sympathy and is generated in accordance with the user's emotional state.

[2411] "Audio output means" refers to a technology that converts text data into audio and conveys it to the user.

[2412] A "growth record" is a record that stores data about a parent's or child's growth through voice recognition and emotion analysis.

[2413] "Physical condition data" is data that includes information related to the user's health condition.

[2414] "Emotional support" refers to providing advice and messages to help users achieve emotional stability and comfort.

[2415] MODE FOR CARRYING OUT THE INVENTION

[2416] This invention is based on an AI system that interacts with parents or children, and incorporates an emotion analysis engine to recognize and respond to the user's emotions. This system combines voice recognition technology, natural language processing technology, database management technology, data analysis technology, and emotion analysis technology to address various issues faced by parents and children.

[2417] Components

[2418] Speech recognition technology: Converts speech into text data. For example, speech recognition software (e.g., Google Cloud Speech-to-Text) is used.

[2419] Natural language processing technology: Analyzes text data and understands its meaning. For example, natural language processing software (e.g., NLP API) is used.

[2420] Database management technology: Technology for effectively storing and managing converted data and analysis results. For example, a database management system (e.g., MySQL) is used.

[2421] Data analysis technology: Technology to analyze collected physical condition data and growth records and extract information useful to users. For example, data analysis libraries (e.g., Pandas, NumPy) are used.

[2422] Emotion analysis technology: Technology that recognizes the user's emotional state and generates appropriate responses based on that information. As a specific example, we will use an emotion analysis engine (e.g., Affectiva API).

[2423] Program processing

[2424] 1. Speech recognition process: The device converts the user's questions or speech into text data using speech recognition software, such as Google Cloud Speech-to-Text.

[2425] 2. Data transmis...

Claims

1. a means for converting questions from a parent or child into text data by speech recognition; means for analyzing the character data using natural language processing technology; means for retrieving an appropriate answer from a database based on the analysis result and outputting the answer by voice; A system including:

2. a means for converting the parent's or child's growth record into character data by voice recognition and storing the character data in a database; a means for managing and displaying the stored growth record; The system of claim 1 , comprising:

3. A means for collecting physical condition data of the parent or child and analyzing the physical condition data using data analysis technology; means for generating recommendations regarding lifestyle rhythms and next actions based on the analysis results and outputting the recommendations by voice; The system of claim 1 , comprising:

4. means for converting everyday conversations between a parent and a child into text data by voice recognition and analyzing the text data using stress analysis technology; means for evaluating a stress level based on the analysis results; means for generating advice according to the evaluated stress level and outputting the advice by voice; The system of claim 1 , comprising:

5. a means for converting complaints from parents or children into text data by speech recognition and analyzing the text data using emotion analysis technology; means for generating a sympathy message based on the analysis result and outputting the sympathy message by voice; The system of claim 1 , comprising:

Images