System

Abstract

To provide a system.SOLUTION: Provided is a system comprising: a stuffed toy that acts as a conversation partner for a child who starts to talk using generative AI and speech synthesis technology; means for generating AI specialized for parent-child households using daily parent-child conversation at home as an additional learning material; means for proposing an optimal external service for the child on the basis of at least either of the parent-child conversation and the conversation between the stuffed toy and the child; means for grasping at least either of health and emotion states of the child from at least either of the parent-child conversation and the conversation between the stuffed toy and proposing an appropriate meal on the basis of the grasped result; and means for retrieving at least either of a restaurant and a delivery service on the basis of the proposed meal.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In the early stages when children start to talk, it is difficult for parents to maintain constant dialogue with their children, and it is also difficult to suggest appropriate external services based on the child's interests, preferences, health status, and lifestyle habits. [Means for solving the problem]

[0005] Using generative AI and speech synthesis technology, we create a stuffed animal that can communicate with children who start talking. Using everyday parent-child conversations at home as additional learning material, the AI ​​becomes specialized for each household. Based on conversations between parents and children, and between the AI ​​and children, we can connect to optimal external services (recommending picture books and toys suitable for the child, or suggesting hospitals and leisure facilities). [Brief explanation of the drawings]

[0006] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 2 is a sequence diagram showing a flow of processing in the data processing system according to the first embodiment of the first form example. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Embodiment 1. [Figure 13] FIG. 10 is a sequence diagram showing a processing flow of a data processing system in a second embodiment of the second form example. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 of Embodiment Example 2. [Figure 15] FIG. 10 is a sequence diagram showing the flow of processing in a data processing system according to a third embodiment of the third embodiment. [Figure 16]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 3 of Embodiment 3. [Figure 17] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the first embodiment of the first form example when an emotion engine is combined. [Figure 18] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Form Example 1 when an emotion engine is combined. [Figure 19] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the second embodiment of the second form example when an emotion engine is combined. [Figure 20] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 of Form Example 2 when an emotion engine is combined. [Figure 21] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the third embodiment of the third form example when an emotion engine is combined. [Figure 22] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 3 of Form Example 3 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0007] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0008] First, the terms used in the following description will be explained.

[0009] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, the processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (TENSOR PROCESSING UNIT (registered trademark)).

[0010] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0011] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0012] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0013] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0014] [First embodiment]

[0015] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0016] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0017] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0018] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0019] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0020] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0021] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0022] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0023] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0024] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0025] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0026] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0027] "Example 1"

[0028] An example of the present invention is a stuffed toy device. This device has an internal microphone and speaker, and communicates with children using generative AI and voice synthesis technology. The microphone picks up the voices of the child and parent and sends the voice data to the AI. The AI ​​analyzes the voice data and generates an appropriate response. The generated response is output from the speaker using voice synthesis technology, realizing a dialogue with the child.

[0029] "Example 2"

[0030] Furthermore, AI learns a child's interests and preferences from parent-child conversations. For example, if it determines that a child is interested in dinosaurs, it will provide topics related to dinosaurs. Also, if it understands a child's favorite foods and activities, it will suggest related picture books and toys. These suggestions help parents provide appropriate educational materials for their children.

[0031] "Example 3"

[0032] AI also understands a child's health condition and lifestyle habits through conversations between parents and children. For example, if it determines that a child has a cold, it will provide information about nearby hospitals. If it determines that a child is not getting enough exercise, it will provide information about nearby parks and leisure facilities. These suggestions help parents manage their children's health.

[0033] The processing flow of each embodiment will be described below.

[0034] "Example 1"

[0035] Step 1: The microphone on the plush toy device picks up the child's or parent's voice.

[0036] Step 2: Send the captured audio data to the AI.

[0037] Step 3: The AI ​​analyzes the voice data and generates an appropriate response.

[0038] Step 4: The generated response is output from the speaker using speech synthesis technology, enabling dialogue with the child.

[0039] "Example 2"

[0040] Step 1: The AI ​​learns about the child's interests and preferences from parent-child conversations.

[0041] Step 2: Based on the information it has learned, the AI ​​will provide topics that match the child's interests and preferences. Step 3: The AI ​​will understand the child's favorite foods and games and suggest related picture books and toys.

[0042] "Example 3"

[0043] Step 1: The AI ​​understands the child's health and lifestyle habits from parent-child conversations and dialogue with the child.

[0044] Step 2: The AI ​​assesses the child's health condition and provides information about nearby hospitals if necessary.

[0045] Step 3: The AI ​​will assess the child's lifestyle and provide information on nearby parks and leisure facilities if necessary.

[0046] Example 1

[0047] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0048] In modern families, children often lack someone they can talk to with peace of mind. It is also difficult to suggest appropriate external services based on parent-child conversations and the child's interests and preferences. It is also difficult to understand a child's health and lifestyle habits and provide appropriate advice based on them.

[0049] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0050] In this invention, the server includes: a means for using generative AI and speech synthesis technology to create a stuffed animal that can communicate with a child who has begun to talk; a means for creating an AI specialized for a particular household using everyday parent-child conversations as additional learning material; a means for connecting the parent-child conversation and the AI-child conversation to optimal external services (recommending picture books and toys suitable for the child, suggesting hospitals and leisure facilities) based on the conversation; a means for collecting voice data using a microphone; a means for transmitting the collected voice data to the server; a means for the server to analyze the voice data and generate an appropriate response using a generative AI model; a means for converting the generated response into voice data using speech synthesis technology; and a means for outputting the converted voice data from a speaker. This provides an environment where children can interact with ease and makes it possible to suggest appropriate external services based on parent-child conversations. It also makes it possible to understand the child's health condition and lifestyle habits and provide appropriate advice based on them.

[0051] "Generative AI" is an artificial intelligence technology that analyzes voice data and generates appropriate responses.

[0052] "Speech synthesis technology" is a technology that converts text data into voice data.

[0053] A "stuffed toy" is a device that has a microphone and speaker inside and allows for communication with children.

[0054] A "microphone" is a device for collecting sound.

[0055] A "speaker" is a device for outputting audio data as sound.

[0056] A "server" is a computer system that analyzes voice data, runs generative AI models, and applies voice synthesis technology.

[0057] "Audio data" refers to digital data of audio collected by a microphone.

[0058] A "prompt sentence" is text data that is input into a generative AI model and serves as the basis for generating a response.

[0059] "External services" are services provided outside the home, such as recommending picture books and toys suitable for children, and suggesting hospitals and leisure facilities.

[0060] "Additional learning" is the process by which the generative AI learns new knowledge based on everyday conversations between parents and children.

[0061] This invention uses generative AI and voice synthesis technology to provide a stuffed toy device that can serve as a conversation partner for children. Specific embodiments of this system are described below.

[0062] Hardware Configuration

[0063] Terminal: The plush toy device includes the following hardware:

[0064] Microphone: A device for collecting the voices of children and parents. It has high sensitivity and the ability to filter out ambient noise.

[0065] Speaker: A device for outputting the generated audio data. It provides high-quality audio output and plays it at a volume that is easy for children to hear.

[0066] Communication module: A device for transmitting audio data to a server. It can use Wi-Fi or Bluetooth.

[0067] Server: The server includes the following software:

[0068] Generative AI model: An artificial intelligence technology that analyzes voice data and generates appropriate responses. For example, OpenAI's GPT-3 (registered trademark) is used.

[0069] Speech recognition technology: Technology that converts voice data into text data. For example, Google (registered trademark) Cloud Speech-to-Text API is used.

[0070] Speech synthesis technology: Technology that converts text data into voice data. For example, Google Text-to-Speech API or Amazon Polly are used.

[0071] Software Configuration

[0072] Terminal: The plush toy device software includes the following functions:

[0073] Audio data collection: Audio data is collected using a microphone.

[0074] Sending audio data: Send the collected audio data to the server.

[0075] Server: The server software includes the following functions:

[0076] Analysis of voice data: The received voice data is input into the generative AI model and analyzed.

[0077] Response generation: The generative AI model generates an appropriate response based on the prompt.

[0078] Speech synthesis: The generated text response is converted into audio data using speech synthesis technology.

[0079] Send audio data: Send the converted audio data to the device.

[0080] Specific examples

[0081] For example, suppose a child asks, "What shall we play today?" In this case, the following processing is performed:

[0082] 1. Collecting voice data: The device's microphone picks up the child's voice and collects voice data.

[0083] 2. Sending audio data: Send the collected audio data to the server.

[0084] 3. Analysis of voice data: The server inputs the received voice data into the generative AI model and analyzes it. The prompt sentence used is "When a child asks, 'What shall we play today?', generate an appropriate response."

[0085] 4. Response generation: The generative AI model generates the response, "Why don't you try playing ball outside today?"

[0086] 5. Speech synthesis: The generated text response is converted into voice data using speech synthesis technology.

[0087] 6. Sending audio data: Send the converted audio data to the device.

[0088] 7. Audio output: The device speaker plays the audio, "Why don't you go outside and play ball today?"

[0089] In this way, the stuffed toy device allows interaction with the child.

[0090] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0091] Step 1:

[0092] Audio data collection

[0093] Device: The microphone in the plush toy picks up the voices of the child and parent. For example, when a child asks, "What shall we play today?", that voice is input into the microphone.

[0094] Input: Child or parent voice

[0095] Output: Digital audio data

[0096] How it works: The microphone is highly sensitive and has the ability to filter out ambient noise. It converts audio into digital data and temporarily stores it in the internal memory.

[0097] Step 2:

[0098] Sending audio data

[0099] Terminal: The collected voice data is sent to the server. Specifically, the communication module in the terminal converts the voice data into packets and sends them to the server via the Internet.

[0100] Input: Digital audio data

[0101] Output: Audio data sent to the server

[0102] Specific operation: The communication module uses Wi-Fi or Bluetooth to convert voice data into packets and transmit them.

[0103] Step 3:

[0104] Analysis of audio data

[0105] Server: Inputs the received voice data into a generative AI model, such as OpenAI's GPT-3. The server converts the voice data into text data and inputs it as a prompt to the generative AI model.

[0106] Input: Audio data sent to the server

[0107] Output: Text data

[0108] Specific operation: Uses speech recognition technology (e.g., Google Cloud Speech-to-Text API) to convert voice data into text data.

[0109] Step 4:

[0110] Generating a response

[0111] Server: The generative AI model generates an appropriate response based on the prompt, for example, "Why don't you try playing ball outside today?"

[0112] Input: Text data (prompt sentence)

[0113] Output: Response text

[0114] How it works: The generative AI model generates responses by taking into account past interaction history and context.

[0115] Step 5:

[0116] Speech synthesis

[0117] Server: The generated text response is converted into voice data using speech synthesis technology, specifically using a speech synthesis service such as Google Text-to-Speech API or Amazon Polly.

[0118] Input: Response text

[0119] Output: Audio data

[0120] Specific operation: Speech synthesis technology converts text data into speech data to achieve natural pronunciation and intonation.

[0121] Step 6:

[0122] Audio output

[0123] Device: Receives the voice data sent from the server and outputs it from the speaker. For example, the device speaker plays the voice "Why don't you play ball outside today?"

[0124] Input: Audio data sent from the server

[0125] Output: Audio (sound coming from the speaker)

[0126] What it does: The speaker provides high-quality audio output at a volume that is easy for children to hear.

[0127] (Application example 1)

[0128] Next, a description will be given of Application Example 1 of Embodiment Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0129] Conventional stuffed toy devices were limited to interacting with children and were not designed to support shopping in physical stores or improve customer service. While they had the ability to learn about parent-child conversations and children's interests and preferences, they were not able to guide customers in physical stores, provide sales information, or respond to customers in real time. This created a need for new ways to improve the customer experience in physical stores.

[0130] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0131] In this invention, the server includes means for using generative AI and speech synthesis technology to create a stuffed animal that will be a conversation partner for a child who starts talking, means for using everyday parent-child conversations at home as additional learning material to create an AI specialized for that household, means for connecting optimal external services (recommending picture books and toys suitable for the child, suggesting hospitals and leisure facilities) based on parent-child conversations and conversations between the AI ​​and the child, means for providing an assistant application that supports shopping in a physical store, means for providing voice guidance on the location and features of products in the store, sales information, etc., and means for responding to customer questions in real time. This makes it possible to improve the customer experience in a physical store.

[0132] "Generative AI" is an artificial intelligence technology that analyzes voice and text data and generates appropriate responses.

[0133] "Speech synthesis" is a technology that converts text data into speech.

[0134] A "stuffed toy" is a device that has a microphone and speaker inside and allows for communication with children.

[0135] "Parent-child conversation" refers to the everyday dialogue that takes place between parents and children at home.

[0136] "Additive learning" is the process of learning by adding new data to existing data.

[0137] "External services" are services provided outside the home, such as recommending picture books and toys, or suggesting hospitals and leisure facilities.

[0138] A "brick and mortar store" is a store that exists in a physical location and offers goods and services.

[0139] An "assistant application" is an application that is installed on a smartphone or other device and supports the user's actions.

[0140] "Product location" refers to the location where a particular product is located within a physical store.

[0141] "Sale information" is information about product discounts and special offers.

[0142] "Real-time response" refers to providing an immediate response to a user's question.

[0143] As an embodiment of this invention, we will explain a system that provides an assistant application that supports shopping in a physical store. This system uses generative AI and speech synthesis technology to respond to customer questions in real time and provide voice guidance on the location and features of products in the store, sale information, etc.

[0144] System configuration

[0145] Hardware:

[0146] Smartphone (microphone, speaker)

[0147] software:

[0148] speech_recognition library: Performs speech recognition.

[0149] pyttsx3 library: Performs speech synthesis.

[0150] openai library: Generate responses using generative AI models.

[0151] Processing flow

[0152] 1. Speech Recognition:

[0153] The user speaks a question into the smartphone's microphone.

[0154] The smartphone converts the voice data into text data using the speech_recognition library.

[0155] 2. Response generation by generative AI:

[0156] The server sends the converted text data to the generative AI model using the openai library.

[0157] The generative AI model generates appropriate responses based on the input text data.

[0158] The generated response is returned to the server in text format.

[0159] 3. Speech synthesis:

[0160] The server converts the generated text response into audio data using the pyttsx3 library.

[0161] Audio data is output from the smartphone speaker, conveying the response to the user.

[0162] Specific examples

[0163] For example, if a user asks, "Where is this item located?" the system operates as follows:

[0164] 1. The user's question, "Where is this product located?" is recorded by the smartphone microphone.

[0165] 2. The speech_recognition library converts the audio data into text data.

[0166] 3. The converted text data is sent to the generative AI model.

[0167] 4. The generative AI model generates an appropriate response, such as "The item is on the third floor, on the left."

[0168] 5. The generated response is converted to audio data using the pyttsx3 library.

[0169] 6. A voice will be output from the smartphone speaker saying, "The product is on the third floor, on the left side."

[0170] Prompt Sentence Examples

[0171] An example of a prompt sent to the generative AI model is as follows:

[0172] Generate appropriate responses to customer questions: Where is this item located?

[0173] This will create a smart shopping assistant that improves the customer experience in physical stores.

[0174] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0175] Step 1:

[0176] The user speaks a question into the smartphone microphone. The input is the user's voice data, and the output is the voice data captured by the smartphone microphone. This voice data is processed in the following steps:

[0177] Step 2:

[0178] The device converts the voice data into text data using the speech_recognition library. The input is the voice data acquired in step 1, and the output is text data. Through this conversion, the user's question is obtained in text format.

[0179] Step 3:

[0180] The terminal sends the converted text data to the server. The input is the text data generated in step 2, and the output is the text data sent to the server. This data is input to the generative AI model in the next step.

[0181] Step 4:

[0182] The server uses the openai library to send text data to the generative AI model and generate an appropriate response. The input is the text data sent in step 3, and the output is the text data of the generated response. This response is an appropriate response to the user's question.

[0183] Step 5:

[0184] The server converts the generated text response into audio data using the pyttsx3 library. The input is the text data generated in step 4, and the output is audio data. This conversion results in the text response in audio format.

[0185] Step 6:

[0186] The server sends the converted voice data to the terminal. The input is the voice data generated in step 5, and the output is the voice data sent to the terminal. This data is provided to the user in the next step.

[0187] Step 7:

[0188] The terminal outputs the voice data from the smartphone speaker and communicates the response to the user. The input is the voice data sent in step 6, and the output is the voice response that the user can hear. This allows the user to receive an appropriate answer to their question by voice.

[0189] Example 2

[0190] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0191] Conventional systems that suggest educational materials for children have the problem of being unable to address the specific interests and preferences of individual children because they make suggestions based on general interests and preferences. Furthermore, there has been no system that utilizes parent-child conversations to learn about a child's interests and preferences and then suggests appropriate educational materials based on that information. This makes it difficult for parents to provide their children with the most appropriate educational materials.

[0192] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0193] In this invention, the server includes: means for using generation AI and speech synthesis technology to create a stuffed animal that will serve as a conversation partner for a child who has begun to talk; means for using everyday parent-child conversations at home as additional learning material to create an AI specialized for that family; means for connecting to optimal external services (proposing educational materials suitable for the child) based on parent-child conversations and conversations between the AI ​​and the child; means for collecting parent-child conversation data and converting the voice data into text data; means for analyzing the text data using natural language processing technology to identify the child's interests and preferences; means for generating suggestions for related topics and educational materials based on the identified interests and preferences; and means for providing the generated suggestions to parents, thereby enabling parents to provide appropriate educational materials based on their child's specific interests and preferences.

[0194] "Generative AI" is artificial intelligence that uses natural language processing technology to analyze text data and generate specific information.

[0195] "Speech synthesis" is a technology that converts text data into voice data.

[0196] A "stuffed animal" is a soft toy made to be a conversation companion for children.

[0197] "Parent-child conversation" refers to the everyday dialogue that takes place between parents and children at home.

[0198] "Additive learning" is the process of updating an AI model by adding new data to existing training data.

[0199] An "external service" is a service that provides resources or information outside the system.

[0200] "Educational materials" is a general term for books, toys, teaching materials, etc. that are used to support children's learning and development.

[0201] "Audio data" refers to data in which audio is recorded in digital format.

[0202] "Text data" refers to data in which character information is recorded in digital format.

[0203] "Natural language processing technology" is a technology that allows computers to understand, interpret, and generate human language.

[0204] "Interests and preferences" are the things that an individual is particularly interested in or likes.

[0205] A "suggestion" is a recommended course of action or option for a particular purpose.

[0206] This invention is a system that learns a child's interests and preferences from parent-child conversations and suggests appropriate educational materials based on this information. A specific embodiment of this system will be described below.

[0207] 1. Program Generation

[0208] The server generates a program to learn the child's interests and preferences from parent-child conversations. The program analyzes the conversation data using natural language processing technology to identify the child's interests and preferences.

[0209] 2. Program processing explanation

[0210] The server processes data using the following hardware and software:

[0211] Hardware:

[0212] Server: A computer with high-performance data processing capabilities

[0213] Device: A device equipped with a microphone for recording parent-child conversations

[0214] software:

[0215] Speech recognition software: Google Speech-to-Text API

[0216] Natural language processing technologies: generative AI models (e.g., OpenAI's GPT-4®)

[0217] 3. Data processing and calculation

[0218] The server processes and calculates the data in the following steps:

[0219] 1. Data Collection:

[0220] The terminal records the conversation between parent and child and transmits the audio data to the server.

[0221] 2. Data conversion:

[0222] The server converts the received voice data into text data using the Google Speech-to-Text API.

[0223] 3. Data Analysis:

[0224] The server inputs the text data into a generative AI model to extract information about the child's interests and preferences.

[0225] 4. Identifying interests and preferences:

[0226] The server analyzes the output of the generative AI model and calculates the frequency of occurrence of certain keywords (e.g., "dinosaur").

[0227] 5. Proposal generation:

[0228] The server retrieves information on related picture books and toys from a database and creates a suggestion list.

[0229] 6. Providing Proposals:

[0230] The server sends the suggestion list to the terminal, which displays the information to the parent.

[0231] 4. Specific Examples

[0232] If conversation data is collected showing that a child frequently uses the word "dinosaur," the server will suggest picture books and toys related to dinosaurs.

[0233] If a child says, "I like pizza," the server will suggest pizza-related picture books and cooking kits.

[0234] 5. Examples of prompts

[0235] "If a child is interested in dinosaurs, what picture books and toys should I suggest?"

[0236] "What educational materials should I provide my child if their favorite food is pizza?"

[0237] The system allows parents to provide appropriate educational materials based on their child's specific interests and preferences.

[0238] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0239] Step 1:

[0240] Data collection

[0241] Input: Parent-child conversation audio data

[0242] Specific operation: The device records the conversation between parent and child. The recorded audio data is sent from the device to the server.

[0243] Output: Audio data sent to the server

[0244] Step 2:

[0245] Data Conversion

[0246] Input: Audio data

[0247] Specific operation: The server sends the received voice data to the Google Speech-to-Text API and converts the voice data into text data.

[0248] Output: Text data

[0249] Step 3:

[0250] Data analysis

[0251] Input: Text data

[0252] How it works: The server inputs the text data into a generative AI model (e.g., OpenAI's GPT-4) to extract information about the child's interests and preferences.

[0253] Output: Information about interests and preferences

[0254] Step 4:

[0255] Identifying interests and preferences

[0256] Input: Information about your interests and preferences

[0257] Specific operation: The server analyzes the output of the generative AI model and calculates the frequency of occurrence of specific keywords (e.g., "dinosaur").

[0258] Output: Identified interests and preferences

[0259] Step 5:

[0260] Proposal Generation

[0261] Input: Identified interests and preferences

[0262] Specific operation: The server retrieves information on related picture books and toys from the database and creates a list of suggestions.

[0263] Output: Suggestion list

[0264] Step 6:

[0265] Providing suggestions

[0266] Input: Suggestion list

[0267] Specific operation: The server sends the proposal list to the terminal, and the terminal displays the information to the parent.

[0268] Output: Suggestion list displayed in parent

[0269] (Application example 2)

[0270] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0271] Conventional educational support systems for children have limited functionality for learning about children's interests and preferences from parent-child conversations, and lack functionality to support the shopping experience in physical stores. Furthermore, there is no way for parents to instantly check detailed information or reviews of suggested products, making it difficult for parents to select appropriate educational materials. Furthermore, there is no navigation function to guide users to the location of products in the store, which makes shopping less efficient.

[0272] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for using generation AI and speech synthesis technology to create a stuffed animal that can be a conversation partner for a child who has started talking; means for using everyday parent-child conversations at home as additional learning material to create an AI specialized for that family; means for connecting optimal external services (recommending picture books and toys suitable for the child, suggesting hospitals and leisure facilities) based on parent-child conversations and conversations between the AI ​​and the child; means for suggesting related products when shopping in a physical store; means for displaying detailed information and reviews about the suggested products; and means including a navigation function for guiding the location of products in the store. This allows the server to learn a child's interests and preferences from parent-child conversations, support the shopping experience in a physical store, and enable efficient and appropriate selection of educational materials.

[0273] "Generative AI" is an artificial intelligence that learns a child's interests and preferences from conversations between parents and children, and then conducts a dialogue based on that information.

[0274] "Speech synthesis technology" is a technology that converts text data into voice data and generates natural speech.

[0275] A "stuffed toy" is a soft toy that children can talk to, and it communicates using generative AI and voice synthesis technology.

[0276] "Parent-child conversation" refers to the everyday dialogue that takes place between parents and children at home.

[0277] "Additional learning materials" are data that the generative AI uses to learn new information through conversations between parents and children and accumulate knowledge specific to that household.

[0278] "External services" are external resources and services such as picture books, toys, hospitals, and leisure facilities that are suggested by the generative AI based on the information it has learned.

[0279] A "brick and mortar store" is a retail establishment that exists in a physical location where customers visit in person to purchase goods.

[0280] "Related products" are educational materials such as picture books and toys that the generative AI suggests based on the child's interests and preferences learned from parent-child conversations.

[0281] "Detailed information" refers to specific information about the features, specifications, and usage of the proposed product.

[0282] "Reviews" refer to other users' ratings and opinions of the proposed product.

[0283] The "navigation function" is a function that guides customers to the location of suggested products within a physical store, helping them find products efficiently.

[0284] A system for implementing this invention includes a means for using generative AI and speech synthesis technology to create a stuffed animal that will act as a conversation partner when a child begins to talk; a means for using everyday parent-child conversations at home as additional learning material to create an AI specialized for that household; a means for connecting to optimal external services (recommending picture books and toys suitable for the child, or suggesting hospitals and leisure facilities) based on the parent-child conversation and the conversation between the AI ​​and the child; a means for suggesting related products when shopping in a physical store; a means for displaying detailed information and reviews about the suggested products; and a means including a navigation function for guiding the location of products within the store.

[0285] System Program

[0286] Hardware used

[0287] Smartphone (microphone, speaker)

[0288] Server (computing resources for running generative AI models)

[0289] Software used

[0290] speech_recognition library: Used to convert speech data into text.

[0291] openai library: Used to extract interests and preferences from text using generative AI models.

[0292] Data processing and calculation

[0293] 1. Acquiring audio data: The conversation between parent and child is recorded using the smartphone's microphone. The user holds the smartphone and the parent and child have a conversation.

[0294] 2. Speech Recognition: Converts voice data to text using the speech_recognition library. The server receives the voice data and converts it to text data.

[0295] 3. Extracting interests and preferences: Using the OpenAI library, we extract children's interests and preferences from the text. The server runs a generative AI model to extract interests and preferences from the text data.

[0296] 4. Save the results: The extracted interests and preferences are saved in JSON format. The server saves the learning results in a database.

[0297] 5. Product Suggestion: Suggests related products when shopping in a physical store. When a user has a smartphone and is shopping in a physical store, the server suggests related products.

[0298] 6. Displaying details and reviews: Displaying details and reviews about the suggested product. The user checks the details and reviews of the suggested product on their smartphone.

[0299] 7. Navigation function: Provides a navigation function that guides the location of products within the store. The user can use their smartphone to check the location of suggested products within the store.

[0300] Specific examples

[0301] Parent-child conversation: "Mom, I want to read a book about dinosaurs!" "Yes, dinosaurs are interesting."

[0302] Prompt Sentence Examples

[0303] Identify your child's interests and preferences from the following conversation:

[0304] Mommy, I want to read a book about dinosaurs!

[0305] Yeah, dinosaurs are fun.

[0306] This system learns about a child's interests and preferences from parent-child conversations, supports the shopping experience in physical stores, and enables the efficient selection of appropriate educational materials.

[0307] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0308] Step 1:

[0309] A user uses a smartphone to record a conversation between parent and child. The smartphone's microphone captures the audio data and sends it to a server. The input is the audio data of the parent-child conversation, and the output is the audio data sent to the server.

[0310] Step 2:

[0311] The server converts the received voice data into text data using the speech_recognition library. The input is voice data and the output is text data. The server analyzes the voice data and performs speech recognition.

[0312] Step 3:

[0313] The server uses a generative AI model to extract children's interests and preferences from text data. The input is text data, and the output is the extracted interest and preference information. The server uses the openai library to generate prompt sentences and input them into the generative AI model.

[0314] Step 4:

[0315] The server saves the extracted interest and preference information in JSON format. The input is the extracted interest and preference information, and the output is the saved JSON data. The server saves the learning results in a database.

[0316] Step 5:

[0317] When a user goes shopping in a physical store, their smartphone connects to a server and suggests related products. The input is the user's location information and extracted information about their interests and preferences, and the output is a list of suggested products. The server obtains the user's location information and searches for related products.

[0318] Step 6:

[0319] The user checks the details and reviews of the suggested products on their smartphone. The input is a list of suggested products, and the output is the details and reviews. The server retrieves the product details and reviews from the database and sends them to the smartphone.

[0320] Step 7:

[0321] The user uses a smartphone to check the location of suggested products in a store. The input is a list of suggested products and store map information, and the output is product location information. The server uses the store map information to provide navigation to the product location.

[0322] Example 3

[0323] Next, a description will be given of a third embodiment of the third embodiment. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0324] In modern families, it is important to understand a child's health condition and lifestyle through parent-child communication and provide appropriate external services. However, it is difficult for parents to constantly understand their child's condition and take appropriate measures. It is also not easy to have appropriate conversations based on the child's interests and preferences. To solve these issues, a system is needed that understands a child's condition through parent-child conversations and dialogue with the child, and suggests appropriate external services.

[0325] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[0326] In this invention, the server includes means for using generation AI and speech synthesis technology to create a dialogue device that will serve as a conversation partner for a child who begins to speak, means for using everyday parent-child conversations at home as additional learning material to create an AI specialized for that household, means for connecting optimal external services (recommending books and toys suitable for the child, suggesting medical institutions and leisure facilities) based on parent-child conversations and conversations between the AI ​​and the child, means for collecting voice data, means for converting the voice data into text data, means for analyzing the text data to understand the child's health condition and lifestyle habits, and means for suggesting external services based on the analysis results. This makes it easier for parents to understand their child's health condition and lifestyle habits and enables them to take appropriate measures quickly.

[0327] "Generative AI" is an artificial intelligence technology that generates new information and dialogue based on voice and text data.

[0328] "Speech synthesis" is a technology that converts text data into voice data and produces speech that sounds like a human voice.

[0329] An "interactive device" is a combination of hardware and software for audibly interacting with a user.

[0330] "Additive learning" is the process of adding new data to an existing artificial intelligence model to train it and improve its performance and accuracy.

[0331] "External services" refers to information about services and facilities that exist outside the system, and provide suggestions and guidance to users.

[0332] "Audio data" refers to digital data of sounds collected through an audio input device such as a microphone.

[0333] "Text data" is character information converted from voice data using voice recognition technology.

[0334] "Natural Language Processing (NLP)" is a technology that enables computers to understand, analyze, and generate human language.

[0335] "Health status" refers to the state of a child's physical and mental health.

[0336] "Lifestyle habits" refers to the behaviors and habits of children in their daily lives.

[0337] "Analysis results" are analysis results obtained using natural language processing or other data analysis techniques.

[0338] This invention is a system that understands a child's health condition and lifestyle habits through parent-child conversations and dialogue with the child, and then suggests appropriate external services. This system uses generative AI models and speech synthesis technology to create a dialogue device that acts as a conversation partner for the child, and uses the parent-child conversations as material for additional learning to build an AI specialized for that household. Furthermore, it provides optimal external services (recommending books and toys, suggesting medical institutions and leisure facilities) based on the parent-child conversations and the conversations between the AI ​​and the child.

[0339] Hardware and software used

[0340] Hardware: microphones, speakers, cameras, servers

[0341] Software: speech recognition software (e.g., Google Speech-to-Text), natural language processing (NLP) software (e.g., OpenAI GPT-4), database management systems (e.g., MySQL®)

[0342] Specific operation of the system

[0343] 1. Collection of audio data

[0344] The device uses a microphone to record conversations between parents and children, and dialogue with the child. The device keeps the microphone on at all times and starts recording when certain keywords are detected.

[0345] 2. Audio data conversion

[0346] The device converts the recorded voice data into text data using voice recognition software (Google Speech-to-Text). The device sends the voice data to the Google Speech-to-Text API in real time and temporarily stores the returned text data.

[0347] 3. Text Data Analysis

[0348] The server analyzes the text data using natural language processing (NLP) software (OpenAI GPT-4) to understand the child's health condition and lifestyle habits. The server receives the text data and sends prompts to OpenAI GPT-4 to obtain the analysis results.

[0349] 4. Health status assessment

[0350] Based on the analysis results, the server determines the child's health condition, such as whether they have a cold or are not getting enough exercise. The server evaluates the analysis results and determines the corresponding health condition if certain keywords or phrases are included.

[0351] 5. Information provision

[0352] Based on the results of the judgment, the server retrieves information about nearby hospitals, parks, and leisure facilities from a database management system (MySQL) and provides it to the parent. The server then sends a query to the MySQL database to retrieve information about appropriate facilities. The retrieved information is then sent to the parent's device.

[0353] Specific examples

[0354] Example 1: When a child says, "My throat hurts"

[0355] The device records this conversation and converts it into text using voice recognition software.

[0356] The server analyzes the text and determines that the child has a cold.

[0357] The server retrieves information about nearby hospitals from a database and provides it to the parents, for example, by sending a notification saying, "The nearest hospital is ABC Clinic. The address is 123 Main St. and the phone number is 555-1234."

[0358] Example 2: If your child says, "I haven't been playing outside much lately"

[0359] The device records this conversation and converts it into text using voice recognition software.

[0360] The server analyzes the text and determines that the child is not getting enough exercise.

[0361] The server retrieves information about nearby parks and recreational facilities from a database and provides it to parents, for example, by sending a notification saying, "The nearest park is XYZ Park. The address is 456 Park Ave."

[0362] Prompt Sentence Examples

[0363] Prompt 1: "Please determine if the child has a cold and provide information about nearby hospitals."

[0364] Prompt 2: "Please determine if your child is physically inactive and provide information about nearby parks and recreational facilities."

[0365] This system allows parents to easily understand their children's health conditions and lifestyle habits, and enables them to take appropriate measures promptly. The flow of the identification process in the third embodiment will be described with reference to FIG.

[0366] Step 1:

[0367] Audio data collection

[0368] The device uses a microphone to record conversations between parents and children, as well as interactions with the child.

[0369] Input: Audio of parent-child conversations and interactions with children.

[0370] What it does: The device keeps the microphone on at all times and starts recording when certain keywords (e.g., "Good morning" or "What's wrong?") are detected.

[0371] Output: Recorded audio data.

[0372] Step 2:

[0373] Audio data conversion

[0374] The voice data recorded by the device is converted into text data using voice recognition software (Google Speech-to-Text).

[0375] Input: Recorded audio data.

[0376] How it works: The device sends voice data to the Google Speech-to-Text API in real time and temporarily stores the returned text data.

[0377] Output: The converted text data.

[0378] Step 3:

[0379] Text data analysis

[0380] The server analyzes the text data using natural language processing (NLP) software (OpenAI GPT-4) to understand the child's health and lifestyle habits.

[0381] Input: The converted text data.

[0382] Specific operation: The server receives the text data and sends a prompt to OpenAI GPT-4 to obtain the analysis result. For example, the prompt might be, "If a child says they have a sore throat, is it possible they have a cold?"

[0383] Output: Analysis results.

[0384] Step 4:

[0385] Health status assessment

[0386] Based on the analysis results, the server determines the child's health condition, such as whether they have a cold or are not getting enough exercise.

[0387] Input: Analysis results.

[0388] What happens: The server evaluates the analysis results and determines the corresponding health status if certain keywords or phrases (e.g., "sore throat" or "not playing outside") are included.

[0389] Output: Health status determination result.

[0390] Step 5:

[0391] Information provision

[0392] Based on the results of the judgment, the server retrieves information about nearby hospitals, parks, and leisure facilities from a database management system (MySQL) and provides it to parents.

[0393] Input: Health status determination results.

[0394] What it does: The server queries the MySQL database to retrieve the appropriate facility information, then sends the retrieved information to the parent's device. For example, it sends a notification containing the hospital's name, address, phone number, etc.

[0395] Output: Facility information provided to parents.

[0396] (Application example 3)

[0397] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0398] In modern families, it is difficult to understand a child's health condition and lifestyle through conversations between parents and children, and to provide appropriate external services. It is also difficult to hold a dialogue based on a child's interests and preferences. Furthermore, there is a lack of means to provide information on nearby medical institutions and sports facilities according to a child's health condition.

[0399] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means. In this invention, the server includes: means for using generation AI and voice synthesis technology to create a stuffed animal that will serve as a conversation partner for a child who has started talking; means for using everyday parent-child conversations at home as additional learning material to create an AI specialized for that family; means for connecting optimal external services (recommending picture books and toys suitable for the child, suggesting hospitals and leisure facilities) based on parent-child conversations and conversations between the AI ​​and the child; means for analyzing parent-child conversations to understand the child's health condition and lifestyle; and means for providing information on nearby medical institutions and sports facilities based on the child's health condition. This makes it possible to understand a child's health condition and lifestyle through parent-child conversations and provide appropriate external services.

[0400] "Generative AI" is an artificial intelligence that analyzes parent-child conversations and dialogue with children, and provides appropriate information and services.

[0401] "Speech synthesis" is a technology that converts text data into speech, and is used to enable stuffed toys that children talk to to have natural conversations.

[0402] "Nutaigiri" is an interactive toy that combines generative AI and voice synthesis technology to act as a conversation partner for children.

[0403] "Additional learning" is the process of using everyday conversations between parents and children as learning data to build an AI that is specific to that household.

[0404] "External services" are information and suggestions about medical institutions, sports facilities, picture books, toys, etc. that are provided based on the child's health condition and interests.

[0405] "Health status" refers to the child's physical condition, such as their physical condition and whether they have any illnesses.

[0406] "Lifestyle habits" refers to a child's behavioral patterns and habits in their daily life.

[0407] "Medical institution" means a hospital, clinic, or other facility that diagnoses and treats children's health problems.

[0408] "Sports facilities" are places such as parks and leisure facilities where children can maintain their health through exercise and play.

[0409] The system for implementing this invention uses generative AI and voice synthesis technology to create a stuffed animal that a child can talk to. The server uses parent-child conversations as material for additional learning and builds an AI specialized for that family. Furthermore, the system analyzes parent-child conversations and dialogue between the AI ​​and the child to understand the child's health condition and lifestyle habits. It then provides information on nearby medical institutions and exercise facilities based on the child's health condition.

[0410] Hardware and software used

[0411] Hardware: Smartphones, stuffed toys (interactive toys)

[0412] Software: Python, OpenAI API, speech synthesis engine

[0413] Data processing and calculation

[0414] The server collects parent-child conversations as text data and analyzes them using a generative AI model. Based on the analysis results, the server identifies the child's health and lifestyle habits and suggests appropriate external services. Specifically, the server performs the following processes:

[0415] 1. Collecting parent-child conversations: Collecting parent-child conversations as audio data using smartphones and stuffed toys.

[0416] 2. Converting voice data to text: Using a speech synthesis engine, the collected voice data is converted into text data.

[0417] 3. Text data analysis: Use OpenAI API to analyze text data and understand children's health conditions and lifestyle habits.

[0418] 4. Suggesting external services: Based on the analysis results, provide information on nearby medical institutions and exercise facilities.

[0419] Specific examples

[0420] For example, consider the case where the following parent-child conversation is collected:

[0421] Example of a parent-child conversation:

[0422] Parent: What did you do at school today?

[0423] Child: He didn't seem very energetic today and was coughing all through class.

[0424] Parent: That's terrible. Maybe you should go see a doctor.

[0425] An example prompt for analyzing this conversation is as follows:

[0426] Example prompt sentence:

[0427] Use the following conversation to determine your child's health:

[0428] Parent: What did you do at school today?

[0429] Child: He didn't seem very energetic today and was coughing all through class.

[0430] Parent: That's terrible. Maybe you should go see a doctor.

[0431] The server inputs this prompt into the OpenAI API and retrieves the analysis results. Based on the analysis results, it determines that the child has a cold and provides information about nearby medical facilities, allowing parents to quickly receive appropriate medical care.

[0432] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[0433] Step 1:

[0434] Collection of parent-child conversations

[0435] The device (smartphone or stuffed toy) collects parent-child conversations as audio data. When the user starts a conversation, the device records the audio using its built-in microphone. The input is the audio data of the parent-child conversation, and the output is the recorded audio file.

[0436] Step 2:

[0437] Converting audio data to text

[0438] The server converts the collected voice data into text data using a speech synthesis engine (e.g., Google Speech-to-Text API). The input is an audio file, and the output is text data. The server sends the audio file to the API and receives the returned text data.

[0439] Step 3:

[0440] Text data analysis

[0441] The server uses a generative AI model (for example, OpenAI's GPT-3) to analyze text data. The input is text data, and the output is the analysis result. The server sends the text data to the generative AI model as a prompt sentence and receives the returned analysis result. Specific examples of prompt sentences are as follows:

[0442] Use the following conversation to determine your child's health:

[0443] Parent: What did you do at school today?

[0444] Child: He didn't seem very energetic today and was coughing all through class.

[0445] Parent: That's terrible. Maybe you should go see a doctor.

[0446] Step 4:

[0447] Health status assessment

[0448] The server determines the child's health status based on the analysis results returned by the generative AI model. The input is the analysis results, and the output is the health status judgment result. For example, if the analysis result is "The child has a cold," the server stores that information.

[0449] Step 5:

[0450] Proposal of external services

[0451] The server provides information on appropriate external services (e.g., nearby medical institutions or exercise facilities) based on the health status assessment results. The input is the health status assessment results, and the output is information on the external services. The server searches the database for information on appropriate external services and provides it to the user.

[0452] Step 6:

[0453] Providing information

[0454] The terminal provides the user with information about the external service received from the server. The input is the information from the external service, and the output is the information provided to the user. The terminal conveys information to the user using a screen display or voice synthesis.

[0455] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0456] "Example 1"

[0457] As one embodiment of the present invention, a system incorporating an emotion engine is provided. In this system, a generation AI is used in combination with an emotion engine that recognizes a user's emotions. Specifically, the emotion engine recognizes the user's emotions from the user's tone of voice, speaking style, choice of words, etc. Then, the generation AI conducts a dialogue based on the emotions. For example, when a child is feeling sad, the emotion engine recognizes the sadness, and the generation AI conducts a dialogue to comfort the child.

[0458] "Example 2"

[0459] The emotion engine also recognizes the child's emotions from parent-child conversations, and the generative AI then conducts a dialogue based on that. For example, if a parent uses harsh words against their child, the emotion engine will recognize from the conversation that the child may be angry. The generative AI will then conduct a dialogue to calm the child's anger.

[0460] "Example 3"

[0461] Furthermore, the emotion engine understands the child's emotions from parent-child conversations and dialogue with the child, and the generation AI then suggests external services based on that. For example, if a child talks about having fun playing in the park, the emotion engine will recognize from the conversation that the child likes parks. Based on that information, the generation AI will suggest external services such as providing information about nearby parks.

[0462] The processing flow of each embodiment will be described below.

[0463] "Example 1"

[0464] Step 1: A dialogue with the user is initiated. This dialogue can be initiated by the user speaking or by the system speaking.

[0465] Step 2: The emotion engine recognizes the user's emotions from their tone of voice, speaking style, and choice of words.

[0466] Step 3: The generative AI creates a dialogue based on the information from the emotion engine. For example, if a child is feeling sad, the generative AI creates a dialogue to comfort the child.

[0467] "Example 2"

[0468] Step 1: A parent-child conversation is initiated. This conversation begins when the parent and child start talking naturally.

[0469] Step 2: The emotion engine recognizes the child's emotions from the parent-child conversation. For example, if a parent uses harsh words against their child, the emotion engine will recognize from the conversation that the child may be angry.

[0470] Step 3: The generative AI creates a dialogue based on information from the emotion engine. For example, if a child is angry, the generative AI creates a dialogue to calm the child down.

[0471] "Example 3"

[0472] Step 1: A dialogue with the child is initiated. This dialogue can be initiated by the child speaking or by the system speaking.

[0473] Step 2: The emotion engine recognizes the child's emotions from the child's conversation. For example, if a child talks about having fun playing in the park, the emotion engine recognizes from the conversation that the child likes the park.

[0474] Step 3: The generation AI suggests external services based on information from the emotion engine. For example, based on information that a child likes parks, the generation AI suggests external services such as providing information about nearby parks.

[0475] Example 1

[0476] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0477] In modern families, there are often few people with whom children can feel comfortable talking. It is also difficult to understand a child's emotions and interests and respond appropriately based on those. Furthermore, there is a lack of systems that can grasp a child's health condition and lifestyle habits and suggest appropriate external services based on that. To solve these issues, a system that can understand a child's emotions through dialogue and generate appropriate responses is needed.

[0478] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0479] In this invention, the server includes means for capturing the voices of the child and parent using a built-in microphone, means for transmitting the captured voice data to the server, means for analyzing the received voice data and recognizing the user's emotion, means for generating an appropriate response based on the recognized emotion, means for converting the generated response into voice data using voice synthesis technology, and means for outputting the converted voice data through a speaker. This makes it possible to understand emotions through dialogue with the child and generate an appropriate response.

[0480] "Generative AI" is a system that uses artificial intelligence technology to generate new information or responses based on input data.

[0481] "Speech synthesis" is a technology that converts text data into voice data.

[0482] A "stuffed toy" is a toy made of soft material in the shape of a doll or animal.

[0483] A "microphone" is a device that converts sound into an electrical signal.

[0484] A "speaker" is a device that converts electrical signals into sound and outputs it.

[0485] A "server" is a computer system that processes and stores data over a network.

[0486] "Audio data" is data that represents audio in digital form.

[0487] An "emotion engine" is a system that analyzes and recognizes a user's emotions from voice and text data.

[0488] "External services" are services or resources that exist outside the system and are proposed or provided to users.

[0489] The "voice data transmission module" is software or hardware for transmitting acquired voice data to a server via a network.

[0490] A "speech synthesis engine" is software or hardware for converting text data into voice data.

[0491] This invention is a system that uses a stuffed toy device to communicate with children. The device has a built-in microphone and speaker, and utilizes generative AI and speech synthesis technology. In addition, by combining it with an emotion engine, it can recognize the user's emotions and respond to them.

[0492] Hardware and software used

[0493] Hardware:

[0494] Built-in microphone: Used to capture the voice of the child or parent.

[0495] Built-in speaker: Used to output the generated audio data.

[0496] Plush toy device: An interactive device with a built-in microphone and speaker.

[0497] software:

[0498] Generative AI models (e.g., GPT-4): Analyze voice data and generate appropriate responses.

[0499] Emotion engine (e.g., Affectiva): Recognizes the user's emotions.

[0500] Voice data transmission module: Transmits the acquired voice data to the server.

[0501] A speech synthesis engine (e.g., Google Text-to-Speech): converts the generated text response into audio data.

[0502] Program processing flow

[0503] 1. Acquiring voice input

[0504] The device uses a built-in microphone to pick up the voices of the child and parent. For example, if a child says, "I had fun at school today," the microphone will capture that voice.

[0505] 2. Sending audio data

[0506] The terminal transmits the captured voice data to a server. Specifically, an audio data transmission module in the terminal transmits the captured voice data to the server via the Internet.

[0507] 3. Analysis of audio data

[0508] The server passes the received voice data to a generative AI for analysis. For example, the server converts the voice data into text and inputs it into a generative AI model (e.g., GPT-4) to understand the content.

[0509] 4. Emotional Recognition

[0510] The server recognizes the user's emotions using an emotion engine. Specifically, the server analyzes the tone and speaking style of the voice data and uses an emotion engine (e.g., Affectiva) to recognize "enjoyment."

[0511] 5. Generating an appropriate response

[0512] The server uses generative AI to generate an appropriate response, for example, a generative AI model might generate the response "That's great! What did you enjoy doing?"

[0513] 6. Speech Synthesis

[0514] The server converts the generated response into voice data using speech synthesis technology. Specifically, the server inputs the generated text response into a speech synthesis engine (e.g., Google Text-to-Speech) to generate voice data.

[0515] 7. Audio Output

[0516] The device outputs the voice data sent from the server through a speaker. For example, the built-in speaker of the device plays a voice message such as "That's great! What did you enjoy doing?"

[0517] Examples of specific examples and prompts

[0518] Examples:

[0519] A child says, "I had fun at school today."

[0520] The stuffed animal device responds, "That's great! What did you have fun doing?"

[0521] Example prompts to input to a generative AI model:

[0522] "How do you respond when your child says, 'I had fun at school today?'"

[0523] This system understands children's emotions through dialogue and generates appropriate responses, contributing to the improvement of children's mental health and communication skills.

[0524] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0525] Step 1:

[0526] Acquiring voice input

[0527] The device uses a built-in microphone to pick up the voices of the child and parent.

[0528] Input: Child or parent voice

[0529] What it does: When a child says, "I had fun at school today," a microphone captures the audio.

[0530] Output: Captured audio data

[0531] Step 2:

[0532] Sending audio data

[0533] The terminal transmits the acquired voice data to the server.

[0534] Input: Captured audio data

[0535] Specific operation: The audio data transmission module in the terminal transmits the captured audio data to a server via the Internet.

[0536] Output: Audio data sent to the server

[0537] Step 3:

[0538] Analysis of audio data

[0539] The server passes the received voice data to the generation AI for analysis.

[0540] Input: Audio data sent to the server

[0541] How it works: The server converts the voice data into text and feeds it into a generative AI model (e.g., GPT-4) to understand the content.

[0542] Output: Parsed text data

[0543] Step 4:

[0544] Emotion recognition

[0545] The server uses an emotion engine to recognize the user's emotion.

[0546] Input: Parsed text data

[0547] Specific operation: The server analyzes the tone and speaking style of the voice data and recognizes "fun" using an emotion engine (e.g., Affectiva).

[0548] Output: Recognized emotion data

[0549] Step 5:

[0550] Generating an appropriate response

[0551] The server uses a generation AI to generate an appropriate response.

[0552] Input: Recognized emotion data

[0553] What happens: The generative AI model generates the response, "That's great! What did you enjoy doing?"

[0554] Output: The generated text response

[0555] Step 6:

[0556] Speech synthesis

[0557] The server converts the generated response into voice data using voice synthesis technology.

[0558] Input: Generated text response

[0559] Specific operation: The server inputs the generated text response into a speech synthesis engine (e.g., Google Text-to-Speech) to generate audio data.

[0560] Output: Generated audio data

[0561] Step 7:

[0562] Audio Output

[0563] The terminal outputs the audio data transmitted from the server through a speaker.

[0564] Input: Generated audio data

[0565] What it does: The device's built-in speaker plays the following audio: "That's great! What did you have fun doing?"

[0566] Output: Played audio

[0567] (Application example 1)

[0568] Next, a description will be given of Application Example 1 of Embodiment Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0569] Conventional stuffed toy devices were limited to interactions with children and were used only at home. Furthermore, they lacked the ability to recognize the user's emotions and engage in dialogue based on them, making it difficult to provide appropriate responses according to the user's emotions. Furthermore, they lacked the means to improve the shopping experience in physical stores, which often left users feeling stressed.

[0570] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means. In this invention, the server includes: means for using generation AI and voice synthesis technology to create a stuffed animal that can be a conversation partner for a child who starts talking; means for using everyday parent-child conversations at home as additional learning material to create an AI specialized for that home; means for connecting optimal external services (recommending picture books and toys suitable for the child, suggesting hospitals and leisure facilities) based on parent-child conversations and conversations between the AI ​​and the child; means for analyzing the user's voice using an application installed on a smartphone in a physical store and providing appropriate product information and recommended products; and means for recognizing the user's emotions and conducting a dialogue based on the analysis. This makes it possible to learn a child's interests and preferences through parent-child conversations at home and improve the shopping experience at a physical store.

[0571] "Generative AI" is an artificial intelligence technology that analyzes user input data and generates appropriate responses.

[0572] "Speech synthesis" is a technology that converts text data into speech.

[0573] A "stuffed toy" is a toy made of soft material in the shape of a doll or animal.

[0574] "Home-specific AI" is artificial intelligence that learns conversation data within a specific home and generates responses optimized for that home.

[0575] An "external service" is a system that provides external information and services that can be accessed from devices and applications within the home.

[0576] A "brick and mortar store" is a store that exists in a physical location and offers goods and services.

[0577] A "smartphone" is a portable device that has the functionality of a mobile phone and is also capable of connecting to the Internet and running applications.

[0578] An "application" is a software program designed to provide a particular function or service.

[0579] "Analyzing the user's voice" is the process of analyzing the voice data uttered by the user and understanding its content.

[0580] "Product information" is detailed information about a specific product.

[0581] "Recommended products" are products suggested based on the user's preferences and needs.

[0582] "Emotion recognition" is the process of determining a user's emotions from the user's tone of voice, speaking style, choice of words, etc.

[0583] "Having a dialogue" means that the user and the system communicate with each other by exchanging information.

[0584] The system for implementing this invention includes: a means for using generative AI and voice synthesis technology to create a stuffed animal that will act as a conversation partner for a child who has started to talk; a means for using everyday parent-child conversations at home as additional learning material to create an AI specialized for that household; a means for connecting to optimal external services based on conversations between parents and children and conversations between the AI ​​and children; a means for analyzing the user's voice using an application installed on a smartphone in a physical store and providing appropriate product information and recommended products; and a means for recognizing the user's emotions and engaging in dialogue based on those emotions.

[0585] Hardware and software used

[0586] Hardware: smartphone, microphone, speaker

[0587] Software: speech_recognition library, transformers library, gTTS library, playsound library

[0588] Data processing and calculation

[0589] 1. Speech recognition: The user's voice is captured through the smartphone's microphone and converted into text using the speech_recognition library, thereby capturing the user's speech as digital data.

[0590] 2. Emotion Recognition: The converted text is analyzed using the emotion engine in the transformers library to recognize the user's emotions, thereby understanding the user's emotional state.

[0591] 3. Response generation: Based on the recognized emotions, a generative AI model is used to generate appropriate responses, enabling dialogue that responds to the user's emotions.

[0592] 4. Speech synthesis: The generated response is converted into speech using the gTTS library and played using the playsound library, providing the user with a spoken response.

[0593] Specific examples

[0594] For example, if a user says, "I'm tired today," the emotion engine will recognize this as "NEGATIVE," and the generative AI model will respond by saying, "I recommend products that will help you relax." This response is converted into voice using speech synthesis technology and played through the smartphone speaker.

[0595] Prompt Sentence Examples

[0596] User: I'm tired today

[0597] Generative AI: Recommend relaxing products. For example, how about some aromatic candles or relaxing herbal tea?

[0598] In this way, it is possible to provide appropriate product information and recommended products according to the user's emotions, thereby improving the shopping experience in physical stores.

[0599] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0600] Step 1:

[0601] A user launches an application on their smartphone and speaks into the microphone. The input is the user's voice, which is captured by the smartphone's microphone. The output is audio data.

[0602] Step 2:

[0603] A smartphone application converts voice data into text data using the speech_recognition library. The input is voice data and the output is text data. Through this conversion, the user's speech content is obtained as digital text.

[0604] Step 3:

[0605] The server analyzes text data using the emotion engine of the transformers library to recognize the user's emotion. The input is text data, and the output is an emotion label (e.g., positive, negative). This analysis identifies the user's emotional state.

[0606] Step 4:

[0607] The server generates an appropriate response using a generative AI model based on the emotion label. The input is text data and emotion label, and the output is a response text. The generative AI model generates dialogue content according to the user's emotion.

[0608] Step 5:

[0609] The server converts the response text to speech using the gTTS library. The input is the response text and the output is speech. This conversion provides the generated response as speech.

[0610] Step 6:

[0611] A smartphone application uses the playsound library to play audio data. The input is the audio data, and the output is an audio response to the user, allowing the user to receive the response by audio.

[0612] Step 7:

[0613] If the user speaks again, the processes from step 1 to step 6 are repeated, thereby enabling continuous dialogue.

[0614] Example 2

[0615] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0616] Conventional parent-child communication support systems have difficulty fully understanding a child's interests, preferences, and emotions, and then conducting appropriate dialogue and making suggestions based on that understanding. They also lack specific support for parents to provide appropriate educational materials and external services to their children. This has led to issues such as difficulty in smooth parent-child communication and in providing effective support for children's growth and learning.

[0617] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0618] In this invention, the server includes means for recording conversations between parents and children and converting the audio data into text data, means for analyzing the text data to learn the child's interests and preferences, means for recognizing the child's emotions from the text data, means for generating dialogue in accordance with the child's interests and emotions, and means for suggesting appropriate educational materials to parents based on the child's interests and preferences. This makes it possible to accurately grasp the child's interests and emotions through parent-child conversations and to provide appropriate dialogue and suggestions based on this.

[0619] "Generative AI" is an artificial intelligence that learns about a child's interests, preferences, and emotions from parent-child conversations and generates dialogue based on that information.

[0620] "Speech synthesis" is a technology that converts text data into voice data and outputs it in a natural voice.

[0621] "Toys" are tools and devices that children play with, and in this system have the function of acting as a conversation partner for children.

[0622] "Parent-child conversation" refers to the everyday dialogue that takes place between parents and children at home.

[0623] "Additive learning" is the process of adding new data to existing training data to improve the performance and accuracy of artificial intelligence.

[0624] "External services" are services and products that exist outside the system, including books and play equipment suitable for children, medical institutions, and entertainment facilities.

[0625] "Audio data" refers to digital data of sound captured through a device such as a microphone.

[0626] "Text data" is character information converted from voice data using voice recognition technology.

[0627] "Natural language processing" is a technology for analyzing text data and understanding its meaning and intent.

[0628] "Emotion recognition" is a technology that identifies human emotions from text data and voice data.

[0629] "Educational materials" are items such as books and toys that are used to support children's learning and development.

[0630] This invention is a system that learns about a child's interests, preferences, and emotions through conversations between parents and children, and then provides appropriate dialogue and suggestions based on that information. A specific embodiment of this system is described below.

[0631] Hardware and software used

[0632] Hardware: microphones, speakers, servers

[0633] Software: speech recognition software (e.g., Google Speech-to-Text), natural language processing engines (e.g., GPT-4), emotion recognition engines (e.g., IBM Watson® Tone Analyzer)

[0634] System configuration

[0635] 1. Acquiring audio data

[0636] The device records conversations between parents and children through a microphone. The device's built-in microphone captures audio and stores it as digital audio data in real time.

[0637] 2. Converting voice data to text

[0638] The server uses voice recognition software to convert the recorded voice data into text data. The server receives the voice data and calls the voice recognition API to generate text data.

[0639] 3. Learning about interests and preferences

[0640] The server analyzes the text data using a natural language processing engine to learn the child's interests and preferences. The server analyzes the text data and extracts specific keywords and phrases to identify the child's interests.

[0641] 4. Emotional Recognition

[0642] The server uses an emotion recognition engine to recognize the child's emotions from the text data. The server sends the text data to the emotion recognition API and obtains emotion labels (e.g., anger, joy, sadness).

[0643] 5. Dialogue Generation

[0644] The server uses a generative AI model to generate dialogue based on the child's interests and emotions. The server inputs prompts into the generative AI model to generate appropriate dialogue.

[0645] 6. Providing Proposals

[0646] The server recommends appropriate educational materials (books, toys, etc.) to parents based on the child's interests and preferences. Based on the information it has learned, the server searches its database for related products and content and makes recommendations to parents.

[0647] Specific examples

[0648] Parent-child conversation:

[0649] Parent: "What do you want to do today?"

[0650] Child: "I want to read a book about dinosaurs!"

[0651] System Action:

[0652] The device records the conversation and the server converts the voice data into text.

[0653] Using a natural language processing engine, the server learns from the statement "I want to read a book about dinosaurs" that the child is interested in dinosaurs.

[0654] The server uses a generative AI model to generate dialogue that provides topics related to dinosaurs.

[0655] The server suggests dinosaur-related books and toys to parents.

[0656] Prompt Sentence Examples

[0657] "If your child is interested in dinosaurs, generate a dialogue that brings up the topic of dinosaurs."

[0658] "If a parent uses harsh language towards a child, generate a dialogue that will calm the child's anger."

[0659] The system supports parent-child communication and makes appropriate suggestions based on children's interests and emotions, helping parents provide better educational materials for their children.

[0660] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0661] Step 1:

[0662] Acquiring audio data

[0663] The device records conversations between parents and children through a microphone.

[0664] Input: Parent-child conversation audio

[0665] How it works: The device's built-in microphone captures audio and stores it as digital audio data in real time.

[0666] Output: Digital audio data

[0667] Step 2:

[0668] Converting audio data to text

[0669] The server uses voice recognition software to convert the recorded voice data into text data.

[0670] Input: Digital audio data

[0671] Specific operation: The server receives the voice data and calls the voice recognition API to generate text data.

[0672] Output: Text data

[0673] Step 3:

[0674] Learning about interests and preferences

[0675] The server uses a natural language processing engine to analyze the text data and learn about the child's interests and preferences.

[0676] Input: Text data

[0677] What it does: The server analyzes the text data and extracts specific keywords and phrases to identify the child's interests.

[0678] Output: Data about the child's interests and preferences

[0679] Step 4:

[0680] Emotion recognition

[0681] The server uses an emotion recognition engine to recognize the child's emotion from the text data.

[0682] Input: Text data

[0683] Specific operation: The server sends text data to the emotion recognition API and obtains emotion labels (e.g., anger, joy, sadness).

[0684] Output: Data about the child's emotions

[0685] Step 5:

[0686] Dialogue generation

[0687] The server uses a generative AI model to generate dialogue that responds to the child's interests and emotions.

[0688] Input: Data about the child's interests and preferences, data about the child's emotions

[0689] Specific operation: The server inputs a prompt sentence into the generative AI model and generates an appropriate dialogue.

[0690] Output: The generated dialogue

[0691] Step 6:

[0692] Providing suggestions

[0693] The server suggests appropriate educational materials (books, toys, etc.) to parents based on the child's interests and preferences.

[0694] Input: Data about your child's interests and preferences

[0695] Specific operation: Based on the information learned by the server, it searches the database for relevant products and content and makes suggestions to the parent.

[0696] Output: A list of suggested educational materials

[0697] (Application example 2)

[0698] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0699] Conventional AI systems based on parent-child conversations were limited to use at home and were unable to respond to children's interests and emotions when shopping in physical stores or when out and about. Furthermore, there was a lack of systems that could suggest products based on children's interests and preferences, or engage in conversations that responded to their emotions, making it difficult to improve the parent-child shopping experience.

[0700] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0701] In this invention, the server includes: means for using generation AI and speech synthesis technology to create a stuffed animal that will be a conversation partner for a child who starts talking; means for using everyday parent-child conversations at home as additional learning material to create an AI specialized for that household; means for connecting optimal external services (recommending picture books and toys suitable for the child, or suggesting hospitals and leisure facilities) based on parent-child conversations and conversations between the AI ​​and the child; means for suggesting products based on the child's interests and preferences when the parent and child shop in a physical store; and means for recognizing the child's emotions and engaging in appropriate dialogue. This improves the parent-child shopping experience and enables product suggestions and dialogue that are responsive to the child's interests and emotions.

[0702] "Generative AI" is an artificial intelligence that learns about a child's interests and preferences from parent-child conversations and generates dialogue based on that information.

[0703] "Speech synthesis technology" is a technology that converts text data into speech, and is used to enable stuffed toys that children can talk to to have natural conversations.

[0704] The "stuffed toy" is a toy created using generative AI and voice synthesis technology to act as a conversation partner for children.

[0705] "Additional learning" is the process by which AI learns knowledge and information specific to a particular household using everyday conversations between parents and children as material.

[0706] "External services" are services such as picture books, toys, hospitals, and leisure facilities that AI suggests based on conversations between parents and children and dialogue between children.

[0707] A "physical store" is a physical store where parents and children actually visit and shop.

[0708] "Product suggestion" is the act of AI suggesting appropriate products to parents and children based on the child's interests and preferences.

[0709] "Emotion recognition" is a technology that identifies a child's emotions from parent-child conversations or dialogue between the child and the parent, and responds accordingly.

[0710] "Dialogue generation" is the process by which AI generates appropriate dialogue based on recognized emotions.

[0711] The system for implementing this invention uses generative AI and voice synthesis technology to create stuffed toys that children can talk to. It also uses everyday parent-child conversations at home as additional learning material to build AI specialized for each household. Furthermore, it provides optimal external services (recommending picture books and toys, suggesting hospitals and leisure facilities) based on parent-child conversations and dialogue between the AI ​​and the child.

[0712] When parents and children shop in physical stores, the system can suggest products based on the child's interests and preferences, and can also recognize the child's emotions and engage in appropriate dialogue.

[0713] Hardware and software used

[0714] Hardware: Smartphones, stuffed toys

[0715] Software: OpenAI API, emotion recognition API, product recommendation system

[0716] Data processing and calculation

[0717] 1. Parent-child conversation acquisition: Using the smartphone's voice recognition function, parent-child conversations are acquired as text data.

[0718] 2. Learning interests and preferences: The acquired conversation data is input into the generation AI to learn the child's interests and preferences.

[0719] 3. Emotion Recognition: Use the emotion recognition API to identify children's emotions from conversation data.

[0720] 4. Product Recommendation: Based on learned interests and preferences, the product recommendation system suggests suitable products.

[0721] 5. Dialogue generation: Based on the recognized emotions, the generative AI generates appropriate dialogue.

[0722] Specific examples

[0723] For example, if the keyword "dinosaur" frequently appears in a parent-child conversation, the AI ​​generator will determine that the child is interested in dinosaurs. While shopping in a physical store, a smartphone application will suggest dinosaur-related products (dinosaur picture books and toys). If the child is angry, the emotion recognition API will identify that emotion, and the AI ​​generator will generate a dialogue to calm the child.

[0724] Prompt Sentence Examples

[0725] "If a child is interested in dinosaurs, what products would you suggest?"

[0726] "Generate calming dialogue when your child is upset."

[0727] In this way, a system can be realized that can understand a child's interests and emotions from parent-child conversations and make appropriate product suggestions and dialogues.

[0728] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0729] Step 1:

[0730] Capture parent-child conversations.

[0731] Input: Parent-child conversation audio

[0732] Output: Text data

[0733] Specific operation: Users (parent and child) converse using the smartphone's voice recognition function. The device then converts the voice into text data using the voice recognition API.

[0734] Step 2:

[0735] Learn about your child's interests and preferences.

[0736] Input: Text data (parent-child conversation)

[0737] Output: Data about the child's interests and preferences

[0738] How it works: The server inputs the acquired text data into a generative AI model to learn the child's interests and preferences. For example, if the keyword "dinosaur" appears frequently, it will determine that the child is interested in dinosaurs.

[0739] Step 3:

[0740] Recognize your child's emotions.

[0741] Input: Text data (parent-child conversation)

[0742] Output: Child's emotion data

[0743] What it does: The server uses an emotion recognition API to identify the child's emotions from the text data, for example, determining whether the child is angry based on the content and tone of the conversation.

[0744] Step 4:

[0745] Suggest products.

[0746] Input: Data about your child's interests and preferences

[0747] Output: Product list

[0748] Specific operation: The server uses a product recommendation system to suggest appropriate products based on the learned interests and preferences. For example, if the user is interested in dinosaurs, it will list dinosaur picture books and toys.

[0749] Step 5:

[0750] Generate dialogue.

[0751] Input: Child's emotion data

[0752] Output: Dialogue text

[0753] Specific behavior: The server uses a generative AI model to generate appropriate dialogue based on the recognized emotion. For example, if a child is angry, it generates dialogue to calm them down.

[0754] Step 6:

[0755] The generated dialogue is synthesized into speech.

[0756] Input: Dialogue text

[0757] Output: Audio data

[0758] Specific operation: The server inputs the generated dialogue text into the speech synthesis API to generate voice data. The device plays this voice data and provides dialogue to the child.

[0759] Step 7:

[0760] The service suggests products when parents and children shop in physical stores.

[0761] Input: Product listings, data about your child's interests and preferences

[0762] Output: Product proposal notification

[0763] Specific operation: When users (parents and children) shop at a physical store, the device displays a notification suggesting appropriate products based on the product list. For example, if there are dinosaur-related products in the store, the device will notify the parent about those products.

[0764] Example 3

[0765] Next, a description will be given of a third embodiment of the third embodiment. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0766] Conventional conversational AI systems have difficulty accurately understanding a child's health condition, lifestyle habits, and emotions, and then suggesting appropriate external services based on that information. They also lack the ability to effectively utilize information obtained from parent-child conversations to provide dialogue and suggestions tailored to the child's interests and preferences. This has meant that parents have been unable to receive sufficient support when managing their children's health and improving their lifestyle habits.

[0767] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[0768] In this invention, the server includes: means for using generation AI and speech synthesis technology to create a dialogue device that will serve as a conversation partner for a child when they start talking; means for using everyday parent-child conversations at home as additional learning material to create an AI specialized for that household; means for connecting optimal external services (recommending books and toys suitable for the child, and suggesting medical institutions and leisure facilities) based on parent-child conversations and conversations between the AI ​​and the child; means for understanding the child's health condition and lifestyle from parent-child conversations and dialogue with the child and suggesting external services based on that; and means for understanding the child's emotions from parent-child conversations using an emotion analysis engine and for the generation AI to suggest external services based on that. This makes it possible to effectively utilize information obtained from parent-child conversations and dialogue with the child to suggest appropriate external services based on the child's health condition, lifestyle, and emotions.

[0769] "Generative AI" is a system that uses artificial intelligence technology to process natural language and generate dialogue.

[0770] "Speech synthesis" is a technology that converts text data into voice data.

[0771] An "interactive device" is hardware or software for interacting with a user through voice or text.

[0772] "Additive training" is the process of adding new data to an existing learning model and retraining it.

[0773] An "external service" is a service that exists outside the system and provides information or functions.

[0774] A "book" is reading material available in print or digital format.

[0775] A "toy" is a tool or device that children use to play.

[0776] "Medical institution" refers to a facility that provides medical services, such as a hospital or clinic.

[0777] "Leisure facilities" are places or facilities that provide entertainment and recreation.

[0778] "Health status" refers to the state of a child's physical health.

[0779] "Lifestyle habits" refers to the behaviors and habits of children in their daily lives.

[0780] An "emotion analysis engine" is software for extracting and analyzing emotions from text data.

[0781] "Parent-child conversation" refers to verbal exchanges between parents and children.

[0782] Dialogue is a verbal exchange between two people.

[0783] This invention is a system that utilizes information obtained from parent-child conversations and dialogue with a child to propose appropriate external services based on the child's health condition, lifestyle habits, and emotions. Specific embodiments of this system are described below.

[0784] Hardware and software used

[0785] Hardware: Servers, devices (smartphones, tablets, PCs)

[0786] Software: Generative AI models (e.g., GPT-4), sentiment analysis engines (e.g., IBM Watson Tone Analyzer), natural language processing (NLP) tools (e.g., SpaCy, NLTK), database management systems (DBMS)

[0787] System configuration

[0788] 1. Data Entry

[0789] The user uses the device to input conversations between parents and children or dialogues with their children. Specifically, the user can input voice using the microphone of a smartphone or tablet, or input text using a keyboard.

[0790] 2. Data Transmission

[0791] The terminal sends the entered data to the server, where it is uploaded via the Internet.

[0792] 3. Data Analysis

[0793] The server then analyzes the received data using natural language processing (NLP) tools (e.g., SpaCy or NLTK) to tokenize the text data and perform grammatical and semantic analysis.

[0794] 4. Assessment of health status and lifestyle habits

[0795] Based on the analysis results, the server uses a generative AI model to determine the child's health condition and lifestyle habits. Specifically, it inputs prompt sentences into the generative AI model (e.g., GPT-4) to obtain a judgment on the child's health condition and lifestyle habits.

[0796] Example: The prompt sentence is "The child has a cough and seems to have a fever." The generative AI model determines that "the child is likely to have a cold."

[0797] 5. Information provision

[0798] The server then suggests appropriate external services based on the results of the generative AI model. Specifically, it searches a database for information on nearby hospitals and parks and provides it to the user.

[0799] Example: If the user is diagnosed as having a cold, the server sends the user a list of nearby hospitals.

[0800] 6. Emotion analysis

[0801] The server uses a sentiment analysis engine (e.g., IBM Watson Tone Analyzer) to understand the child's emotions from the parent-child conversation.

[0802] Example: Recognizing the emotion "fun" from the text "You looked like you had a lot of fun playing in the park yesterday."

[0803] 7. Emotion-based suggestions

[0804] Based on the results of the sentiment analysis engine, the server uses a generative AI model to suggest external services. Specifically, it inputs prompt sentences into the generative AI model and obtains sentiment-based suggestions.

[0805] Example: Based on the emotion "having fun playing in the park," the server provides the user with information about nearby parks.

[0806] Specific examples

[0807] Example 1: If your child has a cold

[0808] User: Says, "My child has a cough and seems to have a fever."

[0809] Terminal: Converts voice data into text and sends it to the server.

[0810] Server: Analyzes the text data and inputs prompt sentences into the generative AI model.

[0811] Generative AI model: Determines that "there is a high possibility that you have a cold."

[0812] Server: Searches the database for information on nearby hospitals and provides it to the user.

[0813] Example 2: If your child is not getting enough exercise

[0814] User: Texts "My kids have been playing at home a lot lately and don't want to go outside."

[0815] Terminal: Sends text data to the server.

[0816] Server: Analyzes the text data and inputs prompt sentences into the generative AI model.

[0817] Generative AI model: Determines that the person is not getting enough exercise.

[0818] Server: Searches a database for information on nearby parks and leisure facilities and provides it to users.

[0819] Example 3: Children enjoying the park

[0820] User: Says, "They looked like they had a lot of fun playing at the park yesterday."

[0821] Terminal: Converts voice data into text and sends it to the server.

[0822] Server: Analyzes the text data and inputs it into the sentiment analysis engine.

[0823] Sentiment analysis engine: Recognizes the emotion of "fun."

[0824] Server: Inputs prompt sentences to the generative AI model based on emotions.

[0825] Generative AI model: Suggests nearby parks.

[0826] Server: Provides users with information about nearby parks.

[0827] This system effectively utilizes information obtained from conversations between parents and children and dialogues with children to help parents manage their children's health. The flow of the identification process in the third embodiment will be described with reference to FIG.

[0828] Step 1:

[0829] Data Entry

[0830] The user uses the terminal to input conversations between the parent and child and dialogues with the child.

[0831] Input: Audio or text data of parent-child conversations and interactions with the child.

[0832] Specific actions: The user uses the microphone on their smartphone or tablet to input voice commands, or the keyboard to input text.

[0833] Output: The audio or text data is saved to the device.

[0834] Step 2:

[0835] Data transmission

[0836] The terminal transmits the input data to the server.

[0837] Input: Audio or text data.

[0838] Specific operation: The device uploads data to a server via the Internet.

[0839] Output: Data is sent to the server.

[0840] Step 3:

[0841] Data analysis

[0842] The server analyzes the received data using natural language processing (NLP) tools.

[0843] Input: The audio or text data sent to the server.

[0844] What happens: The server uses NLP tools (e.g., SpaCy or NLTK) to tokenize the text data and perform grammatical and semantic analysis.

[0845] Output: Parsed text data.

[0846] Step 4:

[0847] Assessment of health status and lifestyle habits

[0848] Based on the analysis results, the server uses a generative AI model to determine the child's health status and lifestyle habits.

[0849] Input: Parsed text data.

[0850] Specific operation: The server inputs prompt sentences into a generative AI model (e.g., GPT-4) to obtain a judgment about the child's health and lifestyle habits.

[0851] Output: Judgments about the child's health and lifestyle.

[0852] Step 5:

[0853] Information provision

[0854] The server suggests appropriate external services based on the results of the generative AI model.

[0855] Input: Judgments about children's health and lifestyle habits.

[0856] Specific operation: The server searches the database for information on nearby hospitals and parks and provides it to the user.

[0857] Output: Information about suggested external services (e.g., a list of nearby hospitals or parks).

[0858] Step 6:

[0859] Emotion analysis

[0860] The server uses an emotion analysis engine to understand the child's emotions from the parent-child conversation.

[0861] Input: Text data of parent-child conversations.

[0862] Specific operation: The server uses a sentiment analysis engine (e.g., IBM Watson Tone Analyzer) to extract sentiment from text data.

[0863] Output: Extracted emotion data.

[0864] Step 7:

[0865] Emotion-based suggestions

[0866] Based on the results of the sentiment analysis engine, the server uses a generative AI model to suggest external services.

[0867] Input: Extracted emotion data.

[0868] How it works: The server inputs prompts into the generative AI model to get sentiment-based suggestions.

[0869] Output: Suggestion of external services based on emotions (e.g., information about nearby parks).

[0870] In this way, the system can effectively utilize information obtained from parent-child conversations and dialogue with the child to suggest appropriate external services based on the child's health condition, lifestyle habits, and emotions.

[0871] (Application example 3)

[0872] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0873] In modern families, it is important to understand a child's health condition and emotions through conversations between parents and children, and to suggest appropriate external services based on that information. However, effective systems to achieve this are not yet fully available. In particular, there is a lack of functionality to suggest appropriate meals based on a child's health condition and emotions, and to search for restaurants and delivery services that offer those meals. As a result, parents often find it difficult to manage their children's health and choose their meals.

[0874] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means. In this invention, the server includes: means for using generation AI and voice synthesis technology to create a stuffed animal that can be a conversation partner for a child who starts talking; means for using everyday parent-child conversations at home as additional learning material to create an AI specialized for that household; means for connecting optimal external services (recommending picture books and toys suitable for the child, suggesting hospitals and leisure facilities) based on the parent-child conversation and the conversation between the AI ​​and the child; means for understanding the child's health condition and emotions from the parent-child conversation and the dialogue with the child and suggesting an appropriate meal based on that; and means for searching for nearby restaurants and delivery services based on the suggested meal. This makes it possible to understand the child's health condition and emotions through parent-child conversation, suggest an appropriate meal based on that, and further search for restaurants and delivery services that provide that meal.

[0875] "Generative AI" is an artificial intelligence that analyzes parent-child conversations and dialogue with children, and generates appropriate external services and suggestions.

[0876] "Speech synthesis" is a technology that converts text data into speech, and is used when stuffed toys communicate with children.

[0877] "Plushie" is a toy that uses generative AI and voice synthesis technology to act as a conversation partner for children.

[0878] "Additional learning" is the process by which AI learns knowledge specific to a particular household using everyday conversations between parents and children as material.

[0879] "External services" are external resources and services such as picture books, toys, hospitals, and leisure facilities suggested by the generative AI.

[0880] "Health status" refers to a child's physical condition, such as their physical condition and whether they have any illnesses.

[0881] "Emotions" refer to a child's mood and psychological state, and are what the generative AI analyzes from conversations.

[0882] "Meals" are nutritious foods and menus suggested based on the child's health and mood.

[0883] "Food and beverage establishments" are establishments such as restaurants and cafes that serve the proposed meals.

[0884] "Delivery service" is a service that delivers suggested meals to your home.

[0885] The system for implementing this invention uses generative AI and voice synthesis technology to create a stuffed animal that a child can talk to, and understands the child's health condition and emotions through parent-child conversations and dialogue with the child, and based on that, suggests appropriate external services. Specific embodiments of this system are described below.

[0886] Hardware and software used

[0887] Hardware: Smartphones, tablets, plush toys

[0888] Software: OpenAI API, Yelp API, Python

[0889] System configuration

[0890] 1. Generative AI and speech synthesis technology:

[0891] The server uses generative AI and voice synthesis technology to create a stuffed animal that the child can talk to.

[0892] The stuffed animal understands the child's health and emotions through conversations between parents and children and through dialogue with the child.

[0893] 2. Additional learning:

[0894] The server uses everyday parent-child conversations at home as additional learning material to generate AI specialized for that household.

[0895] This allows the AI ​​to tailor its interactions to the specific needs and circumstances of the household.

[0896] 3. Suggestions for external services:

[0897] The server provides optimal external services (recommending picture books and toys, suggesting hospitals and leisure facilities) based on conversations between parents and children and dialogue between the AI ​​and the child.

[0898] In addition, the system understands the child's health condition and emotions through conversations between parents and children, and suggests appropriate meals based on that information.

[0899] 4. Search for restaurants and delivery services:

[0900] The server searches for nearby restaurants and delivery services based on the suggested meal.

[0901] This allows parents to easily arrange appropriate meals for their children according to their health and emotional state.

[0902] Data processing and calculation

[0903] 1. Conversation analysis:

[0904] The server uses OpenAI's generative AI model (text-davinci-003) to analyze the child's health and emotions from parent-child conversations.

[0905] The analysis results are output as keywords such as "cold," "lack of exercise," and "looks fun."

[0906] 2. Meal Suggestions:

[0907] The server then suggests appropriate meals based on the analysis results.

[0908] For example, if it is diagnosed as a "cold," it will suggest "nutritious soups and vitamin-rich meals."

[0909] 3. Search for restaurants:

[0910] The server uses the Yelp API to search for nearby restaurants and delivery services based on the suggested meal.

[0911] Specific examples

[0912] Example conversation: "Mom, a lot of kids at school have colds today. I think I might have a bit of a sore throat too."

[0913] Analysis result: "Cold, a little anxious"

[0914] Suggested meal: "Nutritious soups and vitamin-rich meals."

[0915] Nearby restaurants: "Soup specialty restaurant, healthy food restaurant"

[0916] Prompt Sentence Examples

[0917] Analyze the child's health and emotions from the following conversation:

[0918] Mom, there were a lot of kids with colds at school today. I think I have a bit of a sore throat too.

[0919] In this way, it is possible to understand a child's health condition and emotions through conversations between parents and children, suggest appropriate meals based on that information, and even search for restaurants or delivery services that offer those meals.

[0920] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[0921] Step 1:

[0922] The server collects conversations between parents and children. The conversations are input as voice data via a smartphone or tablet. The server uses voice recognition technology to convert this voice data into text data. The converted text data is sent to the generative AI model.

[0923] Step 2:

[0924] The server uses a generative AI model to analyze the child's health status and emotions from the text data. Specifically, it uses OpenAI's generative AI model (text-davinci-003) to analyze the content of the conversation and extract keywords such as "cold," "lack of exercise," and "looking happy." The analysis results are stored on the server.

[0925] Step 3:

[0926] The server then suggests appropriate meals based on the analysis results. For example, if the illness is diagnosed as a "cold," it will suggest "nutritious soups and vitamin-rich meals." This suggestion is made using a generative AI model, which generates a specific meal menu. The generated menu is then sent to the user's device.

[0927] Step 4:

[0928] The server searches for nearby restaurants and delivery services based on the proposed meal. It uses the Yelp API to search for restaurants and delivery services that match the proposed meal menu and retrieves the results. The search results are stored on the server.

[0929] Step 5:

[0930] The server sends the search results to the user's device. The user can then use their smartphone or tablet to check the suggested meal menus and information about the restaurants and delivery services that offer them. Based on this information, the user can then order the appropriate meal.

[0931] Step 6:

[0932] The user orders meals based on the suggested menu and information about restaurants and delivery services. Orders can be placed directly from the user's device, and the server sends the order information to the restaurant or delivery service. This allows users to easily arrange appropriate meals for their children based on their health and emotional state.

[0933] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0934] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0935] Another example of generative AI is Gemini (registered trademark) (Internet search engine). <url: https: gemini.google.com ?hl="ja">) are mentioned.

[0936] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0937] [Second embodiment]

[0938] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0939] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0940] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0941] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0942] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0943] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0944] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0945] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0946] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0947] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0948] In the smart glasses 214, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0949] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0950] "Example 1"

[0951] An example of the present invention is a stuffed toy device. This device has an internal microphone and speaker, and communicates with children using generative AI and voice synthesis technology. The microphone picks up the voices of the child and parent and sends the voice data to the AI. The AI ​​analyzes the voice data and generates an appropriate response. The generated response is output from the speaker using voice synthesis technology, realizing a dialogue with the child.

[0952] "Example 2"

[0953] Furthermore, AI learns a child's interests and preferences from parent-child conversations. For example, if it determines that a child is interested in dinosaurs, it will provide topics related to dinosaurs. Also, if it understands a child's favorite foods and activities, it will suggest related picture books and toys. These suggestions help parents provide appropriate educational materials for their children.

[0954] "Example 3"

[0955] AI also understands a child's health condition and lifestyle habits through conversations between parents and children. For example, if it determines that a child has a cold, it will provide information about nearby hospitals. If it determines that a child is not getting enough exercise, it will provide information about nearby parks and leisure facilities. These suggestions help parents manage their children's health.

[0956] The processing flow of each embodiment will be described below.

[0957] "Example 1"

[0958] Step 1: The microphone on the plush toy device picks up the child's or parent's voice.

[0959] Step 2: Send the captured audio data to the AI.

[0960] Step 3: The AI ​​analyzes the voice data and generates an appropriate response.

[0961] Step 4: The generated response is output from the speaker using speech synthesis technology, enabling dialogue with the child.

[0962] "Example 2"

[0963] Step 1: The AI ​​learns about the child's interests and preferences from parent-child conversations.

[0964] Step 2: Based on the information it has learned, the AI ​​will provide topics that match the child's interests and preferences. Step 3: The AI ​​will understand the child's favorite foods and games and suggest related picture books and toys.

[0965] "Example 3"

[0966] Step 1: The AI ​​understands the child's health and lifestyle habits from parent-child conversations and dialogue with the child.

[0967] Step 2: The AI ​​assesses the child's health condition and provides information about nearby hospitals if necessary.

[0968] Step 3: The AI ​​assesses the child's lifestyle and, if necessary, recommends nearby parks and recreational facilities.

[0969] Provide information.

[0970] Example 1

[0971] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0972] In modern families, children often lack someone they can talk to with peace of mind. It is also difficult to suggest appropriate external services based on parent-child conversations and the child's interests and preferences. It is also difficult to understand a child's health and lifestyle habits and provide appropriate advice based on them.

[0973] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0974] In this invention, the server includes: a means for using generative AI and speech synthesis technology to create a stuffed animal that can communicate with a child who has begun to talk; a means for creating an AI specialized for a particular household using everyday parent-child conversations as additional learning material; a means for connecting the parent-child conversation and the AI-child conversation to optimal external services (recommending picture books and toys suitable for the child, suggesting hospitals and leisure facilities) based on the conversation; a means for collecting voice data using a microphone; a means for transmitting the collected voice data to the server; a means for the server to analyze the voice data and generate an appropriate response using a generative AI model; a means for converting the generated response into voice data using speech synthesis technology; and a means for outputting the converted voice data from a speaker. This provides an environment where children can interact with ease and makes it possible to suggest appropriate external services based on parent-child conversations. It also makes it possible to understand the child's health condition and lifestyle habits and provide appropriate advice based on them.

[0975] "Generative AI" is an artificial intelligence technology that analyzes voice data and generates appropriate responses.

[0976] "Speech synthesis technology" is a technology that converts text data into voice data.

[0977] A "stuffed toy" is a device that has a microphone and speaker inside and allows for communication with children.

[0978] A "microphone" is a device for collecting sound.

[0979] A "speaker" is a device for outputting audio data as sound.

[0980] A "server" is a computer system that analyzes voice data, runs generative AI models, and applies voice synthesis technology.

[0981] "Audio data" refers to digital data of audio collected by a microphone.

[0982] A "prompt sentence" is text data that is input into a generative AI model and serves as the basis for generating a response.

[0983] "External services" are services provided outside the home, such as recommending picture books and toys suitable for children, and suggesting hospitals and leisure facilities.

[0984] "Additional learning" is the process by which the generative AI learns new knowledge based on everyday conversations between parents and children.

[0985] This invention uses generative AI and voice synthesis technology to provide a stuffed toy device that can serve as a conversation partner for children. Specific embodiments of this system are described below.

[0986] Hardware Configuration

[0987] Terminal: The plush toy device includes the following hardware:

[0988] Microphone: A device for collecting the voices of children and parents. It has high sensitivity and the ability to filter out ambient noise.

[0989] Speaker: A device for outputting the generated audio data. It provides high-quality audio output and plays it at a volume that is easy for children to hear.

[0990] Communication module: A device for transmitting audio data to a server. It can use Wi-Fi or Bluetooth.

[0991] Server: The server includes the following software:

[0992] Generative AI model: An artificial intelligence technology that analyzes voice data and generates appropriate responses, such as OpenAI's GPT-3.

[0993] Speech recognition technology: Technology that converts voice data into text data. For example, Google Cloud Speech-to-Text API is used.

[0994] Speech synthesis technology: Technology that converts text data into voice data. For example, Google Text-to-Speech API or Amazon Polly are used.

[0995] Software Configuration

[0996] Terminal: The plush toy device software includes the following functions:

[0997] Audio data collection: Audio data is collected using a microphone.

[0998] Sending audio data: Send the collected audio data to the server.

[0999] Server: The server software includes the following functions:

[1000] Analysis of voice data: The received voice data is input into the generative AI model and analyzed.

[1001] Response generation: The generative AI model generates an appropriate response based on the prompt.

[1002] Speech synthesis: The generated text response is converted into audio data using speech synthesis technology.

[1003] Send audio data: Send the converted audio data to the device.

[1004] Specific examples

[1005] For example, suppose a child asks, "What shall we play today?" In this case, the following processing is performed:

[1006] 1. Collecting voice data: The device's microphone picks up the child's voice and collects voice data.

[1007] 2. Sending audio data: Send the collected audio data to the server.

[1008] 3. Analysis of voice data: The server inputs the received voice data into the generative AI model and analyzes it. The prompt sentence used is "When a child asks, 'What shall we play today?', generate an appropriate response."

[1009] 4. Response generation: The generative AI model generates the response, "Why don't you try playing ball outside today?"

[1010] 5. Speech synthesis: The generated text response is converted into voice data using speech synthesis technology.

[1011] 6. Sending audio data: Send the converted audio data to the device.

[1012] 7. Audio output: The device speaker plays the audio, "Why don't you go outside and play ball today?"

[1013] In this way, the stuffed toy device allows interaction with the child.

[1014] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1015] Step 1:

[1016] Audio data collection

[1017] Device: The microphone in the plush toy picks up the voices of the child and parent. For example, when a child asks, "What shall we play today?", that voice is input into the microphone.

[1018] Input: Child or parent voice

[1019] Output: Digital audio data

[1020] How it works: The microphone is highly sensitive and has the ability to filter out ambient noise. It converts audio into digital data and temporarily stores it in the internal memory.

[1021] Step 2:

[1022] Sending audio data

[1023] Terminal: The collected voice data is sent to the server. Specifically, the communication module in the terminal converts the voice data into packets and sends them to the server via the Internet.

[1024] Input: Digital audio data

[1025] Output: Audio data sent to the server

[1026] Specific operation: The communication module uses Wi-Fi or Bluetooth to convert voice data into packets and transmit them.

[1027] Step 3:

[1028] Analysis of audio data

[1029] Server: Inputs the received voice data into a generative AI model, such as OpenAI's GPT-3. The server converts the voice data into text data and inputs it as a prompt to the generative AI model.

[1030] Input: Audio data sent to the server

[1031] Output: Text data

[1032] Specific operation: Uses speech recognition technology (e.g., Google Cloud Speech-to-Text API) to convert voice data into text data.

[1033] Step 4:

[1034] Generating a response

[1035] Server: The generative AI model generates an appropriate response based on the prompt, for example, "Why don't you try playing ball outside today?"

[1036] Input: Text data (prompt sentence)

[1037] Output: Response text

[1038] How it works: The generative AI model generates responses by taking into account past interaction history and context.

[1039] Step 5:

[1040] Speech synthesis

[1041] Server: The generated text response is converted into voice data using speech synthesis technology, specifically using a speech synthesis service such as Google Text-to-Speech API or Amazon Polly.

[1042] Input: Response text

[1043] Output: Audio data

[1044] Specific operation: Speech synthesis technology converts text data into speech data to achieve natural pronunciation and intonation.

[1045] Step 6:

[1046] Audio output

[1047] Device: Receives the voice data sent from the server and outputs it from the speaker. For example, the device speaker plays the voice "Why don't you play ball outside today?"

[1048] Input: Audio data sent from the server

[1049] Output: Audio (sound coming from the speaker)

[1050] What it does: The speaker provides high-quality audio output at a volume that is easy for children to hear.

[1051] (Application example 1)

[1052] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1053] Conventional stuffed toy devices were limited to interacting with children and were not designed to support shopping in physical stores or improve customer service. While they had the ability to learn about parent-child conversations and children's interests and preferences, they were not able to guide customers in physical stores, provide sales information, or respond to customers in real time. This created a need for new ways to improve the customer experience in physical stores.

[1054] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1055] In this invention, the server includes means for using generative AI and speech synthesis technology to create a stuffed animal that will be a conversation partner for a child who starts talking, means for using everyday parent-child conversations at home as additional learning material to create an AI specialized for that household, means for connecting optimal external services (recommending picture books and toys suitable for the child, suggesting hospitals and leisure facilities) based on parent-child conversations and conversations between the AI ​​and the child, means for providing an assistant application that supports shopping in a physical store, means for providing voice guidance on the location and features of products in the store, sales information, etc., and means for responding to customer questions in real time. This makes it possible to improve the customer experience in a physical store.

[1056] "Generative AI" is an artificial intelligence technology that analyzes voice and text data and generates appropriate responses.

[1057] "Speech synthesis" is a technology that converts text data into speech.

[1058] A "stuffed toy" is a device that has a microphone and speaker inside and allows for communication with children.

[1059] "Parent-child conversation" refers to the everyday dialogue that takes place between parents and children at home.

[1060] "Additive learning" is the process of learning by adding new data to existing data.

[1061] "External services" are services provided outside the home, such as recommending picture books and toys, or suggesting hospitals and leisure facilities.

[1062] A "brick and mortar store" is a store that exists in a physical location and offers goods and services.

[1063] An "assistant application" is an application that is installed on a smartphone or other device and supports the user's actions.

[1064] "Product location" refers to the location where a particular product is located within a physical store.

[1065] "Sale information" is information about product discounts and special offers.

[1066] "Real-time response" refers to providing an immediate response to a user's question.

[1067] As an embodiment of this invention, we will explain a system that provides an assistant application that supports shopping in a physical store. This system uses generative AI and speech synthesis technology to respond to customer questions in real time and provide voice guidance on the location and features of products in the store, sale information, etc.

[1068] System configuration

[1069] Hardware:

[1070] Smartphone (microphone, speaker)

[1071] software:

[1072] speech_recognition library: Performs speech recognition.

[1073] pyttsx3 library: Performs speech synthesis.

[1074] openai library: Generate responses using generative AI models.

[1075] Processing flow

[1076] 1. Speech Recognition:

[1077] The user speaks a question into the smartphone's microphone.

[1078] The smartphone converts the voice data into text data using the speech_recognition library.

[1079] 2. Response generation by generative AI:

[1080] The server sends the converted text data to the generative AI model using the openai library.

[1081] The generative AI model generates appropriate responses based on the input text data.

[1082] The generated response is returned to the server in text format.

[1083] 3. Speech synthesis:

[1084] The server converts the generated text response into audio data using the pyttsx3 library.

[1085] Audio data is output from the smartphone speaker, conveying the response to the user.

[1086] Specific examples

[1087] For example, if a user asks, "Where is this item located?" the system operates as follows:

[1088] 1. The user's question, "Where is this product located?" is recorded by the smartphone microphone.

[1089] 2. The speech_recognition library converts the audio data into text data.

[1090] 3. The converted text data is sent to the generative AI model.

[1091] 4. The generative AI model generates an appropriate response, such as "The item is on the third floor, on the left."

[1092] 5. The generated response is converted to audio data using the pyttsx3 library.

[1093] 6. A voice will be output from the smartphone speaker saying, "The product is on the third floor, on the left side."

[1094] Prompt Sentence Examples

[1095] An example of a prompt sent to the generative AI model is as follows:

[1096] Generate appropriate responses to customer questions: Where is this item located?

[1097] This will create a smart shopping assistant that improves the customer experience in physical stores.

[1098] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1099] Step 1:

[1100] The user speaks a question into the smartphone microphone. The input is the user's voice data, and the output is the voice data captured by the smartphone microphone. This voice data is processed in the following steps:

[1101] Step 2:

[1102] The device converts the voice data into text data using the speech_recognition library. The input is the voice data acquired in step 1, and the output is text data. Through this conversion, the user's question is obtained in text format.

[1103] Step 3:

[1104] The terminal sends the converted text data to the server. The input is the text data generated in step 2, and the output is the text data sent to the server. This data is input to the generative AI model in the next step.

[1105] Step 4:

[1106] The server uses the openai library to send text data to the generative AI model and generate an appropriate response. The input is the text data sent in step 3, and the output is the text data of the generated response. This response is an appropriate response to the user's question.

[1107] Step 5:

[1108] The server converts the generated text response into audio data using the pyttsx3 library. The input is the text data generated in step 4, and the output is audio data. This conversion results in the text response in audio format.

[1109] Step 6:

[1110] The server sends the converted voice data to the terminal. The input is the voice data generated in step 5, and the output is the voice data sent to the terminal. This data is provided to the user in the next step.

[1111] Step 7:

[1112] The terminal outputs the voice data from the smartphone speaker and communicates the response to the user. The input is the voice data sent in step 6, and the output is the voice response that the user can hear. This allows the user to receive an appropriate answer to their question by voice.

[1113] Example 2

[1114] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1115] Conventional systems that suggest educational materials for children have the problem of being unable to address the specific interests and preferences of individual children because they make suggestions based on general interests and preferences. Furthermore, there has been no system that utilizes parent-child conversations to learn about a child's interests and preferences and then suggests appropriate educational materials based on that information. This makes it difficult for parents to provide their children with the most appropriate educational materials.

[1116] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1117] In this invention, the server includes: means for using generation AI and speech synthesis technology to create a stuffed animal that will serve as a conversation partner for a child who has begun to talk; means for using everyday parent-child conversations at home as additional learning material to create an AI specialized for that family; means for connecting to optimal external services (proposing educational materials suitable for the child) based on parent-child conversations and conversations between the AI ​​and the child; means for collecting parent-child conversation data and converting the voice data into text data; means for analyzing the text data using natural language processing technology to identify the child's interests and preferences; means for generating suggestions for related topics and educational materials based on the identified interests and preferences; and means for providing the generated suggestions to parents, thereby enabling parents to provide appropriate educational materials based on their child's specific interests and preferences.

[1118] "Generative AI" is artificial intelligence that uses natural language processing technology to analyze text data and generate specific information.

[1119] "Speech synthesis" is a technology that converts text data into voice data.

[1120] A "stuffed animal" is a soft toy made to be a conversation companion for children.

[1121] "Parent-child conversation" refers to the everyday dialogue that takes place between parents and children at home.

[1122] "Additive learning" is the process of updating an AI model by adding new data to existing training data.

[1123] An "external service" is a service that provides resources or information outside the system.

[1124] "Educational materials" is a general term for books, toys, teaching materials, etc. that are used to support children's learning and development.

[1125] "Audio data" refers to data in which audio is recorded in digital format.

[1126] "Text data" refers to data in which character information is recorded in digital format.

[1127] "Natural language processing technology" is a technology that allows computers to understand, interpret, and generate human language.

[1128] "Interests and preferences" are the things that an individual is particularly interested in or likes.

[1129] A "suggestion" is a recommended course of action or option for a particular purpose.

[1130] This invention is a system that learns a child's interests and preferences from parent-child conversations and suggests appropriate educational materials based on this information. A specific embodiment of this system will be described below.

[1131] 1. Program Generation

[1132] The server generates a program to learn the child's interests and preferences from parent-child conversations. The program analyzes the conversation data using natural language processing technology to identify the child's interests and preferences.

[1133] 2. Program processing explanation

[1134] The server processes data using the following hardware and software:

[1135] Hardware:

[1136] Server: A computer with high-performance data processing capabilities

[1137] Device: A device equipped with a microphone for recording parent-child conversations

[1138] software:

[1139] Speech recognition software: Google Speech-to-Text API

[1140] Natural language processing technology: generative AI models (e.g., OpenAI's GPT-4)

[1141] 3. Data processing and calculation

[1142] The server processes and calculates the data in the following steps:

[1143] 1. Data Collection:

[1144] The terminal records the conversation between parent and child and transmits the audio data to the server.

[1145] 2. Data conversion:

[1146] The server converts the received voice data into text data using the Google Speech-to-Text API.

[1147] 3. Data Analysis:

[1148] The server inputs the text data into a generative AI model to extract information about the child's interests and preferences.

[1149] 4. Identifying interests and preferences:

[1150] The server analyzes the output of the generative AI model and calculates the frequency of occurrence of certain keywords (e.g., "dinosaur").

[1151] 5. Proposal generation:

[1152] The server retrieves information on related picture books and toys from a database and creates a suggestion list.

[1153] 6. Providing Proposals:

[1154] The server sends the suggestion list to the terminal, which displays the information to the parent.

[1155] 4. Specific Examples

[1156] If conversation data is collected showing that a child frequently uses the word "dinosaur," the server will suggest picture books and toys related to dinosaurs.

[1157] If a child says, "I like pizza," the server will suggest pizza-related picture books and cooking kits.

[1158] 5. Examples of prompts

[1159] "If a child is interested in dinosaurs, what picture books and toys should I suggest?"

[1160] "What educational materials should I provide my child if their favorite food is pizza?"

[1161] The system allows parents to provide appropriate educational materials based on their child's specific interests and preferences.

[1162] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1163] Step 1:

[1164] Data collection

[1165] Input: Parent-child conversation audio data

[1166] Specific operation: The device records the conversation between parent and child. The recorded audio data is sent from the device to the server.

[1167] Output: Audio data sent to the server

[1168] Step 2:

[1169] Data Conversion

[1170] Input: Audio data

[1171] Specific operation: The server sends the received voice data to the Google Speech-to-Text API and converts the voice data into text data.

[1172] Output: Text data

[1173] Step 3:

[1174] Data analysis

[1175] Input: Text data

[1176] How it works: The server inputs the text data into a generative AI model (e.g., OpenAI's GPT-4) to extract information about the child's interests and preferences.

[1177] Output: Information about interests and preferences

[1178] Step 4:

[1179] Identifying interests and preferences

[1180] Input: Information about your interests and preferences

[1181] Specific operation: The server analyzes the output of the generative AI model and calculates the frequency of occurrence of specific keywords (e.g., "dinosaur").

[1182] Output: Identified interests and preferences

[1183] Step 5:

[1184] Proposal Generation

[1185] Input: Identified interests and preferences

[1186] Specific operation: The server retrieves information on related picture books and toys from the database and creates a list of suggestions.

[1187] Output: Suggestion list

[1188] Step 6:

[1189] Providing suggestions

[1190] Input: Suggestion list

[1191] Specific operation: The server sends the proposal list to the terminal, and the terminal displays the information to the parent.

[1192] Output: Suggestion list displayed in parent

[1193] (Application example 2)

[1194] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1195] Conventional educational support systems for children have limited functionality for learning about children's interests and preferences from parent-child conversations, and lack functionality to support the shopping experience in physical stores. Furthermore, there is no way for parents to instantly check detailed information or reviews of suggested products, making it difficult for parents to select appropriate educational materials. Furthermore, there is no navigation function to guide users to the location of products in the store, which makes shopping less efficient.

[1196] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for using generation AI and speech synthesis technology to create a stuffed animal that can be a conversation partner for a child who has started talking; means for using everyday parent-child conversations at home as additional learning material to create an AI specialized for that family; means for connecting optimal external services (recommending picture books and toys suitable for the child, suggesting hospitals and leisure facilities) based on parent-child conversations and conversations between the AI ​​and the child; means for suggesting related products when shopping in a physical store; means for displaying detailed information and reviews about the suggested products; and means including a navigation function for guiding the location of products in the store. This allows the server to learn a child's interests and preferences from parent-child conversations, support the shopping experience in a physical store, and enable efficient and appropriate selection of educational materials.

[1197] "Generative AI" is an artificial intelligence that learns a child's interests and preferences from conversations between parents and children, and then conducts a dialogue based on that information.

[1198] "Speech synthesis technology" is a technology that converts text data into voice data and generates natural speech.

[1199] A "stuffed toy" is a soft toy that children can talk to, and it communicates using generative AI and voice synthesis technology.

[1200] "Parent-child conversation" refers to the everyday dialogue that takes place between parents and children at home.

[1201] "Additional learning materials" are data that the generative AI uses to learn new information through conversations between parents and children and accumulate knowledge specific to that household.

[1202] "External services" are external resources and services such as picture books, toys, hospitals, and leisure facilities that are suggested by the generative AI based on the information it has learned.

[1203] A "brick and mortar store" is a retail establishment that exists in a physical location where customers visit in person to purchase goods.

[1204] "Related products" are educational materials such as picture books and toys that the generative AI suggests based on the child's interests and preferences learned from parent-child conversations.

[1205] "Detailed information" refers to specific information about the features, specifications, and usage of the proposed product.

[1206] "Reviews" refer to other users' ratings and opinions of the proposed product.

[1207] The "navigation function" is a function that guides customers to the location of suggested products within a physical store, helping them find products efficiently.

[1208] A system for implementing this invention includes a means for using generative AI and speech synthesis technology to create a stuffed animal that will act as a conversation partner when a child begins to talk; a means for using everyday parent-child conversations at home as additional learning material to create an AI specialized for that household; a means for connecting to optimal external services (recommending picture books and toys suitable for the child, or suggesting hospitals and leisure facilities) based on the parent-child conversation and the conversation between the AI ​​and the child; a means for suggesting related products when shopping in a physical store; a means for displaying detailed information and reviews about the suggested products; and a means including a navigation function for guiding the location of products within the store.

[1209] System Program

[1210] Hardware used

[1211] Smartphone (microphone, speaker)

[1212] Server (computing resources for running generative AI models)

[1213] Software used

[1214] speech_recognition library: Used to convert speech data into text.

[1215] openai library: Used to extract interests and preferences from text using generative AI models.

[1216] Data processing and calculation

[1217] 1. Acquiring audio data: The conversation between parent and child is recorded using the smartphone's microphone. The user holds the smartphone and the parent and child have a conversation.

[1218] 2. Speech Recognition: Converts voice data to text using the speech_recognition library. The server receives the voice data and converts it to text data.

[1219] 3. Extracting interests and preferences: Using the OpenAI library, we extract children's interests and preferences from the text. The server runs a generative AI model to extract interests and preferences from the text data.

[1220] 4. Save the results: The extracted interests and preferences are saved in JSON format. The server saves the learning results in a database.

[1221] 5. Product Suggestion: Suggests related products when shopping in a physical store. When a user has a smartphone and is shopping in a physical store, the server suggests related products.

[1222] 6. Displaying details and reviews: Displaying details and reviews about the suggested product. The user checks the details and reviews of the suggested product on their smartphone.

[1223] 7. Navigation function: Provides a navigation function that guides the location of products within the store. The user can use their smartphone to check the location of suggested products within the store.

[1224] Specific examples

[1225] Parent-child conversation: "Mom, I want to read a book about dinosaurs!" "Yes, dinosaurs are interesting."

[1226] Prompt Sentence Examples

[1227] Identify your child's interests and preferences from the following conversation:

[1228] Mommy, I want to read a book about dinosaurs!

[1229] Yeah, dinosaurs are fun.

[1230] This system learns about a child's interests and preferences from parent-child conversations, supports the shopping experience in physical stores, and enables the efficient selection of appropriate educational materials.

[1231] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1232] Step 1:

[1233] A user uses a smartphone to record a conversation between parent and child. The smartphone's microphone captures the audio data and sends it to a server. The input is the audio data of the parent-child conversation, and the output is the audio data sent to the server.

[1234] Step 2:

[1235] The server converts the received voice data into text data using the speech_recognition library. The input is voice data and the output is text data. The server analyzes the voice data and performs speech recognition.

[1236] Step 3:

[1237] The server uses a generative AI model to extract children's interests and preferences from text data. The input is text data, and the output is the extracted interest and preference information. The server uses the openai library to generate prompt sentences and input them into the generative AI model.

[1238] Step 4:

[1239] The server saves the extracted interest and preference information in JSON format. The input is the extracted interest and preference information, and the output is the saved JSON data. The server saves the learning results in a database.

[1240] Step 5:

[1241] When a user goes shopping in a physical store, their smartphone connects to a server and suggests related products. The input is the user's location information and extracted information about their interests and preferences, and the output is a list of suggested products. The server obtains the user's location information and searches for related products.

[1242] Step 6:

[1243] The user checks the details and reviews of the suggested products on their smartphone. The input is a list of suggested products, and the output is the details and reviews. The server retrieves the product details and reviews from the database and sends them to the smartphone.

[1244] Step 7:

[1245] The user uses a smartphone to check the location of suggested products in a store. The input is a list of suggested products and store map information, and the output is product location information. The server uses the store map information to provide navigation to the product location.

[1246] Example 3

[1247] Next, a description will be given of Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1248] In modern families, it is important to understand a child's health condition and lifestyle through parent-child communication and provide appropriate external services. However, it is difficult for parents to constantly understand their child's condition and take appropriate measures. It is also not easy to have appropriate conversations based on the child's interests and preferences. To solve these issues, a system is needed that understands a child's condition through parent-child conversations and dialogue with the child, and suggests appropriate external services.

[1249] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[1250] In this invention, the server includes means for using generation AI and speech synthesis technology to create a dialogue device that will serve as a conversation partner for a child who begins to speak, means for using everyday parent-child conversations at home as additional learning material to create an AI specialized for that household, means for connecting optimal external services (recommending books and toys suitable for the child, suggesting medical institutions and leisure facilities) based on parent-child conversations and conversations between the AI ​​and the child, means for collecting voice data, means for converting the voice data into text data, means for analyzing the text data to understand the child's health condition and lifestyle habits, and means for suggesting external services based on the analysis results. This makes it easier for parents to understand their child's health condition and lifestyle habits and enables them to take appropriate measures quickly.

[1251] "Generative AI" is an artificial intelligence technology that generates new information and dialogue based on voice and text data.

[1252] "Speech synthesis" is a technology that converts text data into voice data and produces speech that sounds like a human voice.

[1253] An "interactive device" is a combination of hardware and software for audibly interacting with a user.

[1254] "Additive learning" is the process of adding new data to an existing artificial intelligence model to train it and improve its performance and accuracy.

[1255] "External services" refers to information about services and facilities that exist outside the system, and provide suggestions and guidance to users.

[1256] "Audio data" refers to digital data of sounds collected through an audio input device such as a microphone.

[1257] "Text data" is character information converted from voice data using voice recognition technology.

[1258] "Natural Language Processing (NLP)" is a technology that enables computers to understand, analyze, and generate human language.

[1259] "Health status" refers to the state of a child's physical and mental health.

[1260] "Lifestyle habits" refers to the behaviors and habits of children in their daily lives.

[1261] "Analysis results" are analysis results obtained using natural language processing or other data analysis techniques.

[1262] This invention is a system that understands a child's health condition and lifestyle habits through parent-child conversations and dialogue with the child, and then suggests appropriate external services. This system uses generative AI models and speech synthesis technology to create a dialogue device that acts as a conversation partner for the child, and uses the parent-child conversations as material for additional learning to build an AI specialized for that household. Furthermore, it provides optimal external services (recommending books and toys, suggesting medical institutions and leisure facilities) based on the parent-child conversations and the conversations between the AI ​​and the child.

[1263] Hardware and software used

[1264] Hardware: microphones, speakers, cameras, servers

[1265] Software: speech recognition software (e.g., Google Speech-to-Text), natural language processing (NLP) software (e.g., OpenAI GPT-4), database management systems (e.g., MySQL)

[1266] Specific operation of the system

[1267] 1. Collection of audio data

[1268] The device uses a microphone to record conversations between parents and children, and dialogue with the child. The device keeps the microphone on at all times and starts recording when certain keywords are detected.

[1269] 2. Audio data conversion

[1270] The device converts the recorded voice data into text data using voice recognition software (Google Speech-to-Text). The device sends the voice data to the Google Speech-to-Text API in real time and temporarily stores the returned text data.

[1271] 3. Text Data Analysis

[1272] The server analyzes the text data using natural language processing (NLP) software (OpenAI GPT-4) to understand the child's health condition and lifestyle habits. The server receives the text data and sends prompts to OpenAI GPT-4 to obtain the analysis results.

[1273] 4. Health status assessment

[1274] Based on the analysis results, the server determines the child's health condition, such as whether they have a cold or are not getting enough exercise. The server evaluates the analysis results and determines the corresponding health condition if certain keywords or phrases are included.

[1275] 5. Information provision

[1276] Based on the results of the judgment, the server retrieves information about nearby hospitals, parks, and leisure facilities from a database management system (MySQL) and provides it to the parent. The server then sends a query to the MySQL database to retrieve information about appropriate facilities. The retrieved information is then sent to the parent's device.

[1277] Specific examples

[1278] Example 1: When a child says, "My throat hurts"

[1279] The device records this conversation and converts it into text using voice recognition software.

[1280] The server analyzes the text and determines that the child has a cold.

[1281] The server retrieves information about nearby hospitals from a database and provides it to the parents, for example, by sending a notification saying, "The nearest hospital is ABC Clinic. The address is 123 Main St. and the phone number is 555-1234."

[1282] Example 2: If your child says, "I haven't been playing outside much lately"

[1283] The device records this conversation and converts it into text using voice recognition software.

[1284] The server analyzes the text and determines that the child is not getting enough exercise.

[1285] The server retrieves information about nearby parks and recreational facilities from a database and provides it to parents, for example, by sending a notification saying, "The nearest park is XYZ Park. The address is 456 Park Ave."

[1286] Prompt Sentence Examples

[1287] Prompt 1: "Please determine if the child has a cold and provide information about nearby hospitals."

[1288] Prompt 2: "Please determine if your child is physically inactive and provide information about nearby parks and recreational facilities."

[1289] This system allows parents to easily understand their children's health conditions and lifestyle habits, and enables them to take appropriate measures promptly. The flow of the identification process in the third embodiment will be described with reference to FIG.

[1290] Step 1:

[1291] Audio data collection

[1292] The device uses a microphone to record conversations between parents and children, as well as interactions with the child.

[1293] Input: Audio of parent-child conversations and interactions with children.

[1294] What it does: The device keeps the microphone on at all times and starts recording when certain keywords (e.g., "Good morning" or "What's wrong?") are detected.

[1295] Output: Recorded audio data.

[1296] Step 2:

[1297] Audio data conversion

[1298] The voice data recorded by the device is converted into text data using voice recognition software (Google Speech-to-Text).

[1299] Input: Recorded audio data.

[1300] How it works: The device sends voice data to the Google Speech-to-Text API in real time and temporarily stores the returned text data.

[1301] Output: The converted text data.

[1302] Step 3:

[1303] Text data analysis

[1304] The server analyzes the text data using natural language processing (NLP) software (OpenAI GPT-4) to understand the child's health and lifestyle habits.

[1305] Input: The converted text data.

[1306] Specific operation: The server receives the text data and sends a prompt to OpenAI GPT-4 to obtain the analysis result. For example, the prompt might be, "If a child says they have a sore throat, is it possible they have a cold?"

[1307] Output: Analysis results.

[1308] Step 4:

[1309] Health status assessment

[1310] Based on the analysis results, the server determines the child's health condition, such as whether they have a cold or are not getting enough exercise.

[1311] Input: Analysis results.

[1312] What happens: The server evaluates the analysis results and determines the corresponding health status if certain keywords or phrases (e.g., "sore throat" or "not playing outside") are included.

[1313] Output: Health status determination result.

[1314] Step 5:

[1315] Information provision

[1316] Based on the results of the judgment, the server retrieves information about nearby hospitals, parks, and leisure facilities from a database management system (MySQL) and provides it to parents.

[1317] Input: Health status determination results.

[1318] What it does: The server queries the MySQL database to retrieve the appropriate facility information, then sends the retrieved information to the parent's device. For example, it sends a notification containing the hospital's name, address, phone number, etc.

[1319] Output: Facility information provided to parents.

[1320] (Application example 3)

[1321] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1322] In modern families, it is difficult to understand a child's health condition and lifestyle through conversations between parents and children, and to provide appropriate external services. It is also difficult to hold a dialogue based on a child's interests and preferences. Furthermore, there is a lack of means to provide information on nearby medical institutions and sports facilities according to a child's health condition.

[1323] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means. In this invention, the server includes: means for using generation AI and voice synthesis technology to create a stuffed animal that will serve as a conversation partner for a child who has started talking; means for using everyday parent-child conversations at home as additional learning material to create an AI specialized for that family; means for connecting optimal external services (recommending picture books and toys suitable for the child, suggesting hospitals and leisure facilities) based on parent-child conversations and conversations between the AI ​​and the child; means for analyzing parent-child conversations to understand the child's health condition and lifestyle; and means for providing information on nearby medical institutions and sports facilities based on the child's health condition. This makes it possible to understand a child's health condition and lifestyle through parent-child conversations and provide appropriate external services.

[1324] "Generative AI" is an artificial intelligence that analyzes parent-child conversations and dialogue with children, and provides appropriate information and services.

[1325] "Speech synthesis" is a technology that converts text data into speech, and is used to enable stuffed toys that children talk to to have natural conversations.

[1326] "Nutaigiri" is an interactive toy that combines generative AI and voice synthesis technology to act as a conversation partner for children.

[1327] "Additional learning" is the process of using everyday conversations between parents and children as learning data to build an AI that is specific to that household.

[1328] "External services" are information and suggestions about medical institutions, sports facilities, picture books, toys, etc. that are provided based on the child's health condition and interests.

[1329] "Health status" refers to the child's physical condition, such as their physical condition and whether they have any illnesses.

[1330] "Lifestyle habits" refers to a child's behavioral patterns and habits in their daily life.

[1331] "Medical institution" means a hospital, clinic, or other facility that diagnoses and treats children's health problems.

[1332] "Sports facilities" are places such as parks and leisure facilities where children can maintain their health through exercise and play.

[1333] The system for implementing this invention uses generative AI and voice synthesis technology to create a stuffed animal that a child can talk to. The server uses parent-child conversations as material for additional learning and builds an AI specialized for that family. Furthermore, the system analyzes parent-child conversations and dialogue between the AI ​​and the child to understand the child's health condition and lifestyle habits. It then provides information on nearby medical institutions and exercise facilities based on the child's health condition.

[1334] Hardware and software used

[1335] Hardware: Smartphones, stuffed toys (interactive toys)

[1336] Software: Python, OpenAI API, speech synthesis engine

[1337] Data processing and calculation

[1338] The server collects parent-child conversations as text data and analyzes them using a generative AI model. Based on the analysis results, the server identifies the child's health and lifestyle habits and suggests appropriate external services. Specifically, the server performs the following processes:

[1339] 1. Collecting parent-child conversations: Collecting parent-child conversations as audio data using smartphones and stuffed toys.

[1340] 2. Converting voice data to text: Using a speech synthesis engine, the collected voice data is converted into text data.

[1341] 3. Text data analysis: Use OpenAI API to analyze text data and understand children's health conditions and lifestyle habits.

[1342] 4. Suggesting external services: Based on the analysis results, provide information on nearby medical institutions and exercise facilities.

[1343] Specific examples

[1344] For example, consider the case where the following parent-child conversation is collected:

[1345] Example of a parent-child conversation:

[1346] Parent: What did you do at school today?

[1347] Child: He didn't seem very energetic today and was coughing all through class.

[1348] Parent: That's terrible. Maybe you should go see a doctor.

[1349] An example prompt for analyzing this conversation is as follows:

[1350] Example prompt sentence:

[1351] Use the following conversation to determine your child's health:

[1352] Parent: What did you do at school today?

[1353] Child: He didn't seem very energetic today and was coughing all through class.

[1354] Parent: That's terrible. Maybe you should go see a doctor.

[1355] The server inputs this prompt into the OpenAI API and retrieves the analysis results. Based on the analysis results, it determines that the child has a cold and provides information about nearby medical facilities, allowing parents to quickly receive appropriate medical care.

[1356] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[1357] Step 1:

[1358] Collection of parent-child conversations

[1359] The device (smartphone or stuffed toy) collects parent-child conversations as audio data. When the user starts a conversation, the device records the audio using its built-in microphone. The input is the audio data of the parent-child conversation, and the output is the recorded audio file.

[1360] Step 2:

[1361] Converting audio data to text

[1362] The server converts the collected voice data into text data using a speech synthesis engine (e.g., Google Speech-to-Text API). The input is an audio file, and the output is text data. The server sends the audio file to the API and receives the returned text data.

[1363] Step 3:

[1364] Text data analysis

[1365] The server uses a generative AI model (for example, OpenAI's GPT-3) to analyze text data. The input is text data, and the output is the analysis result. The server sends the text data to the generative AI model as a prompt sentence and receives the returned analysis result. Specific examples of prompt sentences are as follows:

[1366] Use the following conversation to determine your child's health:

[1367] Parent: What did you do at school today?

[1368] Child: He didn't seem very energetic today and was coughing all through class.

[1369] Parent: That's terrible. Maybe you should go see a doctor.

[1370] Step 4:

[1371] Health status assessment

[1372] The server determines the child's health status based on the analysis results returned by the generative AI model. The input is the analysis results, and the output is the health status judgment result. For example, if the analysis result is "The child has a cold," the server stores that information.

[1373] Step 5:

[1374] Proposal of external services

[1375] The server provides information on appropriate external services (e.g., nearby medical institutions or exercise facilities) based on the health status assessment results. The input is the health status assessment results, and the output is information on the external services. The server searches the database for information on appropriate external services and provides it to the user.

[1376] Step 6:

[1377] Providing information

[1378] The terminal provides the user with information about the external service received from the server. The input is the information from the external service, and the output is the information provided to the user. The terminal conveys information to the user using a screen display or voice synthesis.

[1379] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1380] "Example 1"

[1381] As one embodiment of the present invention, a system incorporating an emotion engine is provided. In this system, a generation AI is used in combination with an emotion engine that recognizes a user's emotions. Specifically, the emotion engine recognizes the user's emotions from the user's tone of voice, speaking style, choice of words, etc. Then, the generation AI conducts a dialogue based on the emotions. For example, when a child is feeling sad, the emotion engine recognizes the sadness, and the generation AI conducts a dialogue to comfort the child.

[1382] "Example 2"

[1383] The emotion engine also recognizes the child's emotions from parent-child conversations, and the generative AI then conducts a dialogue based on that. For example, if a parent uses harsh words against their child, the emotion engine will recognize from the conversation that the child may be angry. The generative AI will then conduct a dialogue to calm the child's anger.

[1384] "Example 3"

[1385] Furthermore, the emotion engine understands the child's emotions from parent-child conversations and dialogue with the child, and the generation AI then suggests external services based on that. For example, if a child talks about having fun playing in the park, the emotion engine will recognize from the conversation that the child likes parks. Based on that information, the generation AI will suggest external services such as providing information about nearby parks.

[1386] The processing flow of each embodiment will be described below.

[1387] "Example 1"

[1388] Step 1: A dialogue with the user is initiated. This dialogue can be initiated by the user speaking or by the system speaking.

[1389] Step 2: The emotion engine recognizes the user's emotions from their tone of voice, speaking style, and choice of words.

[1390] Step 3: The generative AI creates a dialogue based on the information from the emotion engine. For example, if a child is feeling sad, the generative AI creates a dialogue to comfort the child.

[1391] "Example 2"

[1392] Step 1: A parent-child conversation is initiated. This conversation begins when the parent and child start talking naturally.

[1393] Step 2: The emotion engine recognizes the child's emotions from the parent-child conversation. For example, if a parent uses harsh words against their child, the emotion engine will recognize from the conversation that the child may be angry.

[1394] Step 3: The generative AI creates a dialogue based on information from the emotion engine. For example, if a child is angry, the generative AI creates a dialogue to calm the child down.

[1395] "Example 3"

[1396] Step 1: A dialogue with the child is initiated. This dialogue can be initiated by the child speaking or by the system speaking.

[1397] Step 2: The emotion engine recognizes the child's emotions from the child's conversation. For example, if a child talks about having fun playing in the park, the emotion engine recognizes from the conversation that the child likes the park.

[1398] Step 3: The generation AI suggests external services based on information from the emotion engine. For example, based on information that a child likes parks, the generation AI suggests external services such as providing information about nearby parks.

[1399] Example 1

[1400] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1401] In modern families, there are often few people with whom children can feel comfortable talking. It is also difficult to understand a child's emotions and interests and respond appropriately based on those. Furthermore, there is a lack of systems that can grasp a child's health condition and lifestyle habits and suggest appropriate external services based on that. To solve these issues, a system that can understand a child's emotions through dialogue and generate appropriate responses is needed.

[1402] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1403] In this invention, the server includes means for capturing the voices of the child and parent using a built-in microphone, means for transmitting the captured voice data to the server, means for analyzing the received voice data and recognizing the user's emotion, means for generating an appropriate response based on the recognized emotion, means for converting the generated response into voice data using voice synthesis technology, and means for outputting the converted voice data through a speaker. This makes it possible to understand emotions through dialogue with the child and generate an appropriate response.

[1404] "Generative AI" is a system that uses artificial intelligence technology to generate new information or responses based on input data.

[1405] "Speech synthesis" is a technology that converts text data into voice data.

[1406] A "stuffed toy" is a toy made of soft material in the shape of a doll or animal.

[1407] A "microphone" is a device that converts sound into an electrical signal.

[1408] A "speaker" is a device that converts electrical signals into sound and outputs it.

[1409] A "server" is a computer system that processes and stores data over a network.

[1410] "Audio data" is data that represents audio in digital form.

[1411] An "emotion engine" is a system that analyzes and recognizes a user's emotions from voice and text data.

[1412] "External services" are services or resources that exist outside the system and are proposed or provided to users.

[1413] The "voice data transmission module" is software or hardware for transmitting acquired voice data to a server via a network.

[1414] A "speech synthesis engine" is software or hardware for converting text data into voice data.

[1415] This invention is a system that uses a stuffed toy device to communicate with children. The device has a built-in microphone and speaker, and utilizes generative AI and speech synthesis technology. In addition, by combining it with an emotion engine, it can recognize the user's emotions and respond to them.

[1416] Hardware and software used

[1417] Hardware:

[1418] Built-in microphone: Used to capture the voice of the child or parent.

[1419] Built-in speaker: Used to output the generated audio data.

[1420] Plush toy device: An interactive device with a built-in microphone and speaker.

[1421] software:

[1422] Generative AI models (e.g., GPT-4): Analyze voice data and generate appropriate responses.

[1423] Emotion engine (e.g., Affectiva): Recognizes the user's emotions.

[1424] Voice data transmission module: Transmits the acquired voice data to the server.

[1425] A speech synthesis engine (e.g., Google Text-to-Speech): converts the generated text response into audio data.

[1426] Program processing flow

[1427] 1. Acquiring voice input

[1428] The device uses a built-in microphone to pick up the voices of the child and parent. For example, if a child says, "I had fun at school today," the microphone will capture that voice.

[1429] 2. Sending audio data

[1430] The terminal transmits the captured voice data to a server. Specifically, an audio data transmission module in the terminal transmits the captured voice data to the server via the Internet.

[1431] 3. Analysis of audio data

[1432] The server passes the received voice data to a generative AI for analysis. For example, the server converts the voice data into text and inputs it into a generative AI model (e.g., GPT-4) to understand the content.

[1433] 4. Emotional Recognition

[1434] The server recognizes the user's emotions using an emotion engine. Specifically, the server analyzes the tone and speaking style of the voice data and uses an emotion engine (e.g., Affectiva) to recognize "enjoyment."

[1435] 5. Generating an appropriate response

[1436] The server uses generative AI to generate an appropriate response, for example, a generative AI model might generate the response "That's great! What did you enjoy doing?"

[1437] 6. Speech Synthesis

[1438] The server converts the generated response into voice data using speech synthesis technology. Specifically, the server inputs the generated text response into a speech synthesis engine (e.g., Google Text-to-Speech) to generate voice data.

[1439] 7. Audio Output

[1440] The device outputs the voice data sent from the server through a speaker. For example, the built-in speaker of the device plays a voice message such as "That's great! What did you enjoy doing?"

[1441] Examples of specific examples and prompts

[1442] Examples:

[1443] A child says, "I had fun at school today."

[1444] The stuffed animal device responds, "That's great! What did you have fun doing?"

[1445] Example prompts to input to a generative AI model:

[1446] "How do you respond when your child says, 'I had fun at school today?'"

[1447] This system understands children's emotions through dialogue and generates appropriate responses, contributing to the improvement of children's mental health and communication skills.

[1448] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1449] Step 1:

[1450] Acquiring voice input

[1451] The device uses a built-in microphone to pick up the voices of the child and parent.

[1452] Input: Child or parent voice

[1453] What it does: When a child says, "I had fun at school today," a microphone captures the audio.

[1454] Output: Captured audio data

[1455] Step 2:

[1456] Sending audio data

[1457] The terminal transmits the acquired voice data to the server.

[1458] Input: Captured audio data

[1459] Specific operation: The audio data transmission module in the terminal transmits the captured audio data to a server via the Internet.

[1460] Output: Audio data sent to the server

[1461] Step 3:

[1462] Analysis of audio data

[1463] The server passes the received voice data to the generation AI for analysis.

[1464] Input: Audio data sent to the server

[1465] How it works: The server converts the voice data into text and feeds it into a generative AI model (e.g., GPT-4) to understand the content.

[1466] Output: Parsed text data

[1467] Step 4:

[1468] Emotion recognition

[1469] The server uses an emotion engine to recognize the user's emotion.

[1470] Input: Parsed text data

[1471] Specific operation: The server analyzes the tone and speaking style of the voice data and recognizes "fun" using an emotion engine (e.g., Affectiva).

[1472] Output: Recognized emotion data

[1473] Step 5:

[1474] Generating an appropriate response

[1475] The server uses a generation AI to generate an appropriate response.

[1476] Input: Recognized emotion data

[1477] What happens: The generative AI model generates the response, "That's great! What did you enjoy doing?"

[1478] Output: The generated text response

[1479] Step 6:

[1480] Speech synthesis

[1481] The server converts the generated response into voice data using voice synthesis technology.

[1482] Input: Generated text response

[1483] Specific operation: The server inputs the generated text response into a speech synthesis engine (e.g., Google Text-to-Speech) to generate audio data.

[1484] Output: Generated audio data

[1485] Step 7:

[1486] Audio Output

[1487] The terminal outputs the audio data transmitted from the server through a speaker.

[1488] Input: Generated audio data

[1489] What it does: The device's built-in speaker plays the following audio: "That's great! What did you have fun doing?"

[1490] Output: Played audio

[1491] (Application example 1)

[1492] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1493] Conventional stuffed toy devices were limited to interactions with children and were used only at home. Furthermore, they lacked the ability to recognize the user's emotions and engage in dialogue based on them, making it difficult to provide appropriate responses according to the user's emotions. Furthermore, they lacked the means to improve the shopping experience in physical stores, which often left users feeling stressed.

[1494] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means. In this invention, the server includes: means for using generation AI and voice synthesis technology to create a stuffed animal that can be a conversation partner for a child who starts talking; means for using everyday parent-child conversations at home as additional learning material to create an AI specialized for that home; means for connecting optimal external services (recommending picture books and toys suitable for the child, suggesting hospitals and leisure facilities) based on parent-child conversations and conversations between the AI ​​and the child; means for analyzing the user's voice using an application installed on a smartphone in a physical store and providing appropriate product information and recommended products; and means for recognizing the user's emotions and conducting a dialogue based on the analysis. This makes it possible to learn a child's interests and preferences through parent-child conversations at home and improve the shopping experience at a physical store.

[1495] "Generative AI" is an artificial intelligence technology that analyzes user input data and generates appropriate responses.

[1496] "Speech synthesis" is a technology that converts text data into speech.

[1497] A "stuffed toy" is a toy made of soft material in the shape of a doll or animal.

[1498] "Home-specific AI" is artificial intelligence that learns conversation data within a specific home and generates responses optimized for that home.

[1499] An "external service" is a system that provides external information and services that can be accessed from devices and applications within the home.

[1500] A "brick and mortar store" is a store that exists in a physical location and offers goods and services.

[1501] A "smartphone" is a portable device that has the functionality of a mobile phone and is also capable of connecting to the Internet and running applications.

[1502] An "application" is a software program designed to provide a particular function or service.

[1503] "Analyzing the user's voice" is the process of analyzing the voice data uttered by the user and understanding its content.

[1504] "Product information" is detailed information about a specific product.

[1505] "Recommended products" are products suggested based on the user's preferences and needs.

[1506] "Emotion recognition" is the process of determining a user's emotions from the user's tone of voice, speaking style, choice of words, etc.

[1507] "Having a dialogue" means that the user and the system communicate with each other by exchanging information.

[1508] The system for implementing this invention includes: a means for using generative AI and voice synthesis technology to create a stuffed animal that will act as a conversation partner for a child who has started to talk; a means for using everyday parent-child conversations at home as additional learning material to create an AI specialized for that household; a means for connecting to optimal external services based on conversations between parents and children and conversations between the AI ​​and children; a means for analyzing the user's voice using an application installed on a smartphone in a physical store and providing appropriate product information and recommended products; and a means for recognizing the user's emotions and engaging in dialogue based on those emotions.

[1509] Hardware and software used

[1510] Hardware: smartphone, microphone, speaker

[1511] Software: speech_recognition library, transformers library, gTTS library, playsound library

[1512] Data processing and calculation

[1513] 1. Speech recognition: The user's voice is captured through the smartphone's microphone and converted into text using the speech_recognition library, thereby capturing the user's speech as digital data.

[1514] 2. Emotion Recognition: The converted text is analyzed using the emotion engine in the transformers library to recognize the user's emotions, thereby understanding the user's emotional state.

[1515] 3. Response generation: Based on the recognized emotions, a generative AI model is used to generate appropriate responses, enabling dialogue that responds to the user's emotions.

[1516] 4. Speech synthesis: The generated response is converted into speech using the gTTS library and played using the playsound library, providing the user with a spoken response.

[1517] Specific examples

[1518] For example, if a user says, "I'm tired today," the emotion engine will recognize this as "NEGATIVE," and the generative AI model will respond by saying, "I recommend products that will help you relax." This response is converted into voice using speech synthesis technology and played through the smartphone speaker.

[1519] Prompt Sentence Examples

[1520] User: I'm tired today

[1521] Generative AI: Recommend relaxing products. For example, how about some aromatic candles or relaxing herbal tea?

[1522] In this way, it is possible to provide appropriate product information and recommended products according to the user's emotions, thereby improving the shopping experience in physical stores.

[1523] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1524] Step 1:

[1525] A user launches an application on their smartphone and speaks into the microphone. The input is the user's voice, which is captured by the smartphone's microphone. The output is audio data.

[1526] Step 2:

[1527] A smartphone application converts voice data into text data using the speech_recognition library. The input is voice data and the output is text data. Through this conversion, the user's speech content is obtained as digital text.

[1528] Step 3:

[1529] The server analyzes text data using the emotion engine of the transformers library to recognize the user's emotion. The input is text data, and the output is an emotion label (e.g., positive, negative). This analysis identifies the user's emotional state.

[1530] Step 4:

[1531] The server generates an appropriate response using a generative AI model based on the emotion label. The input is text data and emotion label, and the output is a response text. The generative AI model generates dialogue content according to the user's emotion.

[1532] Step 5:

[1533] The server converts the response text to speech using the gTTS library. The input is the response text and the output is speech. This conversion provides the generated response as speech.

[1534] Step 6:

[1535] A smartphone application uses the playsound library to play audio data. The input is the audio data, and the output is an audio response to the user, allowing the user to receive the response by audio.

[1536] Step 7:

[1537] If the user speaks again, the processes from step 1 to step 6 are repeated, thereby enabling continuous dialogue.

[1538] Example 2

[1539] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1540] Conventional parent-child communication support systems have difficulty fully understanding a child's interests, preferences, and emotions, and then conducting appropriate dialogue and making suggestions based on that understanding. They also lack specific support for parents to provide appropriate educational materials and external services to their children. This has led to issues such as difficulty in smooth parent-child communication and in providing effective support for children's growth and learning.

[1541] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1542] In this invention, the server includes means for recording conversations between parents and children and converting the audio data into text data, means for analyzing the text data to learn the child's interests and preferences, means for recognizing the child's emotions from the text data, means for generating dialogue in accordance with the child's interests and emotions, and means for suggesting appropriate educational materials to parents based on the child's interests and preferences. This makes it possible to accurately grasp the child's interests and emotions through parent-child conversations and to provide appropriate dialogue and suggestions based on this.

[1543] "Generative AI" is an artificial intelligence that learns about a child's interests, preferences, and emotions from parent-child conversations and generates dialogue based on that information.

[1544] "Speech synthesis" is a technology that converts text data into voice data and outputs it in a natural voice.

[1545] "Toys" are tools and devices that children play with, and in this system have the function of acting as a conversation partner for children.

[1546] "Parent-child conversation" refers to the everyday dialogue that takes place between parents and children at home.

[1547] "Additive learning" is the process of adding new data to existing training data to improve the performance and accuracy of artificial intelligence.

[1548] "External services" are services and products that exist outside the system, including books and play equipment suitable for children, medical institutions, and entertainment facilities.

[1549] "Audio data" refers to digital data of sound captured through a device such as a microphone.

[1550] "Text data" is character information converted from voice data using voice recognition technology.

[1551] "Natural language processing" is a technology for analyzing text data and understanding its meaning and intent.

[1552] "Emotion recognition" is a technology that identifies human emotions from text data and voice data.

[1553] "Educational materials" are items such as books and toys that are used to support children's learning and development.

[1554] This invention is a system that learns about a child's interests, preferences, and emotions through conversations between parents and children, and then provides appropriate dialogue and suggestions based on that information. A specific embodiment of this system is described below.

[1555] Hardware and software used

[1556] Hardware: microphones, speakers, servers

[1557] Software: speech recognition software (e.g., Google Speech-to-Text), natural language processing engines (e.g., GPT-4), emotion recognition engines (e.g., IBM Watson Tone Analyzer)

[1558] System configuration

[1559] 1. Acquiring audio data

[1560] The device records conversations between parents and children through a microphone. The device's built-in microphone captures audio and stores it as digital audio data in real time.

[1561] 2. Converting voice data to text

[1562] The server uses voice recognition software to convert the recorded voice data into text data. The server receives the voice data and calls the voice recognition API to generate text data.

[1563] 3. Learning about interests and preferences

[1564] The server analyzes the text data using a natural language processing engine to learn the child's interests and preferences. The server analyzes the text data and extracts specific keywords and phrases to identify the child's interests.

[1565] 4. Emotional Recognition

[1566] The server uses an emotion recognition engine to recognize the child's emotions from the text data. The server sends the text data to the emotion recognition API and obtains emotion labels (e.g., anger, joy, sadness).

[1567] 5. Dialogue Generation

[1568] The server uses a generative AI model to generate dialogue based on the child's interests and emotions. The server inputs prompts into the generative AI model to generate appropriate dialogue.

[1569] 6. Providing Proposals

[1570] The server recommends appropriate educational materials (books, toys, etc.) to parents based on the child's interests and preferences. Based on the information it has learned, the server searches its database for related products and content and makes recommendations to parents.

[1571] Specific examples

[1572] Parent-child conversation:

[1573] Parent: "What do you want to do today?"

[1574] Child: "I want to read a book about dinosaurs!"

[1575] System Action:

[1576] The device records the conversation and the server converts the voice data into text.

[1577] Using a natural language processing engine, the server learns from the statement "I want to read a book about dinosaurs" that the child is interested in dinosaurs.

[1578] The server uses a generative AI model to generate dialogue that provides topics related to dinosaurs.

[1579] The server suggests dinosaur-related books and toys to parents.

[1580] Prompt Sentence Examples

[1581] "If your child is interested in dinosaurs, generate a dialogue that brings up the topic of dinosaurs."

[1582] "If a parent uses harsh language towards a child, generate a dialogue that will calm the child's anger."

[1583] The system supports parent-child communication and makes appropriate suggestions based on children's interests and emotions, helping parents provide better educational materials for their children.

[1584] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1585] Step 1:

[1586] Acquiring audio data

[1587] The device records conversations between parents and children through a microphone.

[1588] Input: Parent-child conversation audio

[1589] How it works: The device's built-in microphone captures audio and stores it as digital audio data in real time.

[1590] Output: Digital audio data

[1591] Step 2:

[1592] Converting audio data to text

[1593] The server uses voice recognition software to convert the recorded voice data into text data.

[1594] Input: Digital audio data

[1595] Specific operation: The server receives the voice data and calls the voice recognition API to generate text data.

[1596] Output: Text data

[1597] Step 3:

[1598] Learning about interests and preferences

[1599] The server uses a natural language processing engine to analyze the text data and learn about the child's interests and preferences.

[1600] Input: Text data

[1601] What it does: The server analyzes the text data and extracts specific keywords and phrases to identify the child's interests.

[1602] Output: Data about the child's interests and preferences

[1603] Step 4:

[1604] Emotion recognition

[1605] The server uses an emotion recognition engine to recognize the child's emotion from the text data.

[1606] Input: Text data

[1607] Specific operation: The server sends text data to the emotion recognition API and obtains emotion labels (e.g., anger, joy, sadness).

[1608] Output: Data about the child's emotions

[1609] Step 5:

[1610] Dialogue generation

[1611] The server uses a generative AI model to generate dialogue that responds to the child's interests and emotions.

[1612] Input: Data about the child's interests and preferences, data about the child's emotions

[1613] Specific operation: The server inputs a prompt sentence into the generative AI model and generates an appropriate dialogue.

[1614] Output: The generated dialogue

[1615] Step 6:

[1616] Providing suggestions

[1617] The server suggests appropriate educational materials (books, toys, etc.) to parents based on the child's interests and preferences.

[1618] Input: Data about your child's interests and preferences

[1619] Specific operation: Based on the information learned by the server, it searches the database for relevant products and content and makes suggestions to the parent.

[1620] Output: A list of suggested educational materials

[1621] (Application example 2)

[1622] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1623] Conventional AI systems based on parent-child conversations were limited to use at home and were unable to respond to children's interests and emotions when shopping in physical stores or when out and about. Furthermore, there was a lack of systems that could suggest products based on children's interests and preferences, or engage in conversations that responded to their emotions, making it difficult to improve the parent-child shopping experience.

[1624] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1625] In this invention, the server includes: means for using generation AI and speech synthesis technology to create a stuffed animal that will be a conversation partner for a child who starts talking; means for using everyday parent-child conversations at home as additional learning material to create an AI specialized for that household; means for connecting optimal external services (recommending picture books and toys suitable for the child, or suggesting hospitals and leisure facilities) based on parent-child conversations and conversations between the AI ​​and the child; means for suggesting products based on the child's interests and preferences when the parent and child shop in a physical store; and means for recognizing the child's emotions and engaging in appropriate dialogue. This improves the parent-child shopping experience and enables product suggestions and dialogue that are responsive to the child's interests and emotions.

[1626] "Generative AI" is an artificial intelligence that learns about a child's interests and preferences from parent-child conversations and generates dialogue based on that information.

[1627] "Speech synthesis technology" is a technology that converts text data into speech, and is used to enable stuffed toys that children can talk to to have natural conversations.

[1628] The "stuffed toy" is a toy created using generative AI and voice synthesis technology to act as a conversation partner for children.

[1629] "Additional learning" is the process by which AI learns knowledge and information specific to a particular household using everyday conversations between parents and children as material.

[1630] "External services" are services such as picture books, toys, hospitals, and leisure facilities that AI suggests based on conversations between parents and children and dialogue between children.

[1631] A "physical store" is a physical store where parents and children actually visit and shop.

[1632] "Product suggestion" is the act of AI suggesting appropriate products to parents and children based on the child's interests and preferences.

[1633] "Emotion recognition" is a technology that identifies a child's emotions from parent-child conversations or dialogue between the child and the parent, and responds accordingly.

[1634] "Dialogue generation" is the process by which AI generates appropriate dialogue based on recognized emotions.

[1635] The system for implementing this invention uses generative AI and voice synthesis technology to create stuffed toys that children can talk to. It also uses everyday parent-child conversations at home as additional learning material to build AI specialized for each household. Furthermore, it provides optimal external services (recommending picture books and toys, suggesting hospitals and leisure facilities) based on parent-child conversations and dialogue between the AI ​​and the child.

[1636] When parents and children shop in physical stores, the system can suggest products based on the child's interests and preferences, and can also recognize the child's emotions and engage in appropriate dialogue.

[1637] Hardware and software used

[1638] Hardware: Smartphones, stuffed toys

[1639] Software: OpenAI API, emotion recognition API, product recommendation system

[1640] Data processing and calculation

[1641] 1. Parent-child conversation acquisition: Using the smartphone's voice recognition function, parent-child conversations are acquired as text data.

[1642] 2. Learning interests and preferences: The acquired conversation data is input into the generation AI to learn the child's interests and preferences.

[1643] 3. Emotion Recognition: Use the emotion recognition API to identify children's emotions from conversation data.

[1644] 4. Product Recommendation: Based on learned interests and preferences, the product recommendation system suggests suitable products.

[1645] 5. Dialogue generation: Based on the recognized emotions, the generative AI generates appropriate dialogue.

[1646] Specific examples

[1647] For example, if the keyword "dinosaur" frequently appears in a parent-child conversation, the AI ​​generator will determine that the child is interested in dinosaurs. While shopping in a physical store, a smartphone application will suggest dinosaur-related products (dinosaur picture books and toys). If the child is angry, the emotion recognition API will identify that emotion, and the AI ​​generator will generate a dialogue to calm the child.

[1648] Prompt Sentence Examples

[1649] "If a child is interested in dinosaurs, what products would you suggest?"

[1650] "Generate calming dialogue when your child is upset."

[1651] In this way, a system can be realized that can understand a child's interests and emotions from parent-child conversations and make appropriate product suggestions and dialogues.

[1652] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1653] Step 1:

[1654] Capture parent-child conversations.

[1655] Input: Parent-child conversation audio

[1656] Output: Text data

[1657] Specific operation: Users (parent and child) converse using the smartphone's voice recognition function. The device then converts the voice into text data using the voice recognition API.

[1658] Step 2:

[1659] Learn about your child's interests and preferences.

[1660] Input: Text data (parent-child conversation)

[1661] Output: Data about the child's interests and preferences

[1662] How it works: The server inputs the acquired text data into a generative AI model to learn the child's interests and preferences. For example, if the keyword "dinosaur" appears frequently, it will determine that the child is interested in dinosaurs.

[1663] Step 3:

[1664] Recognize your child's emotions.

[1665] Input: Text data (parent-child conversation)

[1666] Output: Child's emotion data

[1667] What it does: The server uses an emotion recognition API to identify the child's emotions from the text data, for example, determining whether the child is angry based on the content and tone of the conversation.

[1668] Step 4:

[1669] Suggest products.

[1670] Input: Data about your child's interests and preferences

[1671] Output: Product list

[1672] Specific operation: The server uses a product recommendation system to suggest appropriate products based on the learned interests and preferences. For example, if the user is interested in dinosaurs, it will list dinosaur picture books and toys.

[1673] Step 5:

[1674] Generate dialogue.

[1675] Input: Child's emotion data

[1676] Output: Dialogue text

[1677] Specific behavior: The server uses a generative AI model to generate appropriate dialogue based on the recognized emotion. For example, if a child is angry, it generates dialogue to calm them down.

[1678] Step 6:

[1679] The generated dialogue is synthesized into speech.

[1680] Input: Dialogue text

[1681] Output: Audio data

[1682] Specific operation: The server inputs the generated dialogue text into the speech synthesis API to generate voice data. The device plays this voice data and provides dialogue to the child.

[1683] Step 7:

[1684] The service suggests products when parents and children shop in physical stores.

[1685] Input: Product listings, data about your child's interests and preferences

[1686] Output: Product proposal notification

[1687] Specific operation: When users (parents and children) shop at a physical store, the device displays a notification suggesting appropriate products based on the product list. For example, if there are dinosaur-related products in the store, the device will notify the parent about those products.

[1688] Example 3

[1689] Next, a description will be given of Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1690] Conventional conversational AI systems have difficulty accurately understanding a child's health condition, lifestyle habits, and emotions, and then suggesting appropriate external services based on that information. They also lack the ability to effectively utilize information obtained from parent-child conversations to provide dialogue and suggestions tailored to the child's interests and preferences. This has meant that parents have been unable to receive sufficient support when managing their children's health and improving their lifestyle habits.

[1691] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[1692] In this invention, the server includes: means for using generation AI and speech synthesis technology to create a dialogue device that will serve as a conversation partner for a child when they start talking; means for using everyday parent-child conversations at home as additional learning material to create an AI specialized for that household; means for connecting optimal external services (recommending books and toys suitable for the child, and suggesting medical institutions and leisure facilities) based on parent-child conversations and conversations between the AI ​​and the child; means for understanding the child's health condition and lifestyle from parent-child conversations and dialogue with the child and suggesting external services based on that; and means for understanding the child's emotions from parent-child conversations using an emotion analysis engine and for the generation AI to suggest external services based on that. This makes it possible to effectively utilize information obtained from parent-child conversations and dialogue with the child to suggest appropriate external services based on the child's health condition, lifestyle, and emotions.

[1693] "Generative AI" is a system that uses artificial intelligence technology to process natural language and generate dialogue.

[1694] "Speech synthesis" is a technology that converts text data into voice data.

[1695] An "interactive device" is hardware or software for interacting with a user through voice or text.

[1696] "Additive training" is the process of adding new data to an existing learning model and retraining it.

[1697] An "external service" is a service that exists outside the system and provides information or functions.

[1698] A "book" is reading material available in print or digital format.

[1699] A "toy" is a tool or device that children use to play.

[1700] "Medical institution" refers to a facility that provides medical services, such as a hospital or clinic.

[1701] "Leisure facilities" are places or facilities that provide entertainment and recreation.

[1702] "Health status" refers to the state of a child's physical health.

[1703] "Lifestyle habits" refers to the behaviors and habits of children in their daily lives.

[1704] An "emotion analysis engine" is software for extracting and analyzing emotions from text data.

[1705] "Parent-child conversation" refers to verbal exchanges between parents and children.

[1706] Dialogue is a verbal exchange between two people.

[1707] This invention is a system that utilizes information obtained from parent-child conversations and dialogue with a child to propose appropriate external services based on the child's health condition, lifestyle habits, and emotions. Specific embodiments of this system are described below.

[1708] Hardware and software used

[1709] Hardware: Servers, devices (smartphones, tablets, PCs)

[1710] Software: Generative AI models (e.g., GPT-4), sentiment analysis engines (e.g., IBM Watson Tone Analyzer), natural language processing (NLP) tools (e.g., SpaCy, NLTK), database management systems (DBMS)

[1711] System configuration

[1712] 1. Data Entry

[1713] The user uses the device to input conversations between parents and children or dialogues with their children. Specifically, the user can input voice using the microphone of a smartphone or tablet, or input text using a keyboard.

[1714] 2. Data Transmission

[1715] The terminal sends the entered data to the server, where it is uploaded via the Internet.

[1716] 3. Data Analysis

[1717] The server then analyzes the received data using natural language processing (NLP) tools (e.g., SpaCy or NLTK) to tokenize the text data and perform grammatical and semantic analysis.

[1718] 4. Assessment of health status and lifestyle habits

[1719] Based on the analysis results, the server uses a generative AI model to determine the child's health condition and lifestyle habits. Specifically, it inputs prompt sentences into the generative AI model (e.g., GPT-4) to obtain a judgment on the child's health condition and lifestyle habits.

[1720] Example: The prompt sentence is "The child has a cough and seems to have a fever." The generative AI model determines that "the child is likely to have a cold."

[1721] 5. Information provision

[1722] The server then suggests appropriate external services based on the results of the generative AI model. Specifically, it searches a database for information on nearby hospitals and parks and provides it to the user.

[1723] Example: If the user is diagnosed as having a cold, the server sends the user a list of nearby hospitals.

[1724] 6. Emotion analysis

[1725] The server uses a sentiment analysis engine (e.g., IBM Watson Tone Analyzer) to understand the child's emotions from the parent-child conversation.

[1726] Example: Recognizing the emotion "fun" from the text "You looked like you had a lot of fun playing in the park yesterday."

[1727] 7. Emotion-based suggestions

[1728] Based on the results of the sentiment analysis engine, the server uses a generative AI model to suggest external services. Specifically, it inputs prompt sentences into the generative AI model and obtains sentiment-based suggestions.

[1729] Example: Based on the emotion "having fun playing in the park," the server provides the user with information about nearby parks.

[1730] Specific examples

[1731] Example 1: If your child has a cold

[1732] User: Says, "My child has a cough and seems to have a fever."

[1733] Terminal: Converts voice data into text and sends it to the server.

[1734] Server: Analyzes the text data and inputs prompt sentences into the generative AI model.

[1735] Generative AI model: Determines that "there is a high possibility that you have a cold."

[1736] Server: Searches the database for information on nearby hospitals and provides it to the user.

[1737] Example 2: If your child is not getting enough exercise

[1738] User: Texts "My kids have been playing at home a lot lately and don't want to go outside."

[1739] Terminal: Sends text data to the server.

[1740] Server: Analyzes the text data and inputs prompt sentences into the generative AI model.

[1741] Generative AI model: Determines that the person is not getting enough exercise.

[1742] Server: Searches a database for information on nearby parks and leisure facilities and provides it to users.

[1743] Example 3: Children enjoying the park

[1744] User: Says, "They looked like they had a lot of fun playing at the park yesterday."

[1745] Terminal: Converts voice data into text and sends it to the server.

[1746] Server: Analyzes the text data and inputs it into the sentiment analysis engine.

[1747] Sentiment analysis engine: Recognizes the emotion of "fun."

[1748] Server: Inputs prompt sentences to the generative AI model based on emotions.

[1749] Generative AI model: Suggests nearby parks.

[1750] Server: Provides users with information about nearby parks.

[1751] This system effectively utilizes information obtained from conversations between parents and children and dialogues with children to help parents manage their children's health. The flow of the identification process in the third embodiment will be described with reference to FIG.

[1752] Step 1:

[1753] Data Entry

[1754] The user uses the terminal to input conversations between the parent and child and dialogues with the child.

[1755] Input: Audio or text data of parent-child conversations and interactions with the child.

[1756] Specific actions: The user uses the microphone on their smartphone or tablet to input voice commands, or the keyboard to input text.

[1757] Output: The audio or text data is saved to the device.

[1758] Step 2:

[1759] Data transmission

[1760] The terminal transmits the input data to the server.

[1761] Input: Audio or text data.

[1762] Specific operation: The device uploads data to a server via the Internet.

[1763] Output: Data is sent to the server.

[1764] Step 3:

[1765] Data analysis

[1766] The server analyzes the received data using natural language processing (NLP) tools.

[1767] Input: The audio or text data sent to the server.

[1768] What happens: The server uses NLP tools (e.g., SpaCy or NLTK) to tokenize the text data and perform grammatical and semantic analysis.

[1769] Output: Parsed text data.

[1770] Step 4:

[1771] Assessment of health status and lifestyle habits

[1772] Based on the analysis results, the server uses a generative AI model to determine the child's health status and lifestyle habits.

[1773] Input: Parsed text data.

[1774] Specific operation: The server inputs prompt sentences into a generative AI model (e.g., GPT-4) to obtain a judgment about the child's health and lifestyle habits.

[1775] Output: Judgments about the child's health and lifestyle.

[1776] Step 5:

[1777] Information provision

[1778] The server suggests appropriate external services based on the results of the generative AI model.

[1779] Input: Judgments about children's health and lifestyle habits.

[1780] Specific operation: The server searches the database for information on nearby hospitals and parks and provides it to the user.

[1781] Output: Information about suggested external services (e.g., a list of nearby hospitals or parks).

[1782] Step 6:

[1783] Emotion analysis

[1784] The server uses an emotion analysis engine to understand the child's emotions from the parent-child conversation.

[1785] Input: Text data of parent-child conversations.

[1786] Specific operation: The server uses a sentiment analysis engine (e.g., IBM Watson Tone Analyzer) to extract sentiment from text data.

[1787] Output: Extracted emotion data.

[1788] Step 7:

[1789] Emotion-based suggestions

[1790] Based on the results of the sentiment analysis engine, the server uses a generative AI model to suggest external services.

[1791] Input: Extracted emotion data.

[1792] How it works: The server inputs prompts into the generative AI model to get sentiment-based suggestions.

[1793] Output: Suggestion of external services based on emotions (e.g., information about nearby parks).

[1794] In this way, the system can effectively utilize information obtained from parent-child conversations and dialogue with the child to suggest appropriate external services based on the child's health condition, lifestyle habits, and emotions.

[1795] (Application example 3)

[1796] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1797] In modern families, it is important to understand a child's health condition and emotions through conversations between parents and children, and to suggest appropriate external services based on that information. However, effective systems to achieve this are not yet fully available. In particular, there is a lack of functionality to suggest appropriate meals based on a child's health condition and emotions, and to search for restaurants and delivery services that offer those meals. As a result, parents often find it difficult to manage their children's health and choose their meals.

[1798] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means. In this invention, the server includes: means for using generation AI and voice synthesis technology to create a stuffed animal that can be a conversation partner for a child who starts talking; means for using everyday parent-child conversations at home as additional learning material to create an AI specialized for that household; means for connecting optimal external services (recommending picture books and toys suitable for the child, suggesting hospitals and leisure facilities) based on the parent-child conversation and the conversation between the AI ​​and the child; means for understanding the child's health condition and emotions from the parent-child conversation and the dialogue with the child and suggesting an appropriate meal based on that; and means for searching for nearby restaurants and delivery services based on the suggested meal. This makes it possible to understand the child's health condition and emotions through parent-child conversation, suggest an appropriate meal based on that, and further search for restaurants and delivery services that provide that meal.

[1799] "Generative AI" is an artificial intelligence that analyzes parent-child conversations and dialogue with children, and generates appropriate external services and suggestions.

[1800] "Speech synthesis" is a technology that converts text data into speech, and is used when stuffed toys communicate with children.

[1801] "Plushie" is a toy that uses generative AI and voice synthesis technology to act as a conversation partner for children.

[1802] "Additional learning" is the process by which AI learns knowledge specific to a particular household using everyday conversations between parents and children as material.

[1803] "External services" are external resources and services such as picture books, toys, hospitals, and leisure facilities suggested by the generative AI.

[1804] "Health status" refers to a child's physical condition, such as their physical condition and whether they have any illnesses.

[1805] "Emotions" refer to a child's mood and psychological state, and are what the generative AI analyzes from conversations.

[1806] "Meals" are nutritious foods and menus suggested based on the child's health and mood.

[1807] "Food and beverage establishments" are establishments such as restaurants and cafes that serve the proposed meals.

[1808] "Delivery service" is a service that delivers suggested meals to your home.

[1809] The system for implementing this invention uses generative AI and voice synthesis technology to create a stuffed animal that a child can talk to, and understands the child's health condition and emotions through parent-child conversations and dialogue with the child, and based on that, suggests appropriate external services. Specific embodiments of this system are described below.

[1810] Hardware and software used

[1811] Hardware: Smartphones, tablets, plush toys

[1812] Software: OpenAI API, Yelp API, Python

[1813] System configuration

[1814] 1. Generative AI and speech synthesis technology:

[1815] The server uses generative AI and voice synthesis technology to create a stuffed animal that the child can talk to.

[1816] The stuffed animal understands the child's health and emotions through conversations between parents and children and through dialogue with the child.

[1817] 2. Additional learning:

[1818] The server uses everyday parent-child conversations at home as additional learning material to generate AI specialized for that household.

[1819] This allows the AI ​​to tailor its interactions to the specific needs and circumstances of the household.

[1820] 3. Suggestions for external services:

[1821] The server provides optimal external services (recommending picture books and toys, suggesting hospitals and leisure facilities) based on conversations between parents and children and dialogue between the AI ​​and the child.

[1822] In addition, the system understands the child's health condition and emotions through conversations between parents and children, and suggests appropriate meals based on that information.

[1823] 4. Search for restaurants and delivery services:

[1824] The server searches for nearby restaurants and delivery services based on the suggested meal.

[1825] This allows parents to easily arrange appropriate meals for their children according to their health and emotional state.

[1826] Data processing and calculation

[1827] 1. Conversation analysis:

[1828] The server uses OpenAI's generative AI model (text-davinci-003) to analyze the child's health and emotions from parent-child conversations.

[1829] The analysis results are output as keywords such as "cold," "lack of exercise," and "looks fun."

[1830] 2. Meal Suggestions:

[1831] The server then suggests appropriate meals based on the analysis results.

[1832] For example, if it is diagnosed as a "cold," it will suggest "nutritious soups and vitamin-rich meals."

[1833] 3. Search for restaurants:

[1834] The server uses the Yelp API to search for nearby restaurants and delivery services based on the suggested meal.

[1835] Specific examples

[1836] Example conversation: "Mom, a lot of kids at school have colds today. I think I might have a bit of a sore throat too."

[1837] Analysis result: "Cold, a little anxious"

[1838] Suggested meal: "Nutritious soups and vitamin-rich meals."

[1839] Nearby restaurants: "Soup specialty restaurant, healthy food restaurant"

[1840] Prompt Sentence Examples

[1841] Analyze the child's health and emotions from the following conversation:

[1842] Mom, there were a lot of kids with colds at school today. I think I have a bit of a sore throat too.

[1843] In this way, it is possible to understand a child's health condition and emotions through conversations between parents and children, suggest appropriate meals based on that information, and even search for restaurants or delivery services that offer those meals.

[1844] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[1845] Step 1:

[1846] The server collects conversations between parents and children. The conversations are input as voice data via a smartphone or tablet. The server uses voice recognition technology to convert this voice data into text data. The converted text data is sent to the generative AI model.

[1847] Step 2:

[1848] The server uses a generative AI model to analyze the child's health status and emotions from the text data. Specifically, it uses OpenAI's generative AI model (text-davinci-003) to analyze the content of the conversation and extract keywords such as "cold," "lack of exercise," and "looking happy." The analysis results are stored on the server.

[1849] Step 3:

[1850] The server then suggests appropriate meals based on the analysis results. For example, if the illness is diagnosed as a "cold," it will suggest "nutritious soups and vitamin-rich meals." This suggestion is made using a generative AI model, which generates a specific meal menu. The generated menu is then sent to the user's device.

[1851] Step 4:

[1852] The server searches for nearby restaurants and delivery services based on the proposed meal. It uses the Yelp API to search for restaurants and delivery services that match the proposed meal menu and retrieves the results. The search results are stored on the server.

[1853] Step 5:

[1854] The server sends the search results to the user's device. The user can then use their smartphone or tablet to check the suggested meal menus and information about the restaurants and delivery services that offer them. Based on this information, the user can then order the appropriate meal.

[1855] Step 6:

[1856] The user orders meals based on the suggested menu and information about restaurants and delivery services. Orders can be placed directly from the user's device, and the server sends the order information to the restaurant or delivery service. This allows users to easily arrange appropriate meals for their children based on their health and emotional state.

[1857] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1858] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1859] Another example of generative AI is Gemini (internet search engine). <url: https: gemini.google.com ?hl="ja">) are mentioned.

[1860] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1861] [Third embodiment]

[1862] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1863] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1864] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1865] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1866] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1867] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1868] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1869] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1870] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1871] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1872] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1873] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[1874] "Example 1"

[1875] An example of the present invention is a stuffed toy device. This device has an internal microphone and speaker, and communicates with children using generative AI and voice synthesis technology. The microphone picks up the voices of the child and parent and sends the voice data to the AI. The AI ​​analyzes the voice data and generates an appropriate response. The generated response is output from the speaker using voice synthesis technology, realizing a dialogue with the child.

[1876] "Example 2"

[1877] Furthermore, AI learns a child's interests and preferences from parent-child conversations. For example, if it determines that a child is interested in dinosaurs, it will provide topics related to dinosaurs. Also, if it understands a child's favorite foods and activities, it will suggest related picture books and toys. These suggestions help parents provide appropriate educational materials for their children.

[1878] "Example 3"

[1879] AI also understands a child's health condition and lifestyle habits through conversations between parents and children. For example, if it determines that a child has a cold, it will provide information about nearby hospitals.

[1880] If the app determines that a child is not getting enough exercise, it will provide information about nearby parks and leisure facilities. These suggestions will help parents manage their children's health.

[1881] The processing flow of each embodiment will be described below.

[1882] "Example 1"

[1883] Step 1: The microphone on the plush toy device picks up the child's or parent's voice.

[1884] Step 2: Send the captured audio data to the AI.

[1885] Step 3: The AI ​​analyzes the voice data and generates an appropriate response.

[1886] Step 4: The generated response is output from the speaker using speech synthesis technology, enabling dialogue with the child.

[1887] "Example 2"

[1888] Step 1: The AI ​​learns about the child's interests and preferences from parent-child conversations.

[1889] Step 2: Based on the information it has learned, the AI ​​will provide topics that match the child's interests and preferences. Step 3: The AI ​​will understand the child's favorite foods and games and suggest related picture books and toys.

[1890] "Example 3"

[1891] Step 1: The AI ​​understands the child's health and lifestyle habits from parent-child conversations and dialogue with the child.

[1892] Step 2: The AI ​​assesses the child's health condition and provides information about nearby hospitals if necessary.

[1893] Step 3: The AI ​​will assess the child's lifestyle and provide information on nearby parks and leisure facilities if necessary.

[1894] Example 1

[1895] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1896] In modern families, children often lack someone they can talk to with peace of mind. It is also difficult to suggest appropriate external services based on parent-child conversations and the child's interests and preferences. It is also difficult to understand a child's health and lifestyle habits and provide appropriate advice based on them.

[1897] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1898] In this invention, the server includes: a means for using generative AI and speech synthesis technology to create a stuffed animal that can communicate with a child who has begun to talk; a means for creating an AI specialized for a particular household using everyday parent-child conversations as additional learning material; a means for connecting the parent-child conversation and the AI-child conversation to optimal external services (recommending picture books and toys suitable for the child, suggesting hospitals and leisure facilities) based on the conversation; a means for collecting voice data using a microphone; a means for transmitting the collected voice data to the server; a means for the server to analyze the voice data and generate an appropriate response using a generative AI model; a means for converting the generated response into voice data using speech synthesis technology; and a means for outputting the converted voice data from a speaker. This provides an environment where children can interact with ease and makes it possible to suggest appropriate external services based on parent-child conversations. It also makes it possible to understand the child's health condition and lifestyle habits and provide appropriate advice based on them.

[1899] "Generative AI" is an artificial intelligence technology that analyzes voice data and generates appropriate responses.

[1900] "Speech synthesis technology" is a technology that converts text data into voice data.

[1901] A "stuffed toy" is a device that has a microphone and speaker inside and allows for communication with children.

[1902] A "microphone" is a device for collecting sound.

[1903] A "speaker" is a device for outputting audio data as sound.

[1904] A "server" is a computer system that analyzes voice data, runs generative AI models, and applies voice synthesis technology.

[1905] "Audio data" refers to digital data of audio collected by a microphone.

[1906] A "prompt sentence" is text data that is input into a generative AI model and serves as the basis for generating a response.

[1907] "External services" are services provided outside the home, such as recommending picture books and toys suitable for children, and suggesting hospitals and leisure facilities.

[1908] "Additional learning" is the process by which the generative AI learns new knowledge based on everyday conversations between parents and children.

[1909] This invention uses generative AI and voice synthesis technology to provide a stuffed toy device that can serve as a conversation partner for children. Specific embodiments of this system are described below.

[1910] Hardware Configuration

[1911] Terminal: The plush toy device includes the following hardware:

[1912] Microphone: A device for collecting the voices of children and parents. It has high sensitivity and the ability to filter out ambient noise.

[1913] Speaker: A device for outputting the generated audio data. It provides high-quality audio output and plays it at a volume that is easy for children to hear.

[1914] Communication module: A device for transmitting audio data to a server. It can use Wi-Fi or Bluetooth.

[1915] Server: The server includes the following software:

[1916] Generative AI model: An artificial intelligence technology that analyzes voice data and generates appropriate responses, such as OpenAI's GPT-3.

[1917] Speech recognition technology: Technology that converts voice data into text data. For example, Google Cloud Speech-to-Text API is used.

[1918] Speech synthesis technology: Technology that converts text data into voice data. For example, Google Text-to-Speech API or Amazon Polly are used.

[1919] Software Configuration

[1920] Terminal: The plush toy device software includes the following functions:

[1921] Audio data collection: Audio data is collected using a microphone.

[1922] Sending audio data: Send the collected audio data to the server.

[1923] Server: The server software includes the following functions:

[1924] Analysis of voice data: The received voice data is input into the generative AI model and analyzed.

[1925] Response generation: The generative AI model generates an appropriate response based on the prompt.

[1926] Speech synthesis: The generated text response is converted into audio data using speech synthesis technology.

[1927] Send audio data: Send the converted audio data to the device.

[1928] Specific examples

[1929] For example, suppose a child asks, "What shall we play today?" In this case, the following processing is performed:

[1930] 1. Collecting voice data: The device's microphone picks up the child's voice and collects voice data.

[1931] 2. Sending audio data: Send the collected audio data to the server.

[1932] 3. Analysis of voice data: The server inputs the received voice data into the generative AI model and analyzes it. The prompt sentence used is "When a child asks, 'What shall we play today?', generate an appropriate response."

[1933] 4. Response generation: The generative AI model generates the response, "Why don't you try playing ball outside today?"

[1934] 5. Speech synthesis: The generated text response is converted into voice data using speech synthesis technology.

[1935] 6. Sending audio data: Send the converted audio data to the device.

[1936] 7. Audio output: The device speaker plays the audio, "Why don't you go outside and play ball today?"

[1937] In this way, the stuffed toy device allows interaction with the child.

[1938] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1939] Step 1:

[1940] Audio data collection

[1941] Device: The microphone in the plush toy picks up the voices of the child and parent. For example, when a child asks, "What shall we play today?", that voice is input into the microphone.

[1942] Input: Child or parent voice

[1943] Output: Digital audio data

[1944] How it works: The microphone is highly sensitive and has the ability to filter out ambient noise. It converts audio into digital data and temporarily stores it in the internal memory.

[1945] Step 2:

[1946] Sending audio data

[1947] Terminal: The collected voice data is sent to the server. Specifically, the communication module in the terminal converts the voice data into packets and sends them to the server via the Internet.

[1948] Input: Digital audio data

[1949] Output: Audio data sent to the server

[1950] Specific operation: The communication module uses Wi-Fi or Bluetooth to convert voice data into packets and transmit them.

[1951] Step 3:

[1952] Analysis of audio data

[1953] Server: Inputs the received voice data into a generative AI model, such as OpenAI's GPT-3. The server converts the voice data into text data and inputs it as a prompt to the generative AI model.

[1954] Input: Audio data sent to the server

[1955] Output: Text data

[1956] Specific operation: Uses speech recognition technology (e.g., Google Cloud Speech-to-Text API) to convert voice data into text data.

[1957] Step 4:

[1958] Generating a response

[1959] Server: The generative AI model generates an appropriate response based on the prompt, for example, "Why don't you try playing ball outside today?"

[1960] Input: Text data (prompt sentence)

[1961] Output: Response text

[1962] How it works: The generative AI model generates responses by taking into account past interaction history and context.

[1963] Step 5:

[1964] Speech synthesis

[1965] Server: The generated text response is converted into voice data using speech synthesis technology, specifically using a speech synthesis service such as Google Text-to-Speech API or Amazon Polly.

[1966] Input: Response text

[1967] Output: Audio data

[1968] Specific operation: Speech synthesis technology converts text data into speech data to achieve natural pronunciation and intonation.

[1969] Step 6:

[1970] Audio output

[1971] Device: Receives the voice data sent from the server and outputs it from the speaker. For example, the device speaker plays the voice "Why don't you play ball outside today?"

[1972] Input: Audio data sent from the server

[1973] Output: Audio (sound coming from the speaker)

[1974] What it does: The speaker provides high-quality audio output at a volume that is easy for children to hear.

[1975] (Application example 1)

[1976] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1977] Conventional stuffed toy devices were limited to interacting with children and were not designed to support shopping in physical stores or improve customer service. While they had the ability to learn about parent-child conversations and children's interests and preferences, they were not able to guide customers in physical stores, provide sales information, or respond to customers in real time. This created a need for new ways to improve the customer experience in physical stores.

[1978] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1979] In this invention, the server includes means for using generative AI and speech synthesis technology to create a stuffed animal that will be a conversation partner for a child who starts talking, means for using everyday parent-child conversations at home as additional learning material to create an AI specialized for that household, means for connecting optimal external services (recommending picture books and toys suitable for the child, suggesting hospitals and leisure facilities) based on parent-child conversations and conversations between the AI ​​and the child, means for providing an assistant application that supports shopping in a physical store, means for providing voice guidance on the location and features of products in the store, sales information, etc., and means for responding to customer questions in real time. This makes it possible to improve the customer experience in a physical store.

[1980] "Generative AI" is an artificial intelligence technology that analyzes voice and text data and generates appropriate responses.

[1981] "Speech synthesis" is a technology that converts text data into speech.

[1982] A "stuffed toy" is a device that has a microphone and speaker inside and allows for communication with children.

[1983] "Parent-child conversation" refers to the everyday dialogue that takes place between parents and children at home.

[1984] "Additive learning" is the process of learning by adding new data to existing data.

[1985] "External services" are services provided outside the home, such as recommending picture books and toys, or suggesting hospitals and leisure facilities.

[1986] A "brick and mortar store" is a store that exists in a physical location and offers goods and services.

[1987] An "assistant application" is an application that is installed on a smartphone or other device and supports the user's actions.

[1988] "Product location" refers to the location where a particular product is located within a physical store.

[1989] "Sale information" is information about product discounts and special offers.

[1990] "Real-time response" refers to providing an immediate response to a user's question.

[1991] As an embodiment of this invention, we will explain a system that provides an assistant application that supports shopping in a physical store. This system uses generative AI and speech synthesis technology to respond to customer questions in real time and provide voice guidance on the location and features of products in the store, sale information, etc.

[1992] System configuration

[1993] Hardware:

[1994] Smartphone (microphone, speaker)

[1995] software:

[1996] speech_recognition library: Performs speech recognition.

[1997] pyttsx3 library: Performs speech synthesis.

[1998] openai library: Generate responses using generative AI models.

[1999] Processing flow

[2000] 1. Speech Recognition:

[2001] The user speaks a question into the smartphone's microphone.

[2002] The smartphone converts the voice data into text data using the speech_recognition library.

[2003] 2. Response generation by generative AI:

[2004] The server sends the converted text data to the generative AI model using the openai library.

[2005] The generative AI model generates appropriate responses based on the input text data.

[2006] The generated response is returned to the server in text format.

[2007] 3. Speech synthesis:

[2008] The server converts the generated text response into audio data using the pyttsx3 library.

[2009] Audio data is output from the smartphone speaker, conveying the response to the user.

[2010] Specific examples

[2011] For example, if a user asks, "Where is this item located?" the system operates as follows:

[2012] 1. The user's question, "Where is this product located?" is recorded by the smartphone microphone.

[2013] 2. The speech_recognition library converts the audio data into text data.

[2014] 3. The converted text data is sent to the generative AI model.

[2015] 4. The generative AI model generates an appropriate response, such as "The item is on the third floor, on the left."

[2016] 5. The generated response is converted to audio data using the pyttsx3 library.

[2017] 6. A voice will be output from the smartphone speaker saying, "The product is on the third floor, on the left side."

[2018] Prompt Sentence Examples

[2019] An example of a prompt sent to the generative AI model is as follows:

[2020] Generate appropriate responses to customer questions: Where is this item located?

[2021] This will create a smart shopping assistant that improves the customer experience in physical stores.

[2022] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[2023] Step 1:

[2024] The user speaks a question into the smartphone microphone. The input is the user's voice data, and the output is the voice data captured by the smartphone microphone. This voice data is processed in the following steps:

[2025] Step 2:

[2026] The device converts the voice data into text data using the speech_recognition library. The input is the voice data acquired in step 1, and the output is text data. Through this conversion, the user's question is obtained in text format.

[2027] Step 3:

[2028] The terminal sends the converted text data to the server. The input is the text data generated in step 2, and the output is the text data sent to the server. This data is input to the generative AI model in the next step.

[2029] Step 4:

[2030] The server uses the openai library to send text data to the generative AI model and generate an appropriate response. The input is the text data sent in step 3, and the output is the text data of the generated response. This response is an appropriate response to the user's question.

[2031] Step 5:

[2032] The server converts the generated text response into audio data using the pyttsx3 library. The input is the text data generated in step 4, and the output is audio data. This conversion results in the text response in audio format.

[2033] Step 6:

[2034] The server sends the converted voice data to the terminal. The input is the voice data generated in step 5, and the output is the voice data sent to the terminal. This data is provided to the user in the next step.

[2035] Step 7:

[2036] The terminal outputs the voice data from the smartphone speaker and communicates the response to the user. The input is the voice data sent in step 6, and the output is the voice response that the user can hear. This allows the user to receive an appropriate answer to their question by voice.

[2037] Example 2

[2038] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[2039] Conventional systems that suggest educational materials for children have the problem of being unable to address the specific interests and preferences of individual children because they make suggestions based on general interests and preferences. Furthermore, there has been no system that utilizes parent-child conversations to learn about a child's interests and preferences and then suggests appropriate educational materials based on that information. This makes it difficult for parents to provide their children with the most appropriate educational materials.

[2040] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[2041] In this invention, the server includes: means for using generation AI and speech synthesis technology to create a stuffed animal that will serve as a conversation partner for a child who has begun to talk; means for using everyday parent-child conversations at home as additional learning material to create an AI specialized for that family; means for connecting to optimal external services (proposing educational materials suitable for the child) based on parent-child conversations and conversations between the AI ​​and the child; means for collecting parent-child conversation data and converting the voice data into text data; means for analyzing the text data using natural language processing technology to identify the child's interests and preferences; means for generating suggestions for related topics and educational materials based on the identified interests and preferences; and means for providing the generated suggestions to parents, thereby enabling parents to provide appropriate educational materials based on their child's specific interests and preferences.

[2042] "Generative AI" is artificial intelligence that uses natural language processing technology to analyze text data and generate specific information.

[2043] "Speech synthesis" is a technology that converts text data into voice data.

[2044] A "stuffed animal" is a soft toy made to be a conversation companion for children.

[2045] "Parent-child conversation" refers to the everyday dialogue that takes place between parents and children at home.

[2046] "Additive learning" is the process of updating an AI model by adding new data to existing training data.

[2047] An "external service" is a service that provides resources or information outside the system.

[2048] "Educational materials" is a general term for books, toys, teaching materials, etc. that are used to support children's learning and development.

[2049] "Audio data" refers to data in which audio is recorded in digital format.

[2050] "Text data" refers to data in which character information is recorded in digital format.

[2051] "Natural language processing technology" is a technology that allows computers to understand, interpret, and generate human language.

[2052] "Interests and preferences" are the things that an individual is particularly interested in or likes.

[2053] A "suggestion" is a recommended course of action or option for a particular purpose.

[2054] This invention is a system that learns a child's interests and preferences from parent-child conversations and suggests appropriate educational materials based on this information. A specific embodiment of this system will be described below.

[2055] 1. Program Generation

[2056] The server generates a program to learn the child's interests and preferences from parent-child conversations. The program analyzes the conversation data using natural language processing technology to identify the child's interests and preferences.

[2057] 2. Program processing explanation

[2058] The server processes data using the following hardware and software:

[2059] Hardware:

[2060] Server: A computer with high-performance data processing capabilities

[2061] Device: A device equipped with a microphone for recording parent-child conversations

[2062] software:

[2063] Speech recognition software: Google Speech-to-Text API

[2064] Natural language processing technology: generative AI models (e.g., OpenAI's GPT-4)

[2065] 3. Data processing and calculation

[2066] The server processes and calculates the data in the following steps:

[2067] 1. Data Collection:

[2068] The terminal records the conversation between parent and child and transmits the audio data to the server.

[2069] 2. Data conversion:

[2070] The server converts the received voice data into text data using the Google Speech-to-Text API.

[2071] 3. Data Analysis:

[2072] The server inputs the text data into a generative AI model to extract information about the child's interests and preferences.

[2073] 4. Identifying interests and preferences:

[2074] The server analyzes the output of the generative AI model and calculates the frequency of occurrence of certain keywords (e.g., "dinosaur").

[2075] 5. Proposal generation:

[2076] The server retrieves information on related picture books and toys from a database and creates a suggestion list.

[2077] 6. Providing Proposals:

[2078] The server sends the suggestion list to the terminal, which displays the information to the parent.

[2079] 4. Specific Examples

[2080] If conversation data is collected showing that a child frequently uses the word "dinosaur," the server will suggest picture books and toys related to dinosaurs.

[2081] If a child says, "I like pizza," the server will suggest pizza-related picture books and cooking kits.

[2082] 5. Examples of prompts

[2083] "If a child is interested in dinosaurs, what picture books and toys should I suggest?"

[2084] "What educational materials should I provide my child if their favorite food is pizza?"

[2085] The system allows parents to provide appropriate educational materials based on their child's specific interests and preferences.

[2086] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2087] Step 1:

[2088] Data collection

[2089] Input: Parent-child conversation audio data

[2090] Specific operation: The device records the conversation between parent and child. The recorded audio data is sent from the device to the server.

[2091] Output: Audio data sent to the server

[2092] Step 2:

[2093] Data Conversion

[2094] Input: Audio data

[2095] Specific operation: The server sends the received voice data to the Google Speech-to-Text API and converts the voice data into text data.

[2096] Output: Text data

[2097] Step 3:

[2098] Data analysis

[2099] Input: Text data

[2100] How it works: The server inputs the text data into a generative AI model (e.g., OpenAI's GPT-4) to extract information about the child's interests and preferences.

[2101] Output: Information about interests and preferences

[2102] Step 4:

[2103] Identifying interests and preferences

[2104] Input: Information about your interests and preferences

[2105] Specific operation: The server analyzes the output of the generative AI model and calculates the frequency of occurrence of specific keywords (e.g., "dinosaur").

[2106] Output: Identified interests and preferences

[2107] Step 5:

[2108] Proposal Generation

[2109] Input: Identified interests and preferences

[2110] Specific operation: The server retrieves information on related picture books and toys from the database and creates a list of suggestions.

[2111] Output: Suggestion list

[2112] Step 6:

[2113] Providing suggestions

[2114] Input: Suggestion list

[2115] Specific operation: The server sends the proposal list to the terminal, and the terminal displays the information to the parent.

[2116] Output: Suggestion list displayed in parent

[2117] (Application example 2)

[2118] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[2119] Conventional educational support systems for children have limited functionality for learning about children's interests and preferences from parent-child conversations, and lack functionality to support the shopping experience in physical stores. Furthermore, there is no way for parents to instantly check detailed information or reviews of suggested products, making it difficult for parents to select appropriate educational materials. Furthermore, there is no navigation function to guide users to the location of products in the store, which makes shopping less efficient.

[2120] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for using generation AI and speech synthesis technology to create a stuffed animal that can be a conversation partner for a child who has started talking; means for using everyday parent-child conversations at home as additional learning material to create an AI specialized for that family; means for connecting optimal external services (recommending picture books and toys suitable for the child, suggesting hospitals and leisure facilities) based on parent-child conversations and conversations between the AI ​​and the child; means for suggesting related products when shopping in a physical store; means for displaying detailed information and reviews about the suggested products; and means including a navigation function for guiding the location of products in the store. This allows the server to learn a child's interests and preferences from parent-child conversations, support the shopping experience in a physical store, and enable efficient and appropriate selection of educational materials.

[2121] "Generative AI" is an artificial intelligence that learns a child's interests and preferences from conversations between parents and children, and then conducts a dialogue based on that information.

[2122] "Speech synthesis technology" is a technology that converts text data into voice data and generates natural speech.

[2123] A "stuffed toy" is a soft toy that children can talk to, and it communicates using generative AI and voice synthesis technology.

[2124] "Parent-child conversation" refers to the everyday dialogue that takes place between parents and children at home.

[2125] "Additional learning materials" are data that the generative AI uses to learn new information through conversations between parents and children and accumulate knowledge specific to that household.

[2126] "External services" are external resources and services such as picture books, toys, hospitals, and leisure facilities that are suggested by the generative AI based on the information it has learned.

[2127] A "brick and mortar store" is a retail establishment that exists in a physical location where customers visit in person to purchase goods.

[2128] "Related products" are educational materials such as picture books and toys that the generative AI suggests based on the child's interests and preferences learned from parent-child conversations.

[2129] "Detailed information" refers to specific information about the features, specifications, and usage of the proposed product.

[2130] "Reviews" refer to other users' ratings and opinions of the proposed product.

[2131] The "navigation function" is a function that guides customers to the location of suggested products within a physical store, helping them find products efficiently.

[2132] A system for implementing this invention includes a means for using generative AI and speech synthesis technology to create a stuffed animal that will act as a conversation partner when a child begins to talk; a means for using everyday parent-child conversations at home as additional learning material to create an AI specialized for that household; a means for connecting to optimal external services (recommending picture books and toys suitable for the child, or suggesting hospitals and leisure facilities) based on the parent-child conversation and the conversation between the AI ​​and the child; a means for suggesting related products when shopping in a physical store; a means for displaying detailed information and reviews about the suggested products; and a means including a navigation function for guiding the location of products within the store.

[2133] System Program

[2134] Hardware used

[2135] Smartphone (microphone, speaker)

[2136] Server (computing resources for running generative AI models)

[2137] Software used

[2138] speech_recognition library: Used to convert speech data into text.

[2139] openai library: Used to extract interests and preferences from text using generative AI models.

[2140] Data processing and calculation

[2141] 1. Acquiring audio data: The conversation between parent and child is recorded using the smartphone's microphone. The user holds the smartphone and the parent and child have a conversation.

[2142] 2. Speech Recognition: Converts voice data to text using the speech_recognition library. The server receives the voice data and converts it to text data.

[2143] 3. Extracting interests and preferences: Using the OpenAI library, we extract children's interests and preferences from the text. The server runs a generative AI model to extract interests and preferences from the text data.

[2144] 4. Save the results: The extracted interests and preferences are saved in JSON format. The server saves the learning results in a database.

[2145] 5. Product Suggestion: Suggests related products when shopping in a physical store. When a user has a smartphone and is shopping in a physical store, the server suggests related products.

[2146] 6. Displaying details and reviews: Displaying details and reviews about the suggested product. The user checks the details and reviews of the suggested product on their smartphone.

[2147] 7. Navigation function: Provides a navigation function that guides the location of products within the store. The user can use their smartphone to check the location of suggested products within the store.

[2148] Specific examples

[2149] Parent-child conversation: "Mom, I want to read a book about dinosaurs!" "Yes, dinosaurs are interesting."

[2150] Prompt Sentence Examples

[2151] Identify your child's interests and preferences from the following conversation:

[2152] Mommy, I want to read a book about dinosaurs!

[2153] Yeah, dinosaurs are fun.

[2154] This system learns about a child's interests and preferences from parent-child conversations, supports the shopping experience in physical stores, and enables the efficient selection of appropriate educational materials.

[2155] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2156] Step 1:

[2157] A user uses a smartphone to record a conversation between parent and child. The smartphone's microphone captures the audio data and sends it to a server. The input is the audio data of the parent-child conversation, and the output is the audio data sent to the server.

[2158] Step 2:

[2159] The server converts the received voice data into text data using the speech_recognition library. The input is voice data and the output is text data. The server analyzes the voice data and performs speech recognition.

[2160] Step 3:

[2161] The server uses a generative AI model to extract children's interests and preferences from text data. The input is text data, and the output is the extracted interest and preference information. The server uses the openai library to generate prompt sentences and input them into the generative AI model.

[2162] Step 4:

[2163] The server saves the extracted interest and preference information in JSON format. The input is the extracted interest and preference information, and the output is the saved JSON data. The server saves the learning results in a database.

[2164] Step 5:

[2165] When a user goes shopping in a physical store, their smartphone connects to a server and suggests related products. The input is the user's location information and extracted information about their interests and preferences, and the output is a list of suggested products. The server obtains the user's location information and searches for related products.

[2166] Step 6:

[2167] The user checks the details and reviews of the suggested products on their smartphone. The input is a list of suggested products, and the output is the details and reviews. The server retrieves the product details and reviews from the database and sends them to the smartphone.

[2168] Step 7:

[2169] The user uses a smartphone to check the location of suggested products in a store. The input is a list of suggested products and store map information, and the output is product location information. The server uses the store map information to provide navigation to the product location.

[2170] Example 3

[2171] Next, a third embodiment of the third embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[2172] In modern families, it is important to understand a child's health condition and lifestyle through parent-child communication and provide appropriate external services. However, it is difficult for parents to constantly understand their child's condition and take appropriate measures. It is also not easy to have appropriate conversations based on the child's interests and preferences. To solve these issues, a system is needed that understands a child's condition through parent-child conversations and dialogue with the child, and suggests appropriate external services.

[2173] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[2174] In this invention, the server includes means for using generation AI and speech synthesis technology to create a dialogue device that will serve as a conversation partner for a child who begins to speak, means for using everyday parent-child conversations at home as additional learning material to create an AI specialized for that household, means for connecting optimal external services (recommending books and toys suitable for the child, suggesting medical institutions and leisure facilities) based on parent-child conversations and conversations between the AI ​​and the child, means for collecting voice data, means for converting the voice data into text data, means for analyzing the text data to understand the child's health condition and lifestyle habits, and means for suggesting external services based on the analysis results. This makes it ...

Claims

1. Using generative AI and voice synthesis technology, we have created a stuffed animal that will talk to a child who starts talking. A means for generating an AI specialized for the family of the parent and child using daily conversations between the parent and child at home as additional learning material; a means for proposing an optimal external service to the child based on at least one of the parent-child conversation and the conversation between the stuffed toy and the child; a means for grasping at least one of the health condition and the emotion of the child from at least one of the conversation between the parent and the child and the conversation between the stuffed toy and the child, and for suggesting an appropriate diet based on the grasping result; A means for searching for at least one of restaurants and delivery services based on the suggested meal; A system including:

2. The system according to claim 1 , wherein the generating AI learns at least one of the child's interests and preferences from the parent-child conversation and conducts a dialogue based on the learning results.

3. The system according to claim 1 , wherein the generating AI grasps at least one of the child's health condition and lifestyle habits from at least one of the parent-child conversation and the dialogue with the child, and suggests external services based on the results of the understanding.

Images