System

Abstract

To provide a system which allows children to ask a question in their paces, and acquire information in an understandable manner.SOLUTION: A system comprises: voice recognition means including a voice recognition engine which recognizes voice; reception means for receiving a question from a child; conversion means for converting the question from a child into a text format by the voice recognition engine; estimation means for using a natural language processing engine for estimating an intention of the question, in the question of the child converted into the text format; generation means for generating a prompt sentence which instructs to acquire a proper response to the estimated intention of the question; acquisition means for acquiring a response to the estimated intention of the question, using the generated prompt sentence and the generative AI model; and provision means for converting the acquired response into voice information by a voice synthesis engine, for providing the voice information in an understandable manner for a child.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including a description and related instruction sentence regarding the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2022-180282 A Summary of the Invention [Problem to be solved by the invention]

[0004] Children are curious and ask many questions, but they have limited time to ask their teachers and parents. They cannot research things online like adults can, and they lack the ability to understand things. [Means for solving the problem]

[0005] A system is provided that includes a voice recognition means, a means for accepting a question by voice, a means for inferring the intent of the question, a means for providing information based on the inferred intent, a means for adjusting the tone of the information to be provided, and a means for providing information in the voice of a specific character or person, thereby enabling a child to ask a question at his or her own pace and obtain information in an easily understandable form. [Brief description of the drawings]

[0006] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Diagram 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. FIG. [Diagram 3] FIG. 11 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Diagram 5] FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 13 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 2 is a sequence diagram showing a process flow of the data processing system in the first embodiment of the first form example. [Figure 12] FIG. 11 is a sequence diagram showing a process flow of the data processing system in the application example 1 of the first embodiment. [Figure 13] FIG. 11 is a sequence diagram showing a process flow of the data processing system in the second embodiment of the second embodiment. [Figure 14] FIG. 13 is a sequence diagram showing a process flow of the data processing system in application example 2 of embodiment 2. [Figure 15] FIG. 13 is a sequence diagram showing a process flow of a data processing system in a third embodiment of the present invention. [Figure 16]FIG. 13 is a sequence diagram showing a process flow of the data processing system in the third application example of the third embodiment. [Figure 17] FIG. 13 is a sequence diagram showing the flow of processing in the data processing system in the first embodiment of the first form example when an emotion engine is combined. [Figure 18] FIG. 13 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Form Example 1 when combined with an emotion engine. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0007] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0008] First, the terms used in the following description will be explained.

[0009] In the following embodiments, a signed processor (hereinafter simply referred to as a "processor") may be one arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be one type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (TENSOR PROCESSING UNIT (registered trademark)).

[0010] In the following embodiments, a signed RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by the processor.

[0011] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0012] In the following embodiments, a communication I / F (Interface) with a code is an interface including a communication processor and an antenna. The communication I / F controls communication between multiple computers. An example of a communication standard applied to the communication I / F is a wireless communication standard including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0013] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. In addition, in this specification, the same idea as "A and / or B" is also applied when three or more things are expressed by connecting them with "and / or."

[0014] [First embodiment]

[0015] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0016] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0017] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[0018] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0019] The reception device 38 includes a touch panel 38A and a microphone 38B, and receives user input. The touch panel 38A detects contact with a pointer (e.g., a pen or a finger) to receive user input by the touch of the pointer. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0020] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (e.g., voice and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs voice according to instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a Complementary Metal-Oxide-Semiconductor (CMOS) image sensor or a Charge Coupled Device (CCD) image sensor.

[0021] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54.

[0022] FIG. 2 shows an example of main functions of the data processing device 12 and the smart device 14.

[0023] As shown in Fig. 2, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32. The specific process program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific process program 56 from the storage 32, and executes the read specific process program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific process program 56 executed on the RAM 30.

[0024] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0025] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores a reception output program 60. The reception output program 60 is used together with the specific processing program 56 by the data processing system 10. The processor 46 reads out the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0026] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0027] "Example 1"

[0028] The embodiment of the present invention includes a speech recognition engine for recognizing the child's voice as a speech recognition means. The speech recognition engine identifies the child's pronunciation and vocabulary and converts it into text format. A natural language processing engine is used as a means for inferring the intent of the question. The engine analyzes the text obtained from the speech recognition engine and understands the subject and purpose of the question. Furthermore, as an information providing means, appropriate information is extracted from a knowledge base and provided in a form that is easy for the child to understand. As a means for adjusting the tone of the information, the system has a function for adjusting the method of providing information according to the child's age and level of understanding. Finally, a speech synthesis engine is used as a means for providing information in the voice of a specific character or person. The engine imitates the voice of a character or person that the child likes and provides information in that voice.

[0029] "Example 2"

[0030] As a concrete example, if a child asks "Why is the sky blue?", a speech recognition engine converts this question into text. A natural language processing engine then understands that the question is asking "Why is the sky blue?" and extracts information from a knowledge base about "Why does the sky look blue?" and explains it in terms the child can easily understand. This information is provided in the voice of a character the child likes.

[0031] The process flow of each embodiment will be described below.

[0032] "Example 1"

[0033] Step 1: The child asks the app a question, which is asked by voice.

[0034] Step 2: The speech recognition engine converts the child's speech into text. This engine identifies the child's pronunciation and vocabulary and converts it into a text format.

[0035] Step 3: A natural language processing engine analyzes the text to understand the subject and purpose of the question. Step 4: Extracts the appropriate information from the knowledge base and presents it in a child-friendly format.

[0036] Step 5: Adapt the way you present information to the child's age and level of understanding.

[0037] Step 6: The voice synthesis engine mimics the voice of a character or person that the child likes and provides the information in that voice.

[0038] "Example 2"

[0039] Step 1: A child asks, "Why is the sky blue?"

[0040] Step 2: The speech recognition engine converts this question into text.

[0041] Step 3: The natural language processing engine understands that the question is asking "why about the color of the sky?"

[0042] Step 4: Extract information about "why the sky looks blue" from the knowledge base.

[0043] Step 5: Explain the information in terms your child can understand.

[0044] Step 6: Present that information in the voice of your child's favorite character.

[0045] Example 1

[0046] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal".

[0047] Conventional voice recognition systems have difficulty in accurately recognizing children's voices and providing appropriate information. In addition, the information provided is often not in a form that is easy for children to understand, and there is a lack of functionality to provide information in the voice of a specific character or person. This poses the problem that it is difficult for children to learn with interest.

[0048] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0049] In this invention, the server includes a voice recognition means, a means for accepting a question by voice, a means for guessing the intent of the question, a means for providing information based on the guessed intent, a means for adjusting the tone of the provided information, a means for providing information in the voice of a specific character or person, a means for converting voice data into text, a means for analyzing the text to understand the subject or purpose of the question, a means for extracting appropriate information from a knowledge base, a means for formatting the extracted information into a form that is easy for children to understand, and a means for converting the formatted information into voice. This makes it possible to accurately recognize a child's voice, provide appropriate information, and further provide information in the voice of a specific character or person.

[0050] A "voice recognition means" is a device or software that has the function of analyzing voice data and converting it into text format.

[0051] A "means for accepting voice queries" is a device or software for receiving voice input from a user and processing the voice data.

[0052] The "means for inferring the intent of a question" is a device or software that has the function of analyzing the text converted by the speech recognition means and understanding the subject and purpose of the user's question.

[0053] The "means for providing information" refers to a device or software that has the function of selecting appropriate information based on the intent of the question and providing it to the user.

[0054] The "means for adjusting the tone of information" refers to a device or software that has the function of adjusting the content and expression method of the information provided according to the user's age and level of understanding.

[0055] A "means for providing information in the voice of a specific character or person" is a device or software that has a function of imitating the voice of a character or person that a user likes and providing information in that voice.

[0056] "Means for converting voice data into text" refers to a device or software that has the function of converting voice data into text format using speech recognition means.

[0057] The "means for analyzing text to understand the subject or purpose of the question" refers to a device or software that has the function of analyzing the converted text and understanding the intent of the user's question.

[0058] The "means for extracting appropriate information from a knowledge base" refers to a device or software that has the function of searching and extracting relevant information from a knowledge base based on the intent of a question.

[0059] "Means for formatting the extracted information into a form that is easy for children to understand" refers to a device or software that has the function of processing the extracted information into a form that is easy for children to understand.

[0060] The "means for converting formatted information into speech" refers to a device or software having a function for converting formatted information into speech format using speech synthesis technology.

[0061] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0062] This invention is a system for recognizing a child's voice and providing appropriate information. The system includes a voice recognition means, a means for accepting a question by voice, a means for inferring the intent of the question, a means for providing information, a means for adjusting the tone of the information, a means for providing information in the voice of a specific character or person, a means for converting voice data into text, a means for analyzing the text to understand the subject or purpose of the question, a means for extracting appropriate information from a knowledge base, a means for formatting the extracted information into a form that is easily understandable to children, and a means for converting the formatted information into voice.

[0063] Hardware and Software Used

[0064] The server uses general voice recognition software as a means of voice recognition. Specifically, Google Cloud Speech-to-Text API is one example. This API is used to convert voice data into text.

[0065] To infer the intent of a question, a natural language processing engine is used, specifically a generative AI model such as OpenAI's GPT-4. The engine analyzes the text and understands the subject and purpose of the question.

[0066] To provide this information, a means is needed to extract the appropriate information from a knowledge base, which could be Wikipedia or a specific educational database.

[0067] Adjusting the tone of the information requires a way to shape it according to the child's age and level of understanding, using simple vocabulary and short sentences.

[0068] To provide information in the voice of a specific character or person, a voice synthesis engine is used, specifically, voice synthesis software such as Amazon Polly or Google Cloud Text-to-Speech.

[0069] Examples

[0070] When a user (child) says "Tell me about dinosaurs," the following process takes place:

[0071] 1. The device records the user's voice and sends it to the server.

[0072] 2. The server converts the speech to text using the Google Cloud Speech-to-Text API.

[0073] 3. The server passes the converted text, “Tell me about dinosaurs,” to GPT-4 and analyzes the intent of the question.

[0074] 4. The server extracts information about dinosaurs from Wikipedia and formats the information in a way that is easy for children to understand.

[0075] 5. The server uses Amazon Polly to generate a speech in the voice of the child's favorite character saying, "Dinosaurs were creatures that appeared on Earth approximately 230 million years ago."

[0076] 6. The terminal plays the generated audio and provides information to the user.

[0077] Examples of prompt statements

[0078] In response to the question "Tell me about dinosaurs," the following prompt sentence is input to the generative AI model.

[0079] User Question: Tell me about dinosaurs

[0080] Prompt for generative AI model: Give a simple, child-friendly description of dinosaurs. Include the type, era, characteristics, etc. of the dinosaur.

[0081] Using this prompt, the generative AI model generates text to provide appropriate information to children.

[0082] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0083] Step 1:

[0084] The terminal records the user's voice and transmits it to the server.

[0085] Input: User's voice

[0086] Output: Audio data

[0087] Specific operation: When a user says, "Tell me about dinosaurs," the device's microphone captures the audio and sends the audio data to the server.

[0088] Step 2:

[0089] The server receives the voice data and converts it to text using speech recognition means.

[0090] Input: Audio data

[0091] Output: Text data

[0092] What happens: The server uses the Google Cloud Speech-to-Text API to parse the audio data and convert it into text, such as "Tell me about dinosaurs."

[0093] Step 3:

[0094] The server passes the text data to a natural language processing engine to analyze the intent of the question.

[0095] Input: Text data

[0096] Output: Question intent

[0097] Specific operation: The server uses GPT-4 to analyze the text data and identify the user’s intent to learn about dinosaurs.

[0098] Step 4:

[0099] The server extracts appropriate information from a knowledge base based on the intent of the question.

[0100] Input: Question intent

[0101] Output: Extracted information

[0102] Specific operation: The server searches for information about dinosaurs from knowledge bases such as Wikipedia, and extracts the information that "dinosaurs are creatures that appeared on Earth approximately 230 million years ago, and include species such as Tyrannosaurus and Triceratops."

[0103] Step 5:

[0104] The server formats the extracted information in a way that is easy for children to understand.

[0105] Input: Extracted information

[0106] Output: Formatted information

[0107] Specific operation: The server converts the extracted information into simple vocabulary and short sentences, formatting it into the form of "Dinosaurs are large creatures that lived on Earth a long time ago. There were species called Tyrannosaurus and Triceratops."

[0108] Step 6:

[0109] The server passes the formatted information to a speech synthesis engine, which generates speech in the voice of a particular character or person.

[0110] Input: Formatted information

[0111] Output: Audio data

[0112] What it does: The server uses Amazon Polly to generate a speech sequence in the voice of a child's favorite character saying, "Dinosaurs are big creatures that lived on Earth a long time ago. There were species called Tyrannosaurus and Triceratops."

[0113] Step 7:

[0114] The terminal receives the generated voice data and plays it back to the user.

[0115] Input: Audio data

[0116] Output: Played audio

[0117] Specific operation: The terminal plays the audio data received from the server, allowing the user to listen to information about dinosaurs in the voice of their favorite character.

[0118] (Application example 1)

[0119] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0120] Conventional voice recognition systems are primarily targeted at adult voices, and have the problem of difficulty in accurately recognizing children's voices. It is also difficult to properly understand the content of a child's question and provide information in a format that is easy for children to understand. Furthermore, there is a lack of means to increase children's motivation to learn by providing information in the voices of characters that interest them. To solve these problems, a system is needed that can accurately recognize children's voices, properly analyze the intent of the question, and provide information in a format that is easy for children to understand.

[0121] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0122] In this invention, the server includes a voice recognition means, a means for accepting a question by voice, a means for guessing the intention of the question, a means for providing information based on the guessed intention, a means for adjusting the tone of the information to be provided, a means for providing information in the voice of a specific character or person, a means for providing information in a form that is easy for a child to understand, a means for providing information in the voice of a character that the child likes, and a means including an application to be installed on the smart device. This makes it possible to accurately recognize the child's voice, appropriately analyze the intention of the question, and provide information in a form that is easy for a child to understand. Also, by providing information in the voice of a character that the child is interested in, it is possible to increase the child's motivation to learn.

[0123] "Speech recognition means" refers to technology for converting speech into text.

[0124] The "means for accepting questions by voice" is an interface that enables a user to input questions by voice.

[0125] "Means for inferring the intent of a question" refers to technology that analyzes the subject and purpose of a question from input text.

[0126] The "means for providing information based on an inferred intent" is a technology for providing appropriate information according to the intent of the analyzed question.

[0127] "Means for adjusting the tone of the information provided" refers to technology that adjusts the way information is provided depending on the user's age and level of understanding.

[0128] "Means of providing information in the voice of a specific character or person" refers to technology that provides information by imitating the voice of a character or person that a user likes.

[0129] "Means of providing information in a form that children can easily understand" refers to techniques that provide information in a concise and easy-to-understand manner so that children can easily understand it.

[0130] The "means for providing information in the voice of a character that a child likes" is a technology for providing information using the voice of a character that is of interest to children.

[0131] "Means including an application installed on a smart device" refers to a software application installed on a device such as a smartphone or tablet.

[0132] In order to implement this invention, means including a voice recognition means, a means for accepting a question by voice, a means for inferring the intent of the question, a means for providing information based on the inferred intent, a means for adjusting the tone of the information to be provided, a means for providing information in the voice of a particular character or person, a means for providing information in a form that is easy for children to understand, a means for providing information in the voice of a character that children like, and an application to be installed on a smart device are used.

[0133] Hardware and Software Used

[0134] The server uses the following hardware and software:

[0135] Hardware: Smartphones, tablets

[0136] software:

[0137] Speech recognition engine (Google Cloud Speech-to-Text)

[0138] Natural language processing engine (Google Cloud Natural Language API)

[0139] Knowledge Base (Wolfram Alpha API)

[0140] Speech synthesis engine (Google Cloud Text-to-Speech)

[0141] Processing flow

[0142] 1. Speech recognition: The user speaks a question into their smart device. The speech recognition engine converts this speech into text.

[0143] 2. Natural language processing: The server analyzes the text obtained from the speech recognition engine using a natural language processing engine to infer the intent of the question.

[0144] 3. Information provision: The server extracts appropriate information from a knowledge base based on the inferred intent.

[0145] 4. Tone Adjustment: The server adjusts the tone of the information depending on the user's age and level of understanding.

[0146] 5. Voice synthesis: The server uses a voice synthesis engine to provide information in the voice of the child's favorite character.

[0147] Examples

[0148] For example, if a child asks a smart device, "Tell me about dinosaurs," a speech recognition engine converts this speech into text. Next, a natural language processing engine analyzes this text and infers that the intent of the question is, "I want to know information about dinosaurs." The server extracts information about dinosaurs from a knowledge base and provides the information in a format that is easy for the child to understand. Finally, a speech synthesis engine is used to provide information in the voice of the child's favorite character, such as, "Tyrannosaurus is a dinosaur that lived about 68 million years ago."

[0149] Examples of prompt statements

[0150] Create an application that recognizes children's voices, analyzes the intent of the questions, and provides appropriate information. Use Google Cloud Speech-to-Text API for speech recognition and Google Cloud Natural Language API for natural language processing. Use Wolfram Alpha API to provide information and Google Cloud Text-to-Speech API for speech synthesis. Provide information in the voice of a character that children like.

[0151] In this way, educational applications that allow children to learn in a fun way can be realized.

[0152] The flow of the specific process in the application example 1 will be described with reference to FIG.

[0153] Step 1:

[0154] A user speaks a question into a smart device, and the spoken voice data is picked up by the microphone of the smart device.

[0155] Step 2:

[0156] The terminal sends the acquired voice data to a voice recognition engine (Google Cloud Speech-to-Text). The voice recognition engine converts the voice data into text data. The input is voice data, and the output is text data.

[0157] Step 3:

[0158] The server sends the text data obtained from the speech recognition engine to a natural language processing engine (Google Cloud Natural Language API). The natural language processing engine analyzes the text data and infers the intent of the question. The input is text data, and the output is data that indicates the intent of the question.

[0159] Step 4:

[0160] The server extracts the appropriate information from a knowledge base (Wolfram Alpha API) based on the inferred question intent. The input is data indicating the question intent, and the output is related information data.

[0161] Step 5:

[0162] The server adjusts the tone of the information it provides based on the extracted information data. It changes the way information is presented depending on the user's age and level of understanding. The input is information data and the user's profile data, and the output is the adjusted information data.

[0163] Step 6:

[0164] The server sends the adjusted information data to a voice synthesis engine (Google Cloud Text-to-Speech). The voice synthesis engine converts the information into voice data in the voice of the child's favorite character. The input is the adjusted information data, and the output is voice data.

[0165] Step 7:

[0166] The terminal reproduces the voice data obtained from the voice synthesis engine and provides it to the user, who receives the answer to the question through the voice coming from the smart device.

[0167] Example 2

[0168] Next, a second embodiment of the second embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0169] In conventional voice recognition systems, it can be difficult to provide appropriate answers to user questions. In particular, when children ask questions, it is necessary to provide answers that match their level of understanding, but there is a lack of technology to achieve this. Furthermore, it is necessary to provide answers in the voices of specific characters or people to make it easier to attract children's interest, but conventional systems are not able to fully address this issue.

[0170] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0171] In this invention, the server includes a voice recognition means, a means for accepting a question by voice, a means for analyzing the intent of the question, a means for extracting information based on the analyzed intent, a means for converting the extracted information, a means for voice synthesizing the converted information, and a means for providing information in the voice of a specific character or person. This makes it possible to provide an appropriate and easy-to-understand answer to a user's question. Also, providing information in the voice of a specific character or person can attract the interest of children in particular.

[0172] "Speech recognition means" refers to technology for converting speech into text.

[0173] A "means for accepting voice queries" is a technology for receiving and processing voice input from a user.

[0174] A "means for analyzing the intent of a question" is a technique for understanding the content of a user's question and identifying its intent.

[0175] "Means of information extraction" refers to techniques for retrieving appropriate information from a database or knowledge base based on the analyzed intent.

[0176] "Means for converting information" refers to technology for converting extracted information into a form that is easy for users to understand.

[0177] "Means for synthesizing speech" refers to technology for converting text information into speech.

[0178] "Means for providing information in the voice of a specific character or person" refers to technology for playing back the generated voice in the voice of a specific character or person.

[0179] This invention is a system that allows a user to input a question by voice and provides an appropriate answer to the question. This system includes a voice recognition means, a means for accepting a question by voice, a means for analyzing the intent of the question, a means for extracting information, a means for converting information, a means for synthesizing voice, and a means for providing information in the voice of a specific character or person.

[0180] Hardware and Software Used

[0181] Hardware:

[0182] Device: Any device with a microphone (smartphone, tablet, PC, etc.)

[0183] Server: A server with high-performance computing resources (such as a cloud server)

[0184] software:

[0185] Speech recognition engine: General speech recognition API (e.g. Google Cloud Speech-to-Text API)

[0186] Natural Language Processing Engine: Advanced natural language processing models (e.g. OpenAI's GPT-4)

[0187] Knowledge base: online encyclopedia APIs (e.g. Wikipedia API) or proprietary databases

[0188] Speech synthesis engine: Speech synthesis API (e.g. Amazon Polly)

[0189] Program Processing

[0190] The user speaks a question into the terminal. For example, a child may ask, "Why is the sky blue?"

[0191] 1. Voice Recognition:

[0192] The terminal captures the user's voice through a microphone.

[0193] The server converts the audio to text using the Google Cloud Speech-to-Text API.

[0194] 2. Natural Language Processing:

[0195] The server sends the converted text to a natural language processing engine (OpenAI's GPT-4).

[0196] The natural language processing engine analyzes the intent of the question and understands that you are asking "why about the color of the sky."

[0197] 3. Extracting information from the knowledge base:

[0198] The server accesses a knowledge base (Wikipedia API or its own database) and searches for information about "why the sky looks blue."

[0199] 4. Transformation of information:

[0200] The server converts the retrieved information into language that is easy for children to understand.

[0201] 5. Speech synthesis:

[0202] The server sends the converted information to a speech synthesis engine (Amazon Polly), which generates speech in the voice of the child's favorite character.

[0203] 6. Audio playback:

[0204] The terminal plays the generated audio.

[0205] Examples

[0206] As a concrete example, here is how a child asks "Why is the sky blue?":

[0207] Child: "Why is the sky blue?"

[0208] Terminal: Captures audio and sends it to the server.

[0209] Server: Use the Google Cloud Speech-to-Text API to convert the speech to text "Why is the sky blue?"

[0210] Server: OpenAI's GPT-4 analyzes the intent of the question and retrieves information from the knowledge base.

[0211] Server: Convert that into a simple explanation like, "The sky is blue because the sun's light is scattered by tiny particles in the air."

[0212] Server: Generate the character's voice using Amazon Polly.

[0213] Terminal: Play the generated audio.

[0214] Examples of prompt statements

[0215] "If a child asks, 'Why is the sky blue?', how should I respond?"

[0216] This system starts with the user's voice input and goes through a series of steps including voice recognition, natural language processing, information extraction from a knowledge base, information conversion, voice synthesis, and voice playback, before providing the user with an appropriate answer.

[0217] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0218] Step 1:

[0219] The user speaks a question into the terminal.

[0220] Input: User speech (e.g. "Why is the sky blue?")

[0221] Specific action: A child speaks to the device, "Why is the sky blue?"

[0222] Output: The device captures the audio data.

[0223] Step 2:

[0224] The terminal transmits the captured voice data to the server.

[0225] Input: Captured audio data

[0226] Specific operations: The device sends voice data to the server.

[0227] Output: The server receives the audio data.

[0228] Step 3:

[0229] The server converts the voice data into text using a voice recognition engine.

[0230] Input: Audio data

[0231] What happens: The server uses a speech recognition engine (e.g. a common speech recognition API) to convert the speech into text: "Why is the sky blue?"

[0232] Output: Text data ("Why is the sky blue?")

[0233] Step 4:

[0234] The server sends the text data to a natural language processing engine to analyze the intent of the question.

[0235] Input: Text data ("Why is the sky blue?")

[0236] What happens: The server uses a natural language processing engine (e.g., an advanced natural language processing model) to parse the intent of the question and understands that you are asking "why about the color of the sky?"

[0237] Output: Question intent ("Why is the sky colored?")

[0238] Step 5:

[0239] The server extracts information from a knowledge base based on the parsed intent.

[0240] Input: Question intent ("Reasons for the color of the sky")

[0241] What happens: The server accesses a knowledge base (e.g. an online encyclopedia API or a proprietary database) and searches for information about "why the sky looks blue."

[0242] Output: Extracted information ("The sky is blue because sunlight is scattered by tiny particles in the air")

[0243] Step 6:

[0244] The server translates the extracted information into child-friendly language.

[0245] Input: Extracted information ("The sky is blue because sunlight is scattered by tiny particles in the air.")

[0246] What happens: The server converts that information into a simple explanation, such as "The sky is blue because the sun's light is scattered by tiny particles in the air."

[0247] Output: Transformed information ("The sky is blue because sunlight is scattered by tiny particles in the air.")

[0248] Step 7:

[0249] The server sends the converted information to a speech synthesis engine, which generates speech in the voice of a particular character or person.

[0250] Input: Transformed information ("The sky is blue because sunlight is scattered by tiny particles in the air.")

[0251] Specific behavior: The server uses a speech synthesis engine (e.g., a speech synthesis API) to generate speech in the character's voice.

[0252] Output: The generated audio data

[0253] Step 8:

[0254] The terminal reproduces the generated voice data.

[0255] Input: Generated audio data

[0256] What it does: The device plays a character's voice saying, "The sky looks blue because the sun's light is scattered by tiny particles in the air."

[0257] Output: The user hears the audio.

[0258] (Application example 2)

[0259] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0260] Conventional voice recognition systems have had the problem of difficulty in providing appropriate answers to questions posed by children. In particular, it is necessary to provide information in a form that is easy for children to understand, but existing systems are inadequate in this regard. It is also important to provide answers in the voice of a child's favorite character to attract the child's interest, but this was not achieved with conventional systems. Furthermore, it is necessary to consider the convenience of the application as it is installed on a smartphone.

[0261] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0262] In this invention, the server includes a voice recognition means, a means for accepting a question by voice, a means for guessing the intention of the question, a means for providing information based on the guessed intention, a means for adjusting the tone of the provided information, a means for providing information in the voice of a specific character or person, a means for extracting information from a knowledge base, a means for reconstructing information in words that are easy for children to understand, and an application installed on the smartphone. This makes it possible to provide information in an appropriate and easy-to-understand form in response to a question asked by a child, and further to answer in the voice of a character that is likely to interest children.

[0263] "Speech recognition means" refers to technology that converts speech into text.

[0264] The "means for accepting questions by voice" is an interface that enables a user to input questions by voice.

[0265] "Means for inferring the intent of a question" refers to technology that analyzes the content of an input question and understands its intent.

[0266] "Means for providing information based on inferred intent" refers to technology that selects and provides appropriate information according to the intent of the question.

[0267] The "means for adjusting the tone of the information provided" refers to a technique for adjusting the tone of voice or the manner of expression of the information provided.

[0268] "Means for providing information in the voice of a specific character or person" refers to technology for providing information in audio form by imitating the voice of a specific character or person.

[0269] A "means for extracting information from a knowledge base" is a technique for searching and extracting answers to questions from a knowledge base.

[0270] "Means for reconstructing information in terms that children can easily understand" refers to technology for reconstructing extracted information into a form that children can easily understand.

[0271] An "application installed on a smartphone" is software that runs on a smartphone.

[0272] A system for implementing the present invention includes a voice recognition means, a means for accepting a question via voice, a means for inferring the intent of the question, a means for providing information based on the inferred intent, a means for adjusting the tone of the provided information, a means for providing information in the voice of a particular character or person, a means for extracting information from a knowledge base, a means for reconstructing information in terms that are easy for a child to understand, and an application installed on a smartphone.

[0273] Program processing explanation

[0274] Voice Recognition Method

[0275] When a user speaks a question into the smartphone's microphone, the speech recognition means converts the speech into text using the speech_recognition library.

[0276] A way to guess the intent of a question

[0277] The converted text is then analyzed by a natural language processing engine to infer the intent of the question, using a pipeline from the transformers library.

[0278] A means of extracting information from a knowledge base

[0279] Based on the inferred intent, appropriate information is searched and extracted from a knowledge base, which is a pre-built database.

[0280] A way to restructure information in terms that children can easily understand

[0281] The extracted information is then reconstructed in a way that is easy for children to understand, using a generative AI model.

[0282] A means of providing information in the voice of a specific character or person

[0283] The reconstructed information is then provided as audio in the voice of a specific character or person, using the pyttsx3 library.

[0284] Examples

[0285] For example, if a child asks "Why is the sky blue?", a speech recognition tool converts the question into text. A natural language processing engine then understands that the question is asking "Why is the sky blue?" and extracts information from a knowledge base about "Why does the sky look blue?" and explains it in terms the child can easily understand. This information is provided in the voice of a character the child likes.

[0286] Examples of prompt statements

[0287] If a child asks "Why is the sky blue?", the speech recognition engine converts this question into text, and the natural language processing engine understands that the question is asking "Why is the sky blue?" It then extracts information from its knowledge base about "Why does the sky look blue?" and explains it in terms the child can easily understand, delivered in the voice of a character the child loves.

[0288] In this way, it is possible to provide information in an appropriate and easy-to-understand form in response to a question asked by a child, and further, to respond in the voice of a character that is likely to interest children.

[0289] The flow of the specific process in the application example 2 will be described with reference to FIG.

[0290] Step 1:

[0291] The user inputs a question by voice through the smartphone's microphone. The input voice data is captured by the smartphone's microphone.

[0292] Step 2:

[0293] The smartphone sends the captured voice data to the voice recognition means, which converts the voice data into text using the speech_recognition library. The input is the voice data, and the output is the text data.

[0294] Step 3:

[0295] The server sends the converted text data to a natural language processing engine. The natural language processing engine uses the transformers library pipeline to analyze the text data and infer the intent of the question. The input is text data, and the output is data that indicates the intent of the question.

[0296] Step 4:

[0297] The server searches for appropriate information from a knowledge base based on the inferred question intent. The knowledge base is a pre-constructed database that contains information corresponding to the question intent. The input is data indicating the question intent, and the output is appropriate information.

[0298] Step 5:

[0299] The server sends the extracted information to a generative AI model, which reconstructs the information in terms that are easy for the child to understand. The generative AI model uses prompts to reconstruct the information. The input is the extracted information, and the output is the reconstructed information.

[0300] Step 6:

[0301] The server uses the pyttsx3 library to provide the reconstructed information as audio in the voice of a specific character or person. The input is the reconstructed information and the output is the audio data.

[0302] Step 7:

[0303] The smartphone provides the generated voice data to the user through the speaker. The user can hear the answer provided in the voice of a specific character or person. The input is the voice data and the output is the voice that the user hears.

[0304] Furthermore, an emotion engine that estimates the emotion of the user may be combined. That is, the identification processing unit 290 may estimate the emotion of the user using the emotion identification model 59, and perform identification processing using the emotion of the user.

[0305] "Example 1"

[0306] As one embodiment of the present invention, a system incorporating an emotion engine is provided. The system accepts a question by voice of a child, infers the intention of the question, and provides information based on the inferred intention. Furthermore, the system recognizes the child's emotion and adjusts the method of providing information based on the emotion. Specifically, if the system feels that the child is angry, the system provides information in a calm tone. On the other hand, if the system feels that the child is happy, the system provides information in a bright tone. In this way, the system selects the optimal method of providing information according to the child's emotion.

[0307] "Example 2"

[0308] As a concrete example, consider a child asking "Why is the sky blue?" and feeling joy from the child's voice. In this case, the system uses a speech recognition engine to convert the question into text and a natural language processing engine to infer the intent of the question. Next, it extracts information about "why the sky looks blue" from the knowledge base and explains the information in words that the child can easily understand. Then, because the emotion engine recognizes the child's joy, the system provides the information in a bright tone. Furthermore, providing the information in the voice of the child's favorite character attracts the child's interest and deepens their understanding.

[0309] The process flow of each embodiment will be described below.

[0310] "Example 1"

[0311] Step 1: Accept the child's spoken question.

[0312] Step 2: Convert the question into text using a speech recognition engine.

[0313] Step 3: Use a natural language processing engine to infer the intent of the question.

[0314] Step 4: Extract information based on the intent inferred from the knowledge base.

[0315] Step 5: Recognize the child's emotions using the emotion engine.

[0316] Step 6: Adjust how you present information based on the emotions you identify.

[0317] "Example 2"

[0318] Step 1: A child asks, "Why is the sky blue?"

[0319] Step 2: Convert the question into text using a speech recognition engine.

[0320] Step 3: Use a natural language processing engine to infer the intent of the question.

[0321] Step 4: Extract information about "why the sky looks blue" from the knowledge base.

[0322] Step 5: Use the emotion engine to recognize the child's joy.

[0323] Step 6: Provide information in an upbeat tone based on the perceived pleasure.

[0324] Step 7: Provide information in the voice of a child's favorite character.

[0325] Example 1

[0326] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal".

[0327] Conventional voice recognition systems have difficulty in accurately recognizing children's voices and providing appropriate information. In addition, they lack the ability to adjust the way information is provided according to the child's emotions, making it difficult to provide information in a way that is easy for children to understand and that interests them. Furthermore, providing information in the voice of a specific character or person can attract children's interest, but there are few systems that have such a function.

[0328] The identification process by the identification processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means. In this invention, the server includes a voice recognition means, a means for accepting a question by voice, a means for guessing the intention of the question, a means for providing information based on the guessed intention, a means for adjusting the tone of the information to be provided, a means for providing information in the voice of a specific character or person, and a means for recognizing the child's emotion and adjusting the method of providing information based on the emotion. This makes it possible to accurately recognize the child's voice, understand the intention of the question, and provide information in a form that is easy for the child to understand. In addition, by adjusting the method of providing information according to the child's emotion, it is possible to provide information in a form that is easy for the child to be interested in. Furthermore, by providing information in the voice of a specific character or person, it is possible to attract the child's interest.

[0329] "Speech recognition means" refers to a technology for converting voice data into text data.

[0330] "Means for accepting questions by voice" refers to a technique that enables the system to accept questions input by voice from the user.

[0331] The "means for inferring the intent of a question" is a technology for understanding the subject and purpose of a user's question from text data obtained by a speech recognition means.

[0332] "Means for providing information" refers to technology for providing the user with appropriate information based on the intent of the question.

[0333] "Means for adjusting the tone of the information provided" refers to a technique for adjusting the way information is provided depending on the user's age and level of understanding.

[0334] "Means for providing information in the voice of a specific character or person" refers to technology for providing information by imitating the voice of a character or person that a user likes.

[0335] "Means for recognizing a child's emotions and adjusting the way information is provided based on those emotions" is a technology for recognizing emotions from a child's voice and adjusting the way information is provided based on those emotions.

[0336] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0337] This invention is a system that recognizes a child's voice, analyzes the intent of the question, and provides appropriate information. It also has a function of recognizing the child's emotion and adjusting the way information is provided based on the emotion. This system is implemented using the following hardware and software.

[0338] Hardware and Software Used

[0339] 1. Speech recognition engine: Used to convert voice data into text data. Specifically, we use the Google Cloud Speech-to-Text API.

[0340] 2. Natural language processing engine: Used to analyze the intent of questions from text data. Specifically, we use OpenAI's GPT-3 (registered trademark).

[0341] 3. Knowledge base: Used to provide appropriate information based on the intent of the question, specifically using Wikipedia API and Wolfram Alpha API.

[0342] 4. Speech synthesis engine: Used to convert text data into voice data. Specifically, Amazon Polly or Google Text-to-Speech API is used.

[0343] 5. Emotion recognition engine: Used to recognize emotions from children's voices. Specifically, it uses Microsoft® Azure® Emotion API.

[0344] System Operation

[0345] When a user speaks a question into the terminal, the terminal captures the voice with a microphone and sends it to the server. The server converts the voice into text using a speech recognition engine. It then uses a natural language processing engine to analyze the intent of the question from the text. Based on the analyzed intent, the server searches and retrieves the appropriate information from a knowledge base.

[0346] The way the information is provided is adjusted depending on the user's age and emotions. For example, it provides simple explanations to younger children and more detailed information to older children. It also uses an emotion recognition engine to determine whether a child is angry or happy and adjusts the tone of the information based on that emotion. For example, it provides information in a calm tone if the child is angry and in a bright tone if the child is happy.

[0347] Finally, the speech synthesis engine is used to convert the adjusted information into speech in the voice of a specific character or person. The terminal plays back this speech data and provides it to the user.

[0348] Examples

[0349] Describe what happens when a user asks, "Why is the sky blue?"

[0350] 1. Receiving speech input: The user speaks into the device's microphone, "Why is the sky blue?"

[0351] 2. Speech Recognition: The server receives the speech data and uses the Google Cloud Speech-to-Text API to convert the speech to text: "Why is the sky blue?"

[0352] 3. Question intent analysis: The server inputs the text into GPT-3 and extracts the intent, such as “I want to know the reason for the color of the sky.”

[0353] 4. Information retrieval: The server sends the keyword "sky color" to the Wikipedia API and retrieves the returned information.

[0354] 5. Information Tone Adjustment: The server generates a simple text, "The sky is blue because the sun's light is scattered in the air," and presents it in a light tone.

[0355] 6. Speech synthesis: The server sends text to Amazon Polly and receives speech data in return.

[0356] 7. Audio output: The device plays the audio data and the user hears the explanation in the character's voice.

[0357] Examples of prompt statements

[0358] In response to the question "Why is the sky blue?", the following prompt sentence is input to the generative AI model.

[0359] User Question: Why is the sky blue?

[0360] Age: 7 years

[0361] Emotion: Happy

[0362] Provide your answers in simple terms and in a light tone.

[0363] This prompt sentence is then used to have the generative AI model generate an appropriate answer.

[0364] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0365] Step 1:

[0366] A user speaks a question into a device. Specifically, the user says, "Why is the sky blue?" The input is voice data, and the device's microphone captures this voice. The output is the captured voice data.

[0367] Step 2:

[0368] The terminal transmits the captured audio data to the server. The input is the captured audio data, and the output is the audio data transmitted to the server.

[0369] Step 3:

[0370] The server uses a speech recognition engine to convert the voice data into text data. Specifically, it uses the Google Cloud Speech-to-Text API. The input is voice data, speech recognition is performed as data processing, and the output is the text data "Why is the sky blue?"

[0371] Step 4:

[0372] The server uses a natural language processing engine to analyze the intent of the question from the text data. Specifically, it uses OpenAI's GPT-3. The input is the text data "Why is the sky blue?", intent analysis is performed as a data calculation, and the output is the intent data "I want to know the reason for the color of the sky."

[0373] Step 5:

[0374] The server searches for the appropriate information from the knowledge base. Specifically, it uses the Wikipedia API. The input is the intent data "color of the sky", and a query is sent to the API as a data search, and the output is the information data "The sky is blue because the sunlight is scattered in the air".

[0375] Step 6:

[0376] The server adjusts the way information is provided based on the user's age and emotions. Specifically, it uses an emotion recognition engine to analyze the user's emotions and adjust the tone of the information. The input is information data and the user's age and emotion data, tone adjustment is performed as data processing, and the output is the adjusted information data.

[0377] Step 7:

[0378] The server uses a voice synthesis engine to convert the adjusted information data into voice data. Specifically, it uses Amazon Polly. The input is the adjusted information data, voice synthesis is performed as data processing, and the output is voice data.

[0379] Step 8:

[0380] The device receives the voice data sent from the server and plays it back to the user through the speaker. The input is the voice data, and the output is the voice the user hears. Specifically, the device's speaker plays back information in the character's voice saying, "The sky is blue because the sunlight is scattered in the air."

[0381] (Application example 1)

[0382] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0383] Conventional voice recognition systems have difficulty in accurately recognizing children's voices and providing appropriate information. In addition, they cannot adjust the way information is provided according to the child's emotions, and cannot provide information in a form that is easy for children to understand. Furthermore, they lack the ability to provide information in the voice of a specific character or person, making it difficult to attract children's interest.

[0384] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0385] In this invention, the server includes a voice recognition means, a means for accepting a question by voice, a means for guessing the intention of the question, a means for providing information based on the guessed intention, a means for adjusting the tone of the information to be provided, a means for providing information in the voice of a specific character or person, a means for recognizing emotions, a means for adjusting the method of providing information based on the emotion, a means for extracting information from a knowledge base, a means for using a natural language processing engine, a means for using a voice synthesis engine, and a means for being installed in a smartphone or tablet. This makes it possible to accurately recognize a child's voice, guess the intention of the question, and provide appropriate information. In addition, the method of providing information can be adjusted according to the child's emotions, and by providing information in the voice of a specific character or person, it is possible to attract the child's interest.

[0386] "Speech recognition means" refers to technology that converts speech into text.

[0387] The "means for accepting questions by voice" is an interface that enables a user to input questions by voice.

[0388] "Means of inferring question intent" refers to technology that understands the purpose and subject of a user's question from the entered text.

[0389] "Means for providing information based on inferred intent" refers to technology that provides users with appropriate information according to the intent of the question.

[0390] "Means for adjusting the tone of information provided" refers to technology for adjusting the tone of voice or the manner of expression when providing information.

[0391] "Means of providing information in the voice of a specific character or person" refers to technology that provides information by imitating the voice of a character or person that a user likes.

[0392] "Means of recognizing emotions" refers to technology that analyzes emotions from the user's voice and facial expressions.

[0393] "Means for adjusting the way information is presented based on emotions" refers to technology that changes the way information is presented depending on recognized emotions.

[0394] "Means for extracting information from a knowledge base" refers to technology for searching and extracting necessary information from a database or knowledge base.

[0395] "Means for using a natural language processing engine" refers to means for using natural language processing technology to analyze text data and understand its meaning.

[0396] "Means for using a speech synthesis engine" refers to technology for converting text data into speech.

[0397] "Means of being installed on a smartphone or tablet" refers to technology that is installed as an application on a mobile device.

[0398] A system for implementing the present invention includes a voice recognition means, a means for accepting a question by voice, a means for inferring the intent of the question, a means for providing information based on the inferred intent, a means for adjusting the tone of the provided information, a means for providing information in the voice of a particular character or person, a means for recognizing emotions, a means for adjusting the manner in which information is provided based on emotions, a means for extracting information from a knowledge base, a means for using a natural language processing engine, a means for using a voice synthesis engine, and a means for being installed on a smartphone or tablet.

[0399] Program processing explanation

[0400] The server uses the speech_recognition library as a means of speech recognition to convert the user's voice into text. It uses a microphone as a means of accepting voice questions. It uses the natural language processing engine of the transformers library as a means of inferring the intent of the question, and analyzes the intent of the question from the input text. It extracts appropriate information from a knowledge base as a means of providing information based on the inferred intent.

[0401] As a means of adjusting the tone of the information provided, an emotion recognition engine is used, and the tone of the information is adjusted according to the user's emotions. As a means of providing information in the voice of a specific character or person, the gTTS library is used to perform voice synthesis. As a means of recognizing emotions, the emotion_recognition library is used to analyze emotions from the user's voice and facial expressions. As a means of adjusting the way information is provided based on emotions, the tone of the voice and the way it is expressed are changed according to the recognized emotion.

[0402] As a means of extracting information from a knowledge base, it uses a database or dictionary to search and extract the required information. As a means of using a natural language processing engine, it uses the transformers library to analyze text data and understand its meaning. As a means of using a speech synthesis engine, it uses the gTTS library to convert text data into speech. As a means of being installed on a smartphone or tablet, it is installed as an application on a mobile device.

[0403] Examples

[0404] For example, if a child asks, "What is the sun?", the system operates as follows. First, the child's voice is input through a microphone and converted into text using speech recognition means. Next, the natural language processing engine analyzes the intent of "What is the sun?" and extracts the answer "The sun is the star at the center of our solar system" from the knowledge base. The emotion recognition engine recognizes the child's emotion and determines, for example, that the child is "interested." The speech synthesis engine converts the text "The sun is the star at the center of our solar system" into speech in a bright tone, which is played over the smartphone or tablet's speaker.

[0405] Example of a prompt to be input to a generative AI model:

[0406] A child asks, "What is the sun?" Extract the appropriate answer from your knowledge base and provide it in a way that is easy for the child to understand. Also, provide the information in a tone that reflects the child's emotions.

[0407] The flow of the specific process in the application example 1 will be described with reference to FIG.

[0408] Step 1:

[0409] The user speaks a question into the microphone of the smartphone or tablet. The input voice data is sent to the voice recognition means of the device.

[0410] Step 2:

[0411] The terminal's voice recognition means converts the input voice data into text data. This process uses the speech_recognition library. The voice data is converted into text data and sent to the next processing step.

[0412] Step 3:

[0413] The server's natural language processing engine receives the text data and analyzes the intent of the question using the transformers library. The subject and purpose of the question are extracted from the text data, and a query is generated to search for the appropriate information from the knowledge base.

[0414] Step 4:

[0415] A means for extracting information from the server's knowledge base uses the generated query to search the database and extract the appropriate information, which is then sent to the next processing step.

[0416] Step 5:

[0417] The emotion recognition engine on the server analyzes the emotions from the user's voice and facial expressions. This process is performed using the emotion_recognition library. The emotion data is analyzed to identify the user's emotional state.

[0418] Step 6:

[0419] The means for adjusting the tone of the information provided by the server adjusts the tone and expression of the information based on the analyzed emotion data. For example, the information is provided in a bright tone when the user is interested, and in a calm tone when the user is angry.

[0420] Step 7:

[0421] The server's speech synthesis engine converts the tailored information into speech using the gTTS library, which converts the text data into audio data and provides the information in the voice of a specific character or person.

[0422] Step 8:

[0423] The terminal reproduces the audio data through the speaker, and the user can hear the information provided in the voice of a specific character or person.

[0424] The above processing steps realize a system in which a user can input a question by voice and appropriate information is provided in the voice of a specific character or person.

[0425] Example 2

[0426] Next, a second embodiment of the second embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0427] Conventional voice recognition systems have difficulty in providing appropriate and easy-to-understand answers to children's questions. In addition, they lack the ability to respond with a response that takes into account the child's emotions, or to provide information in the voice of a specific character, making it difficult to attract children's interest. This has created a demand for a system that provides answers to questions that are easy for children to understand and that they can listen to with interest.

[0428] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0429] In this invention, the server includes a voice recognition means, a means for accepting a question by voice, a means for guessing the intent of the question, a means for providing information based on the guessed intent, a means for adjusting the tone of the provided information, a means for providing information in the voice of a specific character or person, a means for analyzing emotions, a means for extracting information from a knowledge base, and a means for converting the extracted information into an easy-to-understand form. This makes it possible to provide appropriate and easy-to-understand answers to children's questions, and further provide responses that take into account the child's emotions and information in the voice of a specific character.

[0430] "Speech recognition means" refers to technology for converting speech into text.

[0431] A "means for accepting voice queries" is a technology for receiving and processing voice input from a user.

[0432] The "means for inferring the intent of a question" is a technique for analyzing the intent of a user's question from speech-recognized text.

[0433] "Means for providing information" refers to technology for providing the user with appropriate information based on the intent of the question.

[0434] The "means for adjusting the tone of information" refers to a technique for adjusting the voice tone of the information to be provided, and providing the information in an appropriate tone according to the user's emotions and situation.

[0435] "Means for providing information in the voice of a specific character or person" refers to technology for providing information in audio form by imitating the voice of a specific character or person.

[0436] "Means for analyzing emotions" refers to technology for analyzing emotions from a user's voice and generating a response according to that emotion.

[0437] "Means for extracting information from a knowledge base" refers to techniques for searching and extracting relevant information from databases and knowledge bases.

[0438] The "means for converting the extracted information into an easily understandable form" refers to a technique for converting the extracted information into a form that is easily understandable to the user.

[0439] This invention is a system for providing appropriate and easy-to-understand answers to questions from children. The system includes a voice recognition means, a means for accepting questions by voice, a means for inferring the intent of the question, a means for providing information, a means for adjusting the tone of the information, a means for providing information in the voice of a specific character or person, a means for analyzing emotions, a means for extracting information from a knowledge base, and a means for converting the extracted information into an easy-to-understand form.

[0440] Hardware and Software Used

[0441] Hardware:

[0442] Device: Smartphone, tablet, computer, or other device that supports voice input

[0443] Server: Cloud server or on-premise server

[0444] software:

[0445] Speech recognition engine: Google Cloud Speech-to-Text

[0446] Natural language processing engine: OpenAI's GPT-3

[0447] Knowledge Base:Wikipedia API

[0448] Sentiment analysis engine: IBM Watson(R) Tone Analyzer

[0449] Speech synthesis engine: Amazon Polly

[0450] System Operation

[0451] When a user (child) asks the question "Why is the sky blue?", the device captures this question using its built-in microphone. The device then converts the speech to text using Google Cloud Speech-to-Text. This text is then sent to the server.

[0452] The server uses OpenAI's GPT-3 to parse the intent of the incoming text - in this case, it understands that you are asking "Why is the sky blue?" The server then uses the Wikipedia API to extract information from its knowledge base about "Why does the sky look blue?"

[0453] The server again uses GPT-3 to convert the extracted information into words that are easy for children to understand. For example, it converts it into "The sky is blue because the sunlight is scattered in the air." In addition, the server uses IBM Watson Tone Analyzer to analyze the child's emotions and recognize that they are feeling joy.

[0454] Finally, the server uses Amazon Polly to provide information in a light-hearted voice of a child's favorite character, and the voice data is sent to the device, which plays it back.

[0455] Examples

[0456] As a concrete example, consider a child asking "Why is the sky blue?" and feeling joy in his voice. In this case, the system performs the following steps:

[0457] 1. The device captures the child's question and converts it into text using Google Cloud Speech-to-Text: "Why is the sky blue?"

[0458] 2. The server uses OpenAI’s GPT-3 to understand that the question is asking “why about the color of the sky?”

[0459] 3. The server uses the Wikipedia API to extract information about "why the sky looks blue."

[0460] 4. The server converts the extracted information into language that children can easily understand.

[0461] 5. The emotion analysis engine recognizes the child's joy, so the server provides the information in a lighter tone.

[0462] 6. The server uses Amazon Polly to provide information in the voice of the child's favorite character.

[0463] 7. The device plays the generated audio.

[0464] Example prompt:

[0465] "When a child asks, 'Why is the sky blue?' Explain to the child why the sky looks blue in terms that the child can easily understand. Also, explain in a bright tone and with the voice of a character that the child likes, so that the child will be happy."

[0466] In this way, the system provides appropriate and engaging responses to the child's questions.

[0467] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0468] Step 1:

[0469] The user asks a question.

[0470] The user (child) asks a question by voice, "Why is the sky blue?" This voice becomes the input for the system.

[0471] Step 2:

[0472] The device captures the audio.

[0473] The device uses a built-in microphone to capture the user's voice, and the captured voice data serves as input for the next processing step.

[0474] Step 3:

[0475] The device uses a speech recognition engine to convert the speech to text.

[0476] The device calls a speech recognition engine (e.g., Google Cloud Speech-to-Text) to convert the captured voice data into text data. The converted text, "Why is the sky blue?", becomes the input for the next processing step.

[0477] Step 4:

[0478] The device sends the text to the server.

[0479] The terminal sends the converted text data to the server, which receives the text data and uses it as input for the next processing step.

[0480] Step 5:

[0481] The server uses a natural language processing engine to analyze the intent of the question.

[0482] The server uses a natural language processing engine (for example, OpenAI's GPT-3) to analyze the intent of the received text data "Why is the sky blue?". The analysis result is the information that "You are asking about the reason for the color of the sky." This analysis result becomes the input for the next processing step.

[0483] Step 6:

[0484] The server extracts the relevant information from a knowledge base.

[0485] The server uses a knowledge base (e.g., the Wikipedia API) to search and extract information about "why the sky looks blue." The extracted information, "The sky looks blue because sunlight is scattered by molecules in the atmosphere," becomes the input for the next processing step.

[0486] Step 7:

[0487] The server converts the extracted information into words that are easy for children to understand.

[0488] The server then uses a natural language processing engine (e.g., GPT-3) to convert the extracted information into child-friendly language. The converted information, "The sky is blue because the sun's light is scattered in the air," becomes the input for the next processing step.

[0489] Step 8:

[0490] The server uses an emotion analysis engine to analyze the child's emotion.

[0491] The server uses an emotion analysis engine (for example, IBM Watson Tone Analyzer) to analyze the emotion from the user's voice. The analysis result is the information that "I feel happy." This analysis result becomes the input for the next processing step.

[0492] Step 9:

[0493] The server uses a speech synthesis engine to convert the information into speech.

[0494] The server uses a speech synthesis engine (e.g., Amazon Polly) to convert the converted information into speech in the voice of a character the child likes. The generated speech data, "The sky is blue because the sunlight is scattered in the air," becomes the input for the next processing step.

[0495] Step 10:

[0496] The device plays the generated audio.

[0497] The server transmits the generated voice data to the terminal, which receives the voice data and plays it through the speaker, allowing the user to listen to the data and obtain the answer to their question.

[0498] (Application example 2)

[0499] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0500] Conventional voice recognition systems have difficulty in accurately understanding the intent of a child when he or she asks a question and providing appropriate information. In addition, the information provided is often not in a form that is easy for children to understand, and it is not possible to provide information in a tone that reflects the child's emotions, making it difficult to attract the child's interest.

[0501] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0502] In this invention, the server includes a voice recognition means, a means for accepting a question by voice, a means for guessing the intention of the question, a means for providing information based on the guessed intention, a means for adjusting the tone of the provided information, a means for providing information in the voice of a specific character or person, an emotion recognition means, a means for adjusting the tone of the information based on the emotion, and a means for extracting information from a knowledge base. This makes it possible to accurately understand the intention of a child's question, provide information in a form that is easy for the child to understand, and provide information in a tone that corresponds to the child's emotion.

[0503] "Speech recognition means" refers to technology for converting speech into text.

[0504] A "means for accepting voice queries" is an interface that allows a user to input queries by voice.

[0505] "Means of inferring the intent of the question" refers to natural language processing technology for understanding the meaning and purpose of the input text.

[0506] "Means for providing information based on inferred intent" refers to technology that extracts appropriate information from a knowledge base according to the intent of the question and provides it to the user.

[0507] "Means of adjusting the tone of the information provided" refers to techniques for adjusting the way information is conveyed depending on the user's emotions and situation.

[0508] "Means for providing information in the voice of a specific character or person" refers to technology that provides information in audio form using the voice of a character or person that the user prefers.

[0509] "Emotion recognition means" is a technology that analyzes and recognizes emotions from the user's voice and facial expressions.

[0510] A "means for adjusting the tone of information based on emotions" is a technique for adjusting the way information is conveyed and its tone depending on the recognized emotions.

[0511] A "means for extracting information from a knowledge base" is a technique for searching and extracting necessary information from a database or knowledge base to provide an answer to a question.

[0512] A system for implementing the present invention includes a voice recognition means, a means for accepting a question via voice, a means for inferring the intent of the question, a means for providing information based on the inferred intent, a means for adjusting the tone of the provided information, a means for providing information in the voice of a particular character or person, a means for recognizing emotions, a means for adjusting the tone of the information based on emotions, and a means for extracting information from a knowledge base.

[0513] System Configuration

[0514] The server uses the following hardware and software:

[0515] Speech recognition method: Uses the Google Cloud Speech-to-Text API to convert speech to text.

[0516] Method of accepting voice questions: Using an application installed on a device such as a smartphone or tablet.

[0517] How to infer the intent of the question: Use the Google Cloud Natural Language API to analyze the intent of the text.

[0518] Providing information based on inferred intent: Use the Wikipedia API or your own database to get information about the question.

[0519] Ways to adjust the tone of information provided: Use the Microsoft Azure Emotion API to analyze the user's emotions and provide information in an appropriate tone.

[0520] A means of providing information in the voice of a specific character or person: Use the Text-to-Speech API to provide information in the voice of a specific character or person.

[0521] Emotion recognition: Uses Microsoft Azure Emotion API to recognize user emotions.

[0522] How to adjust the tone of your information based on emotion: Adjust the delivery and tone of your information depending on the perceived emotion.

[0523] How to extract information from the knowledge base: Use Wikipedia API or your own database to search and extract the required information.

[0524] System Operation

[0525] When a user inputs a question by voice using a terminal such as a smartphone or tablet, the voice recognition means converts the voice into text. Next, the means for guessing the intent of the question analyzes the intent of the text and extracts appropriate information from a knowledge base. The emotion recognition means analyzes the user's emotion and adjusts the tone of the information based on the emotion. Finally, the means for providing information in the voice of a specific character or person provides the information to the user by voice.

[0526] Examples

[0527] For example, consider a child asking "Why is the sky blue?" and feeling joy in his voice. In this case, the system would:

[0528] A speech recognition tool converts the question "Why is the sky blue?" into text.

[0529] A means for inferring the intent of the question analyzes the intent of this question and understands "the reason for the color of the sky."

[0530] Extract information about "why the sky looks blue" from the knowledge base.

[0531] The emotion recognition tool recognizes the child's joy and provides information in a light-hearted tone.

[0532] Information is provided in the voices of children's favorite characters.

[0533] Example of a prompt to be input to a generative AI model:

[0534] If a child asks "Why is the sky blue?" and you can sense joy in their voice, we use a speech recognition engine to convert the question into text and a natural language processing engine to infer the intent of the question. We then extract information about "why the sky looks blue" from our knowledge base and explain that information in terms that the child can easily understand. Then, because the emotion engine recognizes the child's joy, the system provides the information in a lighter tone. Additionally, it provides the information in the voice of a character that the child likes.

[0535] The flow of the specific process in the application example 2 will be described with reference to FIG.

[0536] Step 1:

[0537] The user inputs a question by voice using a device such as a smartphone or tablet.

[0538] Input: User's voice question

[0539] Output: Audio data

[0540] Specific actions: The user speaks a question into the device's microphone.

[0541] Step 2:

[0542] The server uses a voice recognition means to convert the voice data into text.

[0543] Input: Audio data

[0544] Output: Text data

[0545] What it does: Uses the Google Cloud Speech-to-Text API to convert audio data into text.

[0546] Step 3:

[0547] The server analyzes the meaning of the text data using a means for inferring the intent of the question.

[0548] Input: Text data

[0549] Output: Intent data

[0550] Specific behavior: Uses Google Cloud Natural Language API to analyze the meaning of text data.

[0551] Step 4:

[0552] The server uses a means for extracting information from a knowledge base to obtain appropriate information based on the intent data.

[0553] Input: Intent data

[0554] Output: Information data

[0555] Specific behavior: Uses Wikipedia API or its own database to get information about the question.

[0556] Step 5:

[0557] The server uses the emotion recognition means to analyze the emotion of the user.

[0558] Input: User's voice or image data

[0559] Output: Emotion data

[0560] Specific behavior: Analyzes user emotions using Microsoft Azure Emotion API.

[0561] Step 6:

[0562] The server adjusts the tone of the information in response to the emotion data using a means for adjusting the tone of the information based on emotion.

[0563] Input: Emotion data, information data

[0564] Output: Adjusted information data

[0565] Specific Behavior: Adjust your delivery and tone based on the perceived emotion.

[0566] Step 7:

[0567] The server provides the adjusted information data by voice using a means for providing information in the voice of a specific character or person.

[0568] Input: Adjusted information data

[0569] Output: Audio data

[0570] What it does: Uses the Text-to-Speech API to provide information in the voice of a specific character or person.

[0571] Step 8:

[0572] The terminal reproduces the audio data and provides information to the user.

[0573] Input: Audio data

[0574] Output: Providing audio information to the user

[0575] Specific operation: Provide information to the user via audio through the device's speaker.

[0576] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0577] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by making a neural network perform deep learning. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image is input. The data generation model 58 performs inference on the input inference data according to the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0578] Another example of generative AI is Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) are mentioned.

[0579] In the above embodiment, an example was given in which the specific process was performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0580] [Second embodiment]

[0581] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0582] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0583] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[0584] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0585] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0586] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[0587] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54. The transmission and reception of various types of information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0588] Fig. 4 shows an example of main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0589] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0590] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0591] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0592] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0593] "Example 1"

[0594] The embodiment of the present invention includes a speech recognition engine for recognizing the child's voice as a speech recognition means. The speech recognition engine identifies the child's pronunciation and vocabulary and converts it into text format. A natural language processing engine is used as a means for inferring the intent of the question. The engine analyzes the text obtained from the speech recognition engine and understands the subject and purpose of the question. Furthermore, as an information providing means, appropriate information is extracted from a knowledge base and provided in a form that is easy for the child to understand. As a means for adjusting the tone of the information, the system has a function for adjusting the method of providing information according to the child's age and level of understanding. Finally, a speech synthesis engine is used as a means for providing information in the voice of a specific character or person. The engine imitates the voice of a character or person that the child likes and provides information in that voice.

[0595] "Example 2"

[0596] As a concrete example, if a child asks "Why is the sky blue?", a speech recognition engine converts this question into text. A natural language processing engine then understands that the question is asking "Why is the sky blue?" and extracts information from a knowledge base about "Why does the sky look blue?" and explains it in terms the child can easily understand. This information is provided in the voice of a character the child likes.

[0597] The process flow of each embodiment will be described below.

[0598] "Example 1"

[0599] Step 1: The child asks the app a question, which is asked by voice.

[0600] Step 2: The speech recognition engine converts the child's speech into text. This engine identifies the child's pronunciation and vocabulary and converts it into a text format.

[0601] Step 3: A natural language processing engine analyzes the text to understand the subject and purpose of the question. Step 4: Extracts the appropriate information from the knowledge base and presents it in a child-friendly format.

[0602] Step 5: Adapt the way you present information to the child's age and level of understanding.

[0603] Step 6: The voice synthesis engine mimics the voice of a character or person that the child likes and provides the information in that voice.

[0604] "Example 2"

[0605] Step 1: A child asks, "Why is the sky blue?"

[0606] Step 2: The speech recognition engine converts this question into text.

[0607] Step 3: The natural language processing engine understands that the question is asking "why about the color of the sky?"

[0608] Step 4: Extract information about "why the sky looks blue" from the knowledge base.

[0609] Step 5: Explain the information in terms your child can understand.

[0610] Step 6: Present that information in the voice of your child's favorite character.

[0611] Example 1

[0612] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[0613] Conventional voice recognition systems have difficulty in accurately recognizing children's voices and providing appropriate information. In addition, the information provided is often not in a form that is easy for children to understand, and there is a lack of functionality to provide information in the voice of a specific character or person. This poses the problem that it is difficult for children to learn with interest.

[0614] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0615] In this invention, the server includes a voice recognition means, a means for accepting a question by voice, a means for guessing the intent of the question, a means for providing information based on the guessed intent, a means for adjusting the tone of the provided information, a means for providing information in the voice of a specific character or person, a means for converting voice data into text, a means for analyzing the text to understand the subject or purpose of the question, a means for extracting appropriate information from a knowledge base, a means for formatting the extracted information into a form that is easy for children to understand, and a means for converting the formatted information into voice. This makes it possible to accurately recognize a child's voice, provide appropriate information, and further provide information in the voice of a specific character or person.

[0616] A "voice recognition means" is a device or software that has the function of analyzing voice data and converting it into text format.

[0617] A "means for accepting voice queries" is a device or software for receiving voice input from a user and processing the voice data.

[0618] The "means for inferring the intent of a question" is a device or software that has the function of analyzing the text converted by the speech recognition means and understanding the subject and purpose of the user's question.

[0619] The "means for providing information" refers to a device or software that has the function of selecting appropriate information based on the intent of the question and providing it to the user.

[0620] The "means for adjusting the tone of information" refers to a device or software that has the function of adjusting the content and expression method of the information provided according to the user's age and level of understanding.

[0621] A "means for providing information in the voice of a specific character or person" is a device or software that has a function of imitating the voice of a character or person that a user likes and providing information in that voice.

[0622] "Means for converting voice data into text" refers to a device or software that has the function of converting voice data into text format using speech recognition means.

[0623] The "means for analyzing text to understand the subject or purpose of the question" refers to a device or software that has the function of analyzing the converted text and understanding the intent of the user's question.

[0624] The "means for extracting appropriate information from a knowledge base" refers to a device or software that has the function of searching and extracting relevant information from a knowledge base based on the intent of a question.

[0625] "Means for formatting the extracted information into a form that is easy for children to understand" refers to a device or software that has the function of processing the extracted information into a form that is easy for children to understand.

[0626] The "means for converting formatted information into speech" refers to a device or software having a function for converting formatted information into speech format using speech synthesis technology.

[0627] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0628] This invention is a system for recognizing a child's voice and providing appropriate information. The system includes a voice recognition means, a means for accepting a question by voice, a means for inferring the intent of the question, a means for providing information, a means for adjusting the tone of the information, a means for providing information in the voice of a specific character or person, a means for converting voice data into text, a means for analyzing the text to understand the subject or purpose of the question, a means for extracting appropriate information from a knowledge base, a means for formatting the extracted information into a form that is easily understandable to children, and a means for converting the formatted information into voice.

[0629] Hardware and Software Used

[0630] The server uses general voice recognition software, such as Google Cloud Speech-to-Text API, to convert the voice data into text.

[0631] To infer the intent of a question, a natural language processing engine is used, specifically a generative AI model such as OpenAI's GPT-4, which analyzes the text to understand the subject and purpose of the question.

[0632] To provide this information, a means is needed to extract the appropriate information from a knowledge base, which could be Wikipedia or a specific educational database.

[0633] Adjusting the tone of the information requires a way to shape it according to the child's age and level of understanding, using simple vocabulary and short sentences.

[0634] To provide information in the voice of a specific character or person, a voice synthesis engine is used, specifically, voice synthesis software such as Amazon Polly or Google Cloud Text-to-Speech.

[0635] Examples

[0636] When a user (child) says "Tell me about dinosaurs," the following process takes place:

[0637] 1. The device records the user's voice and sends it to the server.

[0638] 2. The server converts the speech to text using the Google Cloud Speech-to-Text API.

[0639] 3. The server passes the converted text, “Tell me about dinosaurs,” to GPT-4 and analyzes the intent of the question.

[0640] 4. The server extracts information about dinosaurs from Wikipedia and formats the information in a way that is easy for children to understand.

[0641] 5. The server uses Amazon Polly to generate a speech in the voice of the child's favorite character saying, "Dinosaurs were creatures that appeared on Earth approximately 230 million years ago."

[0642] 6. The terminal plays the generated audio and provides information to the user.

[0643] Examples of prompt statements

[0644] In response to the question "Tell me about dinosaurs," the following prompt sentence is input to the generative AI model.

[0645] User Question: Tell me about dinosaurs

[0646] Prompt for generative AI model: Give a simple, child-friendly description of dinosaurs. Include the type, era, characteristics, etc. of the dinosaur.

[0647] Using this prompt, the generative AI model generates text to provide appropriate information to children.

[0648] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0649] Step 1:

[0650] The terminal records the user's voice and transmits it to the server.

[0651] Input: User's voice

[0652] Output: Audio data

[0653] Specific operation: When a user says, "Tell me about dinosaurs," the device's microphone captures the audio and sends the audio data to the server.

[0654] Step 2:

[0655] The server receives the voice data and converts it to text using speech recognition means.

[0656] Input: Audio data

[0657] Output: Text data

[0658] What happens: The server uses the Google Cloud Speech-to-Text API to parse the audio data and convert it into text, such as "Tell me about dinosaurs."

[0659] Step 3:

[0660] The server passes the text data to a natural language processing engine to analyze the intent of the question.

[0661] Input: Text data

[0662] Output: Question intent

[0663] Specific operation: The server uses GPT-4 to analyze the text data and identify the user’s intent to learn about dinosaurs.

[0664] Step 4:

[0665] The server extracts appropriate information from a knowledge base based on the intent of the question.

[0666] Input: Question intent

[0667] Output: Extracted information

[0668] Specific operation: The server searches for information about dinosaurs from knowledge bases such as Wikipedia, and extracts the information that "dinosaurs are creatures that appeared on Earth approximately 230 million years ago, and include species such as Tyrannosaurus and Triceratops."

[0669] Step 5:

[0670] The server formats the extracted information in a way that is easy for children to understand.

[0671] Input: Extracted information

[0672] Output: Formatted information

[0673] Specific operation: The server converts the extracted information into simple vocabulary and short sentences, formatting it into the form of "Dinosaurs are large creatures that lived on Earth a long time ago. There were species called Tyrannosaurus and Triceratops."

[0674] Step 6:

[0675] The server passes the formatted information to a speech synthesis engine, which generates speech in the voice of a particular character or person.

[0676] Input: Formatted information

[0677] Output: Audio data

[0678] What it does: The server uses Amazon Polly to generate a speech sequence in the voice of a child's favorite character saying, "Dinosaurs are big creatures that lived on Earth a long time ago. There were species called Tyrannosaurus and Triceratops."

[0679] Step 7:

[0680] The terminal receives the generated voice data and plays it back to the user.

[0681] Input: Audio data

[0682] Output: Played audio

[0683] Specific operation: The terminal plays the audio data received from the server, allowing the user to listen to information about dinosaurs in the voice of their favorite character.

[0684] (Application example 1)

[0685] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[0686] Conventional voice recognition systems are primarily targeted at adult voices, and have the problem of difficulty in accurately recognizing children's voices. It is also difficult to properly understand the content of a child's question and provide information in a format that is easy for children to understand. Furthermore, there is a lack of means to increase children's motivation to learn by providing information in the voices of characters that interest them. To solve these problems, a system is needed that can accurately recognize children's voices, properly analyze the intent of the question, and provide information in a format that is easy for children to understand.

[0687] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0688] In this invention, the server includes a voice recognition means, a means for accepting a question by voice, a means for guessing the intention of the question, a means for providing information based on the guessed intention, a means for adjusting the tone of the information to be provided, a means for providing information in the voice of a specific character or person, a means for providing information in a form that is easy for a child to understand, a means for providing information in the voice of a character that the child likes, and a means including an application to be installed on the smart device. This makes it possible to accurately recognize the child's voice, appropriately analyze the intention of the question, and provide information in a form that is easy for a child to understand. Also, by providing information in the voice of a character that the child is interested in, it is possible to increase the child's motivation to learn.

[0689] "Speech recognition means" refers to technology for converting speech into text.

[0690] The "means for accepting questions by voice" is an interface that enables a user to input questions by voice.

[0691] "Means for inferring the intent of a question" refers to technology that analyzes the subject and purpose of a question from input text.

[0692] The "means for providing information based on an inferred intent" is a technology for providing appropriate information according to the intent of the analyzed question.

[0693] "Means for adjusting the tone of the information provided" refers to technology that adjusts the way information is provided depending on the user's age and level of understanding.

[0694] "Means of providing information in the voice of a specific character or person" refers to technology that provides information by imitating the voice of a character or person that a user likes.

[0695] "Means of providing information in a form that children can easily understand" refers to techniques that provide information in a concise and easy-to-understand manner so that children can easily understand it.

[0696] The "means for providing information in the voice of a character that a child likes" is a technology for providing information using the voice of a character that is of interest to children.

[0697] "Means including an application installed on a smart device" refers to a software application installed on a device such as a smartphone or tablet.

[0698] In order to implement this invention, means including a voice recognition means, a means for accepting a question by voice, a means for inferring the intent of the question, a means for providing information based on the inferred intent, a means for adjusting the tone of the information to be provided, a means for providing information in the voice of a particular character or person, a means for providing information in a form that is easy for children to understand, a means for providing information in the voice of a character that children like, and an application to be installed on a smart device are used.

[0699] Hardware and Software Used

[0700] The server uses the following hardware and software:

[0701] Hardware: Smartphones, tablets

[0702] software:

[0703] Speech recognition engine (Google Cloud Speech-to-Text)

[0704] Natural language processing engine (Google Cloud Natural Language API)

[0705] Knowledge Base (Wolfram Alpha API)

[0706] Speech synthesis engine (Google Cloud Text-to-Speech)

[0707] Processing flow

[0708] 1. Speech recognition: The user speaks a question into their smart device. The speech recognition engine converts this speech into text.

[0709] 2. Natural language processing: The server analyzes the text obtained from the speech recognition engine using a natural language processing engine to infer the intent of the question.

[0710] 3. Information provision: The server extracts appropriate information from a knowledge base based on the inferred intent.

[0711] 4. Tone Adjustment: The server adjusts the tone of the information depending on the user's age and level of understanding.

[0712] 5. Voice synthesis: The server uses a voice synthesis engine to provide information in the voice of the child's favorite character.

[0713] Examples

[0714] For example, if a child asks a smart device, "Tell me about dinosaurs," a speech recognition engine converts this speech into text. Next, a natural language processing engine analyzes this text and infers that the intent of the question is, "I want to know information about dinosaurs." The server extracts information about dinosaurs from a knowledge base and provides the information in a format that is easy for the child to understand. Finally, a speech synthesis engine is used to provide information in the voice of the child's favorite character, such as, "Tyrannosaurus is a dinosaur that lived about 68 million years ago."

[0715] Examples of prompt statements

[0716] Create an application that recognizes children's voices, analyzes the intent of the questions, and provides appropriate information. Use Google Cloud Speech-to-Text API for speech recognition and Google Cloud Natural Language API for natural language processing. Use Wolfram Alpha API to provide information and Google Cloud Text-to-Speech API for speech synthesis. Provide information in the voice of a character that children like.

[0717] In this way, educational applications that allow children to learn in a fun way can be realized.

[0718] The flow of the specific process in the application example 1 will be described with reference to FIG.

[0719] Step 1:

[0720] A user speaks a question into a smart device, and the voice data is picked up by the microphone of the smart device.

[0721] Step 2:

[0722] The terminal sends the acquired voice data to a voice recognition engine (Google Cloud Speech-to-Text). The voice recognition engine converts the voice data into text data. The input is voice data, and the output is text data.

[0723] Step 3:

[0724] The server sends the text data obtained from the speech recognition engine to a natural language processing engine (Google Cloud Natural Language API). The natural language processing engine analyzes the text data and infers the intent of the question. The input is text data, and the output is data that indicates the intent of the question.

[0725] Step 4:

[0726] The server extracts the appropriate information from a knowledge base (Wolfram Alpha API) based on the inferred question intent. The input is data indicating the question intent, and the output is related information data.

[0727] Step 5:

[0728] The server adjusts the tone of the information it provides based on the extracted information data. It changes the way the information is presented depending on the user's age and level of understanding. The input is information data and the user's profile data, and the output is the adjusted information data.

[0729] Step 6:

[0730] The server sends the adjusted information data to a voice synthesis engine (Google Cloud Text-to-Speech). The voice synthesis engine converts the information into voice data in the voice of the child's favorite character. The input is the adjusted information data, and the output is voice data.

[0731] Step 7:

[0732] The terminal reproduces the voice data obtained from the voice synthesis engine and provides it to the user, who receives the answer to the question through the voice coming from the smart device.

[0733] Example 2

[0734] Next, a description will be given of Example 2 of the embodiment 2. In the following description, the data processing device 12 is referred to as a "server" and the smart glasses 214 are referred to as a "terminal".

[0735] In conventional voice recognition systems, it can be difficult to provide appropriate answers to user questions. In particular, when children ask questions, it is necessary to provide answers that match their level of understanding, but there is a lack of technology to achieve this. Furthermore, it is necessary to provide answers in the voices of specific characters or people to make it easier to attract children's interest, but conventional systems are not able to fully address this issue.

[0736] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0737] In this invention, the server includes a voice recognition means, a means for accepting a question by voice, a means for analyzing the intent of the question, a means for extracting information based on the analyzed intent, a means for converting the extracted information, a means for voice synthesizing the converted information, and a means for providing information in the voice of a specific character or person. This makes it possible to provide an appropriate and easy-to-understand answer to a user's question. Also, providing information in the voice of a specific character or person can attract the interest of children in particular.

[0738] "Speech recognition means" refers to technology for converting speech into text.

[0739] A "means for accepting voice queries" is a technology for receiving and processing voice input from a user.

[0740] A "means for analyzing the intent of a question" is a technique for understanding the content of a user's question and identifying its intent.

[0741] "Means of information extraction" refers to techniques for retrieving appropriate information from a database or knowledge base based on the analyzed intent.

[0742] "Means for converting information" refers to technology for converting extracted information into a form that is easy for users to understand.

[0743] "Means for synthesizing speech" refers to technology for converting text information into speech.

[0744] "Means for providing information in the voice of a specific character or person" refers to technology for playing back the generated voice in the voice of a specific character or person.

[0745] This invention is a system that allows a user to input a question by voice and provides an appropriate answer to the question. This system includes a voice recognition means, a means for accepting a question by voice, a means for analyzing the intent of the question, a means for extracting information, a means for converting information, a means for synthesizing voice, and a means for providing information in the voice of a specific character or person.

[0746] Hardware and Software Used

[0747] Hardware:

[0748] Device: Any device with a microphone (smartphone, tablet, PC, etc.)

[0749] Server: A server with high-performance computing resources (such as a cloud server)

[0750] software:

[0751] Speech recognition engine: General speech recognition API (e.g. Google Cloud Speech-to-Text API)

[0752] Natural Language Processing Engine: Advanced natural language processing models (e.g. OpenAI's GPT-4)

[0753] Knowledge base: online encyclopedia APIs (e.g. Wikipedia API) or proprietary databases

[0754] Speech synthesis engine: Speech synthesis API (e.g. Amazon Polly)

[0755] Program Processing

[0756] The user speaks a question into the terminal. For example, a child may ask, "Why is the sky blue?"

[0757] 1. Voice Recognition:

[0758] The terminal captures the user's voice through a microphone.

[0759] The server converts the audio to text using the Google Cloud Speech-to-Text API.

[0760] 2. Natural Language Processing:

[0761] The server sends the converted text to a natural language processing engine (OpenAI's GPT-4).

[0762] The natural language processing engine analyzes the intent of the question and understands that you are asking "why about the color of the sky."

[0763] 3. Extracting information from the knowledge base:

[0764] The server accesses a knowledge base (Wikipedia API or its own database) and searches for information about "why the sky looks blue."

[0765] 4. Transformation of information:

[0766] The server converts the retrieved information into language that is easy for children to understand.

[0767] 5. Speech synthesis:

[0768] The server sends the converted information to a speech synthesis engine (Amazon Polly), which generates speech in the voice of the child's favorite character.

[0769] 6. Audio playback:

[0770] The terminal plays the generated audio.

[0771] Examples

[0772] As a concrete example, here is how a child asks "Why is the sky blue?":

[0773] Child: "Why is the sky blue?"

[0774] Terminal: Captures audio and sends it to the server.

[0775] Server: Use the Google Cloud Speech-to-Text API to convert the speech to text "Why is the sky blue?"

[0776] Server: OpenAI's GPT-4 analyzes the intent of the question and retrieves information from the knowledge base.

[0777] Server: Convert that into a simple explanation like, "The sky is blue because the sun's light is scattered by tiny particles in the air."

[0778] Server: Generate the character's voice using Amazon Polly.

[0779] Terminal: Play the generated audio.

[0780] Examples of prompt statements

[0781] "If a child asks, 'Why is the sky blue?', how should I respond?"

[0782] This system starts with the user's voice input and goes through a series of steps including voice recognition, natural language processing, information extraction from a knowledge base, information conversion, voice synthesis, and voice playback, before providing the user with an appropriate answer.

[0783] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0784] Step 1:

[0785] The user speaks a question into the terminal.

[0786] Input: User speech (e.g. "Why is the sky blue?")

[0787] Specific action: A child speaks to the device, "Why is the sky blue?"

[0788] Output: The device captures the audio data.

[0789] Step 2:

[0790] The terminal transmits the captured voice data to the server.

[0791] Input: Captured audio data

[0792] Specific operations: The device sends voice data to the server.

[0793] Output: The server receives the audio data.

[0794] Step 3:

[0795] The server converts the voice data into text using a voice recognition engine.

[0796] Input: Audio data

[0797] What happens: The server uses a speech recognition engine (e.g. a common speech recognition API) to convert the speech into text: "Why is the sky blue?"

[0798] Output: Text data ("Why is the sky blue?")

[0799] Step 4:

[0800] The server sends the text data to a natural language processing engine to analyze the intent of the question.

[0801] Input: Text data ("Why is the sky blue?")

[0802] What happens: The server uses a natural language processing engine (e.g., an advanced natural language processing model) to parse the intent of the question and understands that you are asking "why about the color of the sky?"

[0803] Output: Question intent ("Why is the sky colored?")

[0804] Step 5:

[0805] The server extracts information from a knowledge base based on the parsed intent.

[0806] Input: Question intent ("Reasons for the color of the sky")

[0807] What happens: The server accesses a knowledge base (e.g. an online encyclopedia API or a proprietary database) and searches for information about "why the sky looks blue."

[0808] Output: Extracted information ("The sky is blue because sunlight is scattered by tiny particles in the air")

[0809] Step 6:

[0810] The server translates the extracted information into child-friendly language.

[0811] Input: Extracted information ("The sky is blue because sunlight is scattered by tiny particles in the air.")

[0812] What happens: The server converts that information into a simple explanation, such as "The sky is blue because the sun's light is scattered by tiny particles in the air."

[0813] Output: Transformed information ("The sky is blue because sunlight is scattered by tiny particles in the air.")

[0814] Step 7:

[0815] The server sends the converted information to a speech synthesis engine, which generates speech in the voice of a particular character or person.

[0816] Input: Transformed information ("The sky is blue because sunlight is scattered by tiny particles in the air.")

[0817] Specific behavior: The server uses a speech synthesis engine (e.g., a speech synthesis API) to generate speech in the character's voice.

[0818] Output: The generated audio data

[0819] Step 8:

[0820] The terminal reproduces the generated voice data.

[0821] Input: Generated audio data

[0822] What it does: The device plays a character's voice saying, "The sky looks blue because the sun's light is scattered by tiny particles in the air."

[0823] Output: The user hears the audio.

[0824] (Application example 2)

[0825] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server", and the smart glasses 214 will be referred to as a "terminal".

[0826] Conventional voice recognition systems have had the problem of difficulty in providing appropriate answers to questions posed by children. In particular, it is necessary to provide information in a form that is easy for children to understand, but existing systems are inadequate in this regard. It is also important to provide answers in the voice of a child's favorite character to attract the child's interest, but this was not achieved with conventional systems. Furthermore, it is necessary to consider the convenience of the application as it is installed on a smartphone.

[0827] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0828] In this invention, the server includes a voice recognition means, a means for accepting a question by voice, a means for guessing the intention of the question, a means for providing information based on the guessed intention, a means for adjusting the tone of the provided information, a means for providing information in the voice of a specific character or person, a means for extracting information from a knowledge base, a means for reconstructing information in words that are easy for children to understand, and an application installed on the smartphone. This makes it possible to provide information in an appropriate and easy-to-understand form in response to a question asked by a child, and further to answer in the voice of a character that is likely to interest children.

[0829] "Speech recognition means" refers to technology that converts speech into text.

[0830] The "means for accepting questions by voice" is an interface that enables a user to input questions by voice.

[0831] "Means for inferring the intent of a question" refers to technology that analyzes the content of an input question and understands its intent.

[0832] "Means for providing information based on inferred intent" refers to technology that selects and provides appropriate information according to the intent of the question.

[0833] The "means for adjusting the tone of the information provided" refers to a technique for adjusting the tone of voice or the manner of expression of the information provided.

[0834] "Means for providing information in the voice of a specific character or person" refers to technology for providing information in audio form by imitating the voice of a specific character or person.

[0835] A "means for extracting information from a knowledge base" is a technique for searching and extracting answers to questions from a knowledge base.

[0836] "Means for reconstructing information in terms that children can easily understand" refers to technology for reconstructing extracted information into a form that children can easily understand.

[0837] An "application installed on a smartphone" is software that runs on a smartphone.

[0838] A system for implementing the present invention includes a voice recognition means, a means for accepting a question via voice, a means for inferring the intent of the question, a means for providing information based on the inferred intent, a means for adjusting the tone of the provided information, a means for providing information in the voice of a particular character or person, a means for extracting information from a knowledge base, a means for reconstructing information in terms that are easy for a child to understand, and an application installed on a smartphone.

[0839] Program processing explanation

[0840] Voice Recognition Method

[0841] When a user speaks a question into the smartphone's microphone, the speech recognition means converts the speech into text using the speech_recognition library.

[0842] A way to guess the intent of a question

[0843] The converted text is then analyzed by a natural language processing engine to infer the intent of the question, using a pipeline from the transformers library.

[0844] A means of extracting information from a knowledge base

[0845] Based on the inferred intent, appropriate information is searched and extracted from a knowledge base, which is a pre-built database.

[0846] A way to restructure information in terms that children can easily understand

[0847] The extracted information is then reconstructed in a way that is easy for children to understand, using a generative AI model.

[0848] A means of providing information in the voice of a specific character or person

[0849] The reconstructed information is then provided as audio in the voice of a specific character or person, using the pyttsx3 library.

[0850] Examples

[0851] For example, if a child asks "Why is the sky blue?", a speech recognition tool converts the question into text. A natural language processing engine then understands that the question is asking "Why is the sky blue?" and extracts information from a knowledge base about "Why does the sky look blue?" and explains it in terms the child can easily understand. This information is provided in the voice of a character the child likes.

[0852] Examples of prompt statements

[0853] If a child asks "Why is the sky blue?", the speech recognition engine converts this question into text, and the natural language processing engine understands that the question is asking "Why is the sky blue?" It then extracts information from its knowledge base about "Why does the sky look blue?" and explains it in terms the child can easily understand, delivered in the voice of a character the child loves.

[0854] In this way, it is possible to provide information in an appropriate and easy-to-understand form in response to a question asked by a child, and further, to respond in the voice of a character that is likely to interest children.

[0855] The flow of the specific process in the application example 2 will be described with reference to FIG.

[0856] Step 1:

[0857] The user inputs a question by voice through the smartphone's microphone. The input voice data is captured by the smartphone's microphone.

[0858] Step 2:

[0859] The smartphone sends the captured voice data to the voice recognition means, which converts the voice data into text using the speech_recognition library. The input is the voice data, and the output is the text data.

[0860] Step 3:

[0861] The server sends the converted text data to a natural language processing engine. The natural language processing engine uses the transformers library pipeline to analyze the text data and infer the intent of the question. The input is text data, and the output is data that indicates the intent of the question.

[0862] Step 4:

[0863] The server searches for appropriate information from a knowledge base based on the inferred question intent. The knowledge base is a pre-constructed database that contains information corresponding to the question intent. The input is data indicating the question intent, and the output is appropriate information.

[0864] Step 5:

[0865] The server sends the extracted information to a generative AI model, which reconstructs the information in terms that are easy for the child to understand. The generative AI model uses prompts to reconstruct the information. The input is the extracted information, and the output is the reconstructed information.

[0866] Step 6:

[0867] The server uses the pyttsx3 library to provide the reconstructed information as audio in the voice of a specific character or person. The input is the reconstructed information and the output is the audio data.

[0868] Step 7:

[0869] The smartphone provides the generated voice data to the user through the speaker. The user can hear the answer provided in the voice of a specific character or person. The input is the voice data and the output is the voice that the user hears.

[0870] In addition, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0871] "Example 1"

[0872] As one embodiment of the present invention, a system incorporating an emotion engine is provided. The system accepts a child's voice question, infers the intent of the question, and provides information based on the inferred intent. Furthermore, the system recognizes the child's emotion and adjusts the way information is provided based on the emotion. Specifically, if the child feels angry, the system

[0873] If the system senses that the child is happy, it will provide information in a brighter tone. In this way, the system can select the optimal way to provide information depending on the child's emotions.

[0874] "Example 2"

[0875] As a concrete example, consider a child asking "Why is the sky blue?" and feeling joy from the child's voice. In this case, the system uses a speech recognition engine to convert the question into text and a natural language processing engine to infer the intent of the question. Next, it extracts information about "why the sky looks blue" from the knowledge base and explains the information in words that the child can easily understand. Then, because the emotion engine recognizes the child's joy, the system provides the information in a bright tone. Furthermore, providing the information in the voice of the child's favorite character attracts the child's interest and deepens their understanding.

[0876] The process flow of each embodiment will be described below.

[0877] "Example 1"

[0878] Step 1: Accept the child's spoken question.

[0879] Step 2: Convert the question into text using a speech recognition engine.

[0880] Step 3: Use a natural language processing engine to infer the intent of the question.

[0881] Step 4: Extract information based on the intent inferred from the knowledge base.

[0882] Step 5: Recognize the child's emotions using the emotion engine.

[0883] Step 6: Adjust how you present information based on the emotions you identify.

[0884] "Example 2"

[0885] Step 1: A child asks, "Why is the sky blue?"

[0886] Step 2: Convert the question into text using a speech recognition engine.

[0887] Step 3: Use a natural language processing engine to infer the intent of the question.

[0888] Step 4: Extract information about "why the sky looks blue" from the knowledge base.

[0889] Step 5: Use the emotion engine to recognize the child's joy.

[0890] Step 6: Provide information in an upbeat tone based on the perceived pleasure.

[0891] Step 7: Provide information in the voice of a child's favorite character.

[0892] Example 1

[0893] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[0894] Conventional voice recognition systems have difficulty in accurately recognizing children's voices and providing appropriate information. In addition, they lack the ability to adjust the way information is provided according to the child's emotions, making it difficult to provide information in a way that is easy for children to understand and that interests them. Furthermore, providing information in the voice of a specific character or person can attract children's interest, but there are few systems that have such a function.

[0895] The identification process by the identification processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means. In this invention, the server includes a voice recognition means, a means for accepting a question by voice, a means for guessing the intention of the question, a means for providing information based on the guessed intention, a means for adjusting the tone of the information to be provided, a means for providing information in the voice of a specific character or person, and a means for recognizing the child's emotion and adjusting the method of providing information based on the emotion. This makes it possible to accurately recognize the child's voice, understand the intention of the question, and provide information in a form that is easy for the child to understand. In addition, by adjusting the method of providing information according to the child's emotion, it is possible to provide information in a form that is easy for the child to be interested in. Furthermore, by providing information in the voice of a specific character or person, it is possible to attract the child's interest.

[0896] "Speech recognition means" refers to a technology for converting voice data into text data.

[0897] "Means for accepting questions by voice" refers to a technique that enables the system to accept questions input by voice from the user.

[0898] The "means for inferring the intent of a question" is a technology for understanding the subject and purpose of a user's question from text data obtained by a speech recognition means.

[0899] "Means for providing information" refers to technology for providing the user with appropriate information based on the intent of the question.

[0900] "Means for adjusting the tone of the information provided" refers to a technique for adjusting the way information is provided depending on the user's age and level of understanding.

[0901] "Means for providing information in the voice of a specific character or person" refers to technology for providing information by imitating the voice of a character or person that a user likes.

[0902] "Means for recognizing a child's emotions and adjusting the way information is provided based on those emotions" is a technology for recognizing emotions from a child's voice and adjusting the way information is provided based on those emotions.

[0903] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0904] This invention is a system that recognizes a child's voice, analyzes the intent of the question, and provides appropriate information. It also has a function of recognizing the child's emotion and adjusting the way information is provided based on the emotion. This system is implemented using the following hardware and software.

[0905] Hardware and Software Used

[0906] 1. Speech recognition engine: Used to convert voice data into text data. Specifically, we use the Google Cloud Speech-to-Text API.

[0907] 2. Natural language processing engine: Used to analyze the intent of questions from text data. Specifically, we use OpenAI's GPT-3.

[0908] 3. Knowledge base: Used to provide appropriate information based on the intent of the question, specifically using Wikipedia API and Wolfram Alpha API.

[0909] 4. Speech synthesis engine: Used to convert text data into voice data. Specifically, Amazon Polly or Google Text-to-Speech API is used.

[0910] 5. Emotion recognition engine: Used to recognize emotions from children's voices. Specifically, it uses Microsoft Azure's Emotion API.

[0911] System Operation

[0912] When a user speaks a question into the terminal, the terminal captures the voice with a microphone and sends it to the server. The server converts the voice into text using a speech recognition engine. It then uses a natural language processing engine to analyze the intent of the question from the text. Based on the analyzed intent, the server searches and retrieves the appropriate information from a knowledge base.

[0913] The way the information is provided is adjusted depending on the user's age and emotions. For example, it provides simple explanations to younger children and more detailed information to older children. It also uses an emotion recognition engine to determine whether a child is angry or happy and adjusts the tone of the information based on that emotion. For example, it provides information in a calm tone if the child is angry and in a bright tone if the child is happy.

[0914] Finally, the speech synthesis engine is used to convert the adjusted information into speech in the voice of a specific character or person. The terminal plays back this speech data and provides it to the user.

[0915] Examples

[0916] Describe what happens when a user asks, "Why is the sky blue?"

[0917] 1. Receiving speech input: The user speaks into the device's microphone, "Why is the sky blue?"

[0918] 2. Speech Recognition: The server receives the speech data and uses the Google Cloud Speech-to-Text API to convert the speech to text: "Why is the sky blue?"

[0919] 3. Question intent analysis: The server inputs the text into GPT-3 and extracts the intent, such as “I want to know the reason for the color of the sky.”

[0920] 4. Information retrieval: The server sends the keyword "sky color" to the Wikipedia API and retrieves the returned information.

[0921] 5. Information Tone Adjustment: The server generates a simple text, "The sky is blue because the sun's light is scattered in the air," and presents it in a light tone.

[0922] 6. Speech synthesis: The server sends text to Amazon Polly and receives speech data in return.

[0923] 7. Audio output: The device plays the audio data and the user hears the explanation in the character's voice.

[0924] Examples of prompt statements

[0925] In response to the question "Why is the sky blue?", the following prompt sentence is input to the generative AI model.

[0926] User Question: Why is the sky blue?

[0927] Age: 7 years

[0928] Emotion: Happy

[0929] Provide your answers in simple terms and in a light tone.

[0930] This prompt sentence is then used to have the generative AI model generate an appropriate answer.

[0931] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0932] Step 1:

[0933] A user speaks a question into a device. Specifically, the user says, "Why is the sky blue?" The input is voice data, and the device's microphone captures this voice. The output is the captured voice data.

[0934] Step 2:

[0935] The terminal transmits the captured audio data to the server. The input is the captured audio data, and the output is the audio data transmitted to the server.

[0936] Step 3:

[0937] The server uses a speech recognition engine to convert the voice data into text data. Specifically, it uses the Google Cloud Speech-to-Text API. The input is voice data, speech recognition is performed as data processing, and the output is the text data "Why is the sky blue?"

[0938] Step 4:

[0939] The server uses a natural language processing engine to analyze the intent of the question from the text data. Specifically, it uses OpenAI's GPT-3. The input is the text data "Why is the sky blue?", intent analysis is performed as a data calculation, and the output is the intent data "I want to know the reason for the color of the sky."

[0940] Step 5:

[0941] The server searches for the appropriate information from the knowledge base. Specifically, it uses the Wikipedia API. The input is the intent data "color of the sky", and a query is sent to the API as a data search, and the output is the information data "The sky is blue because the sunlight is scattered in the air".

[0942] Step 6:

[0943] The server adjusts the way information is provided based on the user's age and emotions. Specifically, it uses an emotion recognition engine to analyze the user's emotions and adjust the tone of the information. The input is information data and the user's age and emotion data, tone adjustment is performed as data processing, and the output is the adjusted information data.

[0944] Step 7:

[0945] The server uses a voice synthesis engine to convert the adjusted information data into voice data. Specifically, it uses Amazon Polly. The input is the adjusted information data, voice synthesis is performed as data processing, and the output is voice data.

[0946] Step 8:

[0947] The device receives the voice data sent from the server and plays it back to the user through the speaker. The input is the voice data, and the output is the voice the user hears. Specifically, the device's speaker plays back information in the character's voice saying, "The sky is blue because the sunlight is scattered in the air."

[0948] (Application example 1)

[0949] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[0950] Conventional voice recognition systems have difficulty in accurately recognizing children's voices and providing appropriate information. In addition, they cannot adjust the way information is provided according to the child's emotions, and cannot provide information in a form that is easy for children to understand. Furthermore, they lack the ability to provide information in the voice of a specific character or person, making it difficult to attract children's interest.

[0951] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0952] In this invention, the server includes a voice recognition means, a means for accepting a question by voice, a means for guessing the intention of the question, a means for providing information based on the guessed intention, a means for adjusting the tone of the information to be provided, a means for providing information in the voice of a specific character or person, a means for recognizing emotions, a means for adjusting the method of providing information based on the emotion, a means for extracting information from a knowledge base, a means for using a natural language processing engine, a means for using a voice synthesis engine, and a means for being installed in a smartphone or tablet. This makes it possible to accurately recognize a child's voice, guess the intention of the question, and provide appropriate information. In addition, the method of providing information can be adjusted according to the child's emotions, and by providing information in the voice of a specific character or person, it is possible to attract the child's interest.

[0953] "Speech recognition means" refers to technology that converts speech into text.

[0954] The "means for accepting questions by voice" is an interface that enables a user to input questions by voice.

[0955] "Means of inferring question intent" refers to technology that understands the purpose and subject of a user's question from the entered text.

[0956] "Means for providing information based on inferred intent" refers to technology that provides users with appropriate information according to the intent of the question.

[0957] "Means for adjusting the tone of information provided" refers to technology for adjusting the tone of voice or the manner of expression when providing information.

[0958] "Means of providing information in the voice of a specific character or person" refers to technology that provides information by imitating the voice of a character or person that a user likes.

[0959] "Means of recognizing emotions" refers to technology that analyzes emotions from the user's voice and facial expressions.

[0960] "Means for adjusting the way information is presented based on emotions" refers to technology that changes the way information is presented depending on recognized emotions.

[0961] "Means for extracting information from a knowledge base" refers to technology for searching and extracting necessary information from a database or knowledge base.

[0962] "Means for using a natural language processing engine" refers to means for using natural language processing technology to analyze text data and understand its meaning.

[0963] "Means for using a speech synthesis engine" refers to technology for converting text data into speech.

[0964] "Means of being installed on a smartphone or tablet" refers to technology that is installed as an application on a mobile device.

[0965] A system for implementing the present invention includes a voice recognition means, a means for accepting a question by voice, a means for inferring the intent of the question, a means for providing information based on the inferred intent, a means for adjusting the tone of the provided information, a means for providing information in the voice of a particular character or person, a means for recognizing emotions, a means for adjusting the manner in which information is provided based on emotions, a means for extracting information from a knowledge base, a means for using a natural language processing engine, a means for using a voice synthesis engine, and a means for being installed on a smartphone or tablet.

[0966] Program processing explanation

[0967] The server uses the speech_recognition library as a means of speech recognition to convert the user's voice into text. It uses a microphone as a means of accepting voice questions. It uses the natural language processing engine of the transformers library as a means of inferring the intent of the question, and analyzes the intent of the question from the input text. It extracts appropriate information from a knowledge base as a means of providing information based on the inferred intent.

[0968] As a means of adjusting the tone of the information provided, an emotion recognition engine is used, and the tone of the information is adjusted according to the user's emotions. As a means of providing information in the voice of a specific character or person, the gTTS library is used to perform voice synthesis. As a means of recognizing emotions, the emotion_recognition library is used to analyze emotions from the user's voice and facial expressions. As a means of adjusting the way information is provided based on emotions, the tone of the voice and the way it is expressed are changed according to the recognized emotion.

[0969] As a means of extracting information from a knowledge base, it uses a database or dictionary to search and extract the required information. As a means of using a natural language processing engine, it uses the transformers library to analyze text data and understand its meaning. As a means of using a speech synthesis engine, it uses the gTTS library to convert text data into speech. As a means of being installed on a smartphone or tablet, it is installed as an application on a mobile device.

[0970] Examples

[0971] For example, if a child asks, "What is the sun?", the system operates as follows. First, the child's voice is input through a microphone and converted into text using speech recognition means. Next, the natural language processing engine analyzes the intent of "What is the sun?" and extracts the answer "The sun is the star at the center of our solar system" from the knowledge base. The emotion recognition engine recognizes the child's emotion and determines, for example, that the child is "interested." The speech synthesis engine converts the text "The sun is the star at the center of our solar system" into speech in a bright tone, which is played over the smartphone or tablet's speaker.

[0972] Example of a prompt to be input to a generative AI model:

[0973] A child asks, "What is the sun?" Extract the appropriate answer from your knowledge base and provide it in a way that is easy for the child to understand. Also, provide the information in a tone that reflects the child's emotions.

[0974] The flow of the specific process in the application example 1 will be described with reference to FIG.

[0975] Step 1:

[0976] The user speaks a question into the microphone of the smartphone or tablet. The input voice data is sent to the voice recognition means of the device.

[0977] Step 2:

[0978] The terminal's voice recognition means converts the input voice data into text data. This process uses the speech_recognition library. The voice data is converted into text data and sent to the next processing step.

[0979] Step 3:

[0980] The server's natural language processing engine receives the text data and analyzes the intent of the question using the transformers library. The subject and purpose of the question are extracted from the text data, and a query is generated to search for the appropriate information from the knowledge base.

[0981] Step 4:

[0982] A means for extracting information from the server's knowledge base uses the generated query to search the database and extract the appropriate information, which is then sent to the next processing step.

[0983] Step 5:

[0984] The emotion recognition engine on the server analyzes the emotions from the user's voice and facial expressions. This process is performed using the emotion_recognition library. The emotion data is analyzed to identify the user's emotional state.

[0985] Step 6:

[0986] The means for adjusting the tone of the information provided by the server adjusts the tone and expression of the information based on the analyzed emotion data. For example, the information is provided in a bright tone when the user is interested, and in a calm tone when the user is angry.

[0987] Step 7:

[0988] The server's speech synthesis engine converts the tailored information into speech using the gTTS library, which converts the text data into audio data and provides the information in the voice of a specific character or person.

[0989] Step 8:

[0990] The terminal reproduces the audio data through the speaker, and the user can hear the information provided in the voice of a specific character or person.

[0991] The above processing steps realize a system in which a user can input a question by voice and appropriate information is provided in the voice of a specific character or person.

[0992] Example 2

[0993] Next, a description will be given of Example 2 of the embodiment 2. In the following description, the data processing device 12 is referred to as a "server" and the smart glasses 214 are referred to as a "terminal".

[0994] Conventional voice recognition systems have difficulty in providing appropriate and easy-to-understand answers to children's questions. In addition, they lack the ability to respond with a response that takes into account the child's emotions, or to provide information in the voice of a specific character, making it difficult to attract children's interest. This has created a demand for a system that provides answers to questions that are easy for children to understand and that they can listen to with interest.

[0995] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0996] In this invention, the server includes a voice recognition means, a means for accepting a question by voice, a means for guessing the intent of the question, a means for providing information based on the guessed intent, a means for adjusting the tone of the provided information, a means for providing information in the voice of a specific character or person, a means for analyzing emotions, a means for extracting information from a knowledge base, and a means for converting the extracted information into an easy-to-understand form. This makes it possible to provide appropriate and easy-to-understand answers to children's questions, and further provide responses that take into account the child's emotions and information in the voice of a specific character.

[0997] "Speech recognition means" refers to technology for converting speech into text.

[0998] A "means for accepting voice queries" is a technology for receiving and processing voice input from a user.

[0999] The "means for inferring the intent of a question" is a technology for analyzing the intent of a user's question from speech-recognized text.

[1000] "Means for providing information" refers to technology for providing the user with appropriate information based on the intent of the question.

[1001] The "means for adjusting the tone of information" refers to a technique for adjusting the voice tone of the information to be provided, and providing the information in an appropriate tone according to the user's emotions and situation.

[1002] "Means for providing information in the voice of a specific character or person" refers to technology for providing information in audio form by imitating the voice of a specific character or person.

[1003] "Means for analyzing emotions" refers to technology for analyzing emotions from the user's voice and generating a response according to those emotions.

[1004] A "means for extracting information from a knowledge base" is a technique for searching and extracting relevant information from a database or knowledge base.

[1005] The "means for converting the extracted information into an easily understandable form" refers to a technique for converting the extracted information into a form that is easily understandable to the user.

[1006] This invention is a system for providing appropriate and easy-to-understand answers to questions from children. The system includes a voice recognition means, a means for accepting questions by voice, a means for inferring the intent of the question, a means for providing information, a means for adjusting the tone of the information, a means for providing information in the voice of a specific character or person, a means for analyzing emotions, a means for extracting information from a knowledge base, and a means for converting the extracted information into an easy-to-understand form.

[1007] Hardware and Software Used

[1008] Hardware:

[1009] Device: Smartphone, tablet, computer, or other device that supports voice input

[1010] Server: Cloud server or on-premise server

[1011] software:

[1012] Speech recognition engine: Google Cloud Speech-to-Text

[1013] Natural language processing engine: OpenAI's GPT-3

[1014] Knowledge Base:Wikipedia API

[1015] Sentiment analysis engine: IBM Watson Tone Analyzer

[1016] Speech synthesis engine: Amazon Polly

[1017] System Operation

[1018] When a user (child) asks the question "Why is the sky blue?", the device captures this question using its built-in microphone. The device then converts the speech to text using Google Cloud Speech-to-Text. This text is then sent to the server.

[1019] The server uses OpenAI's GPT-3 to parse the intent of the incoming text - in this case, it understands that you are asking "Why is the sky blue?" The server then uses the Wikipedia API to extract information from its knowledge base about "Why does the sky look blue?"

[1020] The server again uses GPT-3 to convert the extracted information into words that are easy for children to understand. For example, it converts it into "The sky is blue because the sunlight is scattered in the air." In addition, the server uses IBM Watson Tone Analyzer to analyze the child's emotions and recognize that they are feeling joy.

[1021] Finally, the server uses Amazon Polly to provide information in a light-hearted voice of a child's favorite character, and the voice data is sent to the device, which plays it back.

[1022] Examples

[1023] As a concrete example, consider a child asking "Why is the sky blue?" and feeling joy in his voice. In this case, the system performs the following steps:

[1024] 1. The device captures the child's question and converts it into text using Google Cloud Speech-to-Text: "Why is the sky blue?"

[1025] 2. The server uses OpenAI’s GPT-3 to understand that the question is asking “why about the color of the sky?”

[1026] 3. The server uses the Wikipedia API to extract information about "why the sky looks blue."

[1027] 4. The server converts the extracted information into language that children can easily understand.

[1028] 5. The emotion analysis engine recognizes the child's joy, so the server provides the information in a lighter tone.

[1029] 6. The server uses Amazon Polly to provide information in the voice of the child's favorite character.

[1030] 7. The device plays the generated audio.

[1031] Example prompt:

[1032] "When a child asks, 'Why is the sky blue?' Explain to the child why the sky looks blue in terms that the child can easily understand. Also, explain in a bright tone and with the voice of a character that the child likes, so that the child will be happy."

[1033] In this way, the system provides appropriate and engaging responses to the child's questions.

[1034] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1035] Step 1:

[1036] The user asks a question.

[1037] The user (child) asks a question by voice, "Why is the sky blue?" This voice becomes the input for the system.

[1038] Step 2:

[1039] The device captures the audio.

[1040] The device uses a built-in microphone to capture the user's voice, and the captured voice data serves as input for the next processing step.

[1041] Step 3:

[1042] The device uses a speech recognition engine to convert the speech to text.

[1043] The device calls a speech recognition engine (e.g., Google Cloud Speech-to-Text) to convert the captured voice data into text data. The converted text, "Why is the sky blue?", becomes the input for the next processing step.

[1044] Step 4:

[1045] The device sends the text to the server.

[1046] The terminal sends the converted text data to the server, which receives the text data and uses it as input for the next processing step.

[1047] Step 5:

[1048] The server uses a natural language processing engine to analyze the intent of the question.

[1049] The server uses a natural language processing engine (for example, OpenAI's GPT-3) to analyze the intent of the received text data "Why is the sky blue?". The analysis result is the information that "You are asking about the reason for the color of the sky." This analysis result becomes the input for the next processing step.

[1050] Step 6:

[1051] The server extracts the relevant information from a knowledge base.

[1052] The server uses a knowledge base (e.g., the Wikipedia API) to search and extract information about "why the sky looks blue." The extracted information, "The sky looks blue because sunlight is scattered by molecules in the atmosphere," becomes the input for the next processing step.

[1053] Step 7:

[1054] The server converts the extracted information into words that are easy for children to understand.

[1055] The server then uses a natural language processing engine (e.g., GPT-3) to convert the extracted information into child-friendly language. The converted information, "The sky is blue because the sun's light is scattered in the air," becomes the input for the next processing step.

[1056] Step 8:

[1057] The server uses an emotion analysis engine to analyze the child's emotion.

[1058] The server uses an emotion analysis engine (for example, IBM Watson Tone Analyzer) to analyze the emotion from the user's voice. The analysis result is the information that "I feel happy." This analysis result becomes the input for the next processing step.

[1059] Step 9:

[1060] The server uses a speech synthesis engine to convert the information into speech.

[1061] The server uses a speech synthesis engine (e.g., Amazon Polly) to convert the converted information into speech in the voice of a character the child likes. The generated speech data, "The sky is blue because the sunlight is scattered in the air," becomes the input for the next processing step.

[1062] Step 10:

[1063] The device plays the generated audio.

[1064] The server transmits the generated voice data to the terminal, which receives the voice data and plays it through the speaker, allowing the user to listen to the data and obtain the answer to their question.

[1065] (Application example 2)

[1066] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server", and the smart glasses 214 will be referred to as a "terminal".

[1067] Conventional voice recognition systems have difficulty in accurately understanding the intent of a child when he or she asks a question and providing appropriate information. In addition, the information provided is often not in a form that is easy for children to understand, and it is not possible to provide information in a tone that reflects the child's emotions, making it difficult to attract the child's interest.

[1068] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1069] In this invention, the server includes a voice recognition means, a means for accepting a question by voice, a means for guessing the intention of the question, a means for providing information based on the guessed intention, a means for adjusting the tone of the provided information, a means for providing information in the voice of a specific character or person, an emotion recognition means, a means for adjusting the tone of the information based on the emotion, and a means for extracting information from a knowledge base. This makes it possible to accurately understand the intention of a child's question, provide information in a form that is easy for the child to understand, and provide information in a tone that corresponds to the child's emotion.

[1070] "Speech recognition means" refers to technology for converting speech into text.

[1071] A "means for accepting voice queries" is an interface that allows a user to input queries by voice.

[1072] "Means of inferring the intent of the question" refers to natural language processing technology for understanding the meaning and purpose of the input text.

[1073] "Means for providing information based on inferred intent" refers to technology that extracts appropriate information from a knowledge base according to the intent of the question and provides it to the user.

[1074] "Means of adjusting the tone of the information provided" refers to techniques for adjusting the way information is conveyed depending on the user's emotions and situation.

[1075] "Means for providing information in the voice of a specific character or person" refers to technology that provides information in audio form using the voice of a character or person that the user prefers.

[1076] "Emotion recognition means" is a technology that analyzes and recognizes emotions from the user's voice and facial expressions.

[1077] A "means for adjusting the tone of information based on emotions" is a technique for adjusting the way information is conveyed and its tone depending on the recognized emotions.

[1078] A "means for extracting information from a knowledge base" is a technique for searching and extracting necessary information from a database or knowledge base to provide an answer to a question.

[1079] A system for implementing the present invention includes a voice recognition means, a means for accepting a question via voice, a means for inferring the intent of the question, a means for providing information based on the inferred intent, a means for adjusting the tone of the provided information, a means for providing information in the voice of a particular character or person, a means for recognizing emotions, a means for adjusting the tone of the information based on emotions, and a means for extracting information from a knowledge base.

[1080] System Configuration

[1081] The server uses the following hardware and software:

[1082] Speech recognition method: Uses the Google Cloud Speech-to-Text API to convert speech to text.

[1083] Method of accepting voice questions: Using an application installed on a device such as a smartphone or tablet.

[1084] How to infer the intent of the question: Use the Google Cloud Natural Language API to analyze the intent of the text.

[1085] Providing information based on inferred intent: Use the Wikipedia API or your own database to get information about the question.

[1086] Ways to adjust the tone of information provided: Use the Microsoft Azure Emotion API to analyze the user's emotions and provide information in an appropriate tone.

[1087] A means of providing information in the voice of a specific character or person: Use the Text-to-Speech API to provide information in the voice of a specific character or person.

[1088] Emotion recognition: Uses Microsoft Azure Emotion API to recognize user emotions.

[1089] How to adjust the tone of your information based on emotion: Adjust the delivery and tone of your information depending on the perceived emotion.

[1090] How to extract information from the knowledge base: Use Wikipedia API or your own database to search and extract the required information.

[1091] System Operation

[1092] When a user inputs a question by voice using a terminal such as a smartphone or tablet, the voice recognition means converts the voice into text. Next, the means for guessing the intent of the question analyzes the intent of the text and extracts appropriate information from a knowledge base. The emotion recognition means analyzes the user's emotion and adjusts the tone of the information based on the emotion. Finally, the means for providing information in the voice of a specific character or person provides the information to the user by voice.

[1093] Examples

[1094] For example, consider a child asking "Why is the sky blue?" and feeling joy in his voice. In this case, the system would:

[1095] A speech recognition tool converts the question "Why is the sky blue?" into text.

[1096] A means for inferring the intent of the question analyzes the intent of this question and understands "the reason for the color of the sky."

[1097] Extract information about "why the sky looks blue" from the knowledge base.

[1098] The emotion recognition tool recognizes the child's joy and provides information in a light-hearted tone.

[1099] Information is provided in the voices of children's favorite characters.

[1100] Example of a prompt to be input to a generative AI model:

[1101] If a child asks "Why is the sky blue?" and you can sense joy in their voice, we use a speech recognition engine to convert the question into text and a natural language processing engine to infer the intent of the question. We then extract information about "why the sky looks blue" from our knowledge base and explain that information in terms that the child can easily understand. Then, because the emotion engine recognizes the child's joy, the system provides the information in a lighter tone. Additionally, it provides the information in the voice of a character that the child likes.

[1102] The flow of the specific process in the application example 2 will be described with reference to FIG.

[1103] Step 1:

[1104] The user inputs a question by voice using a device such as a smartphone or tablet.

[1105] Input: User's voice question

[1106] Output: Audio data

[1107] Specific actions: The user speaks a question into the device's microphone.

[1108] Step 2:

[1109] The server uses a voice recognition means to convert the voice data into text.

[1110] Input: Audio data

[1111] Output: Text data

[1112] What it does: Uses the Google Cloud Speech-to-Text API to convert audio data into text.

[1113] Step 3:

[1114] The server analyzes the meaning of the text data using a means for inferring the intent of the question.

[1115] Input: Text data

[1116] Output: Intent data

[1117] Specific behavior: Uses Google Cloud Natural Language API to analyze the meaning of text data.

[1118] Step 4:

[1119] The server uses a means for extracting information from a knowledge base to obtain appropriate information based on the intent data.

[1120] Input: Intent data

[1121] Output: Information data

[1122] Specific behavior: Uses Wikipedia API or its own database to get information about the question.

[1123] Step 5:

[1124] The server uses the emotion recognition means to analyze the emotion of the user.

[1125] Input: User's voice or image data

[1126] Output: Emotion data

[1127] Specific behavior: Analyzes user emotions using Microsoft Azure Emotion API.

[1128] Step 6:

[1129] The server adjusts the tone of the information in response to the emotion data using a means for adjusting the tone of the information based on emotion.

[1130] Input: Emotion data, information data

[1131] Output: Adjusted information data

[1132] Specific Behavior: Adjust your delivery and tone based on the perceived emotion.

[1133] Step 7:

[1134] The server provides the adjusted information data by voice using a means for providing information in the voice of a specific character or person.

[1135] Input: Adjusted information data

[1136] Output: Audio data

[1137] What it does: Uses the Text-to-Speech API to provide information in the voice of a specific character or person.

[1138] Step 8:

[1139] The terminal reproduces the audio data and provides information to the user.

[1140] Input: Audio data

[1141] Output: Providing audio information to the user

[1142] Specific operation: Provide information to the user via audio through the device's speaker.

[1143] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1144] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by making a neural network perform deep learning. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image is input. The data generation model 58 performs inference on the input inference data according to the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1145] Another example of generative AI is Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) are mentioned.

[1146] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1147] [Third embodiment]

[1148] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1149] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1150] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[1151] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1152] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[1153] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[1154] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54. The transmission and reception of various types of information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[1155] Fig. 6 shows an example of main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1156] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1157] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1158] In the headset type terminal 314, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1159] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[1160] "Example 1"

[1161] The embodiment of the present invention includes a speech recognition engine for recognizing the child's voice as a speech recognition means. The speech recognition engine identifies the child's pronunciation and vocabulary and converts it into text format. A natural language processing engine is used as a means for inferring the intent of the question. The engine analyzes the text obtained from the speech recognition engine and understands the subject and purpose of the question. Furthermore, as an information providing means, appropriate information is extracted from a knowledge base and provided in a form that is easy for the child to understand. As a means for adjusting the tone of the information, the system has a function for adjusting the method of providing information according to the child's age and level of understanding. Finally, a speech synthesis engine is used as a means for providing information in the voice of a specific character or person. The engine imitates the voice of a character or person that the child likes and provides information in that voice.

[1162] "Example 2"

[1163] As a concrete example, if a child asks "Why is the sky blue?", a speech recognition engine converts this question into text. A natural language processing engine then understands that the question is asking "Why is the sky blue?" and extracts information from a knowledge base about "Why does the sky look blue?" and explains it in terms the child can easily understand. This information is provided in the voice of a character the child likes.

[1164] The process flow of each embodiment will be described below.

[1165] "Example 1"

[1166] Step 1: The child asks the app a question, which is asked by voice.

[1167] Step 2: The speech recognition engine converts the child's speech into text. This engine identifies the child's pronunciation and vocabulary and converts it into a text format.

[1168] Step 3: A natural language processing engine analyzes the text to understand the subject and purpose of the question. Step 4: Extracts the appropriate information from the knowledge base and presents it in a child-friendly format.

[1169] Step 5: Adapt the way you present information to the child's age and level of understanding.

[1170] Step 6: The voice synthesis engine mimics the voice of a character or person that the child likes and provides the information in that voice.

[1171] "Example 2"

[1172] Step 1: A child asks, "Why is the sky blue?"

[1173] Step 2: The speech recognition engine converts this question into text.

[1174] Step 3: The natural language processing engine understands that the question is asking "why about the color of the sky?"

[1175] Step 4: Extract information about "why the sky looks blue" from the knowledge base.

[1176] Step 5: Explain the information in terms your child can understand.

[1177] Step 6: Present that information in the voice of your child's favorite character.

[1178] Example 1

[1179] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal".

[1180] Conventional voice recognition systems have difficulty in accurately recognizing children's voices and providing appropriate information. In addition, the information provided is often not in a form that is easy for children to understand, and there is a lack of functionality to provide information in the voice of a specific character or person. This poses the problem that it is difficult for children to learn with interest.

[1181] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1182] In this invention, the server includes a voice recognition means, a means for accepting a question by voice, a means for guessing the intent of the question, a means for providing information based on the guessed intent, a means for adjusting the tone of the provided information, a means for providing information in the voice of a specific character or person, a means for converting voice data into text, a means for analyzing the text to understand the subject or purpose of the question, a means for extracting appropriate information from a knowledge base, a means for formatting the extracted information into a form that is easy for children to understand, and a means for converting the formatted information into voice. This makes it possible to accurately recognize a child's voice, provide appropriate information, and further provide information in the voice of a specific character or person.

[1183] A "voice recognition means" is a device or software that has the function of analyzing voice data and converting it into text format.

[1184] A "means for accepting voice queries" is a device or software for receiving voice input from a user and processing the voice data.

[1185] The "means for inferring the intent of a question" is a device or software that has the function of analyzing the text converted by the speech recognition means and understanding the subject and purpose of the user's question.

[1186] The "means for providing information" refers to a device or software that has the function of selecting appropriate information based on the intent of the question and providing it to the user.

[1187] The "means for adjusting the tone of information" refers to a device or software that has the function of adjusting the content and expression method of the information provided according to the user's age and level of understanding.

[1188] A "means for providing information in the voice of a specific character or person" is a device or software that has a function of imitating the voice of a character or person that a user likes and providing information in that voice.

[1189] "Means for converting voice data into text" refers to a device or software that has the function of converting voice data into text format using speech recognition means.

[1190] The "means for analyzing text to understand the subject or purpose of the question" refers to a device or software that has the function of analyzing the converted text and understanding the intent of the user's question.

[1191] The "means for extracting appropriate information from a knowledge base" refers to a device or software that has the function of searching and extracting relevant information from a knowledge base based on the intent of a question.

[1192] "Means for formatting the extracted information into a form that is easy for children to understand" refers to a device or software that has the function of processing the extracted information into a form that is easy for children to understand.

[1193] The "means for converting formatted information into speech" refers to a device or software having a function for converting formatted information into speech format using speech synthesis technology.

[1194] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[1195] This invention is a system for recognizing a child's voice and providing appropriate information. The system includes a voice recognition means, a means for accepting a question by voice, a means for inferring the intent of the question, a means for providing information, a means for adjusting the tone of the information, a means for providing information in the voice of a specific character or person, a means for converting voice data into text, a means for analyzing the text to understand the subject or purpose of the question, a means for extracting appropriate information from a knowledge base, a means for formatting the extracted information into a form that is easily understandable to children, and a means for converting the formatted information into voice.

[1196] Hardware and Software Used

[1197] The server uses general voice recognition software, such as Google Cloud Speech-to-Text API, to convert the voice data into text.

[1198] To infer the intent of a question, a natural language processing engine is used, specifically a generative AI model such as OpenAI's GPT-4, which analyzes the text to understand the subject and purpose of the question.

[1199] To provide this information, a means is needed to extract the appropriate information from a knowledge base, which could be Wikipedia or a specific educational database.

[1200] Adjusting the tone of the information requires a way to shape it according to the child's age and level of understanding, using simple vocabulary and short sentences.

[1201] To provide information in the voice of a specific character or person, a voice synthesis engine is used, specifically, voice synthesis software such as Amazon Polly or Google Cloud Text-to-Speech.

[1202] Examples

[1203] When a user (child) says "Tell me about dinosaurs," the following process takes place:

[1204] 1. The device records the user's voice and sends it to the server.

[1205] 2. The server converts the speech to text using the Google Cloud Speech-to-Text API.

[1206] 3. The server passes the converted text, “Tell me about dinosaurs,” to GPT-4 and analyzes the intent of the question.

[1207] 4. The server extracts information about dinosaurs from Wikipedia and formats the information in a way that is easy for children to understand.

[1208] 5. The server uses Amazon Polly to generate a speech in the voice of the child's favorite character saying, "Dinosaurs were creatures that appeared on Earth approximately 230 million years ago."

[1209] 6. The terminal plays the generated audio and provides information to the user.

[1210] Examples of prompt statements

[1211] In response to the question "Tell me about dinosaurs," the following prompt sentence is input to the generative AI model.

[1212] User Question: Tell me about dinosaurs

[1213] Prompt for generative AI model: Give a simple, child-friendly description of dinosaurs. Include the type, era, characteristics, etc. of the dinosaur.

[1214] Using this prompt, the generative AI model generates text to provide appropriate information to children.

[1215] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1216] Step 1:

[1217] The terminal records the user's voice and transmits it to the server.

[1218] Input: User's voice

[1219] Output: Audio data

[1220] Specific operation: When a user says, "Tell me about dinosaurs," the device's microphone captures the audio and sends the audio data to the server.

[1221] Step 2:

[1222] The server receives the voice data and converts it to text using speech recognition means.

[1223] Input: Audio data

[1224] Output: Text data

[1225] What happens: The server uses the Google Cloud Speech-to-Text API to parse the audio data and convert it into text, such as "Tell me about dinosaurs."

[1226] Step 3:

[1227] The server passes the text data to a natural language processing engine to analyze the intent of the question.

[1228] Input: Text data

[1229] Output: Question intent

[1230] Specific operation: The server uses GPT-4 to analyze the text data and identify the user’s intent to learn about dinosaurs.

[1231] Step 4:

[1232] The server extracts appropriate information from a knowledge base based on the intent of the question.

[1233] Input: Question intent

[1234] Output: Extracted information

[1235] Specific operation: The server searches for information about dinosaurs from knowledge bases such as Wikipedia, and extracts the information that "dinosaurs are creatures that appeared on Earth approximately 230 million years ago, and include species such as Tyrannosaurus and Triceratops."

[1236] Step 5:

[1237] The server formats the extracted information in a way that is easy for children to understand.

[1238] Input: Extracted information

[1239] Output: Formatted information

[1240] Specific operation: The server converts the extracted information into simple vocabulary and short sentences, formatting it into the form of "Dinosaurs are large creatures that lived on Earth a long time ago. There were species called Tyrannosaurus and Triceratops."

[1241] Step 6:

[1242] The server passes the formatted information to a speech synthesis engine, which generates speech in the voice of a particular character or person.

[1243] Input: Formatted information

[1244] Output: Audio data

[1245] What it does: The server uses Amazon Polly to generate a speech sequence in the voice of a child's favorite character saying, "Dinosaurs are big creatures that lived on Earth a long time ago. There were species called Tyrannosaurus and Triceratops."

[1246] Step 7:

[1247] The terminal receives the generated voice data and plays it back to the user.

[1248] Input: Audio data

[1249] Output: Played audio

[1250] Specific operation: The terminal plays the audio data received from the server, allowing the user to listen to information about dinosaurs in the voice of their favorite character.

[1251] (Application example 1)

[1252] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server", and the headset type terminal 314 will be referred to as a "terminal".

[1253] Conventional voice recognition systems are primarily targeted at adult voices, and have the problem of difficulty in accurately recognizing children's voices. It is also difficult to properly understand the content of a child's question and provide information in a format that is easy for children to understand. Furthermore, there is a lack of means to increase children's motivation to learn by providing information in the voices of characters that interest them. To solve these problems, a system is needed that can accurately recognize children's voices, properly analyze the intent of the question, and provide information in a format that is easy for children to understand.

[1254] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1255] In this invention, the server includes a voice recognition means, a means for accepting a question by voice, a means for guessing the intention of the question, a means for providing information based on the guessed intention, a means for adjusting the tone of the information to be provided, a means for providing information in the voice of a specific character or person, a means for providing information in a form that is easy for a child to understand, a means for providing information in the voice of a character that the child likes, and a means including an application to be installed on the smart device. This makes it possible to accurately recognize the child's voice, appropriately analyze the intention of the question, and provide information in a form that is easy for a child to understand. Also, by providing information in the voice of a character that the child is interested in, it is possible to increase the child's motivation to learn.

[1256] "Speech recognition means" refers to technology for converting speech into text.

[1257] The "means for accepting questions by voice" is an interface that enables a user to input questions by voice.

[1258] "Means for inferring the intent of a question" refers to technology that analyzes the subject and purpose of a question from input text.

[1259] The "means for providing information based on an inferred intent" is a technology for providing appropriate information according to the intent of the analyzed question.

[1260] "Means for adjusting the tone of the information provided" refers to technology that adjusts the way information is provided depending on the user's age and level of understanding.

[1261] "Means of providing information in the voice of a specific character or person" refers to technology that provides information by imitating the voice of a character or person that a user likes.

[1262] "Means of providing information in a form that children can easily understand" refers to techniques that provide information in a concise and easy-to-understand manner so that children can easily understand it.

[1263] The "means for providing information in the voice of a character that a child likes" is a technology for providing information using the voice of a character that is of interest to children.

[1264] "Means including an application installed on a smart device" refers to a software application installed on a device such as a smartphone or tablet.

[1265] In order to implement this invention, means including a voice recognition means, a means for accepting a question by voice, a means for inferring the intent of the question, a means for providing information based on the inferred intent, a means for adjusting the tone of the information to be provided, a means for providing information in the voice of a particular character or person, a means for providing information in a form that is easy for children to understand, a means for providing information in the voice of a character that children like, and an application to be installed on a smart device are used.

[1266] Hardware and Software Used

[1267] The server uses the following hardware and software:

[1268] Hardware: Smartphones, tablets

[1269] software:

[1270] Speech recognition engine (Google Cloud Speech-to-Text)

[1271] Natural language processing engine (Google Cloud Natural Language API)

[1272] Knowledge Base (Wolfram Alpha API)

[1273] Speech synthesis engine (Google Cloud Text-to-Speech)

[1274] Processing flow

[1275] 1. Speech recognition: The user speaks a question into their smart device. The speech recognition engine converts this speech into text.

[1276] 2. Natural language processing: The server analyzes the text obtained from the speech recognition engine using a natural language processing engine to infer the intent of the question.

[1277] 3. Information provision: The server extracts appropriate information from a knowledge base based on the inferred intent.

[1278] 4. Tone Adjustment: The server adjusts the tone of the information depending on the user's age and level of understanding.

[1279] 5. Voice synthesis: The server uses a voice synthesis engine to provide information in the voice of the child's favorite character.

[1280] Examples

[1281] For example, if a child asks a smart device, "Tell me about dinosaurs," a speech recognition engine converts this speech into text. Next, a natural language processing engine analyzes this text and infers that the intent of the question is, "I want to know information about dinosaurs." The server extracts information about dinosaurs from a knowledge base and provides the information in a format that is easy for the child to understand. Finally, a speech synthesis engine is used to provide information in the voice of the child's favorite character, such as, "Tyrannosaurus is a dinosaur that lived about 68 million years ago."

[1282] Examples of prompt statements

[1283] Create an application that recognizes children's voices, analyzes the intent of the questions, and provides appropriate information. Use Google Cloud Speech-to-Text API for speech recognition and Google Cloud Natural Language API for natural language processing. Use Wolfram Alpha API to provide information and Google Cloud Text-to-Speech API for speech synthesis. Provide information in the voice of a character that children like.

[1284] In this way, educational applications that allow children to learn in a fun way can be realized.

[1285] The flow of the specific process in the application example 1 will be described with reference to FIG.

[1286] Step 1:

[1287] A user speaks a question into a smart device, and the spoken voice data is picked up by the microphone of the smart device.

[1288] Step 2:

[1289] The terminal sends the acquired voice data to a voice recognition engine (Google Cloud Speech-to-Text). The voice recognition engine converts the voice data into text data. The input is voice data, and the output is text data.

[1290] Step 3:

[1291] The server sends the text data obtained from the speech recognition engine to a natural language processing engine (Google Cloud Natural Language API). The natural language processing engine analyzes the text data and infers the intent of the question. The input is text data, and the output is data that indicates the intent of the question.

[1292] Step 4:

[1293] The server extracts the appropriate information from a knowledge base (Wolfram Alpha API) based on the inferred question intent. The input is data indicating the question intent, and the output is related information data.

[1294] Step 5:

[1295] The server adjusts the tone of the information it provides based on the extracted information data. It changes the way information is presented depending on the user's age and level of understanding. The input is information data and the user's profile data, and the output is the adjusted information data.

[1296] Step 6:

[1297] The server sends the adjusted information data to a voice synthesis engine (Google Cloud Text-to-Speech). The voice synthesis engine converts the information into voice data in the voice of the child's favorite character. The input is the adjusted information data, and the output is voice data.

[1298] Step 7:

[1299] The terminal reproduces the voice data obtained from the voice synthesis engine and provides it to the user, who receives the answer to the question through the voice coming from the smart device.

[1300] Example 2

[1301] Next, a second embodiment of the second form will be described. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal".

[1302] In conventional voice recognition systems, it can be difficult to provide appropriate answers to user questions. In particular, when children ask questions, it is necessary to provide answers that match their level of understanding, but there is a lack of technology to achieve this. Furthermore, it is necessary to provide answers in the voices of specific characters or people to make it easier to attract children's interest, but conventional systems are not able to fully address this issue.

[1303] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1304] In this invention, the server includes a voice recognition means, a means for accepting a question by voice, a means for analyzing the intent of the question, a means for extracting information based on the analyzed intent, a means for converting the extracted information, a means for voice synthesizing the converted information, and a means for providing information in the voice of a specific character or person. This makes it possible to provide an appropriate and easy-to-understand answer to a user's question. Also, providing information in the voice of a specific character or person can attract the interest of children in particular.

[1305] "Speech recognition means" refers to technology for converting speech into text.

[1306] A "means for accepting voice queries" is a technology for receiving and processing voice input from a user.

[1307] A "means for analyzing the intent of a question" is a technique for understanding the content of a user's question and identifying its intent.

[1308] "Means of information extraction" refers to techniques for retrieving appropriate information from a database or knowledge base based on the analyzed intent.

[1309] "Means for converting information" refers to technology for converting extracted information into a form that is easy for users to understand.

[1310] "Means for synthesizing speech" refers to technology for converting text information into speech.

[1311] "Means for providing information in the voice of a specific character or person" refers to technology for playing back the generated voice in the voice of a specific character or person.

[1312] This invention is a system that allows a user to input a question by voice and provides an appropriate answer to the question. This system includes a voice recognition means, a means for accepting a question by voice, a means for analyzing the intent of the question, a means for extracting information, a means for converting information, a means for synthesizing voice, and a means for providing information in the voice of a specific character or person.

[1313] Hardware and Software Used

[1314] Hardware:

[1315] Device: Any device with a microphone (smartphone, tablet, PC, etc.)

[1316] Server: A server with high-performance computing resources (such as a cloud server)

[1317] software:

[1318] Speech recognition engine: General speech recognition API (e.g. Google Cloud Speech-to-Text API)

[1319] Natural Language Processing Engine: Advanced natural language processing models (e.g. OpenAI's GPT-4)

[1320] Knowledge base: online encyclopedia APIs (e.g. Wikipedia API) or proprietary databases

[1321] Speech synthesis engine: Speech synthesis API (e.g. Amazon Polly)

[1322] Program Processing

[1323] The user speaks a question into the terminal. For example, a child may ask, "Why is the sky blue?"

[1324] 1. Voice Recognition:

[1325] The terminal captures the user's voice through a microphone.

[1326] The server converts the audio to text using the Google Cloud Speech-to-Text API.

[1327] 2. Natural Language Processing:

[1328] The server sends the converted text to a natural language processing engine (OpenAI's GPT-4).

[1329] The natural language processing engine analyzes the intent of the question and understands that you are asking "why about the color of the sky."

[1330] 3. Extracting information from the knowledge base:

[1331] The server accesses a knowledge base (Wikipedia API or its own database) and searches for information about "why the sky looks blue."

[1332] 4. Transformation of information:

[1333] The server converts the retrieved information into language that is easy for children to understand.

[1334] 5. Speech synthesis:

[1335] The server sends the converted information to a speech synthesis engine (Amazon Polly), which generates speech in the voice of the child's favorite character.

[1336] 6. Audio playback:

[1337] The terminal plays the generated audio.

[1338] Examples

[1339] As a concrete example, here is how a child asks "Why is the sky blue?":

[1340] Child: "Why is the sky blue?"

[1341] Terminal: Captures audio and sends it to the server.

[1342] Server: Use the Google Cloud Speech-to-Text API to convert the speech to text "Why is the sky blue?"

[1343] Server: OpenAI's GPT-4 analyzes the intent of the question and retrieves information from the knowledge base.

[1344] Server: Convert that into a simple explanation like, "The sky is blue because the sun's light is scattered by tiny particles in the air."

[1345] Server: Generate the character's voice using Amazon Polly.

[1346] Terminal: Play the generated audio.

[1347] Examples of prompt statements

[1348] "If a child asks, 'Why is the sky blue?', how should I respond?"

[1349] This system starts with the user's voice input and goes through a series of steps including voice recognition, natural language processing, information extraction from a knowledge base, information conversion, voice synthesis, and voice playback, before providing the user with an appropriate answer.

[1350] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1351] Step 1:

[1352] The user speaks a question into the terminal.

[1353] Input: User speech (e.g. "Why is the sky blue?")

[1354] Specific action: A child speaks to the device, "Why is the sky blue?"

[1355] Output: The device captures the audio data.

[1356] Step 2:

[1357] The terminal transmits the captured voice data to the server.

[1358] Input: Captured audio data

[1359] Specific operations: The device sends voice data to the server.

[1360] Output: The server receives the audio data.

[1361] Step 3:

[1362] The server converts the voice data into text using a voice recognition engine.

[1363] Input: Audio data

[1364] What happens: The server uses a speech recognition engine (e.g. a common speech recognition API) to convert the speech into text: "Why is the sky blue?"

[1365] Output: Text data ("Why is the sky blue?")

[1366] Step 4:

[1367] The server sends the text data to a natural language processing engine to analyze the intent of the question.

[1368] Input: Text data ("Why is the sky blue?")

[1369] What happens: The server uses a natural language processing engine (e.g., an advanced natural language processing model) to parse the intent of the question and understands that you are asking "why about the color of the sky?"

[1370] Output: Question intent ("Why is the sky colored?")

[1371] Step 5:

[1372] The server extracts information from a knowledge base based on the parsed intent.

[1373] Input: Question intent ("Reasons for the color of the sky")

[1374] What happens: The server accesses a knowledge base (e.g. an online encyclopedia API or a proprietary database) and searches for information about "why the sky looks blue."

[1375] Output: Extracted information ("The sky is blue because sunlight is scattered by tiny particles in the air")

[1376] Step 6:

[1377] The server translates the extracted information into child-friendly language.

[1378] Input: Extracted information ("The sky is blue because sunlight is scattered by tiny particles in the air.")

[1379] What happens: The server converts that information into a simple explanation, such as "The sky is blue because the sun's light is scattered by tiny particles in the air."

[1380] Output: Transformed information ("The sky is blue because sunlight is scattered by tiny particles in the air.")

[1381] Step 7:

[1382] The server sends the converted information to a speech synthesis engine, which generates speech in the voice of a particular character or person.

[1383] Input: Transformed information ("The sky is blue because sunlight is scattered by tiny particles in the air.")

[1384] Specific behavior: The server uses a speech synthesis engine (e.g., a speech synthesis API) to generate speech in the character's voice.

[1385] Output: The generated audio data

[1386] Step 8:

[1387] The terminal reproduces the generated voice data.

[1388] Input: Generated audio data

[1389] What it does: The device plays a character's voice saying, "The sky looks blue because the sun's light is scattered by tiny particles in the air."

[1390] Output: The user hears the audio.

[1391] (Application example 2)

[1392] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server", and the headset type terminal 314 will be referred to as a "terminal".

[1393] Conventional voice recognition systems have a problem in that it is difficult to provide appropriate answers to questions posed by children. In particular, it is necessary to provide information in a form that is easy for children to understand, but existing systems are inadequate in this regard. It is also important to provide answers in the voice of a child's favorite character to attract the child's interest, but this was not achieved in conventional systems. Furthermore, it is necessary to consider the convenience of the application as it is installed on a smartphone.

[1394] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1395] In this invention, the server includes a voice recognition means, a means for accepting a question by voice, a means for guessing the intention of the question, a means for providing information based on the guessed intention, a means for adjusting the tone of the provided information, a means for providing information in the voice of a specific character or person, a means for extracting information from a knowledge base, a means for reconstructing information in words that are easy for children to understand, and an application installed on the smartphone. This makes it possible to provide information in an appropriate and easy-to-understand form in response to a question asked by a child, and further to answer in the voice of a character that is likely to interest children.

[1396] "Speech recognition means" refers to technology that converts speech into text.

[1397] The "means for accepting questions by voice" is an interface that enables a user to input questions by voice.

[1398] "Means for inferring the intent of a question" refers to technology that analyzes the content of an input question and understands its intent.

[1399] "Means for providing information based on inferred intent" refers to technology that selects and provides appropriate information according to the intent of the question.

[1400] The "means for adjusting the tone of the information provided" refers to a technique for adjusting the tone of voice or the manner of expression of the information provided.

[1401] "Means for providing information in the voice of a specific character or person" refers to technology for providing information in audio form by imitating the voice of a specific character or person.

[1402] A "means for extracting information from a knowledge base" is a technique for searching and extracting answers to questions from a knowledge base.

[1403] "Means for reconstructing information in terms that children can easily understand" refers to technology for reconstructing extracted information into a form that children can easily understand.

[1404] An "application installed on a smartphone" is software that runs on a smartphone.

[1405] A system for implementing the present invention includes a voice recognition means, a means for accepting a question via voice, a means for inferring the intent of the question, a means for providing information based on the inferred intent, a means for adjusting the tone of the provided information, a means for providing information in the voice of a particular character or person, a means for extracting information from a knowledge base, a means for reconstructing information in terms that are easy for a child to understand, and an application installed on a smartphone.

[1406] Program processing explanation

[1407] Voice Recognition Method

[1408] When a user speaks a question into the smartphone's microphone, the speech recognition means converts the speech into text using the speech_recognition library.

[1409] A way to guess the intent of a question

[1410] The converted text is then analyzed by a natural language processing engine to infer the intent of the question, using a pipeline from the transformers library.

[1411] A means of extracting information from a knowledge base

[1412] Based on the inferred intent, appropriate information is searched and extracted from a knowledge base, which is a pre-built database.

[1413] A way to restructure information in terms that children can easily understand

[1414] The extracted information is then reconstructed in a way that is easy for children to understand, using a generative AI model.

[1415] A means of providing information in the voice of a specific character or person

[1416] The reconstructed information is then provided as audio in the voice of a specific character or person, using the pyttsx3 library.

[1417] Examples

[1418] For example, if a child asks "Why is the sky blue?", a speech recognition tool converts the question into text. A natural language processing engine then understands that the question is asking "Why is the sky blue?" and extracts information from a knowledge base about "Why does the sky look blue?" and explains it in terms the child can easily understand. This information is provided in the voice of a character the child likes.

[1419] Examples of prompt statements

[1420] If a child asks "Why is the sky blue?", the speech recognition engine converts this question into text, and the natural language processing engine understands that the question is asking "Why is the sky blue?" It then extracts information from its knowledge base about "Why does the sky look blue?" and explains it in terms the child can easily understand, delivered in the voice of a character the child loves.

[1421] In this way, it is possible to provide information in an appropriate and easy-to-understand form in response to a question asked by a child, and further, to respond in the voice of a character that is likely to interest children.

[1422] The flow of the specific process in the application example 2 will be described with reference to FIG.

[1423] Step 1:

[1424] The user inputs a question by voice through the smartphone's microphone. The input voice data is captured by the smartphone's microphone.

[1425] Step 2:

[1426] The smartphone sends the captured voice data to the voice recognition means, which converts the voice data into text using the speech_recognition library. The input is the voice data, and the output is the text data.

[1427] Step 3:

[1428] The server sends the converted text data to a natural language processing engine. The natural language processing engine uses the transformers library pipeline to analyze the text data and infer the intent of the question. The input is text data, and the output is data that indicates the intent of the question.

[1429] Step 4:

[1430] The server searches for appropriate information from a knowledge base based on the inferred question intent. The knowledge base is a pre-constructed database that contains information corresponding to the question intent. The input is data indicating the question intent, and the output is appropriate information.

[1431] Step 5:

[1432] The server sends the extracted information to a generative AI model, which reconstructs the information in terms that are easy for the child to understand. The generative AI model uses prompts to reconstruct the information. The input is the extracted information, and the output is the reconstructed information.

[1433] Step 6:

[1434] The server uses the pyttsx3 library to provide the reconstructed information as audio in the voice of a specific character or person. The input is the reconstructed information and the output is the audio data.

[1435] Step 7:

[1436] The smartphone provides the generated voice data to the user through the speaker. The user can hear the answer provided in the voice of a specific character or person. The input is the voice data and the output is the voice that the user hears.

[1437] In addition, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1438] "Example 1"

[1439] As one embodiment of the present invention, a system incorporating an emotion engine is provided. The system accepts a question by voice of a child, infers the intention of the question, and provides information based on the inferred intention. Furthermore, the system recognizes the child's emotion and adjusts the method of providing information based on the emotion. Specifically, if the system feels that the child is angry, the system provides information in a calm tone. On the other hand, if the system feels that the child is happy, the system provides information in a bright tone. In this way, the system selects the optimal method of providing information according to the child's emotion.

[1440] "Example 2"

[1441] As a concrete example, consider a child asking "Why is the sky blue?" and feeling joy from the child's voice. In this case, the system uses a speech recognition engine to convert the question into text and a natural language processing engine to infer the intent of the question. Next, it extracts information about "why the sky looks blue" from the knowledge base and explains the information in words that the child can easily understand. Then, because the emotion engine recognizes the child's joy, the system provides the information in a bright tone. Furthermore, providing the information in the voice of the child's favorite character attracts the child's interest and deepens their understanding.

[1442] The process flow of each embodiment will be described below.

[1443] "Example 1"

[1444] Step 1: Accept the child's spoken question.

[1445] Step 2: Convert the question into text using a speech recognition engine.

[1446] Step 3: Use a natural language processing engine to infer the intent of the question.

[1447] Step 4: Extract information based on the intent inferred from the knowledge base.

[1448] Step 5: Recognize the child's emotions using the emotion engine.

[1449] Step 6: Adjust how you present information based on the emotions you identify.

[1450] "Example 2"

[1451] Step 1: A child asks, "Why is the sky blue?"

[1452] Step 2: Convert the question into text using a speech recognition engine.

[1453] Step 3: Use a natural language processing engine to infer the intent of the question.

[1454] Step 4: Extract information about "why the sky looks blue" from the knowledge base.

[1455] Step 5: Use the emotion engine to recognize the child's joy.

[1456] Step 6: Provide information in an upbeat tone based on the perceived pleasure.

[1457] Step 7: Provide information in the voice of a child's favorite character.

[1458] Example 1

[1459] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal".

[1460] Conventional voice recognition systems have difficulty in accurately recognizing children's voices and providing appropriate information. In addition, they lack the functionality to adjust the way information is provided according to the child's emotions, making it difficult to provide information in a way that is easy for children to understand and that interests them. Furthermore, providing information in the voice of a specific character or person can attract children's interest, but there are few systems that have such functionality.

[1461] The identification process by the identification processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means. In this invention, the server includes a voice recognition means, a means for accepting a question by voice, a means for guessing the intention of the question, a means for providing information based on the guessed intention, a means for adjusting the tone of the information to be provided, a means for providing information in the voice of a specific character or person, and a means for recognizing the child's emotion and adjusting the method of providing information based on the emotion. This makes it possible to accurately recognize the child's voice, understand the intention of the question, and provide information in a form that is easy for the child to understand. In addition, by adjusting the method of providing information according to the child's emotion, it is possible to provide information in a form that is easy for the child to be interested in. Furthermore, by providing information in the voice of a specific character or person, it is possible to attract the child's interest.

[1462] "Speech recognition means" refers to a technology for converting voice data into text data.

[1463] "Means for accepting questions by voice" refers to a technique that enables the system to accept questions input by voice from the user.

[1464] The "means for inferring the intent of a question" is a technology for understanding the subject and purpose of a user's question from text data obtained by a speech recognition means.

[1465] "Means for providing information" refers to technology for providing the user with appropriate information based on the intent of the question.

[1466] "Means for adjusting the tone of the information provided" refers to a technique for adjusting the way information is provided depending on the user's age and level of understanding.

[1467] "Means for providing information in the voice of a specific character or person" refers to technology for providing information by imitating the voice of a character or person that a user likes.

[1468] "Means for recognizing a child's emotions and adjusting the way information is provided based on those emotions" is a technology for recognizing emotions from a child's voice and adjusting the way information is provided based on those emotions.

[1469] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[1470] This invention is a system that recognizes a child's voice, analyzes the intent of the question, and provides appropriate information. It also has a function of recognizing the child's emotion and adjusting the way information is provided based on the emotion. This system is implemented using the following hardware and software.

[1471] Hardware and Software Used

[1472] 1. Speech recognition engine: Used to convert voice data into text data. Specifically, we use the Google Cloud Speech-to-Text API.

[1473] 2. Natural language processing engine: Used to analyze the intent of questions from text data. Specifically, we use OpenAI's GPT-3.

[1474] 3. Knowledge base: Used to provide appropriate information based on the intent of the question, specifically using Wikipedia API and Wolfram Alpha API.

[1475] 4. Speech synthesis engine: Used to convert text data into voice data. Specifically, Amazon Polly or Google Text-to-Speech API is used.

[1476] 5. Emotion recognition engine: Used to recognize emotions from children's voices. Specifically, it uses Microsoft Azure's Emotion API.

[1477] System Operation

[1478] When a user speaks a question into the terminal, the terminal captures the voice with a microphone and sends it to the server. The server converts the voice into text using a speech recognition engine. It then uses a natural language processing engine to analyze the intent of the question from the text. Based on the analyzed intent, the server searches and retrieves the appropriate information from a knowledge base.

[1479] The way the information is provided is adjusted depending on the user's age and emotions. For example, it provides simple explanations to younger children and more detailed information to older children. It also uses an emotion recognition engine to determine whether a child is angry or happy and adjusts the tone of the information based on that emotion. For example, it provides information in a calm tone if the child is angry and in a bright tone if the child is happy.

[1480] Finally, the speech synthesis engine is used to convert the adjusted information into speech in the voice of a specific character or person. The terminal plays back this speech data and provides it to the user.

[1481] Examples

[1482] Describe what happens when a user asks, "Why is the sky blue?"

[1483] 1. Receiving speech input: The user speaks into the device's microphone, "Why is the sky blue?"

[1484] 2. Speech Recognition: The server receives the speech data and uses the Google Cloud Speech-to-Text API to convert the speech to text: "Why is the sky blue?"

[1485] 3. Question intent analysis: The server inputs the text into GPT-3 and extracts the intent, such as “I want to know the reason for the color of the sky.”

[1486] 4. Information retrieval: The server sends the keyword "sky color" to the Wikipedia API and retrieves the returned information.

[1487] 5. Information Tone Adjustment: The server generates a simple text, "The sky is blue because the sun's light is scattered in the air," and presents it in a light tone.

[1488] 6. Speech synthesis: The server sends text to Amazon Polly and receives speech data in return.

[1489] 7. Audio output: The device plays the audio data and the user hears the explanation in the character's voice.

[1490] Examples of prompt statements

[1491] In response to the question "Why is the sky blue?", the following prompt sentence is input to the generative AI model.

[1492] User Question: Why is the sky blue?

[1493] Age: 7 years

[1494] Emotion: Happy

[1495] Provide your answers in simple terms and in a light tone.

[1496] This prompt sentence is then used to have the generative AI model generate an appropriate answer.

[1497] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1498] Step 1:

[1499] A user speaks a question into a device. Specifically, the user says, "Why is the sky blue?" The input is voice data, and the device's microphone captures this voice. The output is the captured voice data.

[1500] Step 2:

[1501] The terminal transmits the captured audio data to the server. The input is the captured audio data, and the output is the audio data transmitted to the server.

[1502] Step 3:

[1503] The server uses a speech recognition engine to convert the voice data into text data. Specifically, it uses the Google Cloud Speech-to-Text API. The input is voice data, speech recognition is performed as data processing, and the output is the text data "Why is the sky blue?"

[1504] Step 4:

[1505] The server uses a natural language processing engine to analyze the intent of the question from the text data. Specifically, it uses OpenAI's GPT-3. The input is the text data "Why is the sky blue?", intent analysis is performed as a data calculation, and the output is the intent data "I want to know the reason for the color of the sky."

[1506] Step 5:

[1507] The server searches for the appropriate information from the knowledge base. Specifically, it uses the Wikipedia API. The input is the intent data "color of the sky", and a query is sent to the API as a data search, and the output is the information data "The sky is blue because the sunlight is scattered in the air".

[1508] Step 6:

[1509] The server adjusts the way information is provided based on the user's age and emotions. Specifically, it uses an emotion recognition engine to analyze the user's emotions and adjust the tone of the information. The input is information data and the user's age and emotion data, tone adjustment is performed as data processing, and the output is the adjusted information data.

[1510] Step 7:

[1511] The server uses a voice synthesis engine to convert the adjusted information data into voice data. Specifically, it uses Amazon Polly. The input is the adjusted information data, voice synthesis is performed as data processing, and the output is voice data.

[1512] Step 8:

[1513] The device receives the voice data sent from the server and plays it back to the user through the speaker. The input is the voice data, and the output is the voice the user hears. Specifically, the device's speaker plays back information in the character's voice saying, "The sky is blue because the sunlight is scattered in the air."

[1514] (Application example 1)

[1515] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server", and the headset type terminal 314 will be referred to as a "terminal".

[1516] Conventional voice recognition systems have difficulty accurately recognizing children's voices and providing appropriate information. In addition, they are unable to adjust the way information is provided according to the child's emotions, and are unable to provide information in a way that is easy for children to understand. Furthermore, they lack the ability to provide information in the voices of specific characters or people, making it difficult to attract children's interest.

[1517] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1518] In this invention, the server includes a voice recognition means, a means for accepting a question by voice, a means for guessing the intention of the question, a means for providing information based on the guessed intention, a means for adjusting the tone of the information to be provided, a means for providing information in the voice of a specific character or person, a means for recognizing emotions, a means for adjusting the method of providing information based on the emotion, a means for extracting information from a knowledge base, a means for using a natural language processing engine, a means for using a voice synthesis engine, and a means for being installed in a smartphone or tablet. This makes it possible to accurately recognize a child's voice, guess the intention of the question, and provide appropriate information. In addition, the method of providing information can be adjusted according to the child's emotions, and by providing information in the voice of a specific character or person, it is possible to attract the child's interest.

[1519] "Speech recognition means" refers to technology that converts speech into text.

[1520] The "means for accepting questions by voice" is an interface that enables a user to input questions by voice.

[1521] "Means of inferring question intent" refers to technology that understands the purpose and subject of a user's question from the entered text.

[1522] "Means for providing information based on inferred intent" refers to technology that provides users with appropriate information according to the intent of the question.

[1523] "Means for adjusting the tone of information provided" refers to technology for adjusting the tone of voice or the manner of expression when providing information.

[1524] "Means of providing information in the voice of a specific character or person" refers to technology that provides information by imitating the voice of a character or person that a user likes.

[1525] "Means of recognizing emotions" refers to technology that analyzes emotions from the user's voice and facial expressions.

[1526] "Means for adjusting the way information is presented based on emotions" refers to technology that changes the way information is presented depending on recognized emotions.

[1527] "Means for extracting information from a knowledge base" refers to technology for searching and extracting necessary information from a database or knowledge base.

[1528] "Means for using a natural language processing engine" refers to means for using natural language processing technology to analyze text data and understand its meaning.

[1529] "Means for using a speech synthesis engine" refers to technology for converting text data into speech.

[1530] "Means of being installed on a smartphone or tablet" refers to technology that is installed as an application on a mobile device.

[1531] A system for implementing the present invention includes a voice recognition means, a means for accepting a question by voice, a means for inferring the intent of the question, a means for providing information based on the inferred intent, a means for adjusting the tone of the provided information, a means for providing information in the voice of a particular character or person, a means for recognizing emotions, a means for adjusting the manner in which information is provided based on emotions, a means for extracting information from a knowledge base, a means for using a natural language processing engine, a means for using a voice synthesis engine, and a means for being installed on a smartphone or tablet.

[1532] Program processing explanation

[1533] The server uses the speech_recognition library as a means of speech recognition to convert the user's voice into text. It uses a microphone as a means of accepting voice questions. It uses the natural language processing engine of the transformers library as a means of inferring the intent of the question, and analyzes the intent of the question from the input text. It extracts appropriate information from a knowledge base as a means of providing information based on the inferred intent.

[1534] As a means of adjusting the tone of the information provided, an emotion recognition engine is used, and the tone of the information is adjusted according to the user's emotions. As a means of providing information in the voice of a specific character or person, the gTTS library is used to perform voice synthesis. As a means of recognizing emotions, the emotion_recognition library is used to analyze emotions from the user's voice and facial expressions. As a means of adjusting the way information is provided based on emotions, the tone of the voice and the way it is expressed are changed according to the recognized emotion.

[1535] As a means of extracting information from a knowledge base, it uses a database or dictionary to search and extract the required information. As a means of using a natural language processing engine, it uses the transformers library to analyze text data and understand its meaning. As a means of using a speech synthesis engine, it uses the gTTS library to convert text data into speech. As a means of being installed on a smartphone or tablet, it is installed as an application on a mobile device.

[1536] Examples

[1537] For example, if a child asks, "What is the sun?", the system operates as follows. First, the child's voice is input through a microphone and converted into text using speech recognition means. Next, the natural language processing engine analyzes the intent of "What is the sun?" and extracts the answer "The sun is the star at the center of our solar system" from the knowledge base. The emotion recognition engine recognizes the child's emotion and determines, for example, that the child is "interested." The speech synthesis engine converts the text "The sun is the star at the center of our solar system" into speech in a bright tone, which is played over the smartphone or tablet's speaker.

[1538] Example of a prompt to be input to a generative AI model:

[1539] A child asks, "What is the sun?" Extract the appropriate answer from your knowledge base and provide it in a way that is easy for the child to understand. Also, provide the information in a tone that reflects the child's emotions.

[1540] The flow of the specific process in the application example 1 will be described with reference to FIG.

[1541] Step 1:

[1542] The user speaks a question into the microphone of the smartphone or tablet. The input voice data is sent to the voice recognition means of the device.

[1543] Step 2:

[1544] The terminal's voice recognition means converts the input voice data into text data. This process uses the speech_recognition library. The voice data is converted into text data and sent to the next processing step.

[1545] Step 3:

[1546] The server's natural language processing engine receives the text data and analyzes the intent of the question using the transformers library. The subject and purpose of the question are extracted from the text data, and a query is generated to search for the appropriate information from the knowledge base.

[1547] Step 4:

[1548] A means for extracting information from the server's knowledge base uses the generated query to search the database and extract the appropriate information, which is then sent to the next processing step.

[1549] Step 5:

[1550] The emotion recognition engine on the server analyzes the emotions from the user's voice and facial expressions. This process is performed using the emotion_recognition library. The emotion data is analyzed to identify the user's emotional state.

[1551] Step 6:

[1552] The means for adjusting the tone of the information provided by the server adjusts the tone and expression of the information based on the analyzed emotion data. For example, the information is provided in a bright tone when the user is interested, and in a calm tone when the user is angry.

[1553] Step 7:

[1554] The server's speech synthesis engine converts the tailored information into speech using the gTTS library, which converts the text data into audio data and provides the information in the voice of a specific character or person.

[1555] Step 8:

[1556] The terminal reproduces the audio data through the speaker, and the user can hear the information provided in the voice of a specific character or person.

[1557] The above processing steps realize a system in which a user can input a question by voice and appropriate information is provided in the voice of a specific character or person.

[1558] Example 2

[1559] Next, a second embodiment of the second form will be described. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal".

[1560] Conventional voice recognition systems have difficulty in providing appropriate and easy-to-understand answers to children's questions. In addition, they lack the ability to respond with a response that takes into account the child's emotions, or to provide information in the voice of a specific character, making it difficult to attract children's interest. This has created a demand for a system that provides answers to questions that are easy for children to understand and that they can listen to with interest.

[1561] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1562] In this invention, the server includes a voice recognition means, a means for accepting a question by voice, a means for guessing the intent of the question, a means for providing information based on the guessed intent, a means for adjusting the tone of the provided information, a means for providing information in the voice of a specific character or person, a means for analyzing emotions, a means for extracting information from a knowledge base, and a means for converting the extracted information into an easy-to-understand form. This makes it possible to provide appropriate and easy-to-understand answers to children's questions, and further provide responses that take into account the child's emotions and information in the voice of a specific character.

[1563] "Speech recognition means" refers to technology for converting speech into text.

[1564] A "means for accepting voice queries" is a technology for receiving and processing voice input from a user.

[1565] The "means for inferring the intent of a question" is a technology for analyzing the intent of a user's question from speech-recognized text.

[1566] "Means for providing information" refers to technology for providing the user with appropriate information based on the intent of the question.

[1567] The "means for adjusting the tone of information" refers to a technique for adjusting the voice tone of the information to be provided, and providing the information in an appropriate tone according to the user's emotions and situation.

[1568] "Means for providing information in the voice of a specific character or person" refers to technology for providing information in audio form by imitating the voice of a specific character or person.

[1569] "Means for analyzing emotions" refers to technology for analyzing emotions from the user's voice and generating a response according to those emotions.

[1570] A "means for extracting information from a knowledge base" is a technique for searching and extracting relevant information from a database or knowledge base.

[1571] The "means for converting the extracted information into an easily understandable form" refers to a technique for converting the extracted information into a form that is easily understandable to the user.

[1572] This invention is a system for providing appropriate and easy-to-understand answers to questions from children. The system includes a voice recognition means, a means for accepting questions by voice, a means for inferring the intent of the question, a means for providing information, a means for adjusting the tone of the information, a means for providing information in the voice of a specific character or person, a means for analyzing emotions, a means for extracting information from a knowledge base, and a means for converting the extracted information into an easy-to-understand form.

[1573] Hardware and Software Used

[1574] Hardware:

[1575] Device: Smartphone, tablet, computer, or other device that supports voice input

[1576] Server: Cloud server or on-premise server

[1577] software:

[1578] Speech recognition engine: Google Cloud Speech-to-Text

[1579] Natural language processing engine: OpenAI's GPT-3

[1580] Knowledge Base:Wikipedia API

[1581] Sentiment analysis engine: IBM Watson Tone Analyzer

[1582] Speech synthesis engine: Amazon Polly

[1583] System Operation

[1584] When a user (child) asks the question "Why is the sky blue?", the device captures this question using its built-in microphone. The device then converts the speech to text using Google Cloud Speech-to-Text. This text is then sent to the server.

[1585] The server uses OpenAI's GPT-3 to parse the intent of the incoming text - in this case, it understands that you are asking "Why is the sky blue?" The server then uses the Wikipedia API to extract information from its knowledge base about "Why does the sky look blue?"

[1586] The server again uses GPT-3 to convert the extracted information into words that are easy for children to understand. For example, it converts it into "The sky is blue because the sunlight is scattered in the air." In addition, the server uses IBM Watson Tone Analyzer to analyze the child's emotions and recognize that they are feeling joy.

[1587] Finally, the server uses Amazon Polly to provide information in a light-hearted voice of a child's favorite character, and the voice data is sent to the device, which plays it back.

[1588] Examples

[1589] As a concrete example, consider a child asking "Why is the sky blue?" and feeling joy in his voice. In this case, the system performs the following steps:

[1590] 1. The device captures the child's question and converts it into text using Google Cloud Speech-to-Text: "Why is the sky blue?"

[1591] 2. The server uses OpenAI’s GPT-3 to understand that the question is asking “why about the color of the sky?”

[1592] 3. The server uses the Wikipedia API to extract information about "why the sky looks blue."

[1593] 4. The server converts the extracted information into language that children can easily understand.

[1594] 5. The emotion analysis engine recognizes the child's joy, so the server provides the information in a lighter tone.

[1595] 6. The server uses Amazon Polly to provide information in the voice of the child's favorite character.

[1596] 7. The device plays the generated audio.

[1597] Example prompt:

[1598] "When a child asks, 'Why is the sky blue?' Explain to the child why the sky looks blue in terms that the child can easily understand. Also, explain in a bright tone and with the voice of a character that the child likes, so that the child will be happy."

[1599] In this way, the system provides appropriate and engaging responses to the child's questions.

[1600] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1601] Step 1:

[1602] The user asks a question.

[1603] The user (child) asks a question by voice, "Why is the sky blue?" This voice becomes the input for the system.

[1604] Step 2:

[1605] The device captures the audio.

[1606] The device uses a built-in microphone to capture the user's voice, and the captured voice data serves as input for the next processing step.

[1607] Step 3:

[1608] The device uses a speech recognition engine to convert the speech to text.

[1609] The device calls a speech recognition engine (e.g., Google Cloud Speech-to-Text) to convert the captured voice data into text data. The converted text, "Why is the sky blue?", becomes the input for the next processing step.

[1610] Step 4:

[1611] The device sends the text to the server.

[1612] The terminal sends the converted text data to the server, which receives the text data and uses it as input for the next processing step.

[1613] Step 5:

[1614] The server uses a natural language processing engine to analyze the intent of the question.

[1615] The server uses a natural language processing engine (for example, OpenAI's GPT-3) to analyze the intent of the received text data "Why is the sky blue?". The analysis result is the information that "You are asking about the reason for the color of the sky." This analysis result becomes the input for the next processing step.

[1616] Step 6:

[1617] The server extracts the relevant information from a knowledge base.

[1618] The server uses a knowledge base (e.g., the Wikipedia API) to search and extract information about "why the sky looks blue." The extracted information, "The sky looks blue because sunlight is scattered by molecules in the atmosphere," becomes the input for the next processing step.

[1619] Step 7:

[1620] The server converts the extracted information into words that are easy for children to understand.

[1621] The server then uses a natural language processing engine (e.g., GPT-3) to convert the extracted information into child-friendly language. The converted information, "The sky is blue because the sun's light is scattered in the air," becomes the input for the next processing step.

[1622] Step 8:

[1623] The server uses an emotion analysis engine to analyze the child's emotion.

[1624] The server uses an emotion analysis engine (for example, IBM Watson Tone Analyzer) to analyze the emotion from the user's voice. The analysis result is the information that "I feel happy." This analysis result becomes the input for the next processing step.

[1625] Step 9:

[1626] The server uses a speech synthesis engine to convert the information into speech.

[1627] The server uses a speech synthesis engine (e.g., Amazon Polly) to convert the converted information into speech in the voice of a character the child likes. The generated speech data, "The sky is blue because the sunlight is scattered in the air," becomes the input for the next processing step.

[1628] Step 10:

[1629] The device plays the generated audio.

[1630] The server transmits the generated voice data to the terminal, which receives the voice data and plays it through the speaker, allowing the user to listen to the data and obtain the answer to their question.

[1631] (Application example 2)

[1632] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server", and the headset type terminal 314 will be referred to as a "terminal".

[1633] Conventional voice recognition systems have difficulty in accurately understanding the intent of a child when he or she asks a question and providing appropriate information. In addition, the information provided is often not in a form that is easy for children to understand, and it is not possible to provide information in a tone that reflects the child's emotions, making it difficult to attract the child's interest.

[1634] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1635] In this invention, the server includes a voice recognition means, a means for accepting a question by voice, a means for guessing the intention of the question, a means for providing information based on the guessed intention, a means for adjusting the tone of the provided information, a means for providing information in the voice of a specific character or person, an emotion recognition means, a means for adjusting the tone of the information based on the emotion, and a means for extracting information from a knowledge base. This makes it possible to accurately understand the intention of a child's question, provide information in a form that is easy for the child to understand, and provide information in a tone that corresponds to the child's emotion.

[1636] "Speech recognition means" refers to technology for converting speech into text.

[1637] A "means for accepting voice queries" is an interface that allows a user to input queries by voice.

[1638] "Means of inferring the intent of the question" refers to natural language processing technology for understanding the meaning and purpose of the input text.

[1639] "Means for providing information based on inferred intent" refers to technology that extracts appropriate information from a knowledge base according to the intent of the question and provides it to the user.

[1640] "Means of adjusting the tone of the information provided" refers to techniques for adjusting the way information is conveyed depending on the user's emotions and situation.

[1641] "Means for providing information in the voice of a specific character or person" refers to technology that provides information in audio form using the voice of a character or person that the user prefers.

[1642] "Emotion recognition means" is a technology that analyzes and recognizes emotions from the user's voice and facial expressions.

[1643] A "means for adjusting the tone of information based on emotions" is a technique for adjusting the way information is conveyed and its tone depending on the recognized emotions.

[1644] A "means for extracting information from a knowledge base" is a technique for searching and extracting necessary information from a database or knowledge base to provide an answer to a question.

[1645] A system for implementing the present invention includes a voice recognition means, a means for accepting a question via voice, a means for inferring the intent of the question, a means for providing information based on the inferred intent, a means for adjusting the tone of the provided information, a means for providing information in the voice of a particular character or person, a means for recognizing emotions, a means for adjusting the tone of the information based on emotions, and a means for extracting information from a knowledge base.

[1646] System Configuration

[1647] The server uses the following hardware and software:

[1648] Speech recognition method: Uses the Google Cloud Speech-to-Text API to convert speech to text.

[1649] Method of accepting voice questions: Using an application installed on a device such as a smartphone or tablet.

[1650] How to infer the intent of the question: Use the Google Cloud Natural Language API to analyze the intent of the text.

[1651] Providing information based on inferred intent: Use the Wikipedia API or your own database to get information about the question.

[1652] Ways to adjust the tone of information provided: Use the Microsoft Azure Emotion API to analyze the user's emotions and provide information in an appropriate tone.

[1653] A means of providing information in the voice of a specific character or person: Use the Text-to-Speech API to provide information in the voice of a specific character or person.

[1654] Emotion recognition: Uses Microsoft Azure Emotion API to recognize user emotions.

[1655] How to adjust the tone of your information based on emotion: Adjust the delivery and tone of your information depending on the perceived emotion.

[1656] How to extract information from the knowledge base: Use Wikipedia API or your own database to search and extract the required information.

[1657] System Operation

[1658] When a user inputs a question by voice using a terminal such as a smartphone or tablet, the voice recognition means converts the voice into text. Next, the means for guessing the intent of the question analyzes the intent of the text and extracts appropriate information from a knowledge base. The emotion recognition means analyzes the user's emotion and adjusts the tone of the information based on the emotion. Finally, the means for providing information in the voice of a specific character or person provides the information to the user by voice.

[1659] Examples

[1660] For example, consider a child asking "Why is the sky blue?" and feeling joy in his voice. In this case, the system would:

[1661] A speech recognition tool converts the question "Why is the sky blue?" into text.

[1662] A means for inferring the intent of the question analyzes the intent of this question and understands "the reason for the color of the sky."

[1663] Extract information about "why the sky looks blue" from the knowledge base.

[1664] The emotion recognition tool recognizes the child's joy and provides information in a light-hearted tone.

[1665] Information is provided in the voices of children's favorite characters.

[1666] Example of a prompt to be input to a generative AI model:

[1667] If a child asks "Why is the sky blue?" and you can sense joy in their voice, we use a speech recognition engine to convert the question into text and a natural language processing engine to infer the intent of the question. We then extract information about "why the sky looks blue" from our knowledge base and explain that information in terms that the child can easily understand. Then, because the emotion engine recognizes the child's joy, the system provides the information in a lighter tone. Additionally, it provides the information in the voice of a character that the child likes.

[1668] The flow of the specific process in the application example 2 will be described with reference to FIG.

[1669] Step 1:

[1670] The user inputs a question by voice using a device such as a smartphone or tablet.

[1671] Input: User's voice question

[1672] Output: Audio data

[1673] Specific actions: The user speaks a question into the device's microphone.

[1674] Step 2:

[1675] The server uses a voice recognition means to convert the voice data into text.

[1676] Input: Audio data

[1677] Output: Text data

[1678] What it does: Uses the Google Cloud Speech-to-Text API to convert audio data into text.

[1679] Step 3:

[1680] The server analyzes the meaning of the text data using a means for inferring the intent of the question.

[1681] Input: Text data

[1682] Output: Intent data

[1683] Specific behavior: Uses Google Cloud Natural Language API to analyze the meaning of text data.

[1684] Step 4:

[1685] The server uses a means for extracting information from a knowledge base to obtain appropriate information based on the intent data.

[1686] Input: Intent data

[1687] Output: Information data

[1688] Specific behavior: Uses Wikipedia API or its own database to get information about the question.

[1689] Step 5:

[1690] The server uses the emotion recognition means to analyze the emotion of the user.

[1691] Input: User's voice or image data

[1692] Output: Emotion data

[1693] Specific behavior: Analyzes user emotions using Microsoft Azure Emotion API.

[1694] Step 6:

[1695] The server adjusts the tone of the information in response to the emotion data using a means for adjusting the tone of the information based on emotion.

[1696] Input: Emotion data, information data

[1697] Output: Adjusted information data

[1698] Specific Behavior: Adjust your delivery and tone based on the perceived emotion.

[1699] Step 7:

[1700] The server provides the adjusted information data by voice using a means for providing information in the voice of a specific character or person.

[1701] Input: Adjusted information data

[1702] Output: Audio data

[1703] What it does: Uses the Text-to-Speech API to provide information in the voice of a specific character or person.

[1704] Step 8:

[1705] The terminal reproduces the audio data and provides information to the user.

[1706] Input: Audio data

[1707] Output: Providing audio information to the user

[1708] Specific operation: Provide information to the user via audio through the device's speaker.

[1709] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input for the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1710] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by making a neural network perform deep learning. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image is input. The data generation model 58 performs inference on the input inference data according to the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1711] Another example of generative AI is Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) are mentioned.

[1712] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1713] [Fourth embodiment]

[1714] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1715] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1716] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[1717] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. In addition, the microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1718] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[1719] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[1720] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54. The transmission and reception of various types of information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[1721] The control target 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, legs, etc. The posture and behavior of the robot 414 are controlled by controlling the motors of the arms, hands, legs, etc. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1722] Fig. 8 shows an example of main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1723] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1724] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1725] In the robot 414, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1726] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[1727] "Example 1"

[1728] The embodiment of the present invention includes a speech recognition engine for recognizing the child's voice as a speech recognition means. The speech recognition engine identifies the child's pronunciation and vocabulary and converts it into text format. A natural language processing engine is used as a means for inferring the intent of the question. The engine analyzes the text obtained from the speech recognition engine and understands the subject and purpose of the question. Furthermore, as an information providing means, appropriate information is extracted from a knowledge base and provided in a form that is easy for the child to understand. As a means for adjusting the tone of the information, the system has a function for adjusting the method of providing information according to the child's age and level of understanding. Finally, a speech synthesis engine is used as a means for providing information in the voice of a specific character or person. The engine imitates the voice of a character or person that the child likes and provides information in that voice.

[1729] "Example 2"

[1730] As a concrete example, if a child asks "Why is the sky blue?", a speech recognition engine converts this question into text. A natural language processing engine then understands that the question is asking "Why is the sky blue?" and extracts information from a knowledge base about "Why does the sky look blue?" and explains it in terms the child can easily understand. This information is provided in the voice of a character the child likes.

[1731] The process flow of each embodiment will be described below.

[1732] "Example 1"

[1733] Step 1: The child asks the app a question, which is asked by voice.

[1734] Step 2: The speech recognition engine converts the child's speech into text. This engine identifies the child's pronunciation and vocabulary and converts it into a text format.

[1735] Step 3: A natural language processing engine analyzes the text to understand the subject and purpose of the question. Step 4: Extracts the appropriate information from the knowledge base and presents it in a child-friendly format.

[1736] Step 5: Adapt the way you present information to the child's age and level of understanding.

[1737] Step 6: The voice synthesis engine mimics the voice of a character or person that the child likes and provides the information in that voice.

[1738] "Example 2"

[1739] Step 1: A child asks, "Why is the sky blue?"

[1740] Step 2: The speech recognition engine converts this question into text.

[1741] Step 3: The natural language processing engine understands that the question is asking "why about the color of the sky?"

[1742] Step 4: Extract information about "why the sky looks blue" from the knowledge base.

[1743] Step 5: Explain the information in terms your child can understand.

[1744] Step 6: Present that information in the voice of your child's favorite character.

[1745] Example 1

[1746] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal".

[1747] Conventional voice recognition systems have difficulty in accurately recognizing children's voices and providing appropriate information. In addition, the information provided is often not in a form that is easy for children to understand, and there is a lack of functionality to provide information in the voice of a specific character or person. This poses the problem that it is difficult for children to learn with interest.

[1748] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1749] In this invention, the server includes a voice recognition means, a means for accepting a question by voice, a means for guessing the intent of the question, a means for providing information based on the guessed intent, a means for adjusting the tone of the provided information, a means for providing information in the voice of a specific character or person, a means for converting voice data into text, a means for analyzing the text to understand the subject or purpose of the question, a means for extracting appropriate information from a knowledge base, a means for formatting the extracted information into a form that is easy for children to understand, and a means for converting the formatted information into voice. This makes it possible to accurately recognize a child's voice, provide appropriate information, and further provide information in the voice of a specific character or person.

[1750] A "voice recognition means" is a device or software that has the function of analyzing voice data and converting it into text format.

[1751] A "means for accepting voice queries" is a device or software for receiving voice input from a user and processing the voice data.

[1752] The "means for inferring the intent of a question" is a device or software that has the function of analyzing the text converted by the speech recognition means and understanding the subject and purpose of the user's question.

[1753] The "means for providing information" refers to a device or software that has the function of selecting appropriate information based on the intent of the question and providing it to the user.

[1754] The "means for adjusting the tone of information" refers to a device or software that has the function of adjusting the content and expression method of the information provided according to the user's age and level of understanding.

[1755] A "means for providing information in the voice of a specific character or person" is a device or software that has a function of imitating the voice of a character or person that a user likes and providing information in that voice.

[1756] "Means for converting voice data into text" refers to a device or software that has the function of converting voice data into text format using speech recognition means.

[1757] The "means for analyzing text to understand the subject or purpose of the question" refers to a device or software that has the function of analyzing the converted text and understanding the intent of the user's question.

[1758] The "means for extracting appropriate information from a knowledge base" refers to a device or software that has the function of searching and extracting relevant information from a knowledge base based on the intent of a question.

[1759] "Means for formatting the extracted information into a form that is easy for children to understand" refers to a device or software that has the function of processing the extracted information into a form that is easy for children to understand.

[1760] The "means for converting formatted information into speech" refers to a device or software having a function for converting formatted information into speech format using speech synthesis technology.

[1761] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[1762] This invention is a system for recognizing a child's voice and providing appropriate information. The system includes a voice recognition means, a means for accepting a question by voice, a means for inferring the intent of the question, a means for providing information, a means for adjusting the tone of the information, a means for providing information in the voice of a specific character or person, a means for converting voice data into text, a means for analyzing the text to understand the subject or purpose of the question, a means for extracting appropriate information from a knowledge base, a means for formatting the extracted information into a form that is easy for a child to understand, and a means for converting the formatted information into voice.

[1763] Hardware and Software Used

[1764] The server uses general voice recognition software, such as Google Cloud Speech-to-Text API, to convert the voice data into text.

[1765] To infer the intent of a question, a natural language processing engine is used, specifically a generative AI model such as OpenAI's GPT-4, which analyzes the text to understand the subject and purpose of the question.

[1766] To provide this information, a means is needed to extract the appropriate information from a knowledge base, which could be Wikipedia or a specific educational database.

[1767] Adjusting the tone of the information requires a way to shape it according to the child's age and level of understanding, using simple vocabulary and short sentences.

[1768] To provide information in the voice of a specific character or person, a voice synthesis engine is used, specifically, voice synthesis software such as Amazon Polly or Google Cloud Text-to-Speech.

[1769] Examples

[1770] When a user (child) says "Tell me about dinosaurs," the following process takes place:

[1771] 1. The device records the user's voice and sends it to the server.

[1772] 2. The server converts the speech to text using the Google Cloud Speech-to-Text API.

[1773] 3. The server passes the converted text, “Tell me about dinosaurs,” to GPT-4 and analyzes the intent of the question.

[1774] 4. The server extracts information about dinosaurs from Wikipedia and formats the information in a way that is easy for children to understand.

[1775] 5. The server uses Amazon Polly to generate a speech in the voice of the child's favorite character saying, "Dinosaurs were creatures that appeared on Earth approximately 230 million years ago."

[1776] 6. The terminal plays the generated audio and provides information to the user.

[1777] Examples of prompt statements

[1778] In response to the question "Tell me about dinosaurs," the following prompt sentence is input to the generative AI model.

[1779] User Question: Tell me about dinosaurs

[1780] Prompt for generative AI model: Give a simple, child-friendly description of dinosaurs. Include the type, era, characteristics, etc. of the dinosaur.

[1781] Using this prompt, the generative AI model generates text to provide appropriate information to children.

[1782] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1783] Step 1:

[1784] The terminal records the user's voice and transmits it to the server.

[1785] Input: User's voice

[1786] Output: Audio data

[1787] Specific operation: When a user says, "Tell me about dinosaurs," the device's microphone captures the audio and sends the audio data to the server.

[1788] Step 2:

[1789] The server receives the voice data and converts it to text using speech recognition means.

[1790] Input: Audio data

[1791] Output: Text data

[1792] What happens: The server uses the Google Cloud Speech-to-Text API to parse the audio data and convert it into text, such as "Tell me about dinosaurs."

[1793] Step 3:

[1794] The server passes the text data to a natural language processing engine to analyze the intent of the question.

[1795] Input: Text data

[1796] Output: Question intent

[1797] Specific operation: The server uses GPT-4 to analyze the text data and identify the user’s intent to learn about dinosaurs.

[1798] Step 4:

[1799] The server extracts appropriate information from a knowledge base based on the intent of the question.

[1800] Input: Question intent

[1801] Output: Extracted information

[1802] Specific operation: The server searches for information about dinosaurs from knowledge bases such as Wikipedia, and extracts the information that "dinosaurs are creatures that appeared on Earth approximately 230 million years ago, and include species such as Tyrannosaurus and Triceratops."

[1803] Step 5:

[1804] The server formats the extracted information in a way that is easy for children to understand.

[1805] Input: Extracted information

[1806] Output: Formatted information

[1807] Specific operation: The server converts the extracted information into simple vocabulary and short sentences, formatting it in the form of "Dinosaurs are large creatures that lived on Earth a long time ago. There were species called Tyrannosaurus and Triceratops."

[1808] Step 6:

[1809] The server passes the formatted information to a speech synthesis engine, which generates speech in the voice of a particular character or person.

[1810] Input: Formatted information

[1811] Output: Audio data

[1812] What it does: The server uses Amazon Polly to generate a speech sequence in the voice of a child's favorite character saying, "Dinosaurs are big creatures that lived on Earth a long time ago. There were species called Tyrannosaurus and Triceratops."

[1813] Step 7:

[1814] The terminal receives the generated voice data and plays it back to the user.

[1815] Input: Audio data

[1816] Output: Played audio

[1817] Specific operation: The terminal plays the audio data received from the server, allowing the user to listen to information about dinosaurs in the voice of their favorite character.

[1818] (Application example 1)

[1819] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal".

[1820] Conventional voice recognition systems are primarily targeted at adult voices, and have the problem of difficulty in accurately recognizing children's voices. It is also difficult to properly understand the content of a child's question and provide information in a format that is easy for children to understand. Furthermore, there is a lack of means to increase children's motivation to learn by providing information in the voices of characters that interest them. To solve these problems, a system is needed that can accurately recognize children's voices, properly analyze the intent of the question, and provide information in a format that is easy for children to understand.

[1821] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1822] In this invention, the server includes a voice recognition means, a means for accepting a question by voice, a means for guessing the intention of the question, a means for providing information based on the guessed intention, a means for adjusting the tone of the information to be provided, a means for providing information in the voice of a specific character or person, a means for providing information in a form that is easy for a child to understand, a means for providing information in the voice of a character that the child likes, and a means including an application to be installed on the smart device. This makes it possible to accurately recognize the child's voice, appropriately analyze the intention of the question, and provide information in a form that is easy for a child to understand. Also, by providing information in the voice of a character that the child is interested in, it is possible to increase the child's motivation to learn.

[1823] "Speech recognition means" refers to technology for converting speech into text.

[1824] The "means for accepting questions by voice" is an interface that enables a user to input questions by voice.

[1825] "Means for inferring the intent of a question" refers to technology that analyzes the subject and purpose of a question from input text.

[1826] The "means for providing information based on an inferred intent" is a technology for providing appropriate information according to the intent of the analyzed question.

[1827] "Means for adjusting the tone of the information provided" refers to technology that adjusts the way information is provided depending on the user's age and level of understanding.

[1828] "Means of providing information in the voice of a specific character or person" refers to technology that provides information by imitating the voice of a character or person that a user likes.

[1829] "Means of providing information in a form that children can easily understand" refers to techniques that provide information in a concise and easy-to-understand manner so that children can easily understand it.

[1830] The "means for providing information in the voice of a character that a child likes" is a technology for providing information using the voice of a character that is of interest to children.

[1831] "Means including an application installed on a smart device" refers to a software application installed on a device such as a smartphone or tablet.

[1832] In order to implement this invention, means including a voice recognition means, a means for accepting a question by voice, a means for inferring the intent of the question, a means for providing information based on the inferred intent, a means for adjusting the tone of the information to be provided, a means for providing information in the voice of a particular character or person, a means for providing information in a form that is easy for children to understand, a means for providing information in the voice of a character that children like, and an application to be installed on a smart device are used.

[1833] Hardware and Software Used

[1834] The server uses the following hardware and software:

[1835] Hardware: Smartphones, tablets

[1836] software:

[1837] Speech recognition engine (Google Cloud Speech-to-Text)

[1838] Natural language processing engine (Google Cloud Natural Language API)

[1839] Knowledge Base (Wolfram Alpha API)

[1840] Speech synthesis engine (Google Cloud Text-to-Speech)

[1841] Processing flow

[1842] 1. Speech recognition: The user speaks a question into their smart device. The speech recognition engine converts this speech into text.

[1843] 2. Natural language processing: The server analyzes the text obtained from the speech recognition engine using a natural language processing engine to infer the intent of the question.

[1844] 3. Information provision: The server extracts appropriate information from a knowledge base based on the inferred intent.

[1845] 4. Tone Adjustment: The server adjusts the tone of the information depending on the user's age and level of understanding.

[1846] 5. Voice synthesis: The server uses a voice synthesis engine to provide information in the voice of the child's favorite character.

[1847] Examples

[1848] For example, if a child asks a smart device, "Tell me about dinosaurs," a speech recognition engine converts this speech into text. Next, a natural language processing engine analyzes this text and infers that the intent of the question is, "I want to know information about dinosaurs." The server extracts information about dinosaurs from a k...

Claims

1. A voice recognition means including a voice recognition engine for recognizing voice; A reception means for receiving questions from children; A conversion means for converting a question from a child into a text format by the voice recognition engine; A guessing means for guessing the intent of a question from a child's question converted into a text format using a natural language processing engine; A generating means for generating a prompt sentence that instructs the user to obtain an appropriate answer to the inferred question; An acquisition means for acquiring an answer to the inferred intent of the question using the generated prompt sentence and the generation AI model; A means for converting the obtained answers into voice information using a voice synthesis engine and providing the answers in a form that is easy for children to understand; A system including:

2. The providing means provides the answer acquired by the acquiring means in a form that is easy for the child to understand by converting the answer into a voice of a specific character or person. The system of claim 1 .

3. and a recognition means for recognizing an emotional state of the child by the emotion engine. the providing means provides the answer acquired by the acquiring means by using a voice having a tone corresponding to the emotional state of the child recognized by the recognizing means. The system of claim 1 .

Images