System

Abstract

A system is provided.SOLUTION: A system, comprising: means for receiving a request from a user in text or voice; means for parsing the request and generating relevant information using a generative model; means for converting the generated information into audio using a speech synthesis engine; and means for transmitting the generated audio to the user's device for playback.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In today's world, there is a vast amount of information available on the Internet, but most people only search for information that interests them or that they have questions about, limiting their opportunities to learn a wide range of knowledge and culture. Furthermore, in today's busy society, there is a demand for efficient ways to obtain information. Given this background, it is necessary to enable users to learn a wide range of knowledge and culture without having to actively search for information. [Means for solving the problem]

[0005] The present invention solves the above-mentioned problems by providing a system including: means for receiving a request from a user via text or voice; means for analyzing the request and generating related information using a generative model; means for converting the generated information into voice data using a voice synthesis engine; and means for transmitting the generated voice data to the user's terminal and playing it back. In particular, the generated information includes a function for including information for answering additional questions based on the user's request, and a function for adjusting the length of content to match a time specified by the user, allowing users to efficiently acquire a wide range of knowledge.

[0006] "User" means an individual or end-user who uses the System and submits a Request.

[0007] A "Request" is a text or voice-input inquiry about information or a topic that a user wishes to learn about.

[0008] "Generative model" refers to an algorithm or system that uses artificial intelligence techniques to generate information based on user requests.

[0009] "TTS (Text-to-Speech) Engine" refers to technology or software that converts text into natural, fluent speech.

[0010] "Audio Data" refers to audio-format data files or streams generated by a speech synthesis engine.

[0011] "Terminal" refers to the device (e.g., smartphone, tablet, PC, etc.) through which a User accesses the System, sends a request, and receives generated voice data.

[0012] "Server" refers to a computer system or network service that receives requests from users, operates a generative model and a speech synthesis engine to generate information, and transmits speech data.

[0013] "Receiving" refers to the process by which the server captures and reads the request data sent from the user's terminal.

[0014] "Parsing" refers to the process of using a generative model to interpret the user's request and extract and generate the necessary information.

[0015] "Generating information" refers to the generative model automatically creating relevant knowledge and data in response to a user request.

[0016] "Transmission" refers to the process by which the server sends the generated audio data to the user's terminal over the communications network.

[0017] "Playback" refers to the process of making the audio data received by the user's device audible to the user through an audio output device (e.g., speaker, headphones, etc.). [Brief explanation of the drawings]

[0018] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8]FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0019] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0020] First, the terms used in the following description will be explained.

[0021] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0022] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0023] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0024] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0025] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0026] [First embodiment]

[0027] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0028] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0029] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0030] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0031] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0032] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0033] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0034] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0035] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0036] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0037] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0038] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0039] The present invention is a system that allows users to easily learn a wide range of knowledge and culture. In a specific embodiment, the system analyzes a user's request, generates information based on that request, and provides it as a voice using a speech synthesis engine.

[0040] Specific processing of the program

[0041] Accepting requests from users

[0042] 1. User Input

[0043] Users launch the "Radio of Knowledge" application on their device and input a request for information or a topic they want to hear about, for example, "Tell me about the formation of black holes."

[0044] 2. Terminal Processing

[0045] The device receives the user's input and prepares the request to send to the server. In the case of voice input, the device converts the speech to text.

[0046] Parsing the request

[0047] 1. Server Reception

[0048] The server receives the request content sent from the terminal.

[0049] 2. Launching the Generative Model

[0050] The server passes the request to a generative model, which uses natural language processing to analyze the request, for example, extracting detailed information about the topic "black hole formation."

[0051] Information generation

[0052] 1. Information generation within the server

[0053] The generative model generates information appropriate to the request based on the analysis, such as text like "Black holes are formed when an extremely massive star collapses under its own gravity..."

[0054] Text-to-Speech (TTS) to translate content into audio

[0055] 1. Preparing the text

[0056] The server prepares the generated text to be input to the speech synthesis engine.

[0057] 2. Using a TTS engine

[0058] The server sends the text to a speech synthesis engine, which generates natural, fluent speech, adjusting the tone and speed of the speech during the process.

[0059] 3. Generating audio data

[0060] The speech synthesis engine converts the text into speech data, and the generated speech data is returned to the server.

[0061] Content Delivery

[0062] 1. Sending audio data

[0063] The server sends the generated audio data to the user's device.

[0064] 2. Receiving the terminal

[0065] The terminal receives the voice data transmitted from the server.

[0066] 3. Audio playback

[0067] The device plays the audio data, allowing the user to hear it.

[0068] Specific examples

[0069] For example, if a user requests, "Teach me the basic principles of quantum computing," the following process occurs:

[0070] 1. Accepting user requests

[0071] A user types into a terminal, "Tell me about the basic principles of quantum computing."

[0072] 2. Parsing the Request

[0073] The server receives the request and inputs it into the generative model, which analyzes and generates information about the fundamental principles of quantum computing.

[0074] 3. Information Generation

[0075] The generative model generates information such as "quantum computers use qubits to have greater computing power than conventional computers."

[0076] 4. Speech Synthesis

[0077] The server sends the generated text to a speech synthesis engine to generate voice data.

[0078] 5. Content Delivery

[0079] The server sends the audio data to the user's device, which then plays the audio.

[0080] This allows users to easily acquire knowledge and gain a wide range of knowledge and culture without having to actively search for it.

[0081] The processing flow will be explained below.

[0082] Step 1:

[0083] The user starts up the device and opens the "Radio of Knowledge" application. The user enters a request into an input field within the application, either by text or voice. For example, the user might enter "Tell me about the formation of black holes."

[0084] Step 2:

[0085] The device accepts user input. If voice input is selected, the device converts the speech to text using a speech recognition engine.

[0086] Step 3:

[0087] The device sends the request text data to the server via internet communication.

[0088] Step 4:

[0089] The server receives the request sent from the device and prepares to call the generative model API to analyze the received text data.

[0090] Step 5:

[0091] The server launches a generative model to analyze the request. The generative model extracts and generates relevant information based on the request. For example, the server instructs the model to generate information about the "formation of black holes."

[0092] Step 6:

[0093] The generative model generates information in response to the request, for example the generated text might be "Black holes are formed when a very massive star collapses under its own gravity..."

[0094] Step 7:

[0095] The server prepares the generated text to be input to the speech synthesis engine. The server passes the text data to the speech synthesis engine.

[0096] Step 8:

[0097] The text-to-speech engine receives the text data and converts it into speech data. The text-to-speech engine processes the text to produce natural, fluent speech, adjusting the tone and speed of the speech during this process.

[0098] Step 9:

[0099] The generated voice data is returned to the server, which then prepares the received voice data for transmission to the user's device.

[0100] Step 10:

[0101] The server sends the audio data to the user's device via internet communication.

[0102] Step 11:

[0103] The terminal receives the audio data sent from the server. The terminal prepares to play the received audio data.

[0104] Step 12:

[0105] The device plays audio data and allows the user to listen to the audio, allowing the user to hear information based on the request.

[0106] Step 13:

[0107] If a user has an additional question or would like more information during playback, they can enter a new request via text or voice, for example, "Please be more specific."

[0108] Step 14:

[0109] The device receives a new request and prepares to send it to the server. Steps 2 and after are repeated.

[0110] Example 1

[0111] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0112] With conventional technology, it was difficult for users to easily acquire a wide range of knowledge and culture. This was particularly difficult when the process of providing appropriate information in response to a request was complex and required advanced knowledge. Adjusting the quality and tone of the voice data to properly deliver the generated information to the user was also an issue.

[0113] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0114] In this invention, the server includes means for accepting requests from users via text or voice, means for analyzing the requests and generating relevant information using a generative model, means for converting the generated information into voice data using a voice synthesis engine, means for transmitting the generated voice data to the user's terminal and playing it back, means for passing the user's request to the generative model and extracting information, and means for converting the generated voice data by adjusting the tone and speed of the voice. This allows users to easily acquire a wide range of knowledge and culture, and makes it possible to provide appropriate information in response to requests as high-quality voice data.

[0115] "User" means a person or organization that uses the System to request information and receive information provided to it.

[0116] "Device" refers to an electronic device that a user operates to input requests or play audio data, including smartphones, tablets, and PCs.

[0117] A "server" is a computing system that receives a user's request, analyzes it, generates information, converts the generated information into audio data, and distributes it.

[0118] A "Request" is a question or request, entered by text or voice, about the information or topic you would like to know about.

[0119] A "generative model" is an artificial intelligence algorithm or system used to analyze a user request and generate relevant information.

[0120] A "speech synthesis engine" is a technology or function used to convert generated information into voice data. It is also known as a text-to-speech (TTS) engine.

[0121] "Audio data" means digital data converted from text to audio, which is audio played on a user's device.

[0122] "Analysis" is the process of understanding the user's request and extracting and processing the necessary information.

[0123] "Extraction" refers to the process by which the generative model extracts relevant information based on a user request.

[0124] "Tone and speed adjustment" refers to the process of appropriately changing the sound quality and speaking speed of the voice data generated by the speech synthesis engine.

[0125] This invention is a system that allows users to easily learn a wide range of knowledge and culture. It analyzes user requests to generate information, and then converts it into voice using a speech synthesis engine and provides it. This system is mainly composed of three elements: the user, the terminal, and the server.

[0126] First, a user launches the "Wisdom Radio" application on their device and inputs a request for information or a topic they would like to know about. For example, they can input requests by text or voice, such as "Tell me about the formation of black holes" or "Tell me about the basic principles of quantum computing." If input by voice, the device converts the voice into text.

[0127] The device then sends the request to the server. The server receives the request from the user's device and analyzes it. The server uses a generative AI model to extract and generate appropriate information based on the request. For example, the generative AI model might generate a prompt such as "Please provide information about the formation of black holes" or "Please explain the basic principles of quantum computing," and then extract the necessary information based on this prompt.

[0128] The generated information is stored in text format on the server. The server then inputs the generated text into a speech synthesis engine (e.g., a text-to-speech engine) and converts it into audio data. The speech synthesis engine then generates a natural, fluent voice, adjusting the tone and speed of the voice.

[0129] Once the voice data is generated, the server sends it to the user's device. The user's device receives the voice data and plays it back, allowing the user to listen to the requested information. This system allows users to easily acquire a wide range of knowledge and culture, and to receive the information they need by voice without having to actively search for it.

[0130] For example, if a user requests, "Tell me about the formation of black holes," the following process occurs: The user types "Tell me about the formation of black holes" into their device and sends the request to the server. The server inputs the prompt "Please provide information about the formation of black holes" into the generative AI model, and the generative model generates information such as "Black holes are formed when extremely massive stars collapse under their own gravity..." The server then sends the generated text to a speech synthesis engine, which generates natural, fluent audio data. Finally, this audio data is sent to the user's device, where it is played back, allowing the user to listen to information about the formation of black holes.

[0131] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0132] Step 1:

[0133] User Input

[0134] Users launch the "Radio of Knowledge" application on their own device and input requests for information or topics they want to know about via text or voice.

[0135] Input: User request (e.g., "Tell me about the formation of black holes")

[0136] Output: Request in text format (voice input is converted to text)

[0137] What happens: The user types a question into the app's text field and clicks the "Submit" button. If the user uses voice input, the device converts the speech into text and displays it on the screen.

[0138] Step 2:

[0139] Terminal request preparation and sending

[0140] The device receives the user's input and prepares the request to be sent to the server.

[0141] Input: Text data entered by the user (or, in the case of voice input, data already converted to text)

[0142] Output: The request formatted for sending to the server

[0143] Specific operation: The device creates an HTTP request, converts the request content into JSON format, and sends this data to the server when the send button is pressed.

[0144] Step 3:

[0145] Server request reception

[0146] The server receives the request content sent from the terminal.

[0147] Input: Request data sent from the terminal (JSON format)

[0148] Output: Parsed text request

[0149] What happens: The server receives the HTTP request, parses the content, and extracts the text request.

[0150] Step 4:

[0151] Analysis using generative models

[0152] The server passes the request to the generative model, which analyzes the request content using natural language processing.

[0153] Input: Text request (e.g., "Tell me about the formation of black holes")

[0154] Output: A prompt to be passed to the generative model (e.g., "Please provide information about the formation of black holes").

[0155] Specific operation: The server makes an API call to the generative AI model to generate a prompt sentence, which is then input into the generative AI model.

[0156] Step 5:

[0157] Information generation

[0158] Within the server, a generative model generates information appropriate to the request based on the analysis.

[0159] Input: Prompt statement (e.g., "Please provide information about the formation of black holes")

[0160] Output: Generated text information (e.g., "Black holes are formed when a very massive star collapses under its own gravity...")

[0161] Specific operation: The generative AI model generates information based on the prompt sentence and returns the text to the server.

[0162] Step 6:

[0163] Preparing text input for the speech synthesis engine

[0164] The server prepares the generated text to be input to the speech synthesis engine.

[0165] Input: Generated text information

[0166] Output: Text data converted into a format to send to the speech synthesis engine.

[0167] What happens: The server converts the generated text into JSON format and creates an appropriate request to the speech synthesis engine's API.

[0168] Step 7:

[0169] Voice data generation using a voice synthesis engine

[0170] The server sends the text to a speech synthesis engine, which generates natural, fluent speech.

[0171] Input: JSON format text data

[0172] Output: Audio data (e.g. audio file or audio stream)

[0173] How it works: The TTS engine receives text data, generates speech data, and returns the results to the server, adjusting the tone and speed of the speech.

[0174] Step 8:

[0175] Sending audio data

[0176] The server sends the generated audio data to the user's device.

[0177] Input: Generated audio data

[0178] Output: Audio data sent to the user's device

[0179] Specific operation: The server encodes the audio data as an HTTP response and sends it to the user's device.

[0180] Step 9:

[0181] Receiving audio data by the device

[0182] The terminal receives the voice data transmitted from the server.

[0183] Input: Audio data sent from the server

[0184] Output: Audio data in a playable format

[0185] What happens: The device decodes the audio data and converts it into a format that can be played within the application.

[0186] Step 10:

[0187] Device audio playback

[0188] The device plays the audio data, allowing the user to hear it.

[0189] Input: Decoded audio data

[0190] Output: Audio played through speakers or earphones

[0191] Specific behavior: The device's audio player plays the audio data and outputs it to the user as sound.

[0192] (Application example 1)

[0193] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0194] Traditionally, users have had limited means to easily obtain detailed information based on their interests, making it particularly difficult to obtain information while on the move or hands-free. Furthermore, the information generated is often perfunctory, making it difficult for users to learn effectively. Furthermore, if the information generated based on user requests is not properly converted into audio and delivered, learning and comprehension can be hindered.

[0195] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0196] In this invention, the server includes means for receiving a request from a user via text or voice, means for analyzing the request and generating relevant information using a generative model, and means for converting the generated information into voice data using a voice synthesis engine. This allows the user to easily obtain detailed information based on the request in a hands-free manner, and further improves learning and understanding by receiving the generated information in real time via voice. Furthermore, by adding means including a voice synthesis engine for converting the generated text information into voice data and means for inputting prompts into a generative AI model to generate appropriate information, it becomes possible to provide information in a more natural and fluent manner, further improving user convenience.

[0197] "Text or voice input means" means a device or software that provides an interface through which a user can input requests by text or voice.

[0198] "Generative model" refers to a machine learning model or AI system used to generate relevant information based on a user request.

[0199] "Speech synthesis engine" refers to software or hardware for converting text data into natural, fluent speech.

[0200] "Means for transmitting to and playing on a user's device" means a system or protocol that transmits the generated audio data to a user's device and enables the device to play the audio.

[0201] A "prompt" refers to an instruction or question entered into a generative AI model, which triggers the model to generate appropriate information.

[0202] "Generative AI model" refers to an artificial intelligence system that is trained to generate appropriate answers or information from prompts based on machine learning techniques.

[0203] To implement this invention, a user terminal, a server, a generative AI model, and a speech synthesis engine are required.

[0204] The user's device provides an interface that accepts requests input via text or voice. This interface can be realized using the smartphone's screen or voice recognition function. For example, if a user inputs "Tell me about the basic principles of quantum computing," this request will be received by the device.

[0205] The device sends the request to the server, which analyzes it and sends a prompt to the generative AI model. The prompt is a specific instruction or question, such as "User asked: Please tell me about the basic principles of quantum computing. Provide a detailed explanation."

[0206] The generative AI model generates appropriate information based on the received prompt. For example, GPT-3, a model based on machine learning technology, is used as the generative AI model. The generated text information is processed by the server and sent to the speech synthesis engine.

[0207] A speech synthesis engine (e.g., gTTS) converts the generated text into natural, fluent speech, providing information in a way that is easy for users to understand. The tone and speed of the speech can also be adjusted during this process.

[0208] Finally, the generated voice data is sent from the server to the user's device, which then plays it back in real time, allowing the user to easily obtain detailed information based on the request in a hands-free manner, improving learning and comprehension.

[0209] As described above, this invention enables users to easily obtain advanced knowledge and information, and makes it extremely convenient to obtain information while on the move or hands-free.

[0210] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0211] Step 1:

[0212] The user inputs a request via text or voice about a topic they want to learn about. An example of an input request might be, "Teach me about the basic principles of quantum computing." When the device receives the voice input, it uses speech recognition to convert it into text data. This converted text data is sent to the next processing step.

[0213] Input: User's text or voice request

[0214] Output: Text data of the request content

[0215] Step 2:

[0216] The device sends the request content as text data to the server. The server receives this data and begins analyzing the request content. Here, it uses natural language processing technology to analyze and understand the request content.

[0217] Input: Text data of the request content

[0218] Output: Prompt statement used for analysis

[0219] Step 3:

[0220] The server inputs a prompt to the generative AI model. For example, it generates a prompt such as "User asked: Please tell me about the basic principles of quantum computing. Provide a detailed explanation." This prompt is passed to the generative AI model, which then generates information corresponding to the request.

[0221] Input: prompt statement

[0222] Output: Detailed text information generated

[0223] Step 4:

[0224] The generative AI model returns the generated text information to the server, which then sends it to the speech synthesis engine, which generates natural, fluent speech based on the text information. The tone and speed of the speech can then be adjusted as needed.

[0225] Input: Generated detailed text information

[0226] Output: Generated audio data

[0227] Step 5:

[0228] The voice data generated by the speech synthesis engine is returned to the server. The server then sends this voice data to the user's device. The device receives it and plays the voice in real time, allowing the user to receive the requested information by voice.

[0229] Input: Generated audio data

[0230] Output: Audio played on the user's device

[0231] Step 6:

[0232] The device plays the audio data and the user can listen to it. This process allows users to easily obtain detailed information on a particular subject and allows for hands-free learning, especially when on the move.

[0233] Input: Audio data sent to the device

[0234] Output: User experience through the audio data played

[0235] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0236] The present invention is a system that allows users to easily learn a wide range of knowledge and culture. In a specific embodiment, the system analyzes a user's request, generates information based on that request, and provides it as voice using a voice synthesis engine. Furthermore, the present invention incorporates an emotion engine that recognizes the user's emotions, and adds functions for analyzing the request content and adjusting the generated information and voice.

[0237] Specific processing of the program

[0238] Accepting requests from users

[0239] 1. User Input

[0240] Users launch the "Radio of Knowledge" application on their device and input a request for information or a topic they want to hear about, for example, "Tell me about the formation of black holes."

[0241] 2. Terminal Processing

[0242] The device receives the user's input and prepares the request to send to the server. In the case of voice input, the device converts the speech to text.

[0243] Request analysis and emotion recognition

[0244] 1. Server Reception

[0245] The server receives the request content sent from the device and prepares to call the emotion engine and generative model APIs for analysis.

[0246] 2. Activating the Emotional Engine

[0247] The server recognizes emotions from the request content and the user's voice. The emotion engine analyzes the user's emotions (e.g., joy, excitement, anger, sadness, etc.) and passes the results to the generative model.

[0248] 3. Launching the Generative Model

[0249] The server passes the results of the emotion engine and the request content to the generative model, which then analyzes the request content using natural language processing. For example, it extracts detailed information on the topic "formation of black holes."

[0250] Information generation

[0251] 1. Information generation within the server

[0252] The generative model generates information appropriate to the request based on the analysis, such as text like "Black holes are formed when an extremely massive star collapses under its own gravity..."

[0253] 2. Emotion-Based Regulation

[0254] The server adjusts the generated information based on the user's emotions, for example adding detailed technical explanations if the user is excited.

[0255] Text-to-Speech (TTS) to translate content into audio

[0256] 1. Preparing the text

[0257] The server prepares the generated text to be input to the speech synthesis engine.

[0258] 2. Using a TTS engine

[0259] The server sends the text to a speech synthesis engine, which generates natural, fluent speech, adjusting the tone and speed of the speech according to the user's emotions.

[0260] 3. Generating audio data

[0261] The speech synthesis engine converts the text into speech data, and the generated speech data is returned to the server.

[0262] Content Delivery

[0263] 1. Sending audio data

[0264] The server sends the generated audio data to the user's device.

[0265] 2. Receiving the terminal

[0266] The terminal receives the voice data transmitted from the server.

[0267] 3. Audio playback

[0268] The device plays audio data and allows the user to listen to it, allowing the user to hear information based on the request.

[0269] Specific examples

[0270] For example, if a user requests, "Teach me the basic principles of quantum computing," the following process occurs:

[0271] 1. Accepting user requests

[0272] A user types into a terminal, "Tell me about the basic principles of quantum computing."

[0273] 2. Request analysis and emotion recognition

[0274] The server receives the request, analyzes the user's emotions using the emotion engine, and inputs the request content and the resulting emotions into the generative model.

[0275] 3. Information generation and coordination

[0276] The generative model generates information such as "quantum computers use qubits to have greater computing power than classical computers" and adjusts the information as needed based on the results of the emotion engine.

[0277] 4. Speech Synthesis

[0278] The server sends the generated text to a speech synthesis engine to generate speech data, adjusting the tone and speed depending on the user's emotions.

[0279] 5. Content Delivery

[0280] The server sends the audio data to the user's device, which then plays the audio.

[0281] This allows users to easily acquire knowledge and gain a wide range of knowledge and culture without actively searching. In addition, the emotion engine makes it possible to provide information tailored to each individual user.

[0282] The processing flow will be explained below.

[0283] Step 1:

[0284] The user starts up the device and opens the "Radio of Knowledge" application. The user enters a request into an input field within the application, either by text or voice. For example, the user might enter "Tell me about the formation of black holes."

[0285] Step 2:

[0286] The device accepts user input. If voice input is selected, the device converts the speech to text using a speech recognition engine.

[0287] Step 3:

[0288] The device sends the request text data to the server via internet communication.

[0289] Step 4:

[0290] The server receives the request sent from the device and prepares to call the APIs of the generative model and emotion engine to analyze the received text data.

[0291] Step 5:

[0292] The server starts the emotion engine and recognizes emotions from the received request and the user's voice (in the case of voice input). The emotion engine analyzes the user's emotions (e.g., joy, excitement, anger, sadness, etc.) and passes the results to the generative model.

[0293] Step 6:

[0294] The server launches the generative model and inputs the results of the emotion engine and the request content into the model. The generative model analyzes the request content using natural language processing, extracts and generates relevant information. For example, it generates detailed information on the topic of "black hole formation."

[0295] Step 7:

[0296] The generative model generates information in response to the request, for example the generated text might be "Black holes are formed when a very massive star collapses under its own gravity..."

[0297] Step 8:

[0298] The server uses an emotion engine to tailor the generated text based on the user's emotions, for example, adding more technical details if the user is excited, or a simpler explanation if the user is calm.

[0299] Step 9:

[0300] The server prepares the emotion-adjusted text to be input to the text-to-speech engine (TTS). The server passes the text data to the text-to-speech engine.

[0301] Step 10:

[0302] The text-to-speech engine receives the text data and converts it into speech data. The text-to-speech engine converts the text into natural, fluent speech, adjusting the tone and speed of the speech based on the results of the emotion engine.

[0303] Step 11:

[0304] The generated voice data is returned to the server, which then prepares the received voice data for transmission to the user's device.

[0305] Step 12:

[0306] The server sends the audio data to the user's device via internet communication.

[0307] Step 13:

[0308] The terminal receives the audio data sent from the server, and prepares to play the received audio data for the user.

[0309] Step 14:

[0310] The device plays audio data and allows the user to listen to the audio, allowing the user to hear information based on the request.

[0311] Step 15:

[0312] If a user has an additional question or would like more information during playback, they can enter a new request via text or voice, for example, "Please be more specific."

[0313] Step 16:

[0314] The device receives a new request and prepares to send it to the server. Steps 2 and after are repeated.

[0315] This allows users to easily receive information that is adaptive to their emotions, and to acquire knowledge and culture that meets their individual needs.

[0316] Example 2

[0317] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0318] In today's information society, users seek efficient and flexible information gathering in their busy daily lives. However, conventional information provision systems lack the ability to customize information according to the user's emotions and circumstances, making improving the user experience a challenge. Furthermore, even when providing information via voice, the lack of natural dialogue based on the user's emotions and requests can easily lead to unsatisfactory experiences.

[0319] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0320] In this invention, the server includes means for receiving a request from a user in the form of text or voice, means for analyzing the request and generating related information using a generative model, means for converting the generated information into voice data using a voice synthesis engine, means for transmitting the generated voice data to the user's terminal and playing it back, and means for recognizing the user's emotions and adjusting the generated information and voice based on the emotions. This makes it possible to provide information according to the user's emotions and request content, thereby realizing a personalized, high-quality user experience.

[0321] "User" means any person who uses the System to request information or content.

[0322] A "request" is a request entered by a user via text or voice about the information or topic they wish to know about.

[0323] "Text" refers to information entered in written form.

[0324] "Voice" refers to what the user says.

[0325] "Means" refers to a method or device for achieving a particular purpose.

[0326] "Analysis" refers to the process of understanding a user's request and extracting the appropriate information.

[0327] A "generative model" refers to an artificial intelligence algorithm that generates information based on a user's request.

[0328] "Information" refers to knowledge or data provided in response to a User's request.

[0329] A "speech synthesis engine" refers to software or a system that converts text into natural-sounding speech.

[0330] "Audio data" refers to data in audio format generated by a speech synthesis engine.

[0331] "Terminal" refers to a device used by a user, such as a computer, smartphone, or tablet.

[0332] "Emotion" refers to the user's psychological state, including states such as joy, excitement, anger, and sadness.

[0333] "Emotion recognition" refers to the process of determining a user's psychological state from their request or voice.

[0334] "Adjustment" refers to changing the tone of the generated information or voice to match the user's emotions.

[0335] This invention is an information provision system that allows users to easily learn a wide range of knowledge and culture. It has the function of generating related information based on a request input by the user and providing it as voice. It also recognizes the user's emotions and adjusts the generated information and voice. This system will be specifically described in the following embodiment.

[0336] Accepting requests from users

[0337] Users launch the "Wisdom Radio" application using a device such as a smartphone or PC and request the information or topic they are looking for. For example, they can type, "Tell me about the formation of black holes." The device receives the user's input and prepares it to be sent to the server. If voice input is used, the device uses a microphone to capture voice and converts it into text using a speech recognition API.

[0338] Request analysis and emotion recognition

[0339] The server receives requests sent from the device and prepares to call the APIs of the emotion engine and generative model for analysis. The server recognizes emotions from the request content and the user's voice. The emotion engine analyzes the user's emotions (happiness, excitement, anger, sadness, etc.) and passes the results to the generative model.

[0340] Information generation

[0341] The server passes the results of the emotion engine and the request content to the generative model, which analyzes the request using natural language processing. The generative model generates information appropriate to the request content. For example, it generates detailed information such as, "Black holes are formed when extremely massive stars collapse under their own gravity..." The server adjusts the generated information based on the user's emotions, adding detailed technical information if the user is excited, for example.

[0342] Text-to-Speech (TTS) to translate content into audio

[0343] The server prepares the generated text for input to a speech synthesis engine. The server then sends the text to the speech synthesis engine, which generates natural-sounding speech. During this process, the tone and speed of the speech are also adjusted according to the user's emotions. The speech synthesis engine then converts the text into speech data, which is then sent back to the server.

[0344] Content Delivery

[0345] The server sends the generated voice data to the user's terminal. The terminal receives the voice data sent from the server. Finally, the terminal plays the voice data so that the user can listen to the content. This allows the user to obtain information based on the request audibly.

[0346] Specific hardware and software names to be used

[0347] Hardware: Smartphones, servers (cloud services)

[0348] Software: Intelligent Radio application, speech synthesis engine (e.g., Google Cloud Text-to-Speech, Amazon Polly), generative model (e.g., OpenAI GPT-3), emotion engine (e.g., Microsoft Azure Cognitive Services)

[0349] Examples of specific examples and prompts

[0350] For example, if a user requests, "Tell me about the basic principles of quantum computing," the following process takes place: The user types, "Tell me about the basic principles of quantum computing," into their device. The server receives the request and analyzes the user's emotions using an emotion engine. The request and the resulting emotion are input into a generative model, which then generates information such as, "Quantum computers use qubits to have greater computing power than conventional computers." The information is adjusted based on the emotion engine's results, and the server sends the generated text to a speech synthesis engine, which generates audio data. The tone and speed are also adjusted according to the user's emotions. Finally, the server sends the audio data to the user's device, which then plays it back.

[0351] Example prompt sentence:

[0352] A user requests, "Tell me about the basic principles of quantum computing."

[0353] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0354] Step 1:

[0355] A user uses a device to launch the "Radio of Knowledge" application and enters a request for information. The request can be text or voice. For example, they could enter, "Tell me about the basic principles of quantum computing."

[0356] Input: User request (text or voice)

[0357] Output: Text or audio data received by the device

[0358] Specific behavior:

[0359] The user types text into the input area and presses the "Send" button.

[0360] For voice input, audio is recorded using a microphone and converted into text using a speech recognition API.

[0361] Step 2:

[0362] The device sends the user's request to the server, which receives the request and prepares to call the emotion engine and generative model APIs.

[0363] Input: Text or voice data received by the device

[0364] Output: Request data sent to the server

[0365] Specific behavior:

[0366] The terminal encodes the request data and sends an HTTP request to the server.

[0367] The server receives the HTTP request, decodes the data, and prepares it for analysis.

[0368] Step 3:

[0369] The server uses an emotion engine to recognize the user's emotions. It analyzes emotions (e.g., joy, excitement, anger, sadness, etc.) from voice or text data and passes the results to a generative model.

[0370] Input: User text or voice data

[0371] Output: Emotion data analyzed by the emotion engine

[0372] Specific behavior:

[0373] The server calls the emotion engine and passes the user's text or voice data as input.

[0374] The emotion engine analyzes the data, identifies the user's emotion, and returns that data.

[0375] Step 4:

[0376] The server uses the generative model to analyze the user's request and generate appropriate information. It also references the results of the emotion engine to create text based on the request.

[0377] Input: User request data and emotion data

[0378] Output: Generated text data

[0379] Specific behavior:

[0380] The server passes the request content and emotion data to the generative model and receives the generated text data.

[0381] The generative model generates text that matches the request, e.g., "Quantum computers use qubits to achieve greater computing power than classical computers..."

[0382] Step 5:

[0383] The server prepares the generated text for input to a speech synthesis engine, which converts the text data into natural-sounding speech data.

[0384] Input: Generated text data

[0385] Output: Generated audio data

[0386] Specific behavior:

[0387] The server passes the text data to a speech synthesis engine, such as Google Cloud Text-to-Speech or Amazon Polly.

[0388] A speech synthesis engine converts text data into speech, adjusting tone and speed based on emotional data.

[0389] Step 6:

[0390] The server sends the generated audio data to the user's device, which receives and plays the audio data.

[0391] Input: Generated audio data

[0392] Output: Audio data sent to the device

[0393] Specific behavior:

[0394] The server sends the audio data to the user's device via an HTTP request or WebSocket.

[0395] The device receives the audio data and plays the audio using its built-in audio playback function.

[0396] Examples of specific examples and prompts

[0397] For example, if a user requests, "Teach me about the basic principles of quantum computing," the following processing occurs: For example, a user requests, "Teach me about the basic principles of quantum computing."

[0398] (Application example 2)

[0399] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0400] Today's users demand immediate and appropriate provision of a wide range of security-related information, but achieving this requires a wide range of technologies. In particular, it is necessary to provide appropriate countermeasures immediately in emergencies, and conventional methods have difficulty responding flexibly while taking into account the user's feelings. Furthermore, the lack of a function to instruct emergency responses means that prompt and appropriate information cannot be provided when users feel anxious.

[0401] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0402] In this invention, the server includes means for receiving a request from a user in text or voice, means for analyzing the request and generating related information using a generative model, means for adjusting the generated information through means for recognizing the user's emotion, means for converting the generated information into voice data using a voice synthesis engine, means for transmitting the generated voice data to the user's terminal and playing it back, and means for instructing an emergency response based on the generated information. This enables appropriate and individual responses to user requests, and can provide a quick and accurate response, particularly in emergencies.

[0403] "User" means any individual or entity that uses this System to request information or services.

[0404] A "request" refers to information or a question a user enters into a system.

[0405] "Text" refers to information composed of characters.

[0406] "Speech" refers to sound information emitted by the human voice.

[0407] "Means" refers to a method or device for achieving a specific purpose.

[0408] "Analysis" refers to understanding and breaking down the content of input information and carrying out appropriate processing.

[0409] A "generative model" refers to an algorithm or program that uses artificial intelligence (AI) to generate text or speech.

[0410] "Relevant information" refers to appropriate data and knowledge provided based on the user's request.

[0411] "Means for recognizing emotions" refers to technologies or devices for analyzing and determining a user's emotional state.

[0412] A "speech synthesis engine" refers to software or a device that converts text information into voice data.

[0413] "Audio Data" means audio information stored in digital format.

[0414] "Terminal" means a device used by a User to access the System and receive information.

[0415] "Playback" refers to converting audio data into sound and letting the user hear it.

[0416] "Means for providing emergency response instructions" refers to technologies and devices that provide users with appropriate guidelines for action in an emergency.

[0417] The present invention relates to a personalized security assistant system that provides appropriate information to a user when the user inputs security-related requests by text or voice. Specifically, the system is configured as follows.

[0418] 1. Accepting requests from users

[0419] Users use a device such as a smartphone to enter security questions or requests into the system by text or voice, such as "I'm hearing strange noises in my house. What should I do?"

[0420] 2. Request analysis and emotion recognition

[0421] The server analyzes the request received from the device. If the request is voice-activated, it converts it to text using Google Cloud Speech-to-Text. It then uses a generative AI model, such as OpenAI GPT-4, to analyze the request and generate an appropriate response. At the same time, it uses the Microsoft Azure Emotion API to recognize the user's emotions and adjusts the information accordingly.

[0422] 3. Information generation and coordination

[0423] Based on the content analyzed by the generative model, information appropriate to the request is generated. The generated information is adjusted according to the user's emotions through an emotion engine. For example, if the user is feeling anxious, specific and detailed measures will be provided.

[0424] 4. Audio conversion of content using speech synthesis

[0425] The adjusted information is then converted into audio data using Google Cloud Text-to-Speech, with the tone and speed of the audio being adjusted according to the user's emotions.

[0426] 5. Content Delivery and Response Options

[0427] The generated voice data is sent from the server to the user's device, where it is played back. It also provides a means to issue emergency response instructions if necessary, allowing the user to notify relevant authorities with a single touch in the event of an emergency.

[0428] Specific examples

[0429] For example, a user might enter a request like this:

[0430] I hear strange noises in my house, what should I do?

[0431] When this request is received, the following process takes place: The audio is converted to text using Google Cloud Speech-to-Text and input into GPT-4. The generated response is analyzed using the Azure Emotion API, and if anxiety is recognized, countermeasure information is provided as a priority. The adjusted text is converted into audio data using Google Cloud Text-to-Speech and sent to the user's device. By receiving the information via audio, the user can quickly take appropriate action.

[0432] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0433] Step 1:

[0434] The user launches the application on a device such as a smartphone and enters a security request.

[0435] Input: The user's voice or text request.

[0436] What it does: The device uses a microphone to record audio or provides a GUI for entering text.

[0437] Output: Audio or text data.

[0438] Step 2:

[0439] The device converts the speech into text (in the case of voice input).

[0440] Input: Audio data.

[0441] How it works: Converts audio data into text using Google Cloud Speech-to-Text.

[0442] Output: The audio data converted into text data.

[0443] Step 3:

[0444] The terminal transmits the text data to the server.

[0445] Input: Text data converted from speech or entered directly.

[0446] Operation: The device sends text data to the server via an HTTP request.

[0447] Output: The text data sent to the server.

[0448] Step 4:

[0449] The server analyzes the request.

[0450] Input: User request text data.

[0451] How it works: The server inputs text data as prompts into the OpenAI GPT-4 model, which then generates an appropriate response using the generative AI model.

[0452] Output: The generated response text.

[0453] Step 5:

[0454] The server analyzes the user's emotions.

[0455] Input: The user's request text data and voice data (if any).

[0456] How it works: The server uses the Microsoft Azure Emotion API to perform emotion analysis and determine the user's emotional state.

[0457] Output: Parsed emotion data.

[0458] Step 6:

[0459] The server adjusts the generated response text based on the emotion data.

[0460] Input: Generated response text and sentiment data.

[0461] How it works: The server adjusts the response text based on the emotion data, such as adding more information if anxiety is detected.

[0462] Output: The adjusted response text.

[0463] Step 7:

[0464] The server converts the adjusted text into audio data.

[0465] Input: The tailored response text.

[0466] How it works: Uses Google Cloud Text-to-Speech to convert text data into audio data.

[0467] Output: The generated audio data.

[0468] Step 8:

[0469] The server sends the generated audio data to the user's device.

[0470] Input: The generated audio data.

[0471] How it works: The server sends the audio data to the user's device as an HTTP response.

[0472] Output: The audio data sent to the user's device.

[0473] Step 9:

[0474] The device plays the audio data and the user listens to the information.

[0475] Input: Audio data sent from the server.

[0476] Behavior: The device uses its audio player function to play audio data and provides audio to the user.

[0477] Output: The user hears a voice and gets the appropriate security information.

[0478] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0479] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0480] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0481] [Second embodiment]

[0482] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0483] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0484] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0485] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0486] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0487] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0488] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0489] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0490] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0491] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0492] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0493] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0494] The present invention is a system that allows users to easily learn a wide range of knowledge and culture. In a specific embodiment, the system analyzes a user's request, generates information based on that request, and provides it as a voice using a speech synthesis engine.

[0495] Specific processing of the program

[0496] Accepting requests from users

[0497] 1. User Input

[0498] Users launch the "Radio of Knowledge" application on their device and input a request for information or a topic they want to hear about, for example, "Tell me about the formation of black holes."

[0499] 2. Terminal Processing

[0500] The device receives the user's input and prepares the request to send to the server. In the case of voice input, the device converts the speech to text.

[0501] Parsing the request

[0502] 1. Server Reception

[0503] The server receives the request content sent from the terminal.

[0504] 2. Launching the Generative Model

[0505] The server passes the request to a generative model, which uses natural language processing to analyze the request, for example, extracting detailed information about the topic "black hole formation."

[0506] Information generation

[0507] 1. Information generation within the server

[0508] The generative model generates information appropriate to the request based on the analysis, such as text like "Black holes are formed when an extremely massive star collapses under its own gravity..."

[0509] Text-to-Speech (TTS) to translate content into audio

[0510] 1. Preparing the text

[0511] The server prepares the generated text to be input to the speech synthesis engine.

[0512] 2. Using a TTS engine

[0513] The server sends the text to a speech synthesis engine, which generates natural, fluent speech, adjusting the tone and speed of the speech during the process.

[0514] 3. Generating audio data

[0515] The speech synthesis engine converts the text into speech data, and the generated speech data is returned to the server.

[0516] Content Delivery

[0517] 1. Sending audio data

[0518] The server sends the generated audio data to the user's device.

[0519] 2. Receiving the terminal

[0520] The terminal receives the voice data transmitted from the server.

[0521] 3. Audio playback

[0522] The device plays the audio data, allowing the user to hear it.

[0523] Specific examples

[0524] For example, if a user requests, "Teach me the basic principles of quantum computing," the following process occurs:

[0525] 1. Accepting user requests

[0526] A user types into a terminal, "Tell me about the basic principles of quantum computing."

[0527] 2. Parsing the Request

[0528] The server receives the request and inputs it into the generative model, which analyzes and generates information about the fundamental principles of quantum computing.

[0529] 3. Information Generation

[0530] The generative model generates information such as "quantum computers use qubits to have greater computing power than conventional computers."

[0531] 4. Speech Synthesis

[0532] The server sends the generated text to a speech synthesis engine to generate voice data.

[0533] 5. Content Delivery

[0534] The server sends the audio data to the user's device, which then plays the audio.

[0535] This allows users to easily acquire knowledge and gain a wide range of knowledge and culture without having to actively search for it.

[0536] The processing flow will be explained below.

[0537] Step 1:

[0538] The user starts up the device and opens the "Radio of Knowledge" application. The user enters a request into an input field within the application, either by text or voice. For example, the user might enter "Tell me about the formation of black holes."

[0539] Step 2:

[0540] The device accepts user input. If voice input is selected, the device converts the speech to text using a speech recognition engine.

[0541] Step 3:

[0542] The device sends the request text data to the server via internet communication.

[0543] Step 4:

[0544] The server receives the request sent from the device and prepares to call the generative model API to analyze the received text data.

[0545] Step 5:

[0546] The server launches a generative model to analyze the request. The generative model extracts and generates relevant information based on the request. For example, the server instructs the model to generate information about the "formation of black holes."

[0547] Step 6:

[0548] The generative model generates information in response to the request, for example the generated text might be "Black holes are formed when a very massive star collapses under its own gravity..."

[0549] Step 7:

[0550] The server prepares the generated text to be input to the speech synthesis engine. The server passes the text data to the speech synthesis engine.

[0551] Step 8:

[0552] The text-to-speech engine receives the text data and converts it into speech data. The text-to-speech engine processes the text to produce natural, fluent speech, adjusting the tone and speed of the speech during this process.

[0553] Step 9:

[0554] The generated voice data is returned to the server, which then prepares the received voice data for transmission to the user's device.

[0555] Step 10:

[0556] The server sends the audio data to the user's device via internet communication.

[0557] Step 11:

[0558] The terminal receives the audio data sent from the server. The terminal prepares to play the received audio data.

[0559] Step 12:

[0560] The device plays audio data and allows the user to listen to the audio, allowing the user to hear information based on the request.

[0561] Step 13:

[0562] If a user has an additional question or would like more information during playback, they can enter a new request via text or voice, for example, "Please be more specific."

[0563] Step 14:

[0564] The device receives a new request and prepares to send it to the server. Steps 2 and after are repeated.

[0565] Example 1

[0566] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0567] With conventional technology, it was difficult for users to easily acquire a wide range of knowledge and culture. This was particularly difficult when the process of providing appropriate information in response to a request was complex and required advanced knowledge. Adjusting the quality and tone of the voice data to properly deliver the generated information to the user was also an issue.

[0568] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0569] In this invention, the server includes means for accepting requests from users via text or voice, means for analyzing the requests and generating relevant information using a generative model, means for converting the generated information into voice data using a voice synthesis engine, means for transmitting the generated voice data to the user's terminal and playing it back, means for passing the user's request to the generative model and extracting information, and means for converting the generated voice data by adjusting the tone and speed of the voice. This allows users to easily acquire a wide range of knowledge and culture, and makes it possible to provide appropriate information in response to requests as high-quality voice data.

[0570] "User" means a person or organization that uses the System to request information and receive information provided to it.

[0571] "Device" refers to an electronic device that a user operates to input requests or play audio data, including smartphones, tablets, and PCs.

[0572] A "server" is a computing system that receives a user's request, analyzes it, generates information, converts the generated information into audio data, and distributes it.

[0573] A "Request" is a question or request, entered by text or voice, about the information or topic you would like to know about.

[0574] A "generative model" is an artificial intelligence algorithm or system used to analyze a user request and generate relevant information.

[0575] A "speech synthesis engine" is a technology or function used to convert generated information into voice data. It is also known as a text-to-speech (TTS) engine.

[0576] "Audio data" means digital data converted from text to audio, which is audio played on a user's device.

[0577] "Analysis" is the process of understanding the user's request and extracting and processing the necessary information.

[0578] "Extraction" refers to the process by which the generative model extracts relevant information based on a user request.

[0579] "Tone and speed adjustment" refers to the process of appropriately changing the sound quality and speaking speed of the voice data generated by the speech synthesis engine.

[0580] This invention is a system that allows users to easily learn a wide range of knowledge and culture. It analyzes user requests to generate information, and then converts it into voice using a speech synthesis engine and provides it. This system is mainly composed of three elements: the user, the terminal, and the server.

[0581] First, a user launches the "Wisdom Radio" application on their device and inputs a request for information or a topic they would like to know about. For example, they can input requests by text or voice, such as "Tell me about the formation of black holes" or "Tell me about the basic principles of quantum computing." If input by voice, the device converts the voice into text.

[0582] The device then sends the request to the server. The server receives the request from the user's device and analyzes it. The server uses a generative AI model to extract and generate appropriate information based on the request. For example, the generative AI model might generate a prompt such as "Please provide information about the formation of black holes" or "Please explain the basic principles of quantum computing," and then extract the necessary information based on this prompt.

[0583] The generated information is stored in text format on the server. The server then inputs the generated text into a speech synthesis engine (e.g., a text-to-speech engine) and converts it into audio data. The speech synthesis engine then generates a natural, fluent voice, adjusting the tone and speed of the voice.

[0584] Once the voice data is generated, the server sends it to the user's device. The user's device receives the voice data and plays it back, allowing the user to listen to the requested information. This system allows users to easily acquire a wide range of knowledge and culture, and to receive the information they need by voice without having to actively search for it.

[0585] For example, if a user requests, "Tell me about the formation of black holes," the following process occurs: The user types "Tell me about the formation of black holes" into their device and sends the request to the server. The server inputs the prompt "Please provide information about the formation of black holes" into the generative AI model, and the generative model generates information such as "Black holes are formed when extremely massive stars collapse under their own gravity..." The server then sends the generated text to a speech synthesis engine, which generates natural, fluent audio data. Finally, this audio data is sent to the user's device, where it is played back, allowing the user to listen to information about the formation of black holes.

[0586] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0587] Step 1:

[0588] User Input

[0589] Users launch the "Radio of Knowledge" application on their own device and input requests for information or topics they want to know about via text or voice.

[0590] Input: User request (e.g., "Tell me about the formation of black holes")

[0591] Output: Request in text format (voice input is converted to text)

[0592] What happens: The user types a question into the app's text field and clicks the "Submit" button. If the user uses voice input, the device converts the speech into text and displays it on the screen.

[0593] Step 2:

[0594] Terminal request preparation and sending

[0595] The device receives the user's input and prepares the request to be sent to the server.

[0596] Input: Text data entered by the user (or, in the case of voice input, data already converted to text)

[0597] Output: The request formatted for sending to the server

[0598] Specific operation: The device creates an HTTP request, converts the request content into JSON format, and sends this data to the server when the send button is pressed.

[0599] Step 3:

[0600] Server request reception

[0601] The server receives the request content sent from the terminal.

[0602] Input: Request data sent from the terminal (JSON format)

[0603] Output: Parsed text request

[0604] What happens: The server receives the HTTP request, parses the content, and extracts the text request.

[0605] Step 4:

[0606] Analysis using generative models

[0607] The server passes the request to the generative model, which analyzes the request content using natural language processing.

[0608] Input: Text request (e.g., "Tell me about the formation of black holes")

[0609] Output: A prompt to be passed to the generative model (e.g., "Please provide information about the formation of black holes").

[0610] Specific operation: The server makes an API call to the generative AI model to generate a prompt sentence, which is then input into the generative AI model.

[0611] Step 5:

[0612] Information generation

[0613] Within the server, a generative model generates information appropriate to the request based on the analysis.

[0614] Input: Prompt statement (e.g., "Please provide information about the formation of black holes")

[0615] Output: Generated text information (e.g., "Black holes are formed when a very massive star collapses under its own gravity...")

[0616] Specific operation: The generative AI model generates information based on the prompt sentence and returns the text to the server.

[0617] Step 6:

[0618] Preparing text input for the speech synthesis engine

[0619] The server prepares the generated text to be input to the speech synthesis engine.

[0620] Input: Generated text information

[0621] Output: Text data converted into a format to send to the speech synthesis engine.

[0622] What happens: The server converts the generated text into JSON format and creates an appropriate request to the speech synthesis engine's API.

[0623] Step 7:

[0624] Voice data generation using a voice synthesis engine

[0625] The server sends the text to a speech synthesis engine, which generates natural, fluent speech.

[0626] Input: JSON format text data

[0627] Output: Audio data (e.g. audio file or audio stream)

[0628] How it works: The TTS engine receives text data, generates speech data, and returns the results to the server, adjusting the tone and speed of the speech.

[0629] Step 8:

[0630] Sending audio data

[0631] The server sends the generated audio data to the user's device.

[0632] Input: Generated audio data

[0633] Output: Audio data sent to the user's device

[0634] Specific operation: The server encodes the audio data as an HTTP response and sends it to the user's device.

[0635] Step 9:

[0636] Receiving audio data by the device

[0637] The terminal receives the voice data transmitted from the server.

[0638] Input: Audio data sent from the server

[0639] Output: Audio data in a playable format

[0640] What happens: The device decodes the audio data and converts it into a format that can be played within the application.

[0641] Step 10:

[0642] Device audio playback

[0643] The device plays the audio data, allowing the user to hear it.

[0644] Input: Decoded audio data

[0645] Output: Audio played through speakers or earphones

[0646] Specific behavior: The device's audio player plays the audio data and outputs it to the user as sound.

[0647] (Application example 1)

[0648] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0649] Traditionally, users have had limited means to easily obtain detailed information based on their interests, making it particularly difficult to obtain information while on the move or hands-free. Furthermore, the information generated is often perfunctory, making it difficult for users to learn effectively. Furthermore, if the information generated based on user requests is not properly converted into audio and delivered, learning and comprehension can be hindered.

[0650] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0651] In this invention, the server includes means for receiving a request from a user via text or voice, means for analyzing the request and generating relevant information using a generative model, and means for converting the generated information into voice data using a voice synthesis engine. This allows the user to easily obtain detailed information based on the request in a hands-free manner, and further improves learning and understanding by receiving the generated information in real time via voice. Furthermore, by adding means including a voice synthesis engine for converting the generated text information into voice data and means for inputting prompts into a generative AI model to generate appropriate information, it becomes possible to provide information in a more natural and fluent manner, further improving user convenience.

[0652] "Text or voice input means" means a device or software that provides an interface through which a user can input requests by text or voice.

[0653] "Generative model" refers to a machine learning model or AI system used to generate relevant information based on a user request.

[0654] "Speech synthesis engine" refers to software or hardware for converting text data into natural, fluent speech.

[0655] "Means for transmitting to and playing on a user's device" means a system or protocol that transmits the generated audio data to a user's device and enables the device to play the audio.

[0656] A "prompt" refers to an instruction or question entered into a generative AI model, which triggers the model to generate appropriate information.

[0657] "Generative AI model" refers to an artificial intelligence system that is trained to generate appropriate answers or information from prompts based on machine learning techniques.

[0658] To implement this invention, a user terminal, a server, a generative AI model, and a speech synthesis engine are required.

[0659] The user's device provides an interface that accepts requests input via text or voice. This interface can be realized using the smartphone's screen or voice recognition function. For example, if a user inputs "Tell me about the basic principles of quantum computing," this request will be received by the device.

[0660] The device sends the request to the server, which analyzes it and sends a prompt to the generative AI model. The prompt is a specific instruction or question, such as "User asked: Please tell me about the basic principles of quantum computing. Provide a detailed explanation."

[0661] The generative AI model generates appropriate information based on the received prompt. For example, GPT-3, a model based on machine learning technology, is used as the generative AI model. The generated text information is processed by the server and sent to the speech synthesis engine.

[0662] A speech synthesis engine (e.g., gTTS) converts the generated text into natural, fluent speech, providing information in a way that is easy for users to understand. The tone and speed of the speech can also be adjusted during this process.

[0663] Finally, the generated voice data is sent from the server to the user's device, which then plays it back in real time, allowing the user to easily obtain detailed information based on the request in a hands-free manner, improving learning and comprehension.

[0664] As described above, this invention enables users to easily obtain advanced knowledge and information, and makes it extremely convenient to obtain information while on the move or hands-free.

[0665] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0666] Step 1:

[0667] The user inputs a request via text or voice about a topic they want to learn about. An example of an input request might be, "Teach me about the basic principles of quantum computing." When the device receives the voice input, it uses speech recognition to convert it into text data. This converted text data is sent to the next processing step.

[0668] Input: User's text or voice request

[0669] Output: Text data of the request content

[0670] Step 2:

[0671] The device sends the request content as text data to the server. The server receives this data and begins analyzing the request content. Here, it uses natural language processing technology to analyze and understand the request content.

[0672] Input: Text data of the request content

[0673] Output: Prompt statement used for analysis

[0674] Step 3:

[0675] The server inputs a prompt to the generative AI model. For example, it generates a prompt such as "User asked: Please tell me about the basic principles of quantum computing. Provide a detailed explanation." This prompt is passed to the generative AI model, which then generates information corresponding to the request.

[0676] Input: prompt statement

[0677] Output: Detailed text information generated

[0678] Step 4:

[0679] The generative AI model returns the generated text information to the server, which then sends it to the speech synthesis engine, which generates natural, fluent speech based on the text information. The tone and speed of the speech can then be adjusted as needed.

[0680] Input: Generated detailed text information

[0681] Output: Generated audio data

[0682] Step 5:

[0683] The voice data generated by the speech synthesis engine is returned to the server. The server then sends this voice data to the user's device. The device receives it and plays the voice in real time, allowing the user to receive the requested information by voice.

[0684] Input: Generated audio data

[0685] Output: Audio played on the user's device

[0686] Step 6:

[0687] The device plays the audio data and the user can listen to it. This process allows users to easily obtain detailed information on a particular subject and allows for hands-free learning, especially when on the move.

[0688] Input: Audio data sent to the device

[0689] Output: User experience through the audio data played

[0690] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0691] The present invention is a system that allows users to easily learn a wide range of knowledge and culture. In a specific embodiment, the system analyzes a user's request, generates information based on that request, and provides it as voice using a voice synthesis engine. Furthermore, the present invention incorporates an emotion engine that recognizes the user's emotions, and adds functions for analyzing the request content and adjusting the generated information and voice.

[0692] Specific processing of the program

[0693] Accepting requests from users

[0694] 1. User Input

[0695] Users launch the "Radio of Knowledge" application on their device and input a request for information or a topic they want to hear about, for example, "Tell me about the formation of black holes."

[0696] 2. Terminal Processing

[0697] The device receives the user's input and prepares the request to send to the server. In the case of voice input, the device converts the speech to text.

[0698] Request analysis and emotion recognition

[0699] 1. Server Reception

[0700] The server receives the request content sent from the device and prepares to call the emotion engine and generative model APIs for analysis.

[0701] 2. Activating the Emotional Engine

[0702] The server recognizes emotions from the request content and the user's voice. The emotion engine analyzes the user's emotions (e.g., joy, excitement, anger, sadness, etc.) and passes the results to the generative model.

[0703] 3. Launching the Generative Model

[0704] The server passes the results of the emotion engine and the request content to the generative model, which then analyzes the request content using natural language processing. For example, it extracts detailed information on the topic "formation of black holes."

[0705] Information generation

[0706] 1. Information generation within the server

[0707] The generative model generates information appropriate to the request based on the analysis, such as text like "Black holes are formed when an extremely massive star collapses under its own gravity..."

[0708] 2. Emotion-Based Adjustment

[0709] The server adjusts the generated information based on the user's emotions, for example adding detailed technical explanations if the user is excited.

[0710] Text-to-Speech (TTS) to translate content into audio

[0711] 1. Preparing the text

[0712] The server prepares the generated text to be input to the speech synthesis engine.

[0713] 2. Using a TTS engine

[0714] The server sends the text to a speech synthesis engine, which generates natural, fluent speech, adjusting the tone and speed of the speech according to the user's emotions.

[0715] 3. Generating audio data

[0716] The speech synthesis engine converts the text into speech data, and the generated speech data is returned to the server.

[0717] Content Delivery

[0718] 1. Sending audio data

[0719] The server sends the generated audio data to the user's device.

[0720] 2. Receiving the terminal

[0721] The terminal receives the voice data transmitted from the server.

[0722] 3. Audio playback

[0723] The device plays audio data and allows the user to listen to it, allowing the user to hear information based on the request.

[0724] Specific examples

[0725] For example, if a user requests, "Teach me the basic principles of quantum computing," the following process occurs:

[0726] 1. Accepting user requests

[0727] A user types into a terminal, "Tell me about the basic principles of quantum computing."

[0728] 2. Request analysis and emotion recognition

[0729] The server receives the request, analyzes the user's emotions using the emotion engine, and inputs the request content and the resulting emotions into the generative model.

[0730] 3. Information generation and coordination

[0731] The generative model generates information such as "quantum computers use qubits to have greater computing power than classical computers" and adjusts the information as needed based on the results of the emotion engine.

[0732] 4. Speech Synthesis

[0733] The server sends the generated text to a speech synthesis engine to generate speech data, adjusting the tone and speed depending on the user's emotions.

[0734] 5. Content Delivery

[0735] The server sends the audio data to the user's device, which then plays the audio.

[0736] This allows users to easily acquire knowledge and gain a wide range of knowledge and culture without actively searching. In addition, the emotion engine makes it possible to provide information tailored to each individual user.

[0737] The processing flow will be explained below.

[0738] Step 1:

[0739] The user starts up the device and opens the "Radio of Knowledge" application. The user enters a request into an input field within the application, either by text or voice. For example, the user might enter "Tell me about the formation of black holes."

[0740] Step 2:

[0741] The device accepts user input. If voice input is selected, the device converts the speech to text using a speech recognition engine.

[0742] Step 3:

[0743] The device sends the request text data to the server via internet communication.

[0744] Step 4:

[0745] The server receives the request sent from the device and prepares to call the APIs of the generative model and emotion engine to analyze the received text data.

[0746] Step 5:

[0747] The server starts the emotion engine and recognizes emotions from the received request and the user's voice (in the case of voice input). The emotion engine analyzes the user's emotions (e.g., joy, excitement, anger, sadness, etc.) and passes the results to the generative model.

[0748] Step 6:

[0749] The server launches the generative model and inputs the results of the emotion engine and the request content into the model. The generative model analyzes the request content using natural language processing, extracts and generates relevant information. For example, it generates detailed information on the topic of "black hole formation."

[0750] Step 7:

[0751] The generative model generates information in response to the request, for example the generated text might be "Black holes are formed when a very massive star collapses under its own gravity..."

[0752] Step 8:

[0753] The server uses an emotion engine to tailor the generated text based on the user's emotions, for example, adding more technical details if the user is excited, or a simpler explanation if the user is calm.

[0754] Step 9:

[0755] The server prepares the emotion-adjusted text to be input to the text-to-speech engine (TTS). The server passes the text data to the text-to-speech engine.

[0756] Step 10:

[0757] The text-to-speech engine receives the text data and converts it into speech data. The text-to-speech engine converts the text into natural, fluent speech, adjusting the tone and speed of the speech based on the results of the emotion engine.

[0758] Step 11:

[0759] The generated voice data is returned to the server, which then prepares the received voice data for transmission to the user's device.

[0760] Step 12:

[0761] The server sends the audio data to the user's device via internet communication.

[0762] Step 13:

[0763] The terminal receives the audio data sent from the server, and prepares to play the received audio data for the user.

[0764] Step 14:

[0765] The device plays audio data and allows the user to listen to the audio, allowing the user to hear information based on the request.

[0766] Step 15:

[0767] If a user has an additional question or would like more information during playback, they can enter a new request via text or voice, for example, "Please be more specific."

[0768] Step 16:

[0769] The device receives a new request and prepares to send it to the server. Steps 2 and after are repeated.

[0770] This allows users to easily receive information that is adaptive to their emotions, and to acquire knowledge and culture that meets their individual needs.

[0771] Example 2

[0772] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0773] In today's information society, users seek efficient and flexible information gathering in their busy daily lives. However, conventional information provision systems lack the ability to customize information according to the user's emotions and circumstances, making improving the user experience a challenge. Furthermore, even when providing information via voice, the lack of natural dialogue based on the user's emotions and requests can easily lead to unsatisfactory experiences.

[0774] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0775] In this invention, the server includes means for receiving a request from a user in the form of text or voice, means for analyzing the request and generating related information using a generative model, means for converting the generated information into voice data using a voice synthesis engine, means for transmitting the generated voice data to the user's terminal and playing it back, and means for recognizing the user's emotions and adjusting the generated information and voice based on the emotions. This makes it possible to provide information according to the user's emotions and request content, thereby realizing a personalized, high-quality user experience.

[0776] "User" means any person who uses the System to request information or content.

[0777] A "request" is a request entered by a user via text or voice about the information or topic they wish to know about.

[0778] "Text" refers to information entered in written form.

[0779] "Voice" refers to what the user says.

[0780] "Means" refers to a method or device for achieving a particular purpose.

[0781] "Analysis" refers to the process of understanding a user's request and extracting the appropriate information.

[0782] A "generative model" refers to an artificial intelligence algorithm that generates information based on a user's request.

[0783] "Information" refers to knowledge or data provided in response to a User's request.

[0784] A "speech synthesis engine" refers to software or a system that converts text into natural-sounding speech.

[0785] "Audio data" refers to data in audio format generated by a speech synthesis engine.

[0786] "Terminal" refers to a device used by a user, such as a computer, smartphone, or tablet.

[0787] "Emotion" refers to the user's psychological state, including states such as joy, excitement, anger, and sadness.

[0788] "Emotion recognition" refers to the process of determining a user's psychological state from their request or voice.

[0789] "Adjustment" refers to changing the tone of the generated information or voice to match the user's emotions.

[0790] This invention is an information provision system that allows users to easily learn a wide range of knowledge and culture. It has the function of generating related information based on a request input by the user and providing it as voice. It also recognizes the user's emotions and adjusts the generated information and voice. This system will be specifically described in the following embodiment.

[0791] Accepting requests from users

[0792] Users launch the "Wisdom Radio" application using a device such as a smartphone or PC and request the information or topic they are looking for. For example, they can type, "Tell me about the formation of black holes." The device receives the user's input and prepares it to be sent to the server. If voice input is used, the device uses a microphone to capture voice and converts it into text using a speech recognition API.

[0793] Request analysis and emotion recognition

[0794] The server receives requests sent from the device and prepares to call the APIs of the emotion engine and generative model for analysis. The server recognizes emotions from the request content and the user's voice. The emotion engine analyzes the user's emotions (happiness, excitement, anger, sadness, etc.) and passes the results to the generative model.

[0795] Information generation

[0796] The server passes the results of the emotion engine and the request content to the generative model, which analyzes the request using natural language processing. The generative model generates information appropriate to the request content. For example, it generates detailed information such as, "Black holes are formed when extremely massive stars collapse under their own gravity..." The server adjusts the generated information based on the user's emotions, adding detailed technical information if the user is excited, for example.

[0797] Text-to-Speech (TTS) to translate content into audio

[0798] The server prepares the generated text for input to a speech synthesis engine. The server then sends the text to the speech synthesis engine, which generates natural-sounding speech. During this process, the tone and speed of the speech are also adjusted according to the user's emotions. The speech synthesis engine then converts the text into speech data, which is then sent back to the server.

[0799] Content Delivery

[0800] The server sends the generated voice data to the user's terminal. The terminal receives the voice data sent from the server. Finally, the terminal plays the voice data so that the user can listen to the content. This allows the user to obtain information based on the request audibly.

[0801] Specific hardware and software names to be used

[0802] Hardware: Smartphones, servers (cloud services)

[0803] Software: Intelligent Radio application, speech synthesis engine (e.g., Google Cloud Text-to-Speech, Amazon Polly), generative model (e.g., OpenAI GPT-3), emotion engine (e.g., Microsoft Azure Cognitive Services)

[0804] Examples of concrete examples and prompts

[0805] For example, if a user requests, "Tell me about the basic principles of quantum computing," the following process takes place: The user types, "Tell me about the basic principles of quantum computing," into their device. The server receives the request and analyzes the user's emotions using an emotion engine. The request and the resulting emotion are input into a generative model, which then generates information such as, "Quantum computers use qubits to have greater computing power than conventional computers." The information is adjusted based on the emotion engine's results, and the server sends the generated text to a speech synthesis engine, which generates audio data. The tone and speed are also adjusted according to the user's emotions. Finally, the server sends the audio data to the user's device, which then plays it back.

[0806] Example prompt sentence:

[0807] A user requests, "Tell me about the basic principles of quantum computing."

[0808] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0809] Step 1:

[0810] A user uses a device to launch the "Radio of Knowledge" application and enters a request for information. The request can be text or voice. For example, they could enter, "Tell me about the basic principles of quantum computing."

[0811] Input: User request (text or voice)

[0812] Output: Text or audio data received by the device

[0813] Specific behavior:

[0814] The user types text into the input area and presses the "Send" button.

[0815] For voice input, audio is recorded using a microphone and converted into text using a speech recognition API.

[0816] Step 2:

[0817] The device sends the user's request to the server, which receives the request and prepares to call the emotion engine and generative model APIs.

[0818] Input: Text or voice data received by the device

[0819] Output: Request data sent to the server

[0820] Specific behavior:

[0821] The terminal encodes the request data and sends an HTTP request to the server.

[0822] The server receives the HTTP request, decodes the data, and prepares it for analysis.

[0823] Step 3:

[0824] The server uses an emotion engine to recognize the user's emotions. It analyzes emotions (e.g., joy, excitement, anger, sadness, etc.) from voice or text data and passes the results to a generative model.

[0825] Input: User text or voice data

[0826] Output: Emotion data analyzed by the emotion engine

[0827] Specific behavior:

[0828] The server calls the emotion engine and passes the user's text or voice data as input.

[0829] The emotion engine analyzes the data, identifies the user's emotion, and returns that data.

[0830] Step 4:

[0831] The server uses the generative model to analyze the user's request and generate appropriate information. It also references the results of the emotion engine to create text based on the request.

[0832] Input: User request data and emotion data

[0833] Output: Generated text data

[0834] Specific behavior:

[0835] The server passes the request content and emotion data to the generative model and receives the generated text data.

[0836] The generative model generates text that matches the request, e.g., "Quantum computers use qubits to achieve greater computing power than classical computers..."

[0837] Step 5:

[0838] The server prepares the generated text for input to a speech synthesis engine, which converts the text data into natural-sounding speech data.

[0839] Input: Generated text data

[0840] Output: Generated audio data

[0841] Specific behavior:

[0842] The server passes the text data to a speech synthesis engine, such as Google Cloud Text-to-Speech or Amazon Polly.

[0843] A speech synthesis engine converts text data into speech, adjusting tone and speed based on emotional data.

[0844] Step 6:

[0845] The server sends the generated audio data to the user's device, which receives and plays the audio data.

[0846] Input: Generated audio data

[0847] Output: Audio data sent to the device

[0848] Specific behavior:

[0849] The server sends the audio data to the user's device via an HTTP request or WebSocket.

[0850] The device receives the audio data and plays the audio using its built-in audio playback function.

[0851] Examples of specific examples and prompts

[0852] For example, if a user requests, "Teach me about the basic principles of quantum computing," the following processing occurs: For example, a user requests, "Teach me about the basic principles of quantum computing."

[0853] (Application example 2)

[0854] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0855] Today's users demand immediate and appropriate provision of a wide range of security-related information, but achieving this requires a wide range of technologies. In particular, it is necessary to provide appropriate countermeasures immediately in emergencies, and conventional methods have difficulty responding flexibly while taking into account the user's feelings. Furthermore, the lack of a function to instruct emergency responses means that prompt and appropriate information cannot be provided when users feel anxious.

[0856] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0857] In this invention, the server includes means for receiving a request from a user in text or voice, means for analyzing the request and generating related information using a generative model, means for adjusting the generated information through means for recognizing the user's emotion, means for converting the generated information into voice data using a voice synthesis engine, means for transmitting the generated voice data to the user's terminal and playing it back, and means for instructing an emergency response based on the generated information. This enables appropriate and individual responses to user requests, and can provide a quick and accurate response, particularly in emergencies.

[0858] "User" means any individual or entity that uses this System to request information or services.

[0859] A "request" refers to information or a question a user enters into a system.

[0860] "Text" refers to information composed of characters.

[0861] "Speech" refers to sound information emitted by the human voice.

[0862] "Means" refers to a method or device for achieving a specific purpose.

[0863] "Analysis" refers to understanding and breaking down the content of input information and carrying out appropriate processing.

[0864] A "generative model" refers to an algorithm or program that uses artificial intelligence (AI) to generate text or speech.

[0865] "Relevant information" refers to appropriate data and knowledge provided based on the user's request.

[0866] "Means for recognizing emotions" refers to technologies or devices for analyzing and determining a user's emotional state.

[0867] A "speech synthesis engine" refers to software or a device that converts text information into voice data.

[0868] "Audio Data" means audio information stored in digital format.

[0869] "Terminal" means a device used by a User to access the System and receive information.

[0870] "Playback" refers to converting audio data into sound and letting the user hear it.

[0871] "Means for providing emergency response instructions" refers to technologies and devices that provide users with appropriate guidelines for action in an emergency.

[0872] The present invention relates to a personalized security assistant system that provides appropriate information to a user when the user inputs security-related requests by text or voice. Specifically, the system is configured as follows.

[0873] 1. Accepting requests from users

[0874] Users use a device such as a smartphone to enter security questions or requests into the system by text or voice, such as "I'm hearing strange noises in my house. What should I do?"

[0875] 2. Request analysis and emotion recognition

[0876] The server analyzes the request received from the device. If the request is voice-activated, it converts it to text using Google Cloud Speech-to-Text. It then uses a generative AI model, such as OpenAI GPT-4, to analyze the request and generate an appropriate response. At the same time, it uses the Microsoft Azure Emotion API to recognize the user's emotions and adjusts the information accordingly.

[0877] 3. Information generation and coordination

[0878] Based on the content analyzed by the generative model, information appropriate to the request is generated. The generated information is adjusted according to the user's emotions through an emotion engine. For example, if the user is feeling anxious, specific and detailed measures will be provided.

[0879] 4. Audio conversion of content using speech synthesis

[0880] The adjusted information is then converted into audio data using Google Cloud Text-to-Speech, with the tone and speed of the audio being adjusted according to the user's emotions.

[0881] 5. Content Delivery and Response Options

[0882] The generated voice data is sent from the server to the user's device, where it is played back. It also provides a means to issue emergency response instructions if necessary, allowing the user to notify relevant authorities with a single touch in the event of an emergency.

[0883] Specific examples

[0884] For example, a user might enter a request like this:

[0885] I hear strange noises in my house, what should I do?

[0886] When this request is received, the following process takes place: The audio is converted to text using Google Cloud Speech-to-Text and input into GPT-4. The generated response is analyzed using the Azure Emotion API, and if anxiety is recognized, countermeasure information is provided as a priority. The adjusted text is converted into audio data using Google Cloud Text-to-Speech and sent to the user's device. By receiving the information via audio, the user can quickly take appropriate action.

[0887] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0888] Step 1:

[0889] The user launches the application on a device such as a smartphone and enters a security request.

[0890] Input: The user's voice or text request.

[0891] What it does: The device uses a microphone to record audio or provides a GUI for entering text.

[0892] Output: Audio or text data.

[0893] Step 2:

[0894] The device converts the speech into text (in the case of voice input).

[0895] Input: Audio data.

[0896] How it works: Converts audio data into text using Google Cloud Speech-to-Text.

[0897] Output: The audio data converted into text data.

[0898] Step 3:

[0899] The terminal transmits the text data to the server.

[0900] Input: Text data converted from speech or entered directly.

[0901] Operation: The device sends text data to the server via an HTTP request.

[0902] Output: The text data sent to the server.

[0903] Step 4:

[0904] The server analyzes the request.

[0905] Input: User request text data.

[0906] How it works: The server inputs text data as prompts into the OpenAI GPT-4 model, which then generates an appropriate response using the generative AI model.

[0907] Output: The generated response text.

[0908] Step 5:

[0909] The server analyzes the user's emotions.

[0910] Input: The user's request text data and voice data (if any).

[0911] How it works: The server uses the Microsoft Azure Emotion API to perform emotion analysis and determine the user's emotional state.

[0912] Output: Parsed emotion data.

[0913] Step 6:

[0914] The server adjusts the generated response text based on the emotion data.

[0915] Input: Generated response text and sentiment data.

[0916] How it works: The server adjusts the response text based on the emotion data, such as adding more information if anxiety is detected.

[0917] Output: The adjusted response text.

[0918] Step 7:

[0919] The server converts the adjusted text into audio data.

[0920] Input: The tailored response text.

[0921] How it works: Uses Google Cloud Text-to-Speech to convert text data into audio data.

[0922] Output: The generated audio data.

[0923] Step 8:

[0924] The server sends the generated audio data to the user's device.

[0925] Input: The generated audio data.

[0926] How it works: The server sends the audio data to the user's device as an HTTP response.

[0927] Output: The audio data sent to the user's device.

[0928] Step 9:

[0929] The device plays the audio data and the user listens to the information.

[0930] Input: Audio data sent from the server.

[0931] Behavior: The device uses its audio player function to play audio data and provides audio to the user.

[0932] Output: The user hears a voice and gets the appropriate security information.

[0933] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0934] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0935] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0936] [Third embodiment]

[0937] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0938] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0939] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0940] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0941] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0942] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0943] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0944] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0945] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0946] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0947] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0948] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0949] The present invention is a system that allows users to easily learn a wide range of knowledge and culture. In a specific embodiment, the system analyzes a user's request, generates information based on that request, and provides it as a voice using a speech synthesis engine.

[0950] Specific processing of the program

[0951] Accepting requests from users

[0952] 1. User Input

[0953] Users launch the "Radio of Knowledge" application on their device and input a request for information or a topic they want to hear about, for example, "Tell me about the formation of black holes."

[0954] 2. Terminal Processing

[0955] The device receives the user's input and prepares the request to send to the server. In the case of voice input, the device converts the speech to text.

[0956] Parsing the request

[0957] 1. Server Reception

[0958] The server receives the request content sent from the terminal.

[0959] 2. Launching the Generative Model

[0960] The server passes the request to a generative model, which uses natural language processing to analyze the request, for example, extracting detailed information about the topic "black hole formation."

[0961] Information generation

[0962] 1. Information generation within the server

[0963] The generative model generates information appropriate to the request based on the analysis, such as text like "Black holes are formed when an extremely massive star collapses under its own gravity..."

[0964] Text-to-Speech (TTS) to translate content into audio

[0965] 1. Preparing the text

[0966] The server prepares the generated text to be input to the speech synthesis engine.

[0967] 2. Using a TTS engine

[0968] The server sends the text to a speech synthesis engine, which generates natural, fluent speech, adjusting the tone and speed of the speech during the process.

[0969] 3. Generating audio data

[0970] The speech synthesis engine converts the text into speech data, and the generated speech data is returned to the server.

[0971] Content Delivery

[0972] 1. Sending audio data

[0973] The server sends the generated audio data to the user's device.

[0974] 2. Receiving the terminal

[0975] The terminal receives the voice data transmitted from the server.

[0976] 3. Audio playback

[0977] The device plays the audio data, allowing the user to hear it.

[0978] Specific examples

[0979] For example, if a user requests, "Teach me the basic principles of quantum computing," the following process occurs:

[0980] 1. Accepting user requests

[0981] A user types into a terminal, "Tell me about the basic principles of quantum computing."

[0982] 2. Parsing the Request

[0983] The server receives the request and inputs it into the generative model, which analyzes and generates information about the fundamental principles of quantum computing.

[0984] 3. Information Generation

[0985] The generative model generates information such as "quantum computers use qubits to have greater computing power than conventional computers."

[0986] 4. Speech Synthesis

[0987] The server sends the generated text to a speech synthesis engine to generate voice data.

[0988] 5. Content Delivery

[0989] The server sends the audio data to the user's device, which then plays the audio.

[0990] This allows users to easily acquire knowledge and gain a wide range of knowledge and culture without having to actively search for it.

[0991] The processing flow will be explained below.

[0992] Step 1:

[0993] The user starts up the device and opens the "Radio of Knowledge" application. The user enters a request into an input field within the application, either by text or voice. For example, the user might enter "Tell me about the formation of black holes."

[0994] Step 2:

[0995] The device accepts user input. If voice input is selected, the device converts the speech to text using a speech recognition engine.

[0996] Step 3:

[0997] The device sends the request text data to the server via internet communication.

[0998] Step 4:

[0999] The server receives the request sent from the device and prepares to call the generative model API to analyze the received text data.

[1000] Step 5:

[1001] The server launches a generative model to analyze the request. The generative model extracts and generates relevant information based on the request. For example, the server instructs the model to generate information about the "formation of black holes."

[1002] Step 6:

[1003] The generative model generates information in response to the request, for example the generated text might be "Black holes are formed when a very massive star collapses under its own gravity..."

[1004] Step 7:

[1005] The server prepares the generated text to be input to the speech synthesis engine. The server passes the text data to the speech synthesis engine.

[1006] Step 8:

[1007] The text-to-speech engine receives the text data and converts it into speech data. The text-to-speech engine processes the text to produce natural, fluent speech, adjusting the tone and speed of the speech during this process.

[1008] Step 9:

[1009] The generated voice data is returned to the server, which then prepares the received voice data for transmission to the user's device.

[1010] Step 10:

[1011] The server sends the audio data to the user's device via internet communication.

[1012] Step 11:

[1013] The terminal receives the audio data sent from the server. The terminal prepares to play the received audio data.

[1014] Step 12:

[1015] The device plays audio data and allows the user to listen to the audio, allowing the user to hear information based on the request.

[1016] Step 13:

[1017] If a user has an additional question or would like more information during playback, they can enter a new request via text or voice, for example, "Please be more specific."

[1018] Step 14:

[1019] The device receives a new request and prepares to send it to the server. Steps 2 and after are repeated.

[1020] Example 1

[1021] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1022] With conventional technology, it was difficult for users to easily acquire a wide range of knowledge and culture. This was particularly difficult when the process of providing appropriate information in response to a request was complex and required advanced knowledge. Adjusting the quality and tone of the voice data to properly deliver the generated information to the user was also an issue.

[1023] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1024] In this invention, the server includes means for accepting requests from users via text or voice, means for analyzing the requests and generating relevant information using a generative model, means for converting the generated information into voice data using a voice synthesis engine, means for transmitting the generated voice data to the user's terminal and playing it back, means for passing the user's request to the generative model and extracting information, and means for converting the generated voice data by adjusting the tone and speed of the voice. This allows users to easily acquire a wide range of knowledge and culture, and makes it possible to provide appropriate information in response to requests as high-quality voice data.

[1025] "User" means a person or organization that uses the System to request information and receive information provided to it.

[1026] "Device" refers to an electronic device that a user operates to input requests or play audio data, including smartphones, tablets, and PCs.

[1027] A "server" is a computing system that receives a user's request, analyzes it, generates information, converts the generated information into audio data, and distributes it.

[1028] A "Request" is a question or request, entered by text or voice, about the information or topic you would like to know about.

[1029] A "generative model" is an artificial intelligence algorithm or system used to analyze a user request and generate relevant information.

[1030] A "speech synthesis engine" is a technology or function used to convert generated information into voice data. It is also known as a text-to-speech (TTS) engine.

[1031] "Audio data" means digital data converted from text to audio, which is audio played on a user's device.

[1032] "Analysis" is the process of understanding the user's request and extracting and processing the necessary information.

[1033] "Extraction" refers to the process by which the generative model extracts relevant information based on a user request.

[1034] "Tone and speed adjustment" refers to the process of appropriately changing the sound quality and speaking speed of the voice data generated by the speech synthesis engine.

[1035] This invention is a system that allows users to easily learn a wide range of knowledge and culture. It analyzes user requests to generate information, and then converts it into voice using a speech synthesis engine and provides it. This system is mainly composed of three elements: the user, the terminal, and the server.

[1036] First, a user launches the "Wisdom Radio" application on their device and inputs a request for information or a topic they would like to know about. For example, they can input requests by text or voice, such as "Tell me about the formation of black holes" or "Tell me about the basic principles of quantum computing." If input by voice, the device converts the voice into text.

[1037] The device then sends the request to the server. The server receives the request from the user's device and analyzes it. The server uses a generative AI model to extract and generate appropriate information based on the request. For example, the generative AI model might generate a prompt such as "Please provide information about the formation of black holes" or "Please explain the basic principles of quantum computing," and then extract the necessary information based on this prompt.

[1038] The generated information is stored in text format on the server. The server then inputs the generated text into a speech synthesis engine (e.g., a text-to-speech engine) and converts it into audio data. The speech synthesis engine then generates a natural, fluent voice, adjusting the tone and speed of the voice.

[1039] Once the voice data is generated, the server sends it to the user's device. The user's device receives the voice data and plays it back, allowing the user to listen to the requested information. This system allows users to easily acquire a wide range of knowledge and culture, and to receive the information they need by voice without having to actively search for it.

[1040] For example, if a user requests, "Tell me about the formation of black holes," the following process occurs: The user types "Tell me about the formation of black holes" into their device and sends the request to the server. The server inputs the prompt "Please provide information about the formation of black holes" into the generative AI model, and the generative model generates information such as "Black holes are formed when extremely massive stars collapse under their own gravity..." The server then sends the generated text to a speech synthesis engine, which generates natural, fluent audio data. Finally, this audio data is sent to the user's device, where it is played back, allowing the user to listen to information about the formation of black holes.

[1041] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1042] Step 1:

[1043] User Input

[1044] Users launch the "Radio of Knowledge" application on their own device and input requests for information or topics they want to know about via text or voice.

[1045] Input: User request (e.g., "Tell me about the formation of black holes")

[1046] Output: Request in text format (voice input is converted to text)

[1047] What happens: The user types a question into the app's text field and clicks the "Submit" button. If the user uses voice input, the device converts the speech into text and displays it on the screen.

[1048] Step 2:

[1049] Terminal request preparation and sending

[1050] The device receives the user's input and prepares the request to be sent to the server.

[1051] Input: Text data entered by the user (or, in the case of voice input, data already converted to text)

[1052] Output: The request formatted for sending to the server

[1053] Specific operation: The device creates an HTTP request, converts the request content into JSON format, and sends this data to the server when the send button is pressed.

[1054] Step 3:

[1055] Server request reception

[1056] The server receives the request content sent from the terminal.

[1057] Input: Request data sent from the terminal (JSON format)

[1058] Output: Parsed text request

[1059] What happens: The server receives the HTTP request, parses the content, and extracts the text request.

[1060] Step 4:

[1061] Analysis using generative models

[1062] The server passes the request to the generative model, which analyzes the request content using natural language processing.

[1063] Input: Text request (e.g., "Tell me about the formation of black holes")

[1064] Output: A prompt to be passed to the generative model (e.g., "Please provide information about the formation of black holes").

[1065] Specific operation: The server makes an API call to the generative AI model to generate a prompt sentence, which is then input into the generative AI model.

[1066] Step 5:

[1067] Information generation

[1068] Within the server, a generative model generates information appropriate to the request based on the analysis.

[1069] Input: Prompt statement (e.g., "Please provide information about the formation of black holes")

[1070] Output: Generated text information (e.g., "Black holes are formed when a very massive star collapses under its own gravity...")

[1071] Specific operation: The generative AI model generates information based on the prompt sentence and returns the text to the server.

[1072] Step 6:

[1073] Preparing text input for the speech synthesis engine

[1074] The server prepares the generated text to be input to the speech synthesis engine.

[1075] Input: Generated text information

[1076] Output: Text data converted into a format to send to the speech synthesis engine.

[1077] What happens: The server converts the generated text into JSON format and creates an appropriate request to the speech synthesis engine's API.

[1078] Step 7:

[1079] Voice data generation using a voice synthesis engine

[1080] The server sends the text to a speech synthesis engine, which generates natural, fluent speech.

[1081] Input: JSON format text data

[1082] Output: Audio data (e.g. audio file or audio stream)

[1083] How it works: The TTS engine receives text data, generates speech data, and returns the results to the server, adjusting the tone and speed of the speech.

[1084] Step 8:

[1085] Sending audio data

[1086] The server sends the generated audio data to the user's device.

[1087] Input: Generated audio data

[1088] Output: Audio data sent to the user's device

[1089] Specific operation: The server encodes the audio data as an HTTP response and sends it to the user's device.

[1090] Step 9:

[1091] Receiving audio data by the device

[1092] The terminal receives the voice data transmitted from the server.

[1093] Input: Audio data sent from the server

[1094] Output: Audio data in a playable format

[1095] What happens: The device decodes the audio data and converts it into a format that can be played within the application.

[1096] Step 10:

[1097] Device audio playback

[1098] The device plays the audio data, allowing the user to hear it.

[1099] Input: Decoded audio data

[1100] Output: Audio played through speakers or earphones

[1101] Specific behavior: The device's audio player plays the audio data and outputs it to the user as sound.

[1102] (Application example 1)

[1103] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1104] Traditionally, users have had limited means to easily obtain detailed information based on their interests, making it particularly difficult to obtain information while on the move or hands-free. Furthermore, the information generated is often perfunctory, making it difficult for users to learn effectively. Furthermore, if the information generated based on user requests is not properly converted into audio and delivered, learning and comprehension can be hindered.

[1105] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1106] In this invention, the server includes means for receiving a request from a user via text or voice, means for analyzing the request and generating relevant information using a generative model, and means for converting the generated information into voice data using a voice synthesis engine. This allows the user to easily obtain detailed information based on the request in a hands-free manner, and further improves learning and understanding by receiving the generated information in real time via voice. Furthermore, by adding means including a voice synthesis engine for converting the generated text information into voice data and means for inputting prompts into a generative AI model to generate appropriate information, it becomes possible to provide information in a more natural and fluent manner, further improving user convenience.

[1107] "Text or voice input means" means a device or software that provides an interface through which a user can input requests by text or voice.

[1108] "Generative model" refers to a machine learning model or AI system used to generate relevant information based on a user request.

[1109] "Speech synthesis engine" refers to software or hardware for converting text data into natural, fluent speech.

[1110] "Means for transmitting to and playing on a user's device" means a system or protocol that transmits the generated audio data to a user's device and enables the device to play the audio.

[1111] A "prompt" refers to an instruction or question entered into a generative AI model, which triggers the model to generate appropriate information.

[1112] "Generative AI model" refers to an artificial intelligence system that is trained to generate appropriate answers or information from prompts based on machine learning techniques.

[1113] To implement this invention, a user terminal, a server, a generative AI model, and a speech synthesis engine are required.

[1114] The user's device provides an interface that accepts requests input via text or voice. This interface can be realized using the smartphone's screen or voice recognition function. For example, if a user inputs "Tell me about the basic principles of quantum computing," this request will be received by the device.

[1115] The device sends the request to the server, which analyzes it and sends a prompt to the generative AI model. The prompt is a specific instruction or question, such as "User asked: Please tell me about the basic principles of quantum computing. Provide a detailed explanation."

[1116] The generative AI model generates appropriate information based on the received prompt. For example, GPT-3, a model based on machine learning technology, is used as the generative AI model. The generated text information is processed by the server and sent to the speech synthesis engine.

[1117] A speech synthesis engine (e.g., gTTS) converts the generated text into natural, fluent speech, providing information in a way that is easy for users to understand. The tone and speed of the speech can also be adjusted during this process.

[1118] Finally, the generated voice data is sent from the server to the user's device, which then plays it back in real time, allowing the user to easily obtain detailed information based on the request in a hands-free manner, improving learning and comprehension.

[1119] As described above, this invention enables users to easily obtain advanced knowledge and information, and makes it extremely convenient to obtain information while on the move or hands-free.

[1120] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1121] Step 1:

[1122] The user inputs a request via text or voice about a topic they want to learn about. An example of an input request might be, "Teach me about the basic principles of quantum computing." When the device receives the voice input, it uses speech recognition to convert it into text data. This converted text data is sent to the next processing step.

[1123] Input: User's text or voice request

[1124] Output: Text data of the request content

[1125] Step 2:

[1126] The device sends the request content as text data to the server. The server receives this data and begins analyzing the request content. Here, it uses natural language processing technology to analyze and understand the request content.

[1127] Input: Text data of the request content

[1128] Output: Prompt statement used for analysis

[1129] Step 3:

[1130] The server inputs a prompt to the generative AI model. For example, it generates a prompt such as "User asked: Please tell me about the basic principles of quantum computing. Provide a detailed explanation." This prompt is passed to the generative AI model, which then generates information corresponding to the request.

[1131] Input: prompt statement

[1132] Output: Detailed text information generated

[1133] Step 4:

[1134] The generative AI model returns the generated text information to the server, which then sends it to the speech synthesis engine, which generates natural, fluent speech based on the text information. The tone and speed of the speech can then be adjusted as needed.

[1135] Input: Generated detailed text information

[1136] Output: Generated audio data

[1137] Step 5:

[1138] The voice data generated by the speech synthesis engine is returned to the server. The server then sends this voice data to the user's device. The device receives it and plays the voice in real time, allowing the user to receive the requested information by voice.

[1139] Input: Generated audio data

[1140] Output: Audio played on the user's device

[1141] Step 6:

[1142] The device plays the audio data and the user can listen to it. This process allows users to easily obtain detailed information on a particular subject and allows for hands-free learning, especially when on the move.

[1143] Input: Audio data sent to the device

[1144] Output: User experience through the audio data played

[1145] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1146] The present invention is a system that allows users to easily learn a wide range of knowledge and culture. In a specific embodiment, the system analyzes a user's request, generates information based on that request, and provides it as voice using a voice synthesis engine. Furthermore, the present invention incorporates an emotion engine that recognizes the user's emotions, and adds functions for analyzing the request content and adjusting the generated information and voice.

[1147] Specific processing of the program

[1148] Accepting requests from users

[1149] 1. User Input

[1150] Users launch the "Radio of Knowledge" application on their device and input a request for information or a topic they want to hear about, for example, "Tell me about the formation of black holes."

[1151] 2. Terminal Processing

[1152] The device receives the user's input and prepares the request to send to the server. In the case of voice input, the device converts the speech to text.

[1153] Request analysis and emotion recognition

[1154] 1. Server Reception

[1155] The server receives the request content sent from the device and prepares to call the emotion engine and generative model APIs for analysis.

[1156] 2. Activating the Emotional Engine

[1157] The server recognizes emotions from the request content and the user's voice. The emotion engine analyzes the user's emotions (e.g., joy, excitement, anger, sadness, etc.) and passes the results to the generative model.

[1158] 3. Launching the Generative Model

[1159] The server passes the results of the emotion engine and the request content to the generative model, which then analyzes the request content using natural language processing. For example, it extracts detailed information on the topic "formation of black holes."

[1160] Information generation

[1161] 1. Information generation within the server

[1162] The generative model generates information appropriate to the request based on the analysis, such as text like "Black holes are formed when an extremely massive star collapses under its own gravity..."

[1163] 2. Emotion-Based Adjustment

[1164] The server adjusts the generated information based on the user's emotions, for example adding detailed technical explanations if the user is excited.

[1165] Text-to-Speech (TTS) to translate content into audio

[1166] 1. Preparing the text

[1167] The server prepares the generated text to be input to the speech synthesis engine.

[1168] 2. Using a TTS engine

[1169] The server sends the text to a speech synthesis engine, which generates natural, fluent speech, adjusting the tone and speed of the speech according to the user's emotions.

[1170] 3. Generating audio data

[1171] The speech synthesis engine converts the text into speech data, and the generated speech data is returned to the server.

[1172] Content Delivery

[1173] 1. Sending audio data

[1174] The server sends the generated audio data to the user's device.

[1175] 2. Receiving the terminal

[1176] The terminal receives the voice data transmitted from the server.

[1177] 3. Audio playback

[1178] The device plays audio data and allows the user to listen to it, allowing the user to hear information based on the request.

[1179] Specific examples

[1180] For example, if a user requests, "Teach me the basic principles of quantum computing," the following process occurs:

[1181] 1. Accepting user requests

[1182] A user types into a terminal, "Tell me about the basic principles of quantum computing."

[1183] 2. Request analysis and emotion recognition

[1184] The server receives the request, analyzes the user's emotions using the emotion engine, and inputs the request content and the resulting emotions into the generative model.

[1185] 3. Information generation and coordination

[1186] The generative model generates information such as "quantum computers use qubits to have greater computing power than classical computers" and adjusts the information as needed based on the results of the emotion engine.

[1187] 4. Speech Synthesis

[1188] The server sends the generated text to a speech synthesis engine to generate speech data, adjusting the tone and speed depending on the user's emotions.

[1189] 5. Content Delivery

[1190] The server sends the audio data to the user's device, which then plays the audio.

[1191] This allows users to easily acquire knowledge and gain a wide range of knowledge and culture without actively searching. In addition, the emotion engine makes it possible to provide information tailored to each individual user.

[1192] The processing flow will be explained below.

[1193] Step 1:

[1194] The user starts up the device and opens the "Radio of Knowledge" application. The user enters a request into an input field within the application, either by text or voice. For example, the user might enter "Tell me about the formation of black holes."

[1195] Step 2:

[1196] The device accepts user input. If voice input is selected, the device converts the speech to text using a speech recognition engine.

[1197] Step 3:

[1198] The device sends the request text data to the server via internet communication.

[1199] Step 4:

[1200] The server receives the request sent from the device and prepares to call the APIs of the generative model and emotion engine to analyze the received text data.

[1201] Step 5:

[1202] The server starts the emotion engine and recognizes emotions from the received request and the user's voice (in the case of voice input). The emotion engine analyzes the user's emotions (e.g., joy, excitement, anger, sadness, etc.) and passes the results to the generative model.

[1203] Step 6:

[1204] The server launches the generative model and inputs the results of the emotion engine and the request content into the model. The generative model analyzes the request content using natural language processing, extracts and generates relevant information. For example, it generates detailed information on the topic of "black hole formation."

[1205] Step 7:

[1206] The generative model generates information in response to the request, for example the generated text might be "Black holes are formed when a very massive star collapses under its own gravity..."

[1207] Step 8:

[1208] The server uses an emotion engine to tailor the generated text based on the user's emotions, for example, adding more technical details if the user is excited, or a simpler explanation if the user is calm.

[1209] Step 9:

[1210] The server prepares the emotion-adjusted text to be input to the text-to-speech engine (TTS). The server passes the text data to the text-to-speech engine.

[1211] Step 10:

[1212] The text-to-speech engine receives the text data and converts it into speech data. The text-to-speech engine converts the text into natural, fluent speech, adjusting the tone and speed of the speech based on the results of the emotion engine.

[1213] Step 11:

[1214] The generated voice data is returned to the server, which then prepares the received voice data for transmission to the user's device.

[1215] Step 12:

[1216] The server sends the audio data to the user's device via internet communication.

[1217] Step 13:

[1218] The terminal receives the audio data sent from the server, and prepares to play the received audio data for the user.

[1219] Step 14:

[1220] The device plays audio data and allows the user to listen to the audio, allowing the user to hear information based on the request.

[1221] Step 15:

[1222] If a user has an additional question or would like more information during playback, they can enter a new request via text or voice, for example, "Please be more specific."

[1223] Step 16:

[1224] The device receives a new request and prepares to send it to the server. Steps 2 and after are repeated.

[1225] This allows users to easily receive information that is adaptive to their emotions, and to acquire knowledge and culture that meets their individual needs.

[1226] Example 2

[1227] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1228] In today's information society, users seek efficient and flexible information gathering in their busy daily lives. However, conventional information provision systems lack the ability to customize information according to the user's emotions and circumstances, making improving the user experience a challenge. Furthermore, even when providing information via voice, the lack of natural dialogue based on the user's emotions and requests can easily lead to unsatisfactory experiences.

[1229] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1230] In this invention, the server includes means for receiving a request from a user in the form of text or voice, means for analyzing the request and generating related information using a generative model, means for converting the generated information into voice data using a voice synthesis engine, means for transmitting the generated voice data to the user's terminal and playing it back, and means for recognizing the user's emotions and adjusting the generated information and voice based on the emotions. This makes it possible to provide information according to the user's emotions and request content, thereby realizing a personalized, high-quality user experience.

[1231] "User" means any person who uses the System to request information or content.

[1232] A "request" is a request entered by a user via text or voice about the information or topic they wish to know about.

[1233] "Text" refers to information entered in written form.

[1234] "Voice" refers to what the user says.

[1235] "Means" refers to a method or device for achieving a particular purpose.

[1236] "Analysis" refers to the process of understanding a user's request and extracting the appropriate information.

[1237] A "generative model" refers to an artificial intelligence algorithm that generates information based on a user's request.

[1238] "Information" refers to knowledge or data provided in response to a User's request.

[1239] A "speech synthesis engine" refers to software or a system that converts text into natural-sounding speech.

[1240] "Audio data" refers to data in audio format generated by a speech synthesis engine.

[1241] "Terminal" refers to a device used by a user, such as a computer, smartphone, or tablet.

[1242] "Emotion" refers to the user's psychological state, including states such as joy, excitement, anger, and sadness.

[1243] "Emotion recognition" refers to the process of determining a user's psychological state from their request or voice.

[1244] "Adjustment" refers to changing the tone of the generated information or voice to match the user's emotions.

[1245] This invention is an information provision system that allows users to easily learn a wide range of knowledge and culture. It has the function of generating related information based on a request input by the user and providing it as voice. It also recognizes the user's emotions and adjusts the generated information and voice. This system will be specifically described in the following embodiment.

[1246] Accepting requests from users

[1247] Users launch the "Wisdom Radio" application using a device such as a smartphone or PC and request the information or topic they are looking for. For example, they can type, "Tell me about the formation of black holes." The device receives the user's input and prepares it to be sent to the server. If voice input is used, the device uses a microphone to capture voice and converts it into text using a speech recognition API.

[1248] Request analysis and emotion recognition

[1249] The server receives requests sent from the device and prepares to call the APIs of the emotion engine and generative model for analysis. The server recognizes emotions from the request content and the user's voice. The emotion engine analyzes the user's emotions (happiness, excitement, anger, sadness, etc.) and passes the results to the generative model.

[1250] Information generation

[1251] The server passes the results of the emotion engine and the request content to the generative model, which analyzes the request using natural language processing. The generative model generates information appropriate to the request content. For example, it generates detailed information such as, "Black holes are formed when extremely massive stars collapse under their own gravity..." The server adjusts the generated information based on the user's emotions, adding detailed technical information if the user is excited, for example.

[1252] Text-to-Speech (TTS) to translate content into audio

[1253] The server prepares the generated text for input to a speech synthesis engine. The server then sends the text to the speech synthesis engine, which generates natural-sounding speech. During this process, the tone and speed of the speech are also adjusted according to the user's emotions. The speech synthesis engine then converts the text into speech data, which is then sent back to the server.

[1254] Content Delivery

[1255] The server sends the generated voice data to the user's terminal. The terminal receives the voice data sent from the server. Finally, the terminal plays the voice data so that the user can listen to the content. This allows the user to obtain information based on the request audibly.

[1256] Specific hardware and software names to be used

[1257] Hardware: Smartphones, servers (cloud services)

[1258] Software: Intelligent Radio application, speech synthesis engine (e.g., Google Cloud Text-to-Speech, Amazon Polly), generative model (e.g., OpenAI GPT-3), emotion engine (e.g., Microsoft Azure Cognitive Services)

[1259] Examples of concrete examples and prompts

[1260] For example, if a user requests, "Tell me about the basic principles of quantum computing," the following process takes place: The user types, "Tell me about the basic principles of quantum computing," into their device. The server receives the request and analyzes the user's emotions using an emotion engine. The request and the resulting emotion are input into a generative model, which then generates information such as, "Quantum computers use qubits to have greater computing power than conventional computers." The information is adjusted based on the emotion engine's results, and the server sends the generated text to a speech synthesis engine, which generates audio data. The tone and speed are also adjusted according to the user's emotions. Finally, the server sends the audio data to the user's device, which then plays it back.

[1261] Example prompt sentence:

[1262] A user requests, "Tell me about the basic principles of quantum computing."

[1263] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1264] Step 1:

[1265] A user uses a device to launch the "Radio of Knowledge" application and enters a request for information. The request can be text or voice. For example, they could enter, "Tell me about the basic principles of quantum computing."

[1266] Input: User request (text or voice)

[1267] Output: Text or audio data received by the device

[1268] Specific behavior:

[1269] The user types text into the input area and presses the "Send" button.

[1270] For voice input, audio is recorded using a microphone and converted into text using a speech recognition API.

[1271] Step 2:

[1272] The device sends the user's request to the server, which receives the request and prepares to call the emotion engine and generative model APIs.

[1273] Input: Text or voice data received by the device

[1274] Output: Request data sent to the server

[1275] Specific behavior:

[1276] The terminal encodes the request data and sends an HTTP request to the server.

[1277] The server receives the HTTP request, decodes the data, and prepares it for analysis.

[1278] Step 3:

[1279] The server uses an emotion engine to recognize the user's emotions. It analyzes emotions (e.g., joy, excitement, anger, sadness, etc.) from voice or text data and passes the results to a generative model.

[1280] Input: User text or voice data

[1281] Output: Emotion data analyzed by the emotion engine

[1282] Specific behavior:

[1283] The server calls the emotion engine and passes the user's text or voice data as input.

[1284] The emotion engine analyzes the data, identifies the user's emotion, and returns that data.

[1285] Step 4:

[1286] The server uses the generative model to analyze the user's request and generate appropriate information. It also references the results of the emotion engine to create text based on the request.

[1287] Input: User request data and emotion data

[1288] Output: Generated text data

[1289] Specific behavior:

[1290] The server passes the request content and emotion data to the generative model and receives the generated text data.

[1291] The generative model generates text appropriate to the request, e.g., "Quantum computers use qubits to achieve greater computing power than classical computers..."

[1292] Step 5:

[1293] The server prepares the generated text for input to a speech synthesis engine, which converts the text data into natural-sounding speech data.

[1294] Input: Generated text data

[1295] Output: Generated audio data

[1296] Specific behavior:

[1297] The server passes the text data to a speech synthesis engine, such as Google Cloud Text-to-Speech or Amazon Polly.

[1298] A speech synthesis engine converts text data into speech, adjusting tone and speed based on emotional data.

[1299] Step 6:

[1300] The server sends the generated audio data to the user's device, which receives and plays the audio data.

[1301] Input: Generated audio data

[1302] Output: Audio data sent to the device

[1303] Specific behavior:

[1304] The server sends the audio data to the user's device via an HTTP request or WebSocket.

[1305] The device receives the audio data and plays the audio using its built-in audio playback function.

[1306] Examples of concrete examples and prompts

[1307] For example, if a user requests, "Teach me about the basic principles of quantum computing," the following processing occurs: For example, a user requests, "Teach me about the basic principles of quantum computing."

[1308] (Application example 2)

[1309] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1310] Today's users demand immediate and appropriate provision of a wide range of security-related information, but achieving this requires a wide range of technologies. In particular, it is necessary to provide appropriate countermeasures immediately in emergencies, and conventional methods have difficulty responding flexibly while taking into account the user's feelings. Furthermore, the lack of a function to instruct emergency responses means that prompt and appropriate information cannot be provided when users feel anxious.

[1311] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1312] In this invention, the server includes means for receiving a request from a user in text or voice, means for analyzing the request and generating related information using a generative model, means for adjusting the generated information through means for recognizing the user's emotion, means for converting the generated information into voice data using a voice synthesis engine, means for transmitting the generated voice data to the user's terminal and playing it back, and means for instructing an emergency response based on the generated information. This enables appropriate and individual responses to user requests, and can provide a quick and accurate response, particularly in emergencies.

[1313] "User" means any individual or entity that uses this System to request information or services.

[1314] A "request" refers to information or a question a user enters into a system.

[1315] "Text" refers to information composed of characters.

[1316] "Speech" refers to sound information emitted by the human voice.

[1317] "Means" refers to a method or device for achieving a specific purpose.

[1318] "Analysis" refers to understanding and breaking down the content of input information and carrying out appropriate processing.

[1319] A "generative model" refers to an algorithm or program that uses artificial intelligence (AI) to generate text or speech.

[1320] "Relevant information" refers to appropriate data and knowledge provided based on the user's request.

[1321] "Means for recognizing emotions" refers to technologies or devices for analyzing and determining a user's emotional state.

[1322] A "speech synthesis engine" refers to software or a device that converts text information into voice data.

[1323] "Audio Data" means audio information stored in digital format.

[1324] "Terminal" means a device used by a User to access the System and receive information.

[1325] "Playback" refers to converting audio data into sound and letting the user hear it.

[1326] "Means for providing emergency response instructions" refers to technologies and devices that provide users with appropriate guidelines for action in an emergency.

[1327] The present invention relates to a personalized security assistant system that provides appropriate information to a user when the user inputs security-related requests by text or voice. Specifically, the system is configured as follows.

[1328] 1. Accepting requests from users

[1329] Users use a device such as a smartphone to enter security questions or requests into the system by text or voice, such as "I'm hearing strange noises in my house. What should I do?"

[1330] 2. Request analysis and emotion recognition

[1331] The server analyzes the request received from the device. If the request is voice-activated, it converts it to text using Google Cloud Speech-to-Text. It then uses a generative AI model, such as OpenAI GPT-4, to analyze the request and generate an appropriate response. At the same time, it uses the Microsoft Azure Emotion API to recognize the user's emotions and adjusts the information accordingly.

[1332] 3. Information generation and coordination

[1333] Based on the content analyzed by the generative model, information appropriate to the request is generated. The generated information is adjusted according to the user's emotions through an emotion engine. For example, if the user is feeling anxious, specific and detailed measures will be provided.

[1334] 4. Audio conversion of content using speech synthesis

[1335] The adjusted information is then converted into audio data using Google Cloud Text-to-Speech, with the tone and speed of the audio being adjusted according to the user's emotions.

[1336] 5. Content Delivery and Response Options

[1337] The generated voice data is sent from the server to the user's device and played back on the device. It also provides a means to issue emergency response instructions if necessary, allowing the user to notify relevant authorities with a single touch in the event of an emergency.

[1338] Specific examples

[1339] For example, a user might enter a request like this:

[1340] I hear strange noises in my house, what should I do?

[1341] When this request is received, the following process takes place: The audio is converted to text using Google Cloud Speech-to-Text and input into GPT-4. The generated response is analyzed using the Azure Emotion API, and if anxiety is recognized, countermeasure information is provided as a priority. The adjusted text is converted into audio data using Google Cloud Text-to-Speech and sent to the user's device. By receiving the information via audio, the user can quickly take appropriate action.

[1342] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1343] Step 1:

[1344] The user launches the application on a device such as a smartphone and enters a security request.

[1345] Input: The user's voice or text request.

[1346] What it does: The device uses a microphone to record audio or provides a GUI for entering text.

[1347] Output: Audio or text data.

[1348] Step 2:

[1349] The device converts the speech into text (in the case of voice input).

[1350] Input: Audio data.

[1351] How it works: Converts audio data into text using Google Cloud Speech-to-Text.

[1352] Output: The audio data converted into text data.

[1353] Step 3:

[1354] The terminal transmits the text data to the server.

[1355] Input: Text data converted from speech or entered directly.

[1356] Operation: The device sends text data to the server via an HTTP request.

[1357] Output: The text data sent to the server.

[1358] Step 4:

[1359] The server analyzes the request.

[1360] Input: User request text data.

[1361] How it works: The server inputs text data as prompts into the OpenAI GPT-4 model, which then generates an appropriate response using the generative AI model.

[1362] Output: The generated response text.

[1363] Step 5:

[1364] The server analyzes the user's emotions.

[1365] Input: The user's request text data and voice data (if any).

[1366] How it works: The server uses the Microsoft Azure Emotion API to perform emotion analysis and determine the user's emotional state.

[1367] Output: Parsed emotion data.

[1368] Step 6:

[1369] The server adjusts the generated response text based on the emotion data.

[1370] Input: Generated response text and sentiment data.

[1371] How it works: The server adjusts the response text based on the emotion data, such as adding more information if anxiety is detected.

[1372] Output: The adjusted response text.

[1373] Step 7:

[1374] The server converts the adjusted text into audio data.

[1375] Input: The tailored response text.

[1376] How it works: Uses Google Cloud Text-to-Speech to convert text data into audio data.

[1377] Output: The generated audio data.

[1378] Step 8:

[1379] The server sends the generated audio data to the user's device.

[1380] Input: The generated audio data.

[1381] How it works: The server sends the audio data to the user's device as an HTTP response.

[1382] Output: The audio data sent to the user's device.

[1383] Step 9:

[1384] The device plays the audio data and the user listens to the information.

[1385] Input: Audio data sent from the server.

[1386] Behavior: The device uses its audio player function to play audio data and provides audio to the user.

[1387] Output: The user hears a voice and gets the appropriate security information.

[1388] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1389] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1390] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1391] [Fourth embodiment]

[1392] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1393] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1394] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1395] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1396] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1397] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1398] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1399] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1400] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1401] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1402] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1403] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1404] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1405] The present invention is a system that allows users to easily learn a wide range of knowledge and culture. In a specific embodiment, the system analyzes a user's request, generates information based on that request, and provides it as a voice using a speech synthesis engine.

[1406] Specific processing of the program

[1407] Accepting requests from users

[1408] 1. User Input

[1409] Users launch the "Radio of Knowledge" application on their device and input a request for information or a topic they want to hear about, for example, "Tell me about the formation of black holes."

[1410] 2. Terminal Processing

[1411] The device receives the user's input and prepares the request to send to the server. In the case of voice input, the device converts the speech to text.

[1412] Parsing the request

[1413] 1. Server Reception

[1414] The server receives the request content sent from the terminal.

[1415] 2. Launching the Generative Model

[1416] The server passes the request to a generative model, which uses natural language processing to analyze the request, for example, extracting detailed information about the topic "black hole formation."

[1417] Information generation

[1418] 1. Information generation within the server

[1419] The generative model generates information appropriate to the request based on the analysis, such as text like "Black holes are formed when an extremely massive star collapses under its own gravity..."

[1420] Text-to-Speech (TTS) to translate content into audio

[1421] 1. Preparing the text

[1422] The server prepares the generated text to be input to the speech synthesis engine.

[1423] 2. Using a TTS engine

[1424] The server sends the text to a speech synthesis engine, which generates natural, fluent speech, adjusting the tone and speed of the speech during the process.

[1425] 3. Generating audio data

[1426] The speech synthesis engine converts the text into speech data, and the generated speech data is returned to the server.

[1427] Content Delivery

[1428] 1. Sending audio data

[1429] The server sends the generated audio data to the user's device.

[1430] 2. Receiving the terminal

[1431] The terminal receives the voice data transmitted from the server.

[1432] 3. Audio playback

[1433] The device plays the audio data, allowing the user to hear it.

[1434] Specific examples

[1435] For example, if a user requests, "Teach me the basic principles of quantum computing," the following process occurs:

[1436] 1. Accepting user requests

[1437] A user types into a terminal, "Tell me about the basic principles of quantum computing."

[1438] 2. Parsing the Request

[1439] The server receives the request and inputs it into the generative model, which analyzes and generates information about the fundamental principles of quantum computing.

[1440] 3. Information Generation

[1441] The generative model generates information such as "quantum computers use qubits to have greater computing power than conventional computers."

[1442] 4. Speech Synthesis

[1443] The server sends the generated text to a speech synthesis engine to generate voice data.

[1444] 5. Content Delivery

[1445] The server sends the audio data to the user's device, which then plays the audio.

[1446] This allows users to easily acquire knowledge and gain a wide range of knowledge and culture without having to actively search for it.

[1447] The processing flow will be explained below.

[1448] Step 1:

[1449] The user starts up the device and opens the "Radio of Knowledge" application. The user enters a request into an input field within the application, either by text or voice. For example, the user might enter "Tell me about the formation of black holes."

[1450] Step 2:

[1451] The device accepts user input. If voice input is selected, the device converts the speech to text using a speech recognition engine.

[1452] Step 3:

[1453] The device sends the request text data to the server via internet communication.

[1454] Step 4:

[1455] The server receives the request sent from the device and prepares to call the generative model API to analyze the received text data.

[1456] Step 5:

[1457] The server launches a generative model to analyze the request. The generative model extracts and generates relevant information based on the request. For example, the server instructs the model to generate information about the "formation of black holes."

[1458] Step 6:

[1459] The generative model generates information in response to the request, for example the generated text might be "Black holes are formed when a very massive star collapses under its own gravity..."

[1460] Step 7:

[1461] The server prepares the generated text to be input to the speech synthesis engine. The server passes the text data to the speech synthesis engine.

[1462] Step 8:

[1463] The text-to-speech engine receives the text data and converts it into speech data. The text-to-speech engine processes the text to produce natural, fluent speech, adjusting the tone and speed of the speech during this process.

[1464] Step 9:

[1465] The generated voice data is returned to the server, which then prepares the received voice data for transmission to the user's device.

[1466] Step 10:

[1467] The server sends the audio data to the user's device via internet communication.

[1468] Step 11:

[1469] The terminal receives the audio data sent from the server. The terminal prepares to play the received audio data.

[1470] Step 12:

[1471] The device plays audio data and allows the user to listen to the audio, allowing the user to hear information based on the request.

[1472] Step 13:

[1473] If a user has an additional question or would like more information during playback, they can enter a new request via text or voice, for example, "Please be more specific."

[1474] Step 14:

[1475] The device receives a new request and prepares to send it to the server. Steps 2 and after are repeated.

[1476] Example 1

[1477] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1478] With conventional technology, it was difficult for users to easily acquire a wide range of knowledge and culture. This was particularly difficult when the process of providing appropriate information in response to a request was complex and required advanced knowledge. Adjusting the quality and tone of the voice data to properly deliver the generated information to the user was also an issue.

[1479] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1480] In this invention, the server includes means for accepting requests from users via text or voice, means for analyzing the requests and generating relevant information using a generative model, means for converting the generated information into voice data using a voice synthesis engine, means for transmitting the generated voice data to the user's terminal and playing it back, means for passing the user's request to the generative model and extracting information, and means for converting the generated voice data by adjusting the tone and speed of the voice. This allows users to easily acquire a wide range of knowledge and culture, and makes it possible to provide appropriate information in response to requests as high-quality voice data.

[1481] "User" means a person or organization that uses the System to request information and receive information provided to it.

[1482] "Device" refers to an electronic device that a user operates to input requests or play audio data, including smartphones, tablets, and PCs.

[1483] A "server" is a computing system that receives a user's request, analyzes it, generates information, converts the generated information into audio data, and distributes it.

[1484] A "Request" is a question or request, entered by text or voice, about the information or topic you would like to know about.

[1485] A "generative model" is an artificial intelligence algorithm or system used to analyze a user request and generate relevant information.

[1486] A "speech synthesis engine" is a technology or function used to convert generated information into voice data. It is also known as a text-to-speech (TTS) engine.

[1487] "Audio data" means digital data converted from text to audio, which is audio played on a user's device.

[1488] "Analysis" is the process of understanding the user's request and extracting and processing the necessary information.

[1489] "Extraction" refers to the process by which the generative model extracts relevant information based on a user request.

[1490] "Tone and speed adjustment" refers to the process of appropriately changing the sound quality and speaking speed of the voice data generated by the speech synthesis engine.

[1491] This invention is a system that allows users to easily learn a wide range of knowledge and culture. It analyzes user requests to generate information, and then converts it into voice using a speech synthesis engine and provides it. This system is mainly composed of three elements: the user, the terminal, and the server.

[1492] First, a user launches the "Wisdom Radio" application on their device and inputs a request for information or a topic they would like to know about. For example, they can input requests by text or voice, such as "Tell me about the formation of black holes" or "Tell me about the basic principles of quantum computing." If input by voice, the device converts the voice into text.

[1493] The device then sends the request to the server. The server receives the request from the user's device and analyzes it. The server uses a generative AI model to extract and generate appropriate information based on the request. For example, the generative AI model might generate a prompt such as "Please provide information about the formation of black holes" or "Please explain the basic principles of quantum computing," and then extract the necessary information based on this prompt.

[1494] The generated information is stored in text format on the server. The server then inputs the generated text into a speech synthesis engine (e.g., a text-to-speech engine) and converts it into audio data. The speech synthesis engine then generates a natural, fluent voice, adjusting the tone and speed of the voice.

[1495] Once the voice data is generated, the server sends it to the user's device. The user's device receives the voice data and plays it back, allowing the user to listen to the requested information. This system allows users to easily acquire a wide range of knowledge and culture, and to receive the information they need by voice without having to actively search for it.

[1496] For example, if a user requests, "Tell me about the formation of black holes," the following process occurs: The user types "Tell me about the formation of black holes" into their device and sends the request to the server. The server inputs the prompt "Please provide information about the formation of black holes" into the generative AI model, and the generative model generates information such as "Black holes are formed when extremely massive stars collapse under their own gravity..." The server then sends the generated text to a speech synthesis engine, which generates natural, fluent audio data. Finally, this audio data is sent to the user's device, where it is played back, allowing the user to listen to information about the formation of black holes.

[1497] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1498] Step 1:

[1499] User Input

[1500] Users launch the "Radio of Knowledge" application on their own device and input requests for information or topics they want to know about via text or voice.

[1501] Input: User request (e.g., "Tell me about the formation of black holes")

[1502] Output: Request in text format (voice input is converted to text)

[1503] What happens: The user types a question into the app's text field and clicks the "Submit" button. If the user uses voice input, the device converts the speech into text and displays it on the screen.

[1504] Step 2:

[1505] Terminal request preparation and sending

[1506] The device receives the user's input and prepares the request to be sent to the server.

[1507] Input: Text data entered by the user (or, in the case of voice input, data already converted to text)

[1508] Output: The request formatted for sending to the server

[1509] Specific operation: The device creates an HTTP request, converts the request content into JSON format, and sends this data to the server when the send button is pressed.

[1510] Step 3:

[1511] Server request reception

[1512] The server receives the request content sent from the terminal.

[1513] Input: Request data sent from the terminal (JSON format)

[1514] Output: Parsed text request

[1515] What happens: The server receives the HTTP request, parses the content, and extracts the text request.

[1516] Step 4:

[1517] Analysis using generative models

[1518] The server passes the request to the generative model, which analyzes the request content using natural language processing.

[1519] Input: Text request (e.g., "Tell me about the formation of black holes")

[1520] Output: A prompt to be passed to the generative model (e.g., "Please provide information about the formation of black holes").

[1521] Specific operation: The server makes an API call to the generative AI model to generate a prompt sentence, which is then input into the generative AI model.

[1522] Step 5:

[1523] Information generation

[1524] Within the server, a generative model generates information appropriate to the request based on the analysis.

[1525] Input: Prompt statement (e.g., "Please provide information about the formation of black holes")

[1526] Output: Generated text information (e.g., "Black holes are formed when a very massive star collapses under its own gravity...")

[1527] Specific operation: The generative AI model generates information based on the prompt sentence and returns the text to the server.

[1528] Step 6:

[1529] Preparing text input for the speech synthesis engine

[1530] The server prepares the generated text to be input to the speech synthesis engine.

[1531] Input: Generated text information

[1532] Output: Text data converted into a format to send to the speech synthesis engine.

[1533] What happens: The server converts the generated text into JSON format and creates an appropriate request to the speech synthesis engine's API.

[1534] Step 7:

[1535] Voice data generation using a voice synthesis engine

[1536] The server sends the text to a speech synthesis engine, which generates natural, fluent speech.

[1537] Input: JSON format text data

[1538] Output: Audio data (e.g. audio file or audio stream)

[1539] How it works: The TTS engine receives text data, generates speech data, and returns the results to the server, adjusting the tone and speed of the speech.

[1540] Step 8:

[1541] Sending audio data

[1542] The server sends the generated audio data to the user's device.

[1543] Input: Generated audio data

[1544] Output: Audio data sent to the user's device

[1545] Specific operation: The server encodes the audio data as an HTTP response and sends it to the user's device.

[1546] Step 9:

[1547] Receiving audio data by the device

[1548] The terminal receives the voice data transmitted from the server.

[1549] Input: Audio data sent from the server

[1550] Output: Audio data in a playable format

[1551] What happens: The device decodes the audio data and converts it into a format that can be played within the application.

[1552] Step 10:

[1553] Device audio playback

[1554] The device plays the audio data, allowing the user to hear it.

[1555] Input: Decoded audio data

[1556] Output: Audio played through speakers or earphones

[1557] Specific behavior: The device's audio player plays the audio data and outputs it to the user as sound.

[1558] (Application example 1)

[1559] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1560] Traditionally, users have had limited means to easily obtain detailed information based on their interests, making it particularly difficult to obtain information while on the move or hands-free. Furthermore, the information generated is often perfunctory, making it difficult for users to learn effectively. Furthermore, if the information generated based on user requests is not properly converted into audio and delivered, learning and comprehension can be hindered.

[1561] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1562] In this invention, the server includes means for receiving a request from a user via text or voice, means for analyzing the request and generating relevant information using a generative model, and means for converting the generated information into voice data using a voice synthesis engine. This allows the user to easily obtain detailed information based on the request in a hands-free manner, and further improves learning and understanding by receiving the generated information in real time via voice. Furthermore, by adding means including a voice synthesis engine for converting the generated text information into voice data and means for inputting prompts into a generative AI model to generate appropriate information, it becomes possible to provide information in a more natural and fluent manner, further improving user convenience.

[1563] "Text or voice input means" means a device or software that provides an interface through which a user can input requests by text or voice.

[1564] "Generative model" refers to a machine learning model or AI system used to generate relevant information based on a user request.

[1565] "Speech synthesis engine" refers to software or hardware for converting text data into natural, fluent speech.

[1566] "Means for transmitting to and playing on a user's device" means a system or protocol that transmits the generated audio data to a user's device and enables the device to play the audio.

[1567] A "prompt" refers to an instruction or question entered into a generative AI model, which triggers the model to generate appropriate information.

[1568] "Generative AI model" refers to an artificial intelligence system that is trained to generate appropriate answers or information from prompts based on machine learning techniques.

[1569] To implement this invention, a user terminal, a server, a generative AI model, and a speech synthesis engine are required.

[1570] The user's device provides an interface that accepts requests input via text or voice. This interface can be realized using the smartphone's screen or voice recognition function. For example, if a user inputs "Tell me about the basic principles of quantum computing," this request will be received by the device.

[1571] The device sends the request to the server, which analyzes it and sends a prompt to the generative AI model. The prompt is a specific instruction or question, such as "User asked: Please tell me about the basic principles of quantum computing. Provide a detailed explanation."

[1572] The generative AI model generates appropriate information based on the received prompt. For example, GPT-3, a model based on machine learning technology, is used as the generative AI model. The generated text information is processed by the server and sent to the speech synthesis engine.

[1573] A speech synthesis engine (e.g., gTTS) converts the generated text into natural, fluent speech, providing information in a way that is easy for users to understand. The tone and speed of the speech can also be adjusted during this process.

[1574] Finally, the generated voice data is sent from the server to the user's device, which then plays it back in real time, allowing the user to easily obtain detailed information based on the request in a hands-free manner, improving learning and comprehension.

[1575] As described above, this invention enables users to easily obtain advanced knowledge and information, and makes it extremely convenient to obtain information while on the move or hands-free.

[1576] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1577] Step 1:

[1578] The user inputs a request via text or voice about a topic they want to learn about. An example of an input request might be, "Teach me about the basic principles of quantum computing." When the device receives the voice input, it uses speech recognition to convert it into text data. This converted text data is sent to the next processing step.

[1579] Input: User's text or voice request

[1580] Output: Text data of the request content

[1581] Step 2:

[1582] The device sends the request content as text data to the server. The server receives this data and begins analyzing the request content. Here, it uses natural language processing technology to analyze and understand the request content.

[1583] Input: Text data of the request content

[1584] Output: Prompt statement used for analysis

[1585] Step 3:

[1586] The server inputs a prompt to the generative AI model. For example, it generates a prompt such as "User asked: Please tell me about the basic principles of quantum computing. Provide a detailed explanation." This prompt is passed to the generative AI model, which then generates information corresponding to the request.

[1587] Input: prompt statement

[1588] Output: Detailed text information generated

[1589] Step 4:

[1590] The generative AI model returns the generated text information to the server, which then sends it to the speech synthesis engine, which generates natural, fluent speech based on the text information. The tone and speed of the speech can then be adjusted as needed.

[1591] Input: Generated detailed text information

[1592] Output: Generated audio data

[1593] Step 5:

[1594] The voice data generated by the speech synthesis engine is returned to the server. The server then sends this voice data to the user's device. The device receives it and plays the voice in real time, allowing the user to receive the requested information by voice.

[1595] Input: Generated audio data

[1596] Output: Audio played on the user's device

[1597] Step 6:

[1598] The device plays the audio data and the user can listen to it. This process allows users to easily obtain detailed information on a particular subject and allows for hands-free learning, especially when on the move.

[1599] Input: Audio data sent to the device

[1600] Output: User experience through the audio data played

[1601] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1602] The present invention is a system that allows users to easily learn a wide range of knowledge and culture. In a specific embodiment, the system analyzes a user's request, generates information based on that request, and provides it as voice using a voice synthesis engine. Furthermore, the present invention incorporates an emotion engine that recognizes the user's emotions, and adds functions for analyzing the request content and adjusting the generated information and voice.

[1603] Specific processing of the program

[1604] Accepting requests from users

[1605] 1. User Input

[1606] Users launch the "Radio of Knowledge" application on their device and input a request for information or a topic they want to hear about, for example, "Tell me about the formation of black holes."

[1607] 2. Terminal Processing

[1608] The device receives the user's input and prepares the request to send to the server. In the case of voice input, the device converts the speech to text.

[1609] Request analysis and emotion recognition

[1610] 1. Server Reception

[1611] The server receives the request content sent from the device and prepares to call the emotion engine and generative model APIs for analysis.

[1612] 2. Activating the Emotional Engine

[1613] The server recognizes emotions from the request content and the user's voice. The emotion engine analyzes the user's emotions (e.g., joy, excitement, anger, sadness, etc.) and passes the results to the generative model.

[1614] 3. Launching the Generative Model

[1615] The server passes the results of the emotion engine and the request content to the generative model, which then analyzes the request content using natural language processing. For example, it extracts detailed information on the topic "formation of black holes."

[1616] Information generation

[1617] 1. Information generation within the server

[1618] The generative model generates information appropriate to the request based on the analysis, such as text like "Black holes are formed when an extremely massive star collapses under its own gravity..."

[1619] 2. Emotion-Based Adjustment

[1620] The server adjusts the generated information based on the user's emotions, for example adding detailed technical explanations if the user is excited.

[1621] Text-to-Speech (TTS) to translate content into audio

[1622] 1. Preparing the text

[1623] The server prepares the generated text to be input to the speech synthesis engine.

[1624] 2. Using a TTS engine

[1625] The server sends the text to a speech synthesis engine, which generates natural, fluent speech, adjusting the tone and speed of the speech according to the user's emotions.

[1626] 3. Generating audio data

[1627] The speech synthesis engine converts the text into speech data, and the generated speech data is returned to the server.

[1628] Content Delivery

[1629] 1. Sending audio data

[1630] The server sends the generated audio data to the user's device.

[1631] 2. Receiving the terminal

[1632] The terminal receives the voice data transmitted from the server.

[1633] 3. Audio playback

[1634] The device plays audio data and allows the user to listen to it, allowing the user to hear information based on the request.

[1635] Specific examples

[1636] For example, if a user requests, "Teach me the basic principles of quantum computing," the following process occurs:

[1637] 1. Accepting user requests

[1638] A user types into a terminal, "Tell me about the basic principles of quantum computing."

[1639] 2. Request analysis and emotion recognition

[1640] The server receives the request, analyzes the user's emotions using the emotion engine, and inputs the request content and the resulting emotions into the generative model.

[1641] 3. Information generation and coordination

[1642] The generative model generates information such as "quantum computers use qubits to have greater computing power than classical computers" and adjusts the information as needed based on the results of the emotion engine.

[1643] 4. Speech Synthesis

[1644] The server sends the generated text to a speech synthesis engine to generate speech data, adjusting the tone and speed depending on the user's emotions.

[1645] 5. Content Delivery

[1646] The server sends the audio data to the user's device, which then plays the audio.

[1647] This allows users to easily acquire knowledge and gain a wide range of knowledge and culture without actively searching. In addition, the emotion engine makes it possible to provide information tailored to each individual user.

[1648] The processing flow will be explained below.

[1649] Step 1:

[1650] The user starts up the device and opens the "Radio of Knowledge" application. The user enters a request into an input field within the application, either by text or voice. For example, the user might enter "Tell me about the formation of black holes."

[1651] Step 2:

[1652] The device accepts user input. If voice input is selected, the device converts the speech to text using a speech recognition engine.

[1653] Step 3:

[1654] The device sends the request text data to the server via internet communication.

[1655] Step 4:

[1656] The server receives the request sent from the device and prepares to call the APIs of the generative model and emotion engine to analyze the received text data.

[1657] Step 5:

[1658] The server starts the emotion engine and recognizes emotions from the received request and the user's voice (in the case of voice input). The emotion engine analyzes the user's emotions (e.g., joy, excitement, anger, sadness, etc.) and passes the results to the generative model.

[1659] Step 6:

[1660] The server launches the generative model and inputs the results of the emotion engine and the request content into the model. The generative model analyzes the request content using natural language processing, extracts and generates relevant information. For example, it generates detailed information on the topic of "black hole formation."

[1661] Step 7:

[1662] The generative model generates information in response to the request, for example the generated text might be "Black holes are formed when a very massive star collapses under its own gravity..."

[1663] Step 8:

[1664] The server uses an emotion engine to tailor the generated text based on the user's emotions, for example, adding more technical details if the user is excited, or a simpler explanation if the user is calm.

[1665] Step 9:

[1666] The server prepares the emotion-adjusted text to be input to the text-to-speech engine (TTS). The server passes the text data to the text-to-speech engine.

[1667] Step 10:

[1668] The text-to-speech engine receives the text data and converts it into speech data. The text-to-speech engine converts the text into natural, fluent speech, adjusting the tone and speed of the speech based on the results of the emotion engine.

[1669] Step 11:

[1670] The generated voice data is returned to the server, which then prepares the received voice data for transmission to the user's device.

[1671] Step 12:

[1672] The server sends the audio data to the user's device via internet communication.

[1673] Step 13:

[1674] The terminal receives the audio data sent from the server, and prepares to play the received audio data for the user.

[1675] Step 14:

[1676] The device plays audio data and allows the user to listen to the audio, allowing the user to hear information based on the request.

[1677] Step 15:

[1678] If a user has an additional question or would like more information during playback, they can enter a new request via text or voice, for example, "Please be more specific."

[1679] Step 16:

[1680] The device receives a new request and prepares to send it to the server. Steps 2 and after are repeated.

[1681] This allows users to easily receive information that is adaptive to their emotions, and to acquire knowledge and culture that meets their individual needs.

[1682] Example 2

[1683] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1684] In today's information society, users seek efficient and flexible information gathering in their busy daily lives. However, conventional information provision systems lack the ability to customize information according to the user's emotions and circumstances, making improving the user experience a challenge. Furthermore, even when providing information via voice, the lack of natural dialogue based on the user's emotions and requests can easily lead to unsatisfactory experiences.

[1685] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1686] In this invention, the server includes means for receiving a request from a user in the form of text or voice, means for analyzing the request and generating related information using a generative model, means for converting the generated information into voice data using a voice synthesis engine, means for transmitting the generated voice data to the user's terminal and playing it back, and means for recognizing the user's emotions and adjusting the generated information and voice based on the emotions. This makes it possible to provide information according to the user's emotions and request content, thereby realizing a personalized, high-quality user experience.

[1687] "User" means any person who uses the System to request information or content.

[1688] A "request" is a request entered by a user via text or voice about the information or topic they wish to know about.

[1689] "Text" refers to information entered in written form.

[1690] "Voice" refers to what the user says.

[1691] "Means" refers to a method or device for achieving a particular purpose.

[1692] "Analysis" refers to the process of understanding a user's request and extracting the appropriate information.

[1693] A "generative model" refers to an artificial intelligence algorithm that generates information based on a user's request.

[1694] "Information" refers to knowledge or data provided in response to a User's request.

[1695] A "speech synthesis engine" refers to software or a system that converts text into natural-sounding speech.

[1696] "Audio data" refers to data in audio format generated by a speech synthesis engine.

[1697] "Terminal" refers to a device used by a user, such as a computer, smartphone, or tablet.

[1698] "Emotion" refers to the user's psychological state, including states such as joy, excitement, anger, and sadness.

[1699] "Emotion recognition" refers to the process of determining a user's psychological state from their request or voice.

[1700] "Adjustment" refers to changing the tone of the generated information or voice to match the user's emotions.

[1701] This invention is an information provision system that allows users to easily learn a wide range of knowledge and culture. It has the function of generating related information based on a request input by the user and providing it as voice. It also recognizes the user's emotions and adjusts the generated information and voice. This system will be specifically described in the following embodiment.

[1702] Accepting requests from users

[1703] Users launch the "Wisdom Radio" application using a device such as a smartphone or PC and request the information or topic they are looking for. For example, they can type, "Tell me about the formation of black holes." The device receives the user's input and prepares it to be sent to the server. If voice input is used, the device uses a microphone to capture voice and converts it into text using a speech recognition API.

[1704] Request analysis and emotion recognition

[1705] The server receives requests sent from the device and prepares to call the APIs of the emotion engine and generative model for analysis. The server recognizes emotions from the request content and the user's voice. The emotion engine analyzes the user's emotions (happiness, excitement, anger, sadness, etc.) and passes the results to the generative model.

[1706] Information generation

[1707] The server passes the results of the emotion engine and the request content to the generative model, which analyzes the request using natural language processing. The generative model generates information appropriate to the request content. For example, it generates detailed information such as, "Black holes are formed when extremely massive stars collapse under their own gravity..." The server adjusts the generated information based on the user's emotions, adding detailed technical information if the user is excited, for example.

[1708] Text-to-Speech (TTS) to translate content into audio

[1709] The server prepares the generated text for input to a speech synthesis engine. The server then sends the text to the speech synthesis engine, which generates natural-sounding speech. During this process, the tone and speed of the speech are also adjusted according to the user's emotions. The speech synthesis engine then converts the text into speech data, which is then sent back to the server.

[1710] Content Delivery

[1711] The server sends the generated voice data to the user's terminal. The terminal receives the voice data sent from the server. Finally, the terminal plays the voice data so that the user can listen to the content. This allows the user to obtain information based on the request audibly.

[1712] Specific hardware and software names to be used

[1713] Hardware: Smartphones, servers (cloud services)

[1714] Software: Intelligent Radio application, speech synthesis engine (e.g., Google Cloud Text-to-Speech, Amazon Polly), generative model (e.g., OpenAI GPT-3), emotion engine (e.g., Microsoft Azure Cognitive Services)

[1715] Examples of concrete examples and prompts

[1716] For example, if a user requests, "Tell me about the basic principles of quantum computing," the following process takes place: The user types, "Tell me about the basic principles of quantum computing," into their device. The server receives the request and analyzes the user's emotions using an emotion engine. The request and the resulting emotion are input into a generative model, which then generates information such as, "Quantum computers use qubits to have greater computing power than conventional computers." The information is adjusted based on the emotion engine's results, and the server sends the generated text to a speech synthesis engine, which generates audio data. The tone and speed are also adjusted according to the user's emotions. Finally, the server sends the audio data to the user's device, which then plays it back.

[1717] Example prompt sentence:

[1718] A user requests, "Tell me about the basic principles of quantum computing."

[1719] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1720] Step 1:

[1721] A user uses a device to launch the "Radio of Knowledge" application and enters a request for information. The request can be text or voice. For example, they could enter, "Tell me about the basic principles of quantum computing."

[1722] Input: User request (text or voice)

[1723] Output: Text or audio data received by the device

[1724] Specific behavior:

[1725] The user types text into the input area and presses the "Send" button.

[1726] For voice input, audio is recorded using a microphone and converted into text using a speech recognition API.

[1727] Step 2:

[1728] The device sends the user's request to the server, which receives the request and prepares to call the emotion engine and generative model APIs.

[1729] Input: Text or voice data received by the device

[1730] Output: Request data sent to the server

[1731] Specific behavior:

[1732] The terminal encodes the request data and sends an HTTP request to the server.

[1733] The server receives the HTTP request, decodes the data, and prepares it for analysis.

[1734] Step 3:

[1735] The server uses an emotion engine to recognize the user's emotions. It analyzes emotions (e.g., joy, excitement, anger, sadness, etc.) from voice or text data and passes the results to a generative model.

[1736] Input: User text or voice data

[1737] Output: Emotion data analyzed by the emotion engine

[1738] Specific behavior:

[1739] The server calls the emotion engine and passes the user's text or voice data as input.

[1740] The emotion engine analyzes the data, identifies the user's emotion, and returns that data.

[1741] Step 4:

[1742] The server uses the generative model to analyze the user's request and generate appropriate information. It also references the results of the emotion engine to create text based on the request.

[1743] Input: User request data and emotion data

[1744] Output: Generated text data

[1745] Specific behavior:

[1746] The server passes the request content and emotion data to the generative model and receives the generated text data.

[1747] The generative model generates text appropriate to the request, e.g., "Quantum computers use qubits to achieve greater computing power than classical computers..."

[1748] Step 5:

[1749] The server prepares the generated text for input to a speech synthesis engine, which converts the text data into natural-sounding speech data.

[1750] Input: Generated text data

[1751] Output: Generated audio data

[1752] Specific behavior:

[1753] The server passes the text data to a speech synthesis engine, such as Google Cloud Text-to-Speech or Amazon Polly.

[1754] A speech synthesis engine converts text data into speech, adjusting tone and speed based on emotional data.

[1755] Step 6:

[1756] The server sends the generated audio data to the user's device, which receives and plays the audio data.

[1757] Input: Generated audio data

[1758] Output: Audio data sent to the device

[1759] Specific behavior:

[1760] The server sends the audio data to the user's device via an HTTP request or WebSocket.

[1761] The device receives the audio data and plays the audio using its built-in audio playback function.

[1762] Examples of concrete examples and prompts

[1763] For example, if a user requests, "Teach me about the basic principles of quantum computing," the following processing occurs: For example, a user requests, "Teach me about the basic principles of quantum computing."

[1764] (Application example 2)

[1765] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1766] Today's users demand immediate and appropriate provision of a wide range of security-related information, but achieving this requires a wide range of technologies. In particular, it is necessary to provide appropriate countermeasures immediately in emergencies, and conventional methods have difficulty responding flexibly while taking into account the user's feelings. Furthermore, the lack of a function to instruct emergency responses means that prompt and appropriate information cannot be provided when users feel anxious.

[1767] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1768] In this invention, the server includes means for receiving a request from a user in text or voice, means for analyzing the request and generating related information using a generative model, means for adjusting the generated information through means for recognizing the user's emotion, means for converting the generated information into voice data using a voice synthesis engine, means for transmitting the generated voice data to the user's terminal and playing it back, and means for instructing an emergency response based on the generated information. This enables appropriate and individual responses to user requests, and can provide a quick and accurate response, particularly in emergencies.

[1769] "User" means any individual or entity that uses this System to request information or services.

[1770] A "request" refers to information or a question a user enters into a system.

[1771] "Text" refers to information composed of characters.

[1772] "Speech" refers to sound information emitted by the human voice.

[1773] "Means" refers to a method or device for achieving a specific purpose.

[1774] "Analysis" refers to understanding and breaking down the content of input information and carrying out appropriate processing.

[1775] A "generative model" refers to an algorithm or program that uses artificial intelligence (AI) to generate text or speech.

[1776] "Relevant information" refers to appropriate data and knowledge provided based on the user's request.

[1777] "Means for recognizing emotions" refers to technologies or devices for analyzing and determining a user's emotional state.

[1778] A "speech synthesis engine" refers to software or a device that converts text information into voice data.

[1779] "Audio Data" means audio information stored in digital format.

[1780] "Terminal" means a device used by a User to access the System and receive information.

[1781] "Playback" refers to converting audio data into sound and letting the user hear it.

[1782] "Means for providing emergency response instructions" refers to technologies and devices that provide users with appropriate guidelines for action in an emergency.

[1783] The present invention relates to a personalized security assistant system that provides appropriate information to a user when the user inputs security-related requests by text or voice. Specifically, the system is configured as follows.

[1784] 1. Accepting requests from users

[1785] Users use a device such as a smartphone to enter security questions or requests into the system by text or voice, such as "I'm hearing strange noises in my house. What should I do?"

[1786] 2. Request analysis and emotion recognition

[1787] The server analyzes the request received from the device. If the request is voice-activated, it converts it to text using Google Cloud Speech-to-Text. It then uses a generative AI model, such as OpenAI GPT-4, to analyze the request and generate an appropriate response. At the same time, it uses the Microsoft Azure Emotion API to recognize the user's emotions and adjusts the information accordingly.

[1788] 3. Information generation and coordination

[1789] Based on the content analyzed by the generative model, information appropriate to the request is generated. The generated information is adjusted according to the user's emotions through an emotion engine. For example, if the user is feeling anxious, specific and detailed measures will be provided.

[1790] 4. Audio conversion of content using speech synthesis

[1791] The adjusted information is then converted into audio data using Google Cloud Text-to-Speech, with the tone and speed of the audio being adjusted according to the user's emotions.

[1792] 5. Content Delivery and Response Options

[1793] The generated voice data is sent from the server to the user's device and played back on the device. It also provides a means to issue emergency response instructions if necessary, allowing the user to notify relevant authorities with a single touch in the event of an emergency.

[1794] Specific examples

[1795] For example, a user might enter a request like this:

[1796] I hear strange noises in my house, what should I do?

[1797] When this request is received, the following process takes place: The audio is converted to text using Google Cloud Speech-to-Text and input into GPT-4. The generated response is analyzed using the Azure Emotion API, and if anxiety is recognized, countermeasure information is provided as a priority. The adjusted text is converted into audio data using Google Cloud Text-to-Speech and sent to the user's device. By receiving the information via audio, the user can quickly take appropriate action.

[1798] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1799] Step 1:

[1800] The user launches the application on a device such as a smartphone and enters a security request.

[1801] Input: The user's voice or text request.

[1802] What it does: The device uses a microphone to record audio or provides a GUI for entering text.

[1803] Output: Audio or text data.

[1804] Step 2:

[1805] The device converts the speech into text (in the case of voice input).

[1806] Input: Audio data.

[1807] How it works: Converts audio data into text using Google Cloud Speech-to-Text.

[1808] Output: The audio data converted into text data.

[1809] Step 3:

[1810] The terminal transmits the text data to the server.

[1811] Input: Text data converted from speech or entered directly.

[1812] Operation: The device sends text data to the server via an HTTP request.

[1813] Output: The text data sent to the server.

[1814] Step 4:

[1815] The server analyzes the request.

[1816] Input: User request text data.

[1817] How it works: The server inputs text data as prompts into the OpenAI GPT-4 model, which then generates an appropriate response using the generative AI model.

[1818] Output: The generated response text.

[1819] Step 5:

[1820] The server analyzes the user's emotions.

[1821] Input: The user's request text data and voice data (if any).

[1822] How it works: The server uses the Microsoft Azure Emotion API to perform emotion analysis and determine the user's emotional state.

[1823] Output: Parsed emotion data.

[1824] Step 6:

[1825] The server adjusts the generated response text based on the emotion data.

[1826] Input: Generated response text and sentiment data.

[1827] How it works: The server adjusts the response text based on the emotion data, such as adding more information if anxiety is detected.

[1828] Output: The adjusted response text.

[1829] Step 7:

[1830] The server converts the adjusted text into audio data.

[1831] Input: The tailored response text.

[1832] How it works: Uses Google Cloud Text-to-Speech to convert text data into audio data.

[1833] Output: The generated audio data.

[1834] Step 8:

[1835] The server sends the generated audio data to the user's device.

[1836] Input: The generated audio data.

[1837] How it works: The server sends the audio data to the user's device as an HTTP response.

[1838] Output: The audio data sent to the user's device.

[1839] Step 9:

[1840] The device plays the audio data and the user listens to the information.

[1841] Input: Audio data sent from the server.

[1842] Behavior: The device uses its audio player function to play audio data and provides audio to the user.

[1843] Output: The user hears a voice and gets the appropriate security information.

[1844] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1845] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1846] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1847] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1848] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1849] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1850] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1851] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1852] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1853] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1854] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1855] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1856] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1857] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1858] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1859] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1860] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1861] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1862] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1863] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1864] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1865] The following is further disclosed regarding the above embodiment.

[1866] (Claim 1)

[1867] A means of accepting requests from users via text or voice;

[1868] means for analyzing the request and generating relevant information using a generative model;

[1869] A means for converting the generated information into voice data using a voice synthesis engine;

[1870] means for transmitting the generated voice data to a user's terminal and playing it back;

[1871] A system including:

[1872] (Claim 2)

[1873] 10. The system of claim 1, wherein the generated information includes information for answering additional questions based on a user request.

[1874] (Claim 3)

[1875] 10. The system of claim 1, further comprising means for adjusting the length of the content to accommodate a user-specified time (e.g., travel time).

[1876] "Example 1"

[1877] (Claim 1)

[1878] A means of accepting requests from users via text or voice;

[1879] means for analyzing the request and generating relevant information using a generative model;

[1880] A means for converting the generated information into voice data using a voice synthesis engine;

[1881] means for transmitting the generated voice data to a user's terminal and playing it back;

[1882] A means of passing user requests to a generative model to extract information;

[1883] means for converting the generated voice data by adjusting the tone and speed of the voice;

[1884] A system including:

[1885] (Claim 2)

[1886] 10. The system of claim 1, wherein the generated information includes information for answering additional questions based on a user request.

[1887] (Claim 3)

[1888] 10. The system of claim 1, further comprising means for adjusting the length of the content to accommodate a user-specified time (e.g., travel time).

[1889] "Application Example 1"

[1890] (Claim 1)

[1891] A means of accepting requests from users via text or voice;

[1892] means for analyzing the request and generating relevant information using a generative model;

[1893] A means for converting the generated information into voice data using a voice synthesis engine;

[1894] means for transmitting the generated voice data to a user's terminal and playing it back;

[1895] means including a speech synthesis engine for converting the generated text information into speech data;

[1896] A means for inputting prompts into a generative AI model to generate appropriate information; and

[1897] A system including:

[1898] (Claim 2)

[1899] 10. The system of claim 1, wherein the generated information includes information for answering additional questions based on a user request.

[1900] (Claim 3)

[1901] 10. The system of claim 1, further comprising means for adjusting the length of the content to accommodate a user-specified time (e.g., travel time).

[1902] "Example 2: Combining Emotion Engines"

[1903] (Claim 1)

[1904] A means of accepting requests from users via text or voice;

[1905] means for analyzing the request and generating relevant information using a generative model;

[1906] A means for converting the generated information into voice data using a voice synthesis engine;

[1907] means for transmitting the generated voice data to a user's terminal and playing it back;

[1908] means for recognizing a user's emotion and adjusting the generated information and audio based on the emotion;

[1909] A system including:

[1910] (Claim 2)

[1911] 10. The system of claim 1, wherein the generated information includes information for answering additional questions based on a user request.

[1912] (Claim 3)

[1913] 10. The system of claim 1, further comprising means for adjusting the length of the content to accommodate a user-specified time (e.g., travel time).

[1914] "Application example 2 when combining emotion engines"

[1915] (Claim 1)

[1916] A means of accepting requests from users via text or voice;

[1917] means for analyzing the request and generating relevant information using a generative model;

[1918] a means for adjusting the generated information via a means for recognizing a user's emotion;

[1919] A means for converting the generated information into voice data using a voice synthesis engine;

[1920] means for transmitting the generated voice data to a user's terminal and playing it back;

[1921] A system including:

[1922] (Claim 2)

[1923] 10. The system of claim 1, wherein the generated information includes information for answering additional questions based on a user request.

[1924] (Claim 3)

[1925] 10. The system of claim 1, further comprising means for adjusting the length of the content to fit a user-specified time period.

[1926] (Claim 4)

[1927] 10. The system of claim 1, further comprising means for directing an emergency response based on the generated information. [Explanation of symbols]

[1928] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of accepting requests from users via text or voice; means for analyzing the request and generating relevant information using a generative model; A means for converting the generated information into voice data using a voice synthesis engine; means for transmitting the generated voice data to a user's terminal and playing it back; A system including:

2. The system of claim 1 , wherein the generated information includes information for answering additional questions at the user's request.

3. 10. The system of claim 1, further comprising means for adjusting the length of the content to fit a user-specified time.

Images