System

Abstract

A system is provided.SOLUTION: A system, comprising: means for inputting content to be narrated by a user; means for sending the inputted text to a server; means for generating, by the server, an explanatory text using a generative AI model; means for returning, by the server, the generated explanatory text to a device; and means for converting, by the device, the explanatory text into audio and reading the audio aloud by a AI narrator.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In traditional meetings, presenters often spend long periods of time explaining technical terms and background information in detail, placing a heavy burden on the presenter. Furthermore, the audience often loses concentration as they listen to lengthy explanations, resulting in an incomplete understanding of the content. This reduces the efficiency of the entire meeting and is detrimental to all involved. Therefore, new methods are needed to reduce the burden on presenters while promoting audience understanding and maintaining concentration. [Means for solving the problem]

[0005] The present invention includes a means for a user to input content to be narrated and a means for transmitting the input text to a server. The server generates explanatory text using a generative AI model and returns the generated explanatory text to the terminal. The terminal then converts the explanatory text into audio data and provides a means for an AI narrator to read it aloud. This provides a system that reduces the burden on presenters who give long explanations in traditional conferences, maintains the audience's concentration, and promotes understanding.

[0006] "User" refers to a presenter or information provider who inputs the content to be narrated in a conference.

[0007] "Terminal" refers to a device that processes the narration content entered by the user, transmits it to a server, and converts it into audio data.

[0008] "Server" refers to a computer system that has the function of generating explanatory text from the narration content using a generative AI model and returning that explanatory text to the terminal.

[0009] A "generative AI model" refers to an artificial intelligence model that includes algorithms and mechanisms that use natural language processing to generate appropriate descriptions from input text.

[0010] "Explanation" refers to text data generated by a generative AI model to supplement and explain the narration content entered by the user.

[0011] "Audio data" refers to a data format converted to be played as audio based on explanatory text.

[0012] "AI narrator" refers to a speech synthesis engine or system that uses voice data to read explanatory text aloud.

[0013] "Conference software" refers to an application or platform that includes all the functionality required for presenters to input narration content and for AI narration to be performed. [Brief explanation of the drawings]

[0014] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0015] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0016] First, the terms used in the following description will be explained.

[0017] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0018] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0019] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0020] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0021] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0022] [First embodiment]

[0023] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0024] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0025] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0026] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0027] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0028] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0029] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0030] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0031] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0032] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0033] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0034] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0035] This invention relates to a system that aims to reduce the burden on users when narrating technical terms and background information during meetings and improve the audience's concentration and comprehension. The system implements a series of processes: the user inputs the content to be narrated, the content is converted into explanatory text using a generative AI model, and finally, an AI narrator reads it aloud.

[0036] An embodiment of this system is described in detail below.

[0037] 1. User operations

[0038] User: Enters the content to be narrated (e.g., technical terms and their background) through the conferencing software interface. By entering text such as "What is machine learning?" into the input field, the system prepares to generate an explanation for that content.

[0039] 2. From the user to the device

[0040] Terminal: Receives text data entered by the user, converts it into an API request format, and then sends the request to the specified server endpoint.

[0041] 3. From the device to the server

[0042] Server: Receives requests at the API endpoint, analyzes the request content, and passes the analyzed content to the generative AI model.

[0043] 4. Server Processing

[0044] Server: Provides the analyzed text to a generative AI model (e.g., a natural language processing model) as input data. The generative AI model outputs an explanation based on "What is machine learning?" An example of a generated explanation would be "Machine learning is a technology that uses data to enable computers to learn on their own and perform specific tasks."

[0045] 5. From the server to the device

[0046] Server: Converts the generated text data into JSON or XML format and returns it to the requesting device as a response.

[0047] Terminal: Analyzes the received response data and prepares to convert it into voice data.

[0048] 6. Terminal Processing

[0049] Terminal: The received description is converted from text to voice data using a text-to-speech engine.

[0050] 7. AI Narrator Reading

[0051] Device: Passes the generated audio data to the AI ​​narrator and gives instructions to read it out at appropriate times during the meeting.

[0052] AI Narrator: Plays back audio data and provides generated explanations to meeting participants by voice, allowing the presenter to do other tasks while the explanation is being given, and the audience can deepen their understanding by listening to the audio explanation.

[0053] Specific examples

[0054] Meeting Scenarios

[0055] User (Presenter): You have a meeting where you need to explain a new technology called "machine learning" to the leaders of each department.

[0056] Narration content: The presenter enters the narration content "What is machine learning?" into the conference software.

[0057] System Operation

[0058] 1. A user types "What is machine learning?" into the conferencing software.

[0059] 2. The device sends the text data to the server as an API request.

[0060] 3. The server receives the request and generates a description using a generative AI model.

[0061] 4. The server returns the generated description to the device.

[0062] 5. The device receives the description and converts it into audio data.

[0063] 6. The AI ​​narrator plays the audio and reads, "Machine learning is a technology that uses data to teach computers how to perform specific tasks."

[0064] In this way, the present invention provides a system that reduces the burden on the presenter, promotes audience understanding, and helps them maintain their concentration.

[0065] The processing flow will be explained below.

[0066] Step 1:

[0067] A user types what they want to narrate into the conferencing software interface, for example, by typing "What is machine learning" into a text field.

[0068] Step 2:

[0069] The device receives the user's input and converts the text data into an API request, which includes details about the narration and any required explanations.

[0070] Step 3:

[0071] The device sends an API request to the server at the specified endpoint.

[0072] Step 4:

[0073] The server receives the API request, parses the request body, and prepares the parsed text data.

[0074] Step 5:

[0075] The server passes the parsed text data to a generative AI model, which then generates explanatory text based on the input data.

[0076] Step 6:

[0077] The server converts the description returned from the generative AI model into JSON or XML format.

[0078] Step 7:

[0079] The server returns the converted description data to the device as an API response.

[0080] Step 8:

[0081] The device receives the API response and parses the description data.

[0082] Step 9:

[0083] The device passes the analyzed description to a Text-to-Speech (TTS) engine and converts it into audio data.

[0084] Step 10:

[0085] The device passes the generated audio data to the AI ​​narrator, which is instructed on the timing of audio playback.

[0086] Step 11:

[0087] The AI ​​narrator plays back the audio data and provides audible explanations to meeting participants, allowing the presenter to focus on other tasks while the audience listens to the explanations to deepen their understanding.

[0088] Example 1

[0089] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0090] The present invention aims to reduce the burden on users when explaining technical terms and background information during meetings, thereby improving the audience's concentration and comprehension. In such situations, presenters are required to convey specialized information in an easy-to-understand manner, but doing so requires a great deal of effort and time. Currently, there is no effective system for reducing this burden, so the present invention provides a system that solves this problem.

[0091] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0092] In this invention, the server includes a means for a user to input content to be narrated, a means for transmitting the input text to a communication device, a means for the communication device to transmit the input text to an endpoint of the server in the form of an API request, a means for the server to analyze the request and generate explanatory text using a generative AI model, a means for the server to return the generated explanatory text to the communication device, and a means for the communication device to convert the explanatory text into audio data and play the audio. This allows presenters to easily deliver expert explanations and enables audiences to deepen their understanding through the audio explanations.

[0093] A "user" is someone who uses the system to input narration content and initiates the process of generating the description and playing it back.

[0094] A "communication device" refers to equipment or software for sending and receiving data between a user and a server, and is responsible for sending text entered by the user to the server and receiving responses from the server.

[0095] A "server" is a device or program that analyzes requests received from users, generates explanatory text using a generative AI model, and returns the generated explanatory text to the communication device.

[0096] A "generative AI model" is an artificial intelligence model that uses natural language processing algorithms to generate relevant explanatory text from input text.

[0097] An "API request format" refers to a standardized data format used by a communications device to send data to a server, typically including JSON or XML formats.

[0098] A "speech synthesis engine" is software or hardware that converts text data into voice data and has text-to-speech functionality.

[0099] "Explanation" is text data generated by a generative AI model that contains detailed explanations of technical terms and background information entered by the user.

[0100] "Audio data" is a data format that can reproduce explanatory text as audio, generated by a speech synthesis engine.

[0101] "Audio playback" refers to the act of an AI narrator or other audio output device using the generated audio data to play audio.

[0102] The present invention provides a system that aims to reduce the burden on users when narrating technical terms and background information during a meeting, thereby improving the audience's concentration and comprehension. This system allows users to easily deliver technical explanations, and allows the audience to deepen their understanding through the audio explanations. An embodiment of the present invention is described in detail below.

[0103] User operations

[0104] The user inputs the content to be narrated through the conferencing software interface. For example, the user enters the text "What is machine learning?" into the conferencing software's text input field and clicks the submit button. This prepares the system to convert the content into explanatory text.

[0105] Transmitting text data to a communication device

[0106] The communications device receives the text data entered by the user and converts it into an API request format. For example, it converts it into JSON format data. This API request is then sent as an HTTP request to the server's endpoint. An HTTP POST request is used, and the URL is "https: / / api.example.com / generate-explanation."

[0107] Sending the request to the server and parsing it

[0108] The server analyzes the request received at the API endpoint, checks the JSON data of the request, and obtains the value of the "query" key. The obtained text, "What is machine learning?", is passed to the generative AI model.

[0109] Generating explanatory text using a generative AI model

[0110] The server generates an explanation by providing the text analyzed using the generative AI model as input data. The generative AI model (e.g., a natural language processing model) is invoked to generate an explanation based on the prompt "What is machine learning?" The generated explanation is "Machine learning is a technology that uses data to enable computers to learn on their own and perform specific tasks."

[0111] Returning generated data to communication device and converting it into voice

[0112] The server converts the generated explanation into JSON format and returns it to the communication device. The communication device analyzes the received response data and obtains the value of the "explanation" key. A text-to-speech engine is used to convert the obtained text data into audio data. For example, Google's TTS API is used to convert the text into an audio file.

[0113] Audio playback by AI narrator

[0114] The communication device passes the generated audio data to the AI ​​narrator, which then issues instructions to read aloud, at an appropriate time during the meeting, "Machine learning is a technology that uses data to enable computers to learn on their own and perform specific tasks." The AI ​​narrator then plays back the audio data and provides the generated explanation to the meeting participants.

[0115] This process reduces the burden on the user (presenter) to explain technical information in an easy-to-understand manner, and allows the audience to understand the content through their eyes and ears.

[0116] Specific examples

[0117] Meeting Scenarios

[0118] The user (presenter) holds a meeting to explain the new technology "machine learning" to the leaders of each department. The presenter enters the narration content "What is machine learning?" into the conference software.

[0119] System Operation

[0120] During a meeting, a user enters "What is machine learning?" into a text input field and presses the send button. The communication device sends this text data to the server as an API request. The server uses a generative AI model to generate an explanation, which it returns to the communication device: "Machine learning is a technology that uses data to enable computers to learn on their own and perform specific tasks." The communication device then converts this explanation into audio data, which is then played back by an AI narrator.

[0121] In this way, the present invention provides a system that reduces the burden on users and promotes audience understanding.

[0122] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0123] Step 1:

[0124] The user enters the content to be narrated through the conferencing software interface. The user enters "What is machine learning?" in the text input field and presses the send button. Input (text): "What is machine learning?" Output (text data): "What is machine learning?" text data.

[0125] Step 2:

[0126] The device receives text data sent by the user. Since the server cannot understand the received text data as it is, the device converts it into an API request format. Specifically, it converts it into JSON format and creates the data structure "{"query": "What is machine learning"}". Input (text data): Text entered by the user, Output (API request): JSON format data.

[0127] Step 3:

[0128] The terminal sends the converted API request as an HTTP request to the server's API endpoint. The URL used is "https: / / api.example.com / generate-explanation". Input (API request): JSON format data, Output (request to server): HTTP POST request.

[0129] Step 4:

[0130] The server analyzes the request received at the API endpoint. The server parses the JSON data of the request and extracts the value of the "query" key. The extracted text, "What is machine learning?", is passed to the generative AI model. Input (HTTP request): API request data, Output (text data): Analyzed text data.

[0131] Step 5:

[0132] The server uses a generative AI model to generate an explanation based on the analyzed text, "What is machine learning?" The generative AI model uses a natural language processing algorithm to generate an explanation based on the prompt. An example output is the explanation, "Machine learning is a technology that uses data to enable computers to learn on their own and perform specific tasks." Input (text data): Analyzed text data, Output (explanation): Generated explanation.

[0133] Step 6:

[0134] The server converts the generated explanation into JSON format and returns it to the requesting device as an HTTP response. For example, the format is "{"explanation": "Machine learning is a technology that uses data to enable computers to learn on their own and perform specific tasks."}". Input (explanation): Generated explanation, Output (HTTP response): JSON format data.

[0135] Step 7:

[0136] The device analyzes the JSON format response data received from the server. It obtains the value of the "explanation" key and uses a speech synthesis engine to convert the text data into audio data. The synthesized speech engine uses Google's TTS API or similar to generate an audio file. Input (HTTP response): JSON format data, Output (audio data): Generated audio file.

[0137] Step 8:

[0138] The device passes the generated audio data to the AI ​​narrator, who then plays the audio data at the appropriate time during the meeting, reading aloud, "Machine learning is a technology that uses data to enable computers to learn on their own and perform specific tasks." Input (audio data): Generated audio file; Output (audio playback): Audio provided to meeting participants.

[0139] This series of processes reduces the burden of explanation on the presenter and allows the audience to deepen their understanding.

[0140] (Application example 1)

[0141] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0142] In recent years, with the spread of virtual stores, users have been demanding that they be able to easily obtain detailed information about products. However, many current virtual store systems have difficulty quickly and accurately providing the detailed explanations and background information that users require. Furthermore, when users ask questions about products, it is also difficult to provide appropriate answers and explanations in real time. This makes it difficult to increase user satisfaction and purchasing motivation.

[0143] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0144] In this invention, the server includes means for a user to input content to be narrated, means for transmitting the input text to the server, means for generating a description using a generative AI model, means for returning the generated description to the terminal, means for the terminal to convert the description into audio data and have an AI narrator read it aloud, means for the terminal to provide detailed descriptions of products in the virtual store in real time using the generative AI model, and means for customers to input questions through the virtual store interface. This allows users to simply input a question to obtain a detailed description using the generative AI model in real time and to listen to that description in audio.

[0145] A "user" is a user of the system who inputs narration content and questions.

[0146] "Narration" is the act of explaining information orally in a meeting or presentation.

[0147] "Input means" refers to an interface through which a user provides text data to the system.

[0148] "Input text" is character string data such as narration content or questions that a user inputs into the system.

[0149] A "server" is a computing system that processes input data and generates new data using generative AI models.

[0150] A "generative AI model" is an algorithm that uses natural language processing technology and generates explanations and answers based on input text.

[0151] An "explanation" is a document created by a generative AI model that contains detailed information related to the input text.

[0152] A "terminal" is a device operated by a user, and is a device that communicates with a server to send and receive data.

[0153] "Audio data" is data obtained by converting text data into audio, and is data for providing information to the user by audio.

[0154] "AI Narrator" is a virtual assistant that plays audio data and reads the generated explanation aloud.

[0155] A "virtual store" is a virtual store that operates on the Internet and is a platform where users can browse and purchase products online.

[0156] A "detailed description" is a description that includes detailed information about a product or service.

[0157] An "interface" is a part that includes an operation screen and means for exchanging information between the user and the system.

[0158] A "question" is a short sentence or document that allows a user to enter information they want to know about a product or service.

[0159] The present invention relates to a system that automatically generates detailed explanations and provides voice guidance when a user inputs a question about a product in a virtual store. Specific embodiments for carrying out the present invention will be described below.

[0160] First, a user enters a question through a virtual store interface, which could be a web browser or a mobile application. Suppose a user enters a question such as, "What are the features of this smartwatch?"

[0161] The terminal (client device) converts the text data entered by the user into an API request format and sends it to the specified server. This process uses an HTTP request, and the request content is encoded in JSON format or similar.

[0162] The server receives requests at an API endpoint. The received request content is passed to a generative AI model. The generative AI model uses natural language processing technology (e.g., BERT or GPT-3). The model generates an appropriate explanation for the input question.

[0163] The generated description is sent back to the device as a response from the server. The device then analyzes the received description and converts it into audio data. This process uses a speech synthesis engine such as the Google Text-to-Speech (gTTS) engine.

[0164] Once the audio data is generated, an AI narrator on the device plays it back and provides the generated explanation to the user, allowing users to simply input their question, receive a detailed explanation in real time using the generative AI model, and listen to that explanation in audio.

[0165] For example, if a user wants to know more about a new product, a "smartwatch," in a virtual store, they can type, "What are the features of this smartwatch?" The system sends this question to the server, and using a generative AI model, generates a description such as, "This smartwatch is a gadget with a long battery life, waterproof functionality, and comprehensive health tracking capabilities." The description is then converted into audio data and provided to the user by an AI narrator.

[0166] This allows the user to quickly and accurately obtain detailed information about products in the virtual store and use it as a basis for making a purchase decision.

[0167] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0168] Step 1:

[0169] The user inputs a question through the virtual store interface. The user inputs a question into the interface, such as "What are the features of this smartwatch?" The input data is sent to the terminal as a text string. In this step, the input data is the user's question text, and the output data is the text string received by the terminal.

[0170] Step 2:

[0171] The device converts the question text received from the user into an API request format and sends it to the specified server. Specifically, it encodes the text data into JSON format and sends it as an HTTP POST request. In this step, the input data is the user's question text, and the output data is the request converted into JSON format.

[0172] Step 3:

[0173] The server receives requests at the API endpoint. It analyzes the request content and passes it to the generative AI model. This process uses the Python requests library, etc. Specifically, it extracts the necessary text from the JSON-formatted data and passes it to the generative AI model. In this step, the input data is the JSON-formatted request, and the output data is the text data passed to the generative AI model.

[0174] Step 4:

[0175] A generative AI model generates a detailed explanation based on the input question text. For example, a natural language processing model such as GPT-3 or BERT is used. Specifically, it processes the input text as a prompt and outputs a generated explanation. In this step, the input data is the question text, and the output data is the generated explanation.

[0176] Step 5:

[0177] The server returns the generated description to the requesting terminal as a response. The response data is encoded in JSON format and sent as an HTTP response. In this step, the input data is the generated description, and the output data is a JSON-formatted response.

[0178] Step 6:

[0179] The device analyzes the received response data and converts it into voice data. Specifically, it uses a speech synthesis engine such as the Google Text-to-Speech (gTTS) engine to convert the text data into voice data. In this step, the input data is a JSON-formatted response, and the output data is voice data.

[0180] Step 7:

[0181] The AI ​​narrator plays the generated audio data and provides the user with audio information. Specifically, the audio is played through the device's speaker. In this step, the input data is the audio data, and the output data is the audio information provided to the user.

[0182] In this way, the user can quickly and accurately obtain detailed information about products in the virtual store and hear their explanations in audio.

[0183] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0184] This invention relates to a system that aims to reduce the burden on users when narrating technical terms and background information during meetings, thereby improving the audience's concentration and comprehension. The system implements a series of processes: the user inputs the content to be narrated, the content is converted into explanatory text using a generative AI model, and finally, an AI narrator reads it aloud. It also combines an emotion engine to analyze the user's emotions and dynamically adjust the explanatory text and audio data accordingly.

[0185] Specific embodiments will be described below.

[0186] 1. User operations

[0187] User: Enters the content to be narrated through the conferencing software interface. For example, the user enters the text "What is machine learning?" into the input field.

[0188] 2. From the user to the device

[0189] Terminal: Receives text data entered by the user and passes it to the emotion engine. The emotion engine analyzes the user's emotions from the text and voice data. It converts the data containing the analysis results into an API request format and sends it to the server endpoint.

[0190] 3. From the device to the server

[0191] Server: Receives the API request, analyzes the request body, and prepares the analyzed text data and emotion data.

[0192] 4. Server Processing

[0193] Server: Passes the analyzed text data and emotion data to the generative AI model, which generates a description based on the input data and adjusts the tone and content of the description based on the emotion data as needed.

[0194] 5. From the server to the device

[0195] Server: Converts the generated description into an appropriate format (for example, JSON or XML) and sends it back to the requesting device as a response.

[0196] Terminal: Analyzes the received response data and obtains the explanatory data.

[0197] 6. Terminal Processing

[0198] Device: The acquired description is passed to a Text-to-Speech (TTS) engine and converted into audio data. It is also possible to adjust the tone and speed of the audio based on emotional data.

[0199] 7. AI Narrator Reading

[0200] Device: Passes the generated voice data to the AI ​​narrator and issues instructions to read it aloud at the appropriate time.

[0201] AI Narrator: Plays back audio data and provides audio explanations to meeting participants, allowing the presenter to focus on other tasks while the audience listens to the explanations to deepen their understanding.

[0202] Specific examples

[0203] Meeting Scenarios

[0204] User (Presenter): You have a meeting where you need to explain a new technology called "machine learning" to the leaders of each department.

[0205] Narration content: The presenter enters the narration content "What is machine learning?" into the conference software.

[0206] System Operation

[0207] 1. A user types "What is machine learning?" into the conferencing software.

[0208] 2. The device receives the text data and sends an API request to the server, including the analysis results from the emotion engine.

[0209] 3. The server receives the request and generates a description using a generative AI model, adjusting the tone and content of the description as needed based on the sentiment data.

[0210] 4. The server returns the generated description to the device.

[0211] 5. The device receives the description and converts it into audio data, adjusting the tone and speed of the audio based on the emotional data.

[0212] 6. The AI ​​narrator plays the audio and reads, "Machine learning is a technology that uses data to teach computers how to perform specific tasks."

[0213] In this way, the presenter can reduce the burden on the presenter, promote the audience's understanding, and maintain their concentration. Furthermore, by combining it with an emotion engine, the presenter can provide flexible narration that responds to the user's emotions.

[0214] The processing flow will be explained below.

[0215] Step 1:

[0216] A user types what they want to narrate into the conferencing software interface, for example, by typing "What is machine learning" into a text field.

[0217] Step 2:

[0218] The terminal receives the user's input text and passes the data to the emotion engine, which analyzes the user's emotions from the text data and voice data and generates an analysis result.

[0219] Step 3:

[0220] The device converts the data, including the analysis results of the emotion engine, into an API request format, which includes the narration content and emotion data.

[0221] Step 4:

[0222] The device sends an API request to the server at the specified endpoint.

[0223] Step 5:

[0224] The server receives the API request, parses the request body, and prepares the parsed text data and emotion data.

[0225] Step 6:

[0226] The server passes the analyzed text data and emotion data to a generative AI model, which generates a description based on the user's input and adjusts the tone and content of the description based on the emotion data as needed.

[0227] Step 7:

[0228] The server converts the generated description into JSON or XML format and sends it back to the requesting device as a response.

[0229] Step 8:

[0230] The device receives the API response and parses and extracts the description data.

[0231] Step 9:

[0232] The device passes the extracted description to a Text-to-Speech (TTS) engine and converts it into audio data, which can adjust the tone and speed of the voice based on emotional data.

[0233] Step 10:

[0234] The device passes the generated audio data to the AI ​​narrator and instructs it on the timing of audio playback.

[0235] Step 11:

[0236] The AI ​​narrator plays back the audio data and provides audible explanations to meeting participants, allowing the presenter to focus on other tasks while the audience listens to the explanations to deepen their understanding.

[0237] Specific examples

[0238] A user types "What is machine learning?" into the conferencing software.

[0239] The device receives the user's input, and the emotion engine analyzes the user's emotion. For example, if the user is nervous, it analyzes it as "nervous."

[0240] The device sends an API request containing text data and emotion data to the server.

[0241] The server receives the request and uses a generative AI model to generate a description such as, "Machine learning is a technique that uses data to enable computers to teach themselves and perform specific tasks."

[0242] The server returns the generated explanation to the terminal.

[0243] The device receives the text and passes it to a TTS engine to convert it into speech data. Based on the emotional data, the tone and speed of the speech are adjusted to create a tone that alleviates tension.

[0244] The AI ​​narrator plays the audio and reads, "Machine learning is a technique that uses data to enable computers to teach themselves and perform specific tasks."

[0245] Example 2

[0246] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0247] Currently, narrating technical terms and background information during meetings can be a significant burden for presenters, potentially reducing the audience's concentration and comprehension. Furthermore, it is difficult to adapt flexibly to the user's emotions, limiting the effectiveness of narration. There is a need for a system that can resolve these issues and promote effective communication and understanding.

[0248] The specification process by the specification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes a means for a user to input content to be narrated, a means for receiving the input text and analyzing emotions using an analysis engine, a means for converting the analyzed data into an API request format and sending it to the server, a means for the server to generate explanatory text using a generation AI model, a means for adjusting the tone and content of the generated explanatory text based on the emotional data, a means for returning the generated explanatory text to the terminal, and a means for the terminal to convert the explanatory text into audio data and have an AI narrator read it aloud. This reduces the burden on the presenter in narrating and improves the audience's concentration and comprehension.

[0249] "User" refers to the person who uses this system to provide narration during a meeting.

[0250] "Narration content" refers to text data such as technical terms and background information that a user wants to explain during a conference.

[0251] "Terminal" refers to the computer or mobile device that a user uses to enter narration content.

[0252] An "analysis engine" refers to software or hardware that analyzes a user's emotions from input text data or voice data.

[0253] "Emotion data" refers to information about the user's emotions analyzed by the analysis engine, and includes emotion labels such as "neutral" and "excited."

[0254] "API request" refers to the protocol used by the device to send data to the server, including parsed text data and emotion data.

[0255] "Server" refers to the computational resources on the network that are responsible for processing the analyzed data and generating narratives using generative AI models.

[0256] "Generative AI model" refers to the artificial intelligence technology used by the server to generate the description, including, for example, a natural language processing model.

[0257] "Explanatory text" refers to text generated by a generative AI model that clearly explains the content of the user's narration.

[0258] "Text-to-Speech (TTS) engine" refers to software for converting text data into voice data.

[0259] "AI narrator" refers to software or hardware that plays back generated audio data and provides audio narration.

[0260] "Audio Data" refers to the audio files generated by the Text-to-Speech engine and played by the AI ​​narrator.

[0261] "Tone" refers to the emotional resonance or tone of a voice or text, and is adjusted based on emotional data.

[0262] This invention relates to a system that reduces the burden on users when narrating technical terms and background information during meetings, improving audience concentration and comprehension. The system implements a series of processes: the user inputs the content to be narrated, the content is converted into explanatory text using a generative AI model, and finally, an AI narrator reads it aloud. It also incorporates an emotion engine to analyze the user's emotions and dynamically adjust the explanatory text and audio data accordingly.

[0263] Specifically, a user enters the content to be narrated into a text field through the conferencing software interface. For example, consider a user entering "What is machine learning?" This input is received by the device and then analyzed by an emotion engine (e.g., IBM Watson Tone Analyzer). The emotion engine analyzes the user's emotions and generates emotion data such as "neutral" or "excited."

[0264] The analysis results are converted into an API request format and sent to a server. The server receives this API request and generates a description using a generative AI model (e.g., OpenAI GPT-3). The tone and content of the generated description may be adjusted based on the emotional data. For example, a description might be generated that reads, "Machine learning is a technology that uses data to enable computers to learn on their own and perform specific tasks."

[0265] The generated description is converted into an appropriate format, such as JSON, by the server and sent back to the device. The device receives and analyzes this data, then passes it to a Text-to-Speech (TTS) engine (e.g., Google Cloud Text-to-Speech) to convert it into audio data. The tone and speed of the voice are also adjusted based on the emotion data. For example, if a "neutral" emotion label is added, the voice will be generated in a calm and composed tone.

[0266] Finally, the device passes the generated audio data to the AI ​​narrator, which then issues instructions to read it aloud at the appropriate time. For example, the user can manually instruct the narrator to read it aloud during the presentation. The AI ​​narrator plays back the audio data and reads out loud, "Machine learning is a technology that uses data to enable computers to learn on their own and perform specific tasks," making it easier for the audience to understand the content.

[0267] Specific examples

[0268] Meeting Scenarios

[0269] A meeting is held in which the user (presenter) needs to explain a new technology called "machine learning" to the leaders of each department.

[0270] Narration content: The presenter enters the narration content "What is machine learning?" into the conference software.

[0271] Prompt Sentence Examples

[0272] The user types the following into the conferencing software:

[0273] What is machine learning?

[0274] This system reduces the burden on presenters, promotes audience understanding, and helps maintain concentration. In addition, by combining it with an emotion engine, it enables flexible narration that responds to the user's emotions.

[0275] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0276] System processing flow

[0277] Step 1:

[0278] The user enters the content to be narrated into a text field through the conferencing software interface. For example, they might enter "What is machine learning?" This input is then sent to the terminal as text data required for the next process.

[0279] Input: User-entered text "What is machine learning?"

[0280] Output: Text data "What is machine learning?"

[0281] Step 2:

[0282] The device receives the text data entered by the user. The received text data is then passed to an emotion engine (such as IBM Watson Tone Analyzer or a similar analysis engine) for analysis. The emotion engine analyzes the text data and generates an emotion label, such as "neutral" or "excited."

[0283] Input: Text data "What is machine learning?"

[0284] Output: Emotion data (e.g. "Neutral")

[0285] Step 3:

[0286] The device converts the analyzed text data and emotion data into an API request format. This API request format data is in a format for sending to the server. The converted data is then sent to the server.

[0287] Input: Text data "What is machine learning?", emotion data "Neutral"

[0288] Output: Data in API request format

[0289] Step 4:

[0290] The server receives the API request and parses the request body, taking the parsed text and sentiment data and feeding it into a generative AI model (such as OpenAI GPT-3 or a similar natural language processing model). This model combines the original text and sentiment data to generate a description.

[0291] Input: Data in API request format (text data "What is machine learning?", sentiment data "Neutral")

[0292] Output: Generated description (e.g., "Machine learning is a technique that uses data to teach computers how to perform specific tasks.")

[0293] Step 5:

[0294] The server converts the generated description into an appropriate format (e.g., JSON or XML) and sends it back to the requesting device as a response.

[0295] Input: Generated description

[0296] Output: JSON format data returned to the terminal

[0297] Step 6:

[0298] The device analyzes the response data received and obtains the generated explanation. This explanation is then passed to a Text-to-Speech (TTS) engine (such as Google Cloud Text-to-Speech or a similar voice synthesis engine) to convert it into voice data. At this time, the tone and speed of the voice are also adjusted based on the emotional data.

[0299] Input: JSON format data (generated description)

[0300] Output: Audio data

[0301] Step 7:

[0302] The device then passes the generated audio data to the AI ​​narrator and issues instructions to read it aloud at the appropriate time. For example, the user can manually instruct the AI ​​narrator to read it aloud during a presentation.

[0303] The AI ​​narrator plays the audio and provides audible explanations to meeting participants, such as, "Machine learning is a technology that uses data to enable computers to teach themselves and perform specific tasks."

[0304] Input: Audio data

[0305] Output: Audio instructions are provided to meeting participants

[0306] This series of steps allows users to provide easy-to-understand explanations to the audience while reducing the burden of narration. In addition, the introduction of an emotion engine enables flexible narration that responds to the user's emotions.

[0307] (Application example 2)

[0308] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0309] In manufacturing sites, there is a need for robots to efficiently and reliably provide instructions on work tasks and procedures. Detailed, easy-to-understand explanations are necessary for workers to understand the setup and operation procedures of new machines, but conventional systems have not been able to adequately provide this. There is also a need for technology that can reduce the burden on workers and reduce errors.

[0310] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a means for a user to input the content to be narrated, a means for transmitting the input text to the server, a means for the server to generate explanatory text using a generative AI model, a means for the server to return the generated explanatory text to the terminal, a means for the terminal to convert the explanatory text into audio data and for an AI narrator to read it aloud, and a means for the robot to provide audio and visual work instructions. This provides detailed and easy-to-understand explanations of work procedures at the manufacturing site, making it easier for workers to understand and reducing operational errors.

[0311] "User" refers to the person or operator who operates the system and inputs the narration content.

[0312] "Narration content" refers to the text information of explanations and comments that the user inputs to the system.

[0313] "Input text" refers to text data of the narration content input by the user to the system.

[0314] A "server" refers to a computer on a network that receives requests from users, uses a generative AI model to generate explanatory text, and returns it to the device.

[0315] A "generative AI model" refers to an artificial intelligence model that uses natural language processing technology to convert input text into explanatory text.

[0316] "Explanation" refers to the text information of explanations or descriptions generated by a generative AI model from input text.

[0317] "Terminal" refers to a device that receives user input and communicates with a server to convert generated descriptions into audio data.

[0318] "Audio data" refers to data in audio format that has been converted from explanatory text using a speech synthesis engine.

[0319] "AI Narrator" refers to a speech synthesis system that uses generated voice data to read explanations aloud on behalf of the user.

[0320] A "speech synthesis engine" refers to software or hardware for converting input text data into voice data.

[0321] "Robot" refers to an automated mechanical device that is placed on a manufacturing floor and is capable of providing audio and visual instructions for work.

[0322] A specific system for implementing this invention includes an interface where a user inputs narration content, a terminal for processing text data, a server that generates explanatory text using a generative AI model, and a robot that provides audio and visual explanations.

[0323] System configuration

[0324] 1. User Interface:

[0325] An interface for users to input narration content (e.g., input devices such as PCs and tablets).

[0326] The interface has the function of sending input text to the server.

[0327] 2. Terminal:

[0328] Receives input text and sends it to the server.

[0329] It has the function of receiving the description data returned from the server and converting it into audio data (example of use: Python library gTTS, etc.).

[0330] 3. Server:

[0331] Use a generative AI model (e.g., GPT-3) to generate a description from the input text.

[0332] A response including the description and associated emotion data is sent back to the device.

[0333] The server uses natural language processing technology to analyze the user's input and generate effective explanations.

[0334] 4. Speech synthesis engine:

[0335] It is responsible for converting explanatory text into audio data (e.g. gTTS library).

[0336] By adjusting the tone and speed of the generated voice data, the narration is performed according to the user's emotions.

[0337] 5. Robots:

[0338] An automated mechanical device that provides audio and visual instructions for work.

[0339] It has the function of playing back audio data and visually displaying specific work procedures.

[0340] System operation example

[0341] Consider a scenario where a user is explaining the setup procedure for a new manufacturing machine. The system works as follows:

[0342] 1. User Input:

[0343] The user enters the following into the interface: "I'd like to explain the setup procedure for a new machine."

[0344] 2. Sending input text:

[0345] The device sends the input text to the server, which also obtains emotional data via an emotion analysis engine.

[0346] 3. Description generation:

[0347] The server generates an explanation using a generative AI model, for example, using the following prompt:

[0348] Here is the procedure for setting up a new manufacturing machine.

[0349] 4. Return of description:

[0350] The server sends the generated instructions back to the terminal. For example, it generates instructions like, "First, connect the machine to the power source. Next, select "Settings" from the main menu to perform the initial setup and follow the instructions. After that, install each part in the correct position. Check that all parts are installed correctly."

[0351] 5. Generate audio data:

[0352] The device converts the received text into audio data, which the robot reads aloud. A speech synthesis engine narrates in a tone and speed that matches the user's emotions.

[0353] 6. Robot explanation:

[0354] The robot provides audio and visual instructions, allowing the worker to carry out the setup procedure with clear guidance.

[0355] In this way, work procedures at the manufacturing site can be explained in detail and in an easy-to-understand manner, promoting understanding among workers and reducing operational errors.

[0356] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0357] Step 1:

[0358] The user enters the narration content into the interface, for example, "I will explain the setup procedure for a new machine." The input text is sent to the terminal.

[0359] Step 2:

[0360] The device sends the received input text to the emotion engine, which analyzes the emotional data. The emotion engine analyzes the text data and extracts emotional data (e.g., notification tone and emotional intensity). This generates an API request containing the emotional data and sends it to the server.

[0361] Step 3:

[0362] The server receives the API request and parses the request body. It extracts the input text and emotion data and passes them to the generative AI model. The server then uses the generative AI model to generate an explanation. Specifically, the following prompt sentence is used as input:

[0363] Here is the procedure for setting up a new manufacturing machine.

[0364] The generative AI model generates explanatory text based on input text and sentiment data.

[0365] Step 4:

[0366] The server converts the generated description into an appropriate format (e.g., JSON) and sends a response containing the description and emotion data back to the device.

[0367] Step 5:

[0368] The device analyzes the response data received from the server, extracts explanatory text data and passes it to the speech synthesis engine, and adjusts the tone and speed of the generated speech based on the emotional data.

[0369] Step 6:

[0370] The device uses a speech synthesis engine (e.g., gTTS) to convert the description into audio data, which is then ready to be played by the AI ​​narrator.

[0371] Step 7:

[0372] The AI ​​narrator plays audio data at specified times, and the robot uses this audio data to explain the work.The robot also visually demonstrates the work steps and provides specific instructions to the worker.

[0373] Through the above processing steps, the system reduces the burden on users, effectively communicates work procedures at the manufacturing site, and reduces operational errors.

[0374] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0375] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0376] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0377] [Second embodiment]

[0378] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0379] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0380] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0381] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0382] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0383] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0384] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0385] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0386] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0387] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0388] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0389] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0390] This invention relates to a system that aims to reduce the burden on users when narrating technical terms and background information during meetings and improve the audience's concentration and comprehension. The system implements a series of processes: the user inputs the content to be narrated, the content is converted into explanatory text using a generative AI model, and finally, an AI narrator reads it aloud.

[0391] An embodiment of this system is described in detail below.

[0392] 1. User operations

[0393] User: Enters the content to be narrated (e.g., technical terms and their background) through the conferencing software interface. By entering text such as "What is machine learning?" into the input field, the system prepares to generate an explanation for that content.

[0394] 2. From the user to the device

[0395] Terminal: Receives text data entered by the user, converts it into an API request format, and then sends the request to the specified server endpoint.

[0396] 3. From the device to the server

[0397] Server: Receives requests at the API endpoint, analyzes the request content, and passes the analyzed content to the generative AI model.

[0398] 4. Server Processing

[0399] Server: Provides the analyzed text to a generative AI model (e.g., a natural language processing model) as input data. The generative AI model outputs an explanation based on "What is machine learning?" An example of a generated explanation would be "Machine learning is a technology that uses data to enable computers to learn on their own and perform specific tasks."

[0400] 5. From the server to the device

[0401] Server: Converts the generated text data into JSON or XML format and returns it to the requesting device as a response.

[0402] Terminal: Analyzes the received response data and prepares to convert it into voice data.

[0403] 6. Terminal Processing

[0404] Terminal: The received description is converted from text to voice data using a text-to-speech engine.

[0405] 7. AI Narrator Reading

[0406] Device: Passes the generated audio data to the AI ​​narrator and gives instructions to read it out at appropriate times during the meeting.

[0407] AI Narrator: Plays back audio data and provides generated explanations to meeting participants by voice, allowing the presenter to do other tasks while the explanation is being given, and the audience can deepen their understanding by listening to the audio explanation.

[0408] Specific examples

[0409] Meeting Scenarios

[0410] User (Presenter): You have a meeting where you need to explain a new technology called "machine learning" to the leaders of each department.

[0411] Narration content: The presenter enters the narration content "What is machine learning?" into the conference software.

[0412] System Operation

[0413] 1. A user types "What is machine learning?" into the conferencing software.

[0414] 2. The device sends the text data to the server as an API request.

[0415] 3. The server receives the request and generates a description using a generative AI model.

[0416] 4. The server returns the generated description to the device.

[0417] 5. The device receives the description and converts it into audio data.

[0418] 6. The AI ​​narrator plays the audio and reads, "Machine learning is a technology that uses data to teach computers how to perform specific tasks."

[0419] In this way, the present invention provides a system that reduces the burden on the presenter, promotes audience understanding, and helps them maintain their concentration.

[0420] The processing flow will be explained below.

[0421] Step 1:

[0422] A user types what they want to narrate into the conferencing software interface, for example, by typing "What is machine learning" into a text field.

[0423] Step 2:

[0424] The device receives the user's input and converts the text data into an API request, which includes details about the narration and any required explanations.

[0425] Step 3:

[0426] The device sends an API request to the server at the specified endpoint.

[0427] Step 4:

[0428] The server receives the API request, parses the request body, and prepares the parsed text data.

[0429] Step 5:

[0430] The server passes the parsed text data to a generative AI model, which then generates explanatory text based on the input data.

[0431] Step 6:

[0432] The server converts the description returned from the generative AI model into JSON or XML format.

[0433] Step 7:

[0434] The server returns the converted description data to the device as an API response.

[0435] Step 8:

[0436] The device receives the API response and parses the description data.

[0437] Step 9:

[0438] The device passes the analyzed description to a Text-to-Speech (TTS) engine and converts it into audio data.

[0439] Step 10:

[0440] The device passes the generated audio data to the AI ​​narrator, which is instructed on the timing of audio playback.

[0441] Step 11:

[0442] The AI ​​narrator plays back the audio data and provides audible explanations to meeting participants, allowing the presenter to focus on other tasks while the audience listens to the explanations to deepen their understanding.

[0443] Example 1

[0444] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0445] The present invention aims to reduce the burden on users when explaining technical terms and background information during meetings, thereby improving the audience's concentration and comprehension. In such situations, presenters are required to convey specialized information in an easy-to-understand manner, but doing so requires a great deal of effort and time. Currently, there is no effective system for reducing this burden, so the present invention provides a system that solves this problem.

[0446] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0447] In this invention, the server includes a means for a user to input content to be narrated, a means for transmitting the input text to a communication device, a means for the communication device to transmit the input text to an endpoint of the server in the form of an API request, a means for the server to analyze the request and generate explanatory text using a generative AI model, a means for the server to return the generated explanatory text to the communication device, and a means for the communication device to convert the explanatory text into audio data and play the audio. This allows presenters to easily deliver expert explanations and enables audiences to deepen their understanding through the audio explanations.

[0448] A "user" is someone who uses the system to input narration content and initiates the process of generating the description and playing it back.

[0449] A "communication device" refers to equipment or software for sending and receiving data between a user and a server, and is responsible for sending text entered by the user to the server and receiving responses from the server.

[0450] A "server" is a device or program that analyzes requests received from users, generates explanatory text using a generative AI model, and returns the generated explanatory text to the communication device.

[0451] A "generative AI model" is an artificial intelligence model that uses natural language processing algorithms to generate relevant explanatory text from input text.

[0452] An "API request format" refers to a standardized data format used by a communications device to send data to a server, typically including JSON or XML formats.

[0453] A "speech synthesis engine" is software or hardware that converts text data into voice data and has text-to-speech functionality.

[0454] "Explanation" is text data generated by a generative AI model that contains detailed explanations of technical terms and background information entered by the user.

[0455] "Audio data" is a data format that can reproduce explanatory text as audio, generated by a speech synthesis engine.

[0456] "Audio playback" refers to the act of an AI narrator or other audio output device using the generated audio data to play audio.

[0457] The present invention provides a system that aims to reduce the burden on users when narrating technical terms and background information during a meeting, thereby improving the audience's concentration and comprehension. This system allows users to easily deliver technical explanations, and allows the audience to deepen their understanding through the audio explanations. An embodiment of the present invention is described in detail below.

[0458] User operations

[0459] The user inputs the content to be narrated through the conferencing software interface. For example, the user enters the text "What is machine learning?" into the conferencing software's text input field and clicks the submit button. This prepares the system to convert the content into explanatory text.

[0460] Transmitting text data to a communication device

[0461] The communications device receives the text data entered by the user and converts it into an API request format. For example, it converts it into JSON format data. This API request is then sent as an HTTP request to the server's endpoint. An HTTP POST request is used, and the URL is "https: / / api.example.com / generate-explanation."

[0462] Sending the request to the server and parsing it

[0463] The server analyzes the request received at the API endpoint, checks the JSON data of the request, and obtains the value of the "query" key. The obtained text, "What is machine learning?", is passed to the generative AI model.

[0464] Generating explanatory text using a generative AI model

[0465] The server generates an explanation by providing the text analyzed using the generative AI model as input data. The generative AI model (e.g., a natural language processing model) is invoked to generate an explanation based on the prompt "What is machine learning?" The generated explanation is "Machine learning is a technology that uses data to enable computers to learn on their own and perform specific tasks."

[0466] Returning generated data to communication device and converting it into voice

[0467] The server converts the generated explanation into JSON format and returns it to the communication device. The communication device analyzes the received response data and obtains the value of the "explanation" key. A text-to-speech engine is used to convert the obtained text data into audio data. For example, Google's TTS API is used to convert the text into an audio file.

[0468] Audio playback by AI narrator

[0469] The communication device passes the generated audio data to the AI ​​narrator, which then issues instructions to read aloud, at an appropriate time during the meeting, "Machine learning is a technology that uses data to enable computers to learn on their own and perform specific tasks." The AI ​​narrator then plays back the audio data and provides the generated explanation to the meeting participants.

[0470] This process reduces the burden on the user (presenter) to explain technical information in an easy-to-understand manner, and allows the audience to understand the content through their eyes and ears.

[0471] Specific examples

[0472] Meeting Scenarios

[0473] The user (presenter) holds a meeting to explain the new technology "machine learning" to the leaders of each department. The presenter enters the narration content "What is machine learning?" into the conference software.

[0474] System Operation

[0475] During a meeting, a user enters "What is machine learning?" into a text input field and presses the send button. The communication device sends this text data to the server as an API request. The server uses a generative AI model to generate an explanation, which it returns to the communication device: "Machine learning is a technology that uses data to enable computers to learn on their own and perform specific tasks." The communication device then converts this explanation into audio data, which is then played back by an AI narrator.

[0476] In this way, the present invention provides a system that reduces the burden on users and promotes audience understanding.

[0477] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0478] Step 1:

[0479] The user enters the content to be narrated through the conferencing software interface. The user enters "What is machine learning?" in the text input field and presses the send button. Input (text): "What is machine learning?" Output (text data): "What is machine learning?" text data.

[0480] Step 2:

[0481] The device receives text data sent by the user. Since the server cannot understand the received text data as it is, the device converts it into an API request format. Specifically, it converts it into JSON format and creates the data structure "{"query": "What is machine learning"}". Input (text data): Text entered by the user, Output (API request): JSON format data.

[0482] Step 3:

[0483] The terminal sends the converted API request as an HTTP request to the server's API endpoint. The URL used is "https: / / api.example.com / generate-explanation". Input (API request): JSON format data, Output (request to server): HTTP POST request.

[0484] Step 4:

[0485] The server analyzes the request received at the API endpoint. The server parses the JSON data of the request and extracts the value of the "query" key. The extracted text, "What is machine learning?", is passed to the generative AI model. Input (HTTP request): API request data, Output (text data): Analyzed text data.

[0486] Step 5:

[0487] The server uses a generative AI model to generate an explanation based on the analyzed text, "What is machine learning?" The generative AI model uses a natural language processing algorithm to generate an explanation based on the prompt. An example output is the explanation, "Machine learning is a technology that uses data to enable computers to learn on their own and perform specific tasks." Input (text data): Analyzed text data, Output (explanation): Generated explanation.

[0488] Step 6:

[0489] The server converts the generated explanation into JSON format and returns it to the requesting device as an HTTP response. For example, the format is "{"explanation": "Machine learning is a technology that uses data to enable computers to learn on their own and perform specific tasks."}". Input (explanation): Generated explanation, Output (HTTP response): JSON format data.

[0490] Step 7:

[0491] The device analyzes the JSON format response data received from the server. It obtains the value of the "explanation" key and uses a speech synthesis engine to convert the text data into audio data. The synthesized speech engine uses Google's TTS API or similar to generate an audio file. Input (HTTP response): JSON format data, Output (audio data): Generated audio file.

[0492] Step 8:

[0493] The device passes the generated audio data to the AI ​​narrator, who then plays the audio data at the appropriate time during the meeting, reading aloud, "Machine learning is a technology that uses data to enable computers to learn on their own and perform specific tasks." Input (audio data): Generated audio file; Output (audio playback): Audio provided to meeting participants.

[0494] This series of processes reduces the burden of explanation on the presenter and allows the audience to deepen their understanding.

[0495] (Application example 1)

[0496] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0497] In recent years, with the spread of virtual stores, users have been demanding that they be able to easily obtain detailed information about products. However, many current virtual store systems have difficulty quickly and accurately providing the detailed explanations and background information that users require. Furthermore, when users ask questions about products, it is also difficult to provide appropriate answers and explanations in real time. This makes it difficult to increase user satisfaction and purchasing motivation.

[0498] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0499] In this invention, the server includes means for a user to input content to be narrated, means for transmitting the input text to the server, means for generating a description using a generative AI model, means for returning the generated description to the terminal, means for the terminal to convert the description into audio data and have an AI narrator read it aloud, means for the terminal to provide detailed descriptions of products in the virtual store in real time using the generative AI model, and means for customers to input questions through the virtual store interface. This allows users to simply input a question to obtain a detailed description using the generative AI model in real time and to listen to that description in audio.

[0500] A "user" is a user of the system who inputs narration content and questions.

[0501] "Narration" is the act of explaining information orally in a meeting or presentation.

[0502] "Input means" refers to an interface through which a user provides text data to the system.

[0503] "Input text" is character string data such as narration content or questions that a user inputs into the system.

[0504] A "server" is a computing system that processes input data and generates new data using generative AI models.

[0505] A "generative AI model" is an algorithm that uses natural language processing technology and generates explanations and answers based on input text.

[0506] An "explanation" is a document created by a generative AI model that contains detailed information related to the input text.

[0507] A "terminal" is a device operated by a user, and is a device that communicates with a server to send and receive data.

[0508] "Audio data" is data obtained by converting text data into audio, and is data for providing information to the user by audio.

[0509] "AI Narrator" is a virtual assistant that plays audio data and reads the generated explanation aloud.

[0510] A "virtual store" is a virtual store that operates on the Internet and is a platform where users can browse and purchase products online.

[0511] A "detailed description" is a description that includes detailed information about a product or service.

[0512] An "interface" is a part that includes an operation screen and means for exchanging information between the user and the system.

[0513] A "question" is a short sentence or document that allows a user to enter information they want to know about a product or service.

[0514] The present invention relates to a system that automatically generates detailed explanations and provides voice guidance when a user inputs a question about a product in a virtual store. Specific embodiments for carrying out the present invention will be described below.

[0515] First, a user enters a question through a virtual store interface, which could be a web browser or a mobile application. Suppose a user enters a question such as, "What are the features of this smartwatch?"

[0516] The terminal (client device) converts the text data entered by the user into an API request format and sends it to the specified server. This process uses an HTTP request, and the request content is encoded in JSON format or similar.

[0517] The server receives requests at an API endpoint. The received request content is passed to a generative AI model. The generative AI model uses natural language processing technology (e.g., BERT or GPT-3). The model generates an appropriate explanation for the input question.

[0518] The generated description is sent back to the device as a response from the server. The device then analyzes the received description and converts it into audio data. This process uses a speech synthesis engine such as the Google Text-to-Speech (gTTS) engine.

[0519] Once the audio data is generated, an AI narrator on the device plays it back and provides the generated explanation to the user, allowing users to simply input their question, receive a detailed explanation in real time using the generative AI model, and listen to that explanation in audio.

[0520] For example, if a user wants to know more about a new product, a "smartwatch," in a virtual store, they can type, "What are the features of this smartwatch?" The system sends this question to the server, and using a generative AI model, generates a description such as, "This smartwatch is a gadget with a long battery life, waterproof functionality, and comprehensive health tracking capabilities." The description is then converted into audio data and provided to the user by an AI narrator.

[0521] This allows the user to quickly and accurately obtain detailed information about products in the virtual store and use it as a basis for making a purchase decision.

[0522] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0523] Step 1:

[0524] The user inputs a question through the virtual store interface. The user inputs a question into the interface, such as "What are the features of this smartwatch?" The input data is sent to the terminal as a text string. In this step, the input data is the user's question text, and the output data is the text string received by the terminal.

[0525] Step 2:

[0526] The device converts the question text received from the user into an API request format and sends it to the specified server. Specifically, it encodes the text data into JSON format and sends it as an HTTP POST request. In this step, the input data is the user's question text, and the output data is the request converted into JSON format.

[0527] Step 3:

[0528] The server receives requests at the API endpoint. It analyzes the request content and passes it to the generative AI model. This process uses the Python requests library, etc. Specifically, it extracts the necessary text from the JSON-formatted data and passes it to the generative AI model. In this step, the input data is the JSON-formatted request, and the output data is the text data passed to the generative AI model.

[0529] Step 4:

[0530] A generative AI model generates a detailed explanation based on the input question text. For example, a natural language processing model such as GPT-3 or BERT is used. Specifically, it processes the input text as a prompt and outputs a generated explanation. In this step, the input data is the question text, and the output data is the generated explanation.

[0531] Step 5:

[0532] The server returns the generated description to the requesting terminal as a response. The response data is encoded in JSON format and sent as an HTTP response. In this step, the input data is the generated description, and the output data is a JSON-formatted response.

[0533] Step 6:

[0534] The device analyzes the received response data and converts it into voice data. Specifically, it uses a speech synthesis engine such as the Google Text-to-Speech (gTTS) engine to convert the text data into voice data. In this step, the input data is a JSON-formatted response, and the output data is voice data.

[0535] Step 7:

[0536] The AI ​​narrator plays the generated audio data and provides the user with audio information. Specifically, the audio is played through the device's speaker. In this step, the input data is the audio data, and the output data is the audio information provided to the user.

[0537] In this way, the user can quickly and accurately obtain detailed information about products in the virtual store and hear their explanations in audio.

[0538] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0539] This invention relates to a system that aims to reduce the burden on users when narrating technical terms and background information during meetings, thereby improving the audience's concentration and comprehension. The system implements a series of processes: the user inputs the content to be narrated, the content is converted into explanatory text using a generative AI model, and finally, an AI narrator reads it aloud. It also combines an emotion engine to analyze the user's emotions and dynamically adjust the explanatory text and audio data accordingly.

[0540] Specific embodiments will be described below.

[0541] 1. User operations

[0542] User: Enters the content to be narrated through the conferencing software interface. For example, the user enters the text "What is machine learning?" into the input field.

[0543] 2. From the user to the device

[0544] Terminal: Receives text data entered by the user and passes it to the emotion engine. The emotion engine analyzes the user's emotions from the text and voice data. It converts the data containing the analysis results into an API request format and sends it to the server endpoint.

[0545] 3. From the device to the server

[0546] Server: Receives the API request, analyzes the request body, and prepares the analyzed text data and emotion data.

[0547] 4. Server Processing

[0548] Server: Passes the analyzed text data and emotion data to the generative AI model, which generates a description based on the input data and adjusts the tone and content of the description based on the emotion data as needed.

[0549] 5. From the server to the device

[0550] Server: Converts the generated description into an appropriate format (for example, JSON or XML) and sends it back to the requesting device as a response.

[0551] Terminal: Analyzes the received response data and obtains the explanatory data.

[0552] 6. Terminal Processing

[0553] Device: The acquired description is passed to a Text-to-Speech (TTS) engine and converted into audio data. It is also possible to adjust the tone and speed of the audio based on emotional data.

[0554] 7. AI Narrator Reading

[0555] Device: Passes the generated voice data to the AI ​​narrator and issues instructions to read it aloud at the appropriate time.

[0556] AI Narrator: Plays back audio data and provides audio explanations to meeting participants, allowing the presenter to focus on other tasks while the audience listens to the explanations to deepen their understanding.

[0557] Specific examples

[0558] Meeting Scenarios

[0559] User (Presenter): You have a meeting where you need to explain a new technology called "machine learning" to the leaders of each department.

[0560] Narration content: The presenter enters the narration content "What is machine learning?" into the conference software.

[0561] System Operation

[0562] 1. A user types "What is machine learning?" into the conferencing software.

[0563] 2. The device receives the text data and sends an API request to the server, including the analysis results from the emotion engine.

[0564] 3. The server receives the request and generates a description using a generative AI model, adjusting the tone and content of the description as needed based on the sentiment data.

[0565] 4. The server returns the generated description to the device.

[0566] 5. The device receives the description and converts it into audio data, adjusting the tone and speed of the audio based on the emotional data.

[0567] 6. The AI ​​narrator plays the audio and reads, "Machine learning is a technology that uses data to teach computers how to perform specific tasks."

[0568] In this way, the presenter can reduce the burden on the presenter, promote the audience's understanding, and maintain their concentration. Furthermore, by combining it with an emotion engine, the presenter can provide flexible narration that responds to the user's emotions.

[0569] The processing flow will be explained below.

[0570] Step 1:

[0571] A user types what they want to narrate into the conferencing software interface, for example, by typing "What is machine learning" into a text field.

[0572] Step 2:

[0573] The terminal receives the user's input text and passes the data to the emotion engine, which analyzes the user's emotions from the text data and voice data and generates an analysis result.

[0574] Step 3:

[0575] The device converts the data, including the analysis results of the emotion engine, into an API request format, which includes the narration content and emotion data.

[0576] Step 4:

[0577] The device sends an API request to the server at the specified endpoint.

[0578] Step 5:

[0579] The server receives the API request, parses the request body, and prepares the parsed text data and emotion data.

[0580] Step 6:

[0581] The server passes the analyzed text data and emotion data to a generative AI model, which generates a description based on the user's input and adjusts the tone and content of the description based on the emotion data as needed.

[0582] Step 7:

[0583] The server converts the generated description into JSON or XML format and sends it back to the requesting device as a response.

[0584] Step 8:

[0585] The device receives the API response and parses and extracts the description data.

[0586] Step 9:

[0587] The device passes the extracted description to a Text-to-Speech (TTS) engine and converts it into audio data, which can adjust the tone and speed of the voice based on emotional data.

[0588] Step 10:

[0589] The device passes the generated audio data to the AI ​​narrator and instructs it on the timing of audio playback.

[0590] Step 11:

[0591] The AI ​​narrator plays back the audio data and provides audible explanations to meeting participants, allowing the presenter to focus on other tasks while the audience listens to the explanations to deepen their understanding.

[0592] Specific examples

[0593] A user types "What is machine learning?" into the conferencing software.

[0594] The device receives the user's input, and the emotion engine analyzes the user's emotion. For example, if the user is nervous, it analyzes it as "nervous."

[0595] The device sends an API request containing text data and emotion data to the server.

[0596] The server receives the request and uses a generative AI model to generate a description such as, "Machine learning is a technique that uses data to enable computers to teach themselves and perform specific tasks."

[0597] The server returns the generated explanation to the terminal.

[0598] The device receives the text and passes it to a TTS engine to convert it into speech data. Based on the emotional data, the tone and speed of the speech are adjusted to create a tone that alleviates tension.

[0599] The AI ​​narrator plays the audio and reads, "Machine learning is a technique that uses data to enable computers to teach themselves and perform specific tasks."

[0600] Example 2

[0601] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0602] Currently, narrating technical terms and background information during meetings can be a significant burden for presenters, potentially reducing the audience's concentration and comprehension. Furthermore, it is difficult to adapt flexibly to the user's emotions, limiting the effectiveness of narration. There is a need for a system that can resolve these issues and promote effective communication and understanding.

[0603] The specification process by the specification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes a means for a user to input content to be narrated, a means for receiving the input text and analyzing emotions using an analysis engine, a means for converting the analyzed data into an API request format and sending it to the server, a means for the server to generate explanatory text using a generation AI model, a means for adjusting the tone and content of the generated explanatory text based on the emotional data, a means for returning the generated explanatory text to the terminal, and a means for the terminal to convert the explanatory text into audio data and have an AI narrator read it aloud. This reduces the burden on the presenter in narrating and improves the audience's concentration and comprehension.

[0604] "User" refers to the person who uses this system to provide narration during a meeting.

[0605] "Narration content" refers to text data such as technical terms and background information that a user wants to explain during a conference.

[0606] "Terminal" refers to the computer or mobile device that a user uses to enter narration content.

[0607] An "analysis engine" refers to software or hardware that analyzes a user's emotions from input text data or voice data.

[0608] "Emotion data" refers to information about the user's emotions analyzed by the analysis engine, and includes emotion labels such as "neutral" and "excited."

[0609] "API request" refers to the protocol used by the device to send data to the server, including parsed text data and emotion data.

[0610] "Server" refers to the computational resources on the network that are responsible for processing the analyzed data and generating narratives using generative AI models.

[0611] "Generative AI model" refers to the artificial intelligence technology used by the server to generate the description, including, for example, a natural language processing model.

[0612] "Explanatory text" refers to text generated by a generative AI model that clearly explains the content of the user's narration.

[0613] "Text-to-Speech (TTS) engine" refers to software for converting text data into voice data.

[0614] "AI narrator" refers to software or hardware that plays back generated audio data and provides audio narration.

[0615] "Audio Data" refers to the audio files generated by the Text-to-Speech engine and played by the AI ​​narrator.

[0616] "Tone" refers to the emotional resonance or tone of a voice or text, and is adjusted based on emotional data.

[0617] This invention relates to a system that reduces the burden on users when narrating technical terms and background information during meetings, improving audience concentration and comprehension. The system implements a series of processes: the user inputs the content to be narrated, the content is converted into explanatory text using a generative AI model, and finally, an AI narrator reads it aloud. It also incorporates an emotion engine to analyze the user's emotions and dynamically adjust the explanatory text and audio data accordingly.

[0618] Specifically, a user enters the content to be narrated into a text field through the conferencing software interface. For example, consider a user entering "What is machine learning?" This input is received by the device and then analyzed by an emotion engine (e.g., IBM Watson Tone Analyzer). The emotion engine analyzes the user's emotions and generates emotion data such as "neutral" or "excited."

[0619] The analysis results are converted into an API request format and sent to a server. The server receives this API request and generates a description using a generative AI model (e.g., OpenAI GPT-3). The tone and content of the generated description may be adjusted based on the emotional data. For example, a description might be generated that reads, "Machine learning is a technology that uses data to enable computers to learn on their own and perform specific tasks."

[0620] The generated description is converted into an appropriate format, such as JSON, by the server and sent back to the device. The device receives and analyzes this data, then passes it to a Text-to-Speech (TTS) engine (e.g., Google Cloud Text-to-Speech) to convert it into audio data. The tone and speed of the voice are also adjusted based on the emotion data. For example, if a "neutral" emotion label is added, the voice will be generated in a calm and composed tone.

[0621] Finally, the device passes the generated audio data to the AI ​​narrator, which then issues instructions to read it aloud at the appropriate time. For example, the user can manually instruct the narrator to read it aloud during the presentation. The AI ​​narrator plays back the audio data and reads out loud, "Machine learning is a technology that uses data to enable computers to learn on their own and perform specific tasks," making it easier for the audience to understand the content.

[0622] Specific examples

[0623] Meeting Scenarios

[0624] A meeting is held in which the user (presenter) needs to explain a new technology called "machine learning" to the leaders of each department.

[0625] Narration content: The presenter enters the narration content "What is machine learning?" into the conference software.

[0626] Prompt Sentence Examples

[0627] The user types the following into the conferencing software:

[0628] What is machine learning?

[0629] This system reduces the burden on presenters, promotes audience understanding, and helps maintain concentration. In addition, by combining it with an emotion engine, it enables flexible narration that responds to the user's emotions.

[0630] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0631] System processing flow

[0632] Step 1:

[0633] The user enters the content to be narrated into a text field through the conferencing software interface. For example, they might enter "What is machine learning?" This input is then sent to the terminal as text data required for the next process.

[0634] Input: User-entered text "What is machine learning?"

[0635] Output: Text data "What is machine learning?"

[0636] Step 2:

[0637] The device receives the text data entered by the user. The received text data is then passed to an emotion engine (such as IBM Watson Tone Analyzer or a similar analysis engine) for analysis. The emotion engine analyzes the text data and generates an emotion label, such as "neutral" or "excited."

[0638] Input: Text data "What is machine learning?"

[0639] Output: Emotion data (e.g. "Neutral")

[0640] Step 3:

[0641] The device converts the analyzed text data and emotion data into an API request format. This API request format data is in a format for sending to the server. The converted data is then sent to the server.

[0642] Input: Text data "What is machine learning?", emotion data "Neutral"

[0643] Output: Data in API request format

[0644] Step 4:

[0645] The server receives the API request and parses the request body, taking the parsed text and sentiment data and feeding it into a generative AI model (such as OpenAI GPT-3 or a similar natural language processing model). This model combines the original text and sentiment data to generate a description.

[0646] Input: Data in API request format (text data "What is machine learning?", sentiment data "Neutral")

[0647] Output: Generated description (e.g., "Machine learning is a technique that uses data to teach computers how to perform specific tasks.")

[0648] Step 5:

[0649] The server converts the generated description into an appropriate format (e.g., JSON or XML) and sends it back to the requesting device as a response.

[0650] Input: Generated description

[0651] Output: JSON format data returned to the terminal

[0652] Step 6:

[0653] The device analyzes the response data received and obtains the generated explanation. This explanation is then passed to a Text-to-Speech (TTS) engine (such as Google Cloud Text-to-Speech or a similar voice synthesis engine) to convert it into voice data. At this time, the tone and speed of the voice are also adjusted based on the emotional data.

[0654] Input: JSON format data (generated description)

[0655] Output: Audio data

[0656] Step 7:

[0657] The device then passes the generated audio data to the AI ​​narrator and issues instructions to read it aloud at the appropriate time. For example, the user can manually instruct the AI ​​narrator to read it aloud during a presentation.

[0658] The AI ​​narrator plays the audio and provides audible explanations to meeting participants, such as, "Machine learning is a technology that uses data to enable computers to teach themselves and perform specific tasks."

[0659] Input: Audio data

[0660] Output: Audio instructions are provided to meeting participants

[0661] This series of steps allows users to provide easy-to-understand explanations to the audience while reducing the burden of narration. In addition, the introduction of an emotion engine enables flexible narration that responds to the user's emotions.

[0662] (Application example 2)

[0663] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0664] In manufacturing sites, there is a need for robots to efficiently and reliably provide instructions on work tasks and procedures. Detailed, easy-to-understand explanations are necessary for workers to understand the setup and operation procedures of new machines, but conventional systems have not been able to adequately provide this. There is also a need for technology that can reduce the burden on workers and reduce errors.

[0665] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a means for a user to input the content to be narrated, a means for transmitting the input text to the server, a means for the server to generate explanatory text using a generative AI model, a means for the server to return the generated explanatory text to the terminal, a means for the terminal to convert the explanatory text into audio data and for an AI narrator to read it aloud, and a means for the robot to provide audio and visual work instructions. This provides detailed and easy-to-understand explanations of work procedures at the manufacturing site, making it easier for workers to understand and reducing operational errors.

[0666] "User" refers to the person or operator who operates the system and inputs the narration content.

[0667] "Narration content" refers to the text information of explanations and comments that the user inputs to the system.

[0668] "Input text" refers to text data of the narration content input by the user to the system.

[0669] A "server" refers to a computer on a network that receives requests from users, uses a generative AI model to generate explanatory text, and returns it to the device.

[0670] A "generative AI model" refers to an artificial intelligence model that uses natural language processing technology to convert input text into explanatory text.

[0671] "Explanation" refers to the text information of explanations or descriptions generated by a generative AI model from input text.

[0672] "Terminal" refers to a device that receives user input and communicates with a server to convert generated descriptions into audio data.

[0673] "Audio data" refers to data in audio format that has been converted from explanatory text using a speech synthesis engine.

[0674] "AI Narrator" refers to a speech synthesis system that uses generated voice data to read explanations aloud on behalf of the user.

[0675] A "speech synthesis engine" refers to software or hardware for converting input text data into voice data.

[0676] "Robot" refers to an automated mechanical device that is placed on a manufacturing floor and is capable of providing audio and visual instructions for work.

[0677] A specific system for implementing this invention includes an interface where a user inputs narration content, a terminal for processing text data, a server that generates explanatory text using a generative AI model, and a robot that provides audio and visual explanations.

[0678] System configuration

[0679] 1. User Interface:

[0680] An interface for users to input narration content (e.g., input devices such as PCs and tablets).

[0681] The interface has the function of sending input text to the server.

[0682] 2. Terminal:

[0683] Receives input text and sends it to the server.

[0684] It has the function of receiving the description data returned from the server and converting it into audio data (example of use: Python library gTTS, etc.).

[0685] 3. Server:

[0686] Use a generative AI model (e.g., GPT-3) to generate a description from the input text.

[0687] A response including the description and associated emotion data is sent back to the device.

[0688] The server uses natural language processing technology to analyze the user's input and generate effective explanations.

[0689] 4. Speech synthesis engine:

[0690] It is responsible for converting explanatory text into audio data (e.g. gTTS library).

[0691] By adjusting the tone and speed of the generated voice data, the narration is performed according to the user's emotions.

[0692] 5. Robots:

[0693] An automated mechanical device that provides audio and visual instructions for work.

[0694] It has the function of playing back audio data and visually displaying specific work procedures.

[0695] System operation example

[0696] Consider a scenario where a user is explaining the setup procedure for a new manufacturing machine. The system works as follows:

[0697] 1. User Input:

[0698] The user enters the following into the interface: "I'd like to explain the setup procedure for a new machine."

[0699] 2. Sending input text:

[0700] The device sends the input text to the server, which also obtains emotional data via an emotion analysis engine.

[0701] 3. Description generation:

[0702] The server generates an explanation using a generative AI model, for example, using the following prompt:

[0703] Here is the procedure for setting up a new manufacturing machine.

[0704] 4. Return of description:

[0705] The server sends the generated instructions back to the terminal. For example, it generates instructions like, "First, connect the machine to the power source. Next, select "Settings" from the main menu to perform the initial setup and follow the instructions. After that, install each part in the correct position. Check that all parts are installed correctly."

[0706] 5. Generate audio data:

[0707] The device converts the received text into audio data, which the robot reads aloud. A speech synthesis engine narrates in a tone and speed that matches the user's emotions.

[0708] 6. Robot explanation:

[0709] The robot provides audio and visual instructions, allowing the worker to carry out the setup procedure with clear guidance.

[0710] In this way, work procedures at the manufacturing site can be explained in detail and in an easy-to-understand manner, promoting understanding among workers and reducing operational errors.

[0711] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0712] Step 1:

[0713] The user enters the narration content into the interface, for example, "I will explain the setup procedure for a new machine." The input text is sent to the terminal.

[0714] Step 2:

[0715] The device sends the received input text to the emotion engine, which analyzes the emotional data. The emotion engine analyzes the text data and extracts emotional data (e.g., notification tone and emotional intensity). This generates an API request containing the emotional data and sends it to the server.

[0716] Step 3:

[0717] The server receives the API request and parses the request body. It extracts the input text and emotion data and passes them to the generative AI model. The server then uses the generative AI model to generate an explanation. Specifically, the following prompt sentence is used as input:

[0718] Here is the procedure for setting up a new manufacturing machine.

[0719] The generative AI model generates explanatory text based on input text and sentiment data.

[0720] Step 4:

[0721] The server converts the generated description into an appropriate format (e.g., JSON) and sends a response containing the description and emotion data back to the device.

[0722] Step 5:

[0723] The device analyzes the response data received from the server, extracts explanatory text data and passes it to the speech synthesis engine, and adjusts the tone and speed of the generated speech based on the emotional data.

[0724] Step 6:

[0725] The device uses a speech synthesis engine (e.g., gTTS) to convert the description into audio data, which is then ready to be played by the AI ​​narrator.

[0726] Step 7:

[0727] The AI ​​narrator plays audio data at specified times, and the robot uses this audio data to explain the work.The robot also visually demonstrates the work steps and provides specific instructions to the worker.

[0728] Through the above processing steps, the system reduces the burden on users, effectively communicates work procedures at the manufacturing site, and reduces operational errors.

[0729] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0730] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0731] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0732] [Third embodiment]

[0733] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0734] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0735] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0736] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0737] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0738] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0739] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0740] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0741] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0742] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0743] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0744] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0745] This invention relates to a system that aims to reduce the burden on users when narrating technical terms and background information during meetings and improve the audience's concentration and comprehension. The system implements a series of processes: the user inputs the content to be narrated, the content is converted into explanatory text using a generative AI model, and finally, an AI narrator reads it aloud.

[0746] An embodiment of this system is described in detail below.

[0747] 1. User operations

[0748] User: Enters the content to be narrated (e.g., technical terms and their background) through the conferencing software interface. By entering text such as "What is machine learning?" into the input field, the system prepares to generate an explanation for that content.

[0749] 2. From the user to the device

[0750] Terminal: Receives text data entered by the user, converts it into an API request format, and then sends the request to the specified server endpoint.

[0751] 3. From the device to the server

[0752] Server: Receives requests at the API endpoint, analyzes the request content, and passes the analyzed content to the generative AI model.

[0753] 4. Server Processing

[0754] Server: Provides the analyzed text to a generative AI model (e.g., a natural language processing model) as input data. The generative AI model outputs an explanation based on "What is machine learning?" An example of a generated explanation would be "Machine learning is a technology that uses data to enable computers to learn on their own and perform specific tasks."

[0755] 5. From the server to the device

[0756] Server: Converts the generated text data into JSON or XML format and returns it to the requesting device as a response.

[0757] Terminal: Analyzes the received response data and prepares to convert it into voice data.

[0758] 6. Terminal Processing

[0759] Terminal: The received description is converted from text to voice data using a text-to-speech engine.

[0760] 7. AI Narrator Reading

[0761] Device: Passes the generated audio data to the AI ​​narrator and gives instructions to read it out at appropriate times during the meeting.

[0762] AI Narrator: Plays back audio data and provides generated explanations to meeting participants by voice, allowing the presenter to do other tasks while the explanation is being given, and the audience can deepen their understanding by listening to the audio explanation.

[0763] Specific examples

[0764] Meeting Scenarios

[0765] User (Presenter): You have a meeting where you need to explain a new technology called "machine learning" to the leaders of each department.

[0766] Narration content: The presenter enters the narration content "What is machine learning?" into the conference software.

[0767] System Operation

[0768] 1. A user types "What is machine learning?" into the conferencing software.

[0769] 2. The device sends the text data to the server as an API request.

[0770] 3. The server receives the request and generates a description using a generative AI model.

[0771] 4. The server returns the generated description to the device.

[0772] 5. The device receives the description and converts it into audio data.

[0773] 6. The AI ​​narrator plays the audio and reads, "Machine learning is a technology that uses data to teach computers how to perform specific tasks."

[0774] In this way, the present invention provides a system that reduces the burden on the presenter, promotes audience understanding, and helps them maintain their concentration.

[0775] The processing flow will be explained below.

[0776] Step 1:

[0777] A user types what they want to narrate into the conferencing software interface, for example, by typing "What is machine learning" into a text field.

[0778] Step 2:

[0779] The device receives the user's input and converts the text data into an API request, which includes details about the narration and any required explanations.

[0780] Step 3:

[0781] The device sends an API request to the server at the specified endpoint.

[0782] Step 4:

[0783] The server receives the API request, parses the request body, and prepares the parsed text data.

[0784] Step 5:

[0785] The server passes the parsed text data to a generative AI model, which then generates explanatory text based on the input data.

[0786] Step 6:

[0787] The server converts the description returned from the generative AI model into JSON or XML format.

[0788] Step 7:

[0789] The server returns the converted description data to the device as an API response.

[0790] Step 8:

[0791] The device receives the API response and parses the description data.

[0792] Step 9:

[0793] The device passes the analyzed description to a Text-to-Speech (TTS) engine and converts it into audio data.

[0794] Step 10:

[0795] The device passes the generated audio data to the AI ​​narrator, which is instructed on the timing of audio playback.

[0796] Step 11:

[0797] The AI ​​narrator plays back the audio data and provides audible explanations to meeting participants, allowing the presenter to focus on other tasks while the audience listens to the explanations to deepen their understanding.

[0798] Example 1

[0799] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0800] The present invention aims to reduce the burden on users when explaining technical terms and background information during meetings, thereby improving the audience's concentration and comprehension. In such situations, presenters are required to convey specialized information in an easy-to-understand manner, but doing so requires a great deal of effort and time. Currently, there is no effective system for reducing this burden, so the present invention provides a system that solves this problem.

[0801] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0802] In this invention, the server includes a means for a user to input content to be narrated, a means for transmitting the input text to a communication device, a means for the communication device to transmit the input text to an endpoint of the server in the form of an API request, a means for the server to analyze the request and generate explanatory text using a generative AI model, a means for the server to return the generated explanatory text to the communication device, and a means for the communication device to convert the explanatory text into audio data and play the audio. This allows presenters to easily deliver expert explanations and enables audiences to deepen their understanding through the audio explanations.

[0803] A "user" is someone who uses the system to input narration content and initiates the process of generating the description and playing it back.

[0804] A "communication device" refers to equipment or software for sending and receiving data between a user and a server, and is responsible for sending text entered by the user to the server and receiving responses from the server.

[0805] A "server" is a device or program that analyzes requests received from users, generates explanatory text using a generative AI model, and returns the generated explanatory text to the communication device.

[0806] A "generative AI model" is an artificial intelligence model that uses natural language processing algorithms to generate relevant explanatory text from input text.

[0807] An "API request format" refers to a standardized data format used by a communications device to send data to a server, typically including JSON or XML formats.

[0808] A "speech synthesis engine" is software or hardware that converts text data into voice data and has text-to-speech functionality.

[0809] "Explanation" is text data generated by a generative AI model that contains detailed explanations of technical terms and background information entered by the user.

[0810] "Audio data" is a data format that can reproduce explanatory text as audio, generated by a speech synthesis engine.

[0811] "Audio playback" refers to the act of an AI narrator or other audio output device using the generated audio data to play audio.

[0812] The present invention provides a system that aims to reduce the burden on users when narrating technical terms and background information during a meeting, thereby improving the audience's concentration and comprehension. This system allows users to easily deliver technical explanations, and allows the audience to deepen their understanding through the audio explanations. An embodiment of the present invention is described in detail below.

[0813] User operations

[0814] The user inputs the content to be narrated through the conferencing software interface. For example, the user enters the text "What is machine learning?" into the conferencing software's text input field and clicks the submit button. This prepares the system to convert the content into explanatory text.

[0815] Transmitting text data to a communication device

[0816] The communications device receives the text data entered by the user and converts it into an API request format. For example, it converts it into JSON format data. This API request is then sent as an HTTP request to the server's endpoint. An HTTP POST request is used, and the URL is "https: / / api.example.com / generate-explanation."

[0817] Sending the request to the server and parsing it

[0818] The server analyzes the request received at the API endpoint, checks the JSON data of the request, and obtains the value of the "query" key. The obtained text, "What is machine learning?", is passed to the generative AI model.

[0819] Generating explanatory text using a generative AI model

[0820] The server generates an explanation by providing the text analyzed using the generative AI model as input data. The generative AI model (e.g., a natural language processing model) is invoked to generate an explanation based on the prompt "What is machine learning?" The generated explanation is "Machine learning is a technology that uses data to enable computers to learn on their own and perform specific tasks."

[0821] Returning generated data to communication device and converting it into voice

[0822] The server converts the generated explanation into JSON format and returns it to the communication device. The communication device analyzes the received response data and obtains the value of the "explanation" key. A text-to-speech engine is used to convert the obtained text data into audio data. For example, Google's TTS API is used to convert the text into an audio file.

[0823] Audio playback by AI narrator

[0824] The communication device passes the generated audio data to the AI ​​narrator, which then issues instructions to read aloud, at an appropriate time during the meeting, "Machine learning is a technology that uses data to enable computers to learn on their own and perform specific tasks." The AI ​​narrator then plays back the audio data and provides the generated explanation to the meeting participants.

[0825] This process reduces the burden on the user (presenter) to explain technical information in an easy-to-understand manner, and allows the audience to understand the content through their eyes and ears.

[0826] Specific examples

[0827] Meeting Scenarios

[0828] The user (presenter) holds a meeting to explain the new technology "machine learning" to the leaders of each department. The presenter enters the narration content "What is machine learning?" into the conference software.

[0829] System Operation

[0830] During a meeting, a user enters "What is machine learning?" into a text input field and presses the send button. The communication device sends this text data to the server as an API request. The server uses a generative AI model to generate an explanation, which it returns to the communication device: "Machine learning is a technology that uses data to enable computers to learn on their own and perform specific tasks." The communication device then converts this explanation into audio data, which is then played back by an AI narrator.

[0831] In this way, the present invention provides a system that reduces the burden on users and promotes audience understanding.

[0832] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0833] Step 1:

[0834] The user enters the content to be narrated through the conferencing software interface. The user enters "What is machine learning?" in the text input field and presses the send button. Input (text): "What is machine learning?" Output (text data): "What is machine learning?" text data.

[0835] Step 2:

[0836] The device receives text data sent by the user. Since the server cannot understand the received text data as it is, the device converts it into an API request format. Specifically, it converts it into JSON format and creates the data structure "{"query": "What is machine learning"}". Input (text data): Text entered by the user, Output (API request): JSON format data.

[0837] Step 3:

[0838] The terminal sends the converted API request as an HTTP request to the server's API endpoint. The URL used is "https: / / api.example.com / generate-explanation". Input (API request): JSON format data, Output (request to server): HTTP POST request.

[0839] Step 4:

[0840] The server analyzes the request received at the API endpoint. The server parses the JSON data of the request and extracts the value of the "query" key. The extracted text, "What is machine learning?", is passed to the generative AI model. Input (HTTP request): API request data, Output (text data): Analyzed text data.

[0841] Step 5:

[0842] The server uses a generative AI model to generate an explanation based on the analyzed text, "What is machine learning?" The generative AI model uses a natural language processing algorithm to generate an explanation based on the prompt. An example output is the explanation, "Machine learning is a technology that uses data to enable computers to learn on their own and perform specific tasks." Input (text data): Analyzed text data, Output (explanation): Generated explanation.

[0843] Step 6:

[0844] The server converts the generated explanation into JSON format and returns it to the requesting device as an HTTP response. For example, the format is "{"explanation": "Machine learning is a technology that uses data to enable computers to learn on their own and perform specific tasks."}". Input (explanation): Generated explanation, Output (HTTP response): JSON format data.

[0845] Step 7:

[0846] The device analyzes the JSON format response data received from the server. It obtains the value of the "explanation" key and uses a speech synthesis engine to convert the text data into audio data. The synthesized speech engine uses Google's TTS API or similar to generate an audio file. Input (HTTP response): JSON format data, Output (audio data): Generated audio file.

[0847] Step 8:

[0848] The device passes the generated audio data to the AI ​​narrator, who then plays the audio data at the appropriate time during the meeting, reading aloud, "Machine learning is a technology that uses data to enable computers to learn on their own and perform specific tasks." Input (audio data): Generated audio file; Output (audio playback): Audio provided to meeting participants.

[0849] This series of processes reduces the burden of explanation on the presenter and allows the audience to deepen their understanding.

[0850] (Application example 1)

[0851] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0852] In recent years, with the spread of virtual stores, users have been demanding that they be able to easily obtain detailed information about products. However, many current virtual store systems have difficulty quickly and accurately providing the detailed explanations and background information that users require. Furthermore, when users ask questions about products, it is also difficult to provide appropriate answers and explanations in real time. This makes it difficult to increase user satisfaction and purchasing motivation.

[0853] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0854] In this invention, the server includes means for a user to input content to be narrated, means for transmitting the input text to the server, means for generating a description using a generative AI model, means for returning the generated description to the terminal, means for the terminal to convert the description into audio data and have an AI narrator read it aloud, means for the terminal to provide detailed descriptions of products in the virtual store in real time using the generative AI model, and means for customers to input questions through the virtual store interface. This allows users to simply input a question to obtain a detailed description using the generative AI model in real time and to listen to that description in audio.

[0855] A "user" is a user of the system who inputs narration content and questions.

[0856] "Narration" is the act of explaining information orally in a meeting or presentation.

[0857] "Input means" refers to an interface through which a user provides text data to the system.

[0858] "Input text" is character string data such as narration content or questions that a user inputs into the system.

[0859] A "server" is a computing system that processes input data and generates new data using generative AI models.

[0860] A "generative AI model" is an algorithm that uses natural language processing technology and generates explanations and answers based on input text.

[0861] An "explanation" is a document created by a generative AI model that contains detailed information related to the input text.

[0862] A "terminal" is a device operated by a user, and is a device that communicates with a server to send and receive data.

[0863] "Audio data" is data obtained by converting text data into audio, and is data for providing information to the user by audio.

[0864] "AI Narrator" is a virtual assistant that plays audio data and reads the generated explanation aloud.

[0865] A "virtual store" is a virtual store that operates on the Internet and is a platform where users can browse and purchase products online.

[0866] A "detailed description" is a description that includes detailed information about a product or service.

[0867] An "interface" is a part that includes an operation screen and means for exchanging information between the user and the system.

[0868] A "question" is a short sentence or document that allows a user to enter information they want to know about a product or service.

[0869] The present invention relates to a system that automatically generates detailed explanations and provides voice guidance when a user inputs a question about a product in a virtual store. Specific embodiments for carrying out the present invention will be described below.

[0870] First, a user enters a question through a virtual store interface, which could be a web browser or a mobile application. Suppose a user enters a question such as, "What are the features of this smartwatch?"

[0871] The terminal (client device) converts the text data entered by the user into an API request format and sends it to the specified server. This process uses an HTTP request, and the request content is encoded in JSON format or similar.

[0872] The server receives requests at an API endpoint. The received request content is passed to a generative AI model. The generative AI model uses natural language processing technology (e.g., BERT or GPT-3). The model generates an appropriate explanation for the input question.

[0873] The generated description is sent back to the device as a response from the server. The device then analyzes the received description and converts it into audio data. This process uses a speech synthesis engine such as the Google Text-to-Speech (gTTS) engine.

[0874] Once the audio data is generated, an AI narrator on the device plays it back and provides the generated explanation to the user, allowing users to simply input their question, receive a detailed explanation in real time using the generative AI model, and listen to that explanation in audio.

[0875] For example, if a user wants to know more about a new product, a "smartwatch," in a virtual store, they can type, "What are the features of this smartwatch?" The system sends this question to the server, and using a generative AI model, generates a description such as, "This smartwatch is a gadget with a long battery life, waterproof functionality, and comprehensive health tracking capabilities." The description is then converted into audio data and provided to the user by an AI narrator.

[0876] This allows the user to quickly and accurately obtain detailed information about products in the virtual store and use it as a basis for making a purchase decision.

[0877] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0878] Step 1:

[0879] The user inputs a question through the virtual store interface. The user inputs a question into the interface, such as "What are the features of this smartwatch?" The input data is sent to the terminal as a text string. In this step, the input data is the user's question text, and the output data is the text string received by the terminal.

[0880] Step 2:

[0881] The device converts the question text received from the user into an API request format and sends it to the specified server. Specifically, it encodes the text data into JSON format and sends it as an HTTP POST request. In this step, the input data is the user's question text, and the output data is the request converted into JSON format.

[0882] Step 3:

[0883] The server receives requests at the API endpoint. It analyzes the request content and passes it to the generative AI model. This process uses the Python requests library, etc. Specifically, it extracts the necessary text from the JSON-formatted data and passes it to the generative AI model. In this step, the input data is the JSON-formatted request, and the output data is the text data passed to the generative AI model.

[0884] Step 4:

[0885] A generative AI model generates a detailed explanation based on the input question text. For example, a natural language processing model such as GPT-3 or BERT is used. Specifically, it processes the input text as a prompt and outputs a generated explanation. In this step, the input data is the question text, and the output data is the generated explanation.

[0886] Step 5:

[0887] The server returns the generated description to the requesting terminal as a response. The response data is encoded in JSON format and sent as an HTTP response. In this step, the input data is the generated description, and the output data is a JSON-formatted response.

[0888] Step 6:

[0889] The device analyzes the received response data and converts it into voice data. Specifically, it uses a speech synthesis engine such as the Google Text-to-Speech (gTTS) engine to convert the text data into voice data. In this step, the input data is a JSON-formatted response, and the output data is voice data.

[0890] Step 7:

[0891] The AI ​​narrator plays the generated audio data and provides the user with audio information. Specifically, the audio is played through the device's speaker. In this step, the input data is the audio data, and the output data is the audio information provided to the user.

[0892] In this way, the user can quickly and accurately obtain detailed information about products in the virtual store and hear their explanations in audio.

[0893] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0894] This invention relates to a system that aims to reduce the burden on users when narrating technical terms and background information during meetings, thereby improving the audience's concentration and comprehension. The system implements a series of processes: the user inputs the content to be narrated, the content is converted into explanatory text using a generative AI model, and finally, an AI narrator reads it aloud. It also combines an emotion engine to analyze the user's emotions and dynamically adjust the explanatory text and audio data accordingly.

[0895] Specific embodiments will be described below.

[0896] 1. User operations

[0897] User: Enters the content to be narrated through the conferencing software interface. For example, the user enters the text "What is machine learning?" into the input field.

[0898] 2. From the user to the device

[0899] Terminal: Receives text data entered by the user and passes it to the emotion engine. The emotion engine analyzes the user's emotions from the text and voice data. It converts the data containing the analysis results into an API request format and sends it to the server endpoint.

[0900] 3. From the device to the server

[0901] Server: Receives the API request, analyzes the request body, and prepares the analyzed text data and emotion data.

[0902] 4. Server Processing

[0903] Server: Passes the analyzed text data and emotion data to the generative AI model, which generates a description based on the input data and adjusts the tone and content of the description based on the emotion data as needed.

[0904] 5. From the server to the device

[0905] Server: Converts the generated description into an appropriate format (for example, JSON or XML) and sends it back to the requesting device as a response.

[0906] Terminal: Analyzes the received response data and obtains the explanatory data.

[0907] 6. Terminal Processing

[0908] Device: The acquired description is passed to a Text-to-Speech (TTS) engine and converted into audio data. It is also possible to adjust the tone and speed of the audio based on emotional data.

[0909] 7. AI Narrator Reading

[0910] Device: Passes the generated voice data to the AI ​​narrator and issues instructions to read it aloud at the appropriate time.

[0911] AI Narrator: Plays back audio data and provides audio explanations to meeting participants, allowing the presenter to focus on other tasks while the audience listens to the explanations to deepen their understanding.

[0912] Specific examples

[0913] Meeting Scenarios

[0914] User (Presenter): You have a meeting where you need to explain a new technology called "machine learning" to the leaders of each department.

[0915] Narration content: The presenter enters the narration content "What is machine learning?" into the conference software.

[0916] System Operation

[0917] 1. A user types "What is machine learning?" into the conferencing software.

[0918] 2. The device receives the text data and sends an API request to the server, including the analysis results from the emotion engine.

[0919] 3. The server receives the request and generates a description using a generative AI model, adjusting the tone and content of the description as needed based on the sentiment data.

[0920] 4. The server returns the generated description to the device.

[0921] 5. The device receives the description and converts it into audio data, adjusting the tone and speed of the audio based on the emotional data.

[0922] 6. The AI ​​narrator plays the audio and reads, "Machine learning is a technology that uses data to teach computers how to perform specific tasks."

[0923] In this way, the presenter can reduce the burden on the presenter, promote the audience's understanding, and maintain their concentration. Furthermore, by combining it with an emotion engine, the presenter can provide flexible narration that responds to the user's emotions.

[0924] The processing flow will be explained below.

[0925] Step 1:

[0926] A user types what they want to narrate into the conferencing software interface, for example, by typing "What is machine learning" into a text field.

[0927] Step 2:

[0928] The terminal receives the user's input text and passes the data to the emotion engine, which analyzes the user's emotions from the text data and voice data and generates an analysis result.

[0929] Step 3:

[0930] The device converts the data, including the analysis results of the emotion engine, into an API request format, which includes the narration content and emotion data.

[0931] Step 4:

[0932] The device sends an API request to the server at the specified endpoint.

[0933] Step 5:

[0934] The server receives the API request, parses the request body, and prepares the parsed text data and emotion data.

[0935] Step 6:

[0936] The server passes the analyzed text data and emotion data to a generative AI model, which generates a description based on the user's input and adjusts the tone and content of the description based on the emotion data as needed.

[0937] Step 7:

[0938] The server converts the generated description into JSON or XML format and sends it back to the requesting device as a response.

[0939] Step 8:

[0940] The device receives the API response and parses and extracts the description data.

[0941] Step 9:

[0942] The device passes the extracted description to a Text-to-Speech (TTS) engine and converts it into audio data, which can adjust the tone and speed of the voice based on emotional data.

[0943] Step 10:

[0944] The device passes the generated audio data to the AI ​​narrator and instructs it on the timing of audio playback.

[0945] Step 11:

[0946] The AI ​​narrator plays back the audio data and provides audible explanations to meeting participants, allowing the presenter to focus on other tasks while the audience listens to the explanations to deepen their understanding.

[0947] Specific examples

[0948] A user types "What is machine learning?" into the conferencing software.

[0949] The device receives the user's input, and the emotion engine analyzes the user's emotion. For example, if the user is nervous, it analyzes it as "nervous."

[0950] The device sends an API request containing text data and emotion data to the server.

[0951] The server receives the request and uses a generative AI model to generate a description such as, "Machine learning is a technique that uses data to enable computers to teach themselves and perform specific tasks."

[0952] The server returns the generated explanation to the terminal.

[0953] The device receives the text and passes it to a TTS engine to convert it into speech data. Based on the emotional data, the tone and speed of the speech are adjusted to create a tone that alleviates tension.

[0954] The AI ​​narrator plays the audio and reads, "Machine learning is a technique that uses data to enable computers to teach themselves and perform specific tasks."

[0955] Example 2

[0956] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0957] Currently, narrating technical terms and background information during meetings can be a significant burden for presenters, potentially reducing the audience's concentration and comprehension. Furthermore, it is difficult to adapt flexibly to the user's emotions, limiting the effectiveness of narration. There is a need for a system that can resolve these issues and promote effective communication and understanding.

[0958] The specification process by the specification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes a means for a user to input content to be narrated, a means for receiving the input text and analyzing emotions using an analysis engine, a means for converting the analyzed data into an API request format and sending it to the server, a means for the server to generate explanatory text using a generation AI model, a means for adjusting the tone and content of the generated explanatory text based on the emotional data, a means for returning the generated explanatory text to the terminal, and a means for the terminal to convert the explanatory text into audio data and have an AI narrator read it aloud. This reduces the burden on the presenter in narrating and improves the audience's concentration and comprehension.

[0959] "User" refers to the person who uses this system to provide narration during a meeting.

[0960] "Narration content" refers to text data such as technical terms and background information that a user wants to explain during a conference.

[0961] "Terminal" refers to the computer or mobile device that a user uses to enter narration content.

[0962] An "analysis engine" refers to software or hardware that analyzes a user's emotions from input text data or voice data.

[0963] "Emotion data" refers to information about the user's emotions analyzed by the analysis engine, and includes emotion labels such as "neutral" and "excited."

[0964] "API request" refers to the protocol used by the device to send data to the server, including parsed text data and emotion data.

[0965] "Server" refers to the computational resources on the network that are responsible for processing the analyzed data and generating narratives using generative AI models.

[0966] "Generative AI model" refers to the artificial intelligence technology used by the server to generate the description, including, for example, a natural language processing model.

[0967] "Explanatory text" refers to text generated by a generative AI model that clearly explains the content of the user's narration.

[0968] "Text-to-Speech (TTS) engine" refers to software for converting text data into voice data.

[0969] "AI narrator" refers to software or hardware that plays back generated audio data and provides audio narration.

[0970] "Audio Data" refers to the audio files generated by the Text-to-Speech engine and played by the AI ​​narrator.

[0971] "Tone" refers to the emotional resonance or tone of a voice or text, and is adjusted based on emotional data.

[0972] This invention relates to a system that reduces the burden on users when narrating technical terms and background information during meetings, improving audience concentration and comprehension. The system implements a series of processes: the user inputs the content to be narrated, the content is converted into explanatory text using a generative AI model, and finally, an AI narrator reads it aloud. It also incorporates an emotion engine to analyze the user's emotions and dynamically adjust the explanatory text and audio data accordingly.

[0973] Specifically, a user enters the content to be narrated into a text field through the conferencing software interface. For example, consider a user entering "What is machine learning?" This input is received by the device and then analyzed by an emotion engine (e.g., IBM Watson Tone Analyzer). The emotion engine analyzes the user's emotions and generates emotion data such as "neutral" or "excited."

[0974] The analysis results are converted into an API request format and sent to a server. The server receives this API request and generates a description using a generative AI model (e.g., OpenAI GPT-3). The tone and content of the generated description may be adjusted based on the emotional data. For example, a description might be generated that reads, "Machine learning is a technology that uses data to enable computers to learn on their own and perform specific tasks."

[0975] The generated description is converted into an appropriate format, such as JSON, by the server and sent back to the device. The device receives and analyzes this data, then passes it to a Text-to-Speech (TTS) engine (e.g., Google Cloud Text-to-Speech) to convert it into audio data. The tone and speed of the voice are also adjusted based on the emotion data. For example, if a "neutral" emotion label is added, the voice will be generated in a calm and composed tone.

[0976] Finally, the device passes the generated audio data to the AI ​​narrator, which then issues instructions to read it aloud at the appropriate time. For example, the user can manually instruct the narrator to read it aloud during the presentation. The AI ​​narrator plays back the audio data and reads out loud, "Machine learning is a technology that uses data to enable computers to learn on their own and perform specific tasks," making it easier for the audience to understand the content.

[0977] Specific examples

[0978] Meeting Scenarios

[0979] A meeting is held in which the user (presenter) needs to explain a new technology called "machine learning" to the leaders of each department.

[0980] Narration content: The presenter enters the narration content "What is machine learning?" into the conference software.

[0981] Prompt Sentence Examples

[0982] The user types the following into the conferencing software:

[0983] What is machine learning?

[0984] This system reduces the burden on presenters, promotes audience understanding, and helps maintain concentration. In addition, by combining it with an emotion engine, it enables flexible narration that responds to the user's emotions.

[0985] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0986] System processing flow

[0987] Step 1:

[0988] The user enters the content to be narrated into a text field through the conferencing software interface. For example, they might enter "What is machine learning?" This input is then sent to the terminal as text data required for the next process.

[0989] Input: User-entered text "What is machine learning?"

[0990] Output: Text data "What is machine learning?"

[0991] Step 2:

[0992] The device receives the text data entered by the user. The received text data is then passed to an emotion engine (such as IBM Watson Tone Analyzer or a similar analysis engine) for analysis. The emotion engine analyzes the text data and generates an emotion label, such as "neutral" or "excited."

[0993] Input: Text data "What is machine learning?"

[0994] Output: Emotion data (e.g. "Neutral")

[0995] Step 3:

[0996] The device converts the analyzed text data and emotion data into an API request format. This API request format data is in a format for sending to the server. The converted data is then sent to the server.

[0997] Input: Text data "What is machine learning?", emotion data "Neutral"

[0998] Output: Data in API request format

[0999] Step 4:

[1000] The server receives the API request and parses the request body, taking the parsed text and sentiment data and feeding it into a generative AI model (such as OpenAI GPT-3 or a similar natural language processing model). This model combines the original text and sentiment data to generate a description.

[1001] Input: Data in API request format (text data "What is machine learning?", sentiment data "Neutral")

[1002] Output: Generated description (e.g., "Machine learning is a technique that uses data to teach computers how to perform specific tasks.")

[1003] Step 5:

[1004] The server converts the generated description into an appropriate format (e.g., JSON or XML) and sends it back to the requesting device as a response.

[1005] Input: Generated description

[1006] Output: JSON format data returned to the terminal

[1007] Step 6:

[1008] The device analyzes the response data received and obtains the generated explanation. This explanation is then passed to a Text-to-Speech (TTS) engine (such as Google Cloud Text-to-Speech or a similar voice synthesis engine) to convert it into voice data. At this time, the tone and speed of the voice are also adjusted based on the emotional data.

[1009] Input: JSON format data (generated description)

[1010] Output: Audio data

[1011] Step 7:

[1012] The device then passes the generated audio data to the AI ​​narrator and issues instructions to read it aloud at the appropriate time. For example, the user can manually instruct the AI ​​narrator to read it aloud during a presentation.

[1013] The AI ​​narrator plays the audio and provides audible explanations to meeting participants, such as, "Machine learning is a technology that uses data to enable computers to teach themselves and perform specific tasks."

[1014] Input: Audio data

[1015] Output: Audio instructions are provided to meeting participants

[1016] This series of steps allows users to provide easy-to-understand explanations to the audience while reducing the burden of narration. In addition, the introduction of an emotion engine enables flexible narration that responds to the user's emotions.

[1017] (Application example 2)

[1018] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1019] In manufacturing sites, there is a need for robots to efficiently and reliably provide instructions on work tasks and procedures. Detailed, easy-to-understand explanations are necessary for workers to understand the setup and operation procedures of new machines, but conventional systems have not been able to adequately provide this. There is also a need for technology that can reduce the burden on workers and reduce errors.

[1020] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a means for a user to input the content to be narrated, a means for transmitting the input text to the server, a means for the server to generate explanatory text using a generative AI model, a means for the server to return the generated explanatory text to the terminal, a means for the terminal to convert the explanatory text into audio data and for an AI narrator to read it aloud, and a means for the robot to provide audio and visual work instructions. This provides detailed and easy-to-understand explanations of work procedures at the manufacturing site, making it easier for workers to understand and reducing operational errors.

[1021] "User" refers to the person or operator who operates the system and inputs the narration content.

[1022] "Narration content" refers to the text information of explanations and comments that the user inputs to the system.

[1023] "Input text" refers to text data of the narration content input by the user to the system.

[1024] A "server" refers to a computer on a network that receives requests from users, uses a generative AI model to generate explanatory text, and returns it to the device.

[1025] A "generative AI model" refers to an artificial intelligence model that uses natural language processing technology to convert input text into explanatory text.

[1026] "Explanation" refers to the text information of explanations or descriptions generated by a generative AI model from input text.

[1027] "Terminal" refers to a device that receives user input and communicates with a server to convert generated descriptions into audio data.

[1028] "Audio data" refers to data in audio format that has been converted from explanatory text using a speech synthesis engine.

[1029] "AI Narrator" refers to a speech synthesis system that uses generated voice data to read explanations aloud on behalf of the user.

[1030] A "speech synthesis engine" refers to software or hardware for converting input text data into voice data.

[1031] "Robot" refers to an automated mechanical device that is placed on a manufacturing floor and is capable of providing audio and visual instructions for work.

[1032] A specific system for implementing this invention includes an interface where a user inputs narration content, a terminal for processing text data, a server that generates explanatory text using a generative AI model, and a robot that provides audio and visual explanations.

[1033] System configuration

[1034] 1. User Interface:

[1035] An interface for users to input narration content (e.g., input devices such as PCs and tablets).

[1036] The interface has the function of sending input text to the server.

[1037] 2. Terminal:

[1038] Receives input text and sends it to the server.

[1039] It has the function of receiving the description data returned from the server and converting it into audio data (example of use: Python library gTTS, etc.).

[1040] 3. Server:

[1041] Use a generative AI model (e.g., GPT-3) to generate a description from the input text.

[1042] A response including the description and associated emotion data is sent back to the device.

[1043] The server uses natural language processing technology to analyze the user's input and generate effective explanations.

[1044] 4. Speech synthesis engine:

[1045] It is responsible for converting explanatory text into audio data (e.g. gTTS library).

[1046] By adjusting the tone and speed of the generated voice data, the narration is performed according to the user's emotions.

[1047] 5. Robots:

[1048] An automated mechanical device that provides audio and visual instructions for work.

[1049] It has the function of playing back audio data and visually displaying specific work procedures.

[1050] System operation example

[1051] Consider a scenario where a user is explaining the setup procedure for a new manufacturing machine. The system works as follows:

[1052] 1. User Input:

[1053] The user enters the following into the interface: "I'd like to explain the setup procedure for a new machine."

[1054] 2. Sending input text:

[1055] The device sends the input text to the server, which also obtains emotional data via an emotion analysis engine.

[1056] 3. Description generation:

[1057] The server generates an explanation using a generative AI model, for example, using the following prompt:

[1058] Here is the procedure for setting up a new manufacturing machine.

[1059] 4. Return of description:

[1060] The server sends the generated instructions back to the terminal. For example, it generates instructions like, "First, connect the machine to the power source. Next, select "Settings" from the main menu to perform the initial setup and follow the instructions. After that, install each part in the correct position. Check that all parts are installed correctly."

[1061] 5. Generate audio data:

[1062] The device converts the received text into audio data, which the robot reads aloud. A speech synthesis engine narrates in a tone and speed that matches the user's emotions.

[1063] 6. Robot explanation:

[1064] The robot provides audio and visual instructions, allowing the worker to carry out the setup procedure with clear guidance.

[1065] In this way, work procedures at the manufacturing site can be explained in detail and in an easy-to-understand manner, promoting understanding among workers and reducing operational errors.

[1066] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1067] Step 1:

[1068] The user enters the narration content into the interface, for example, "I will explain the setup procedure for a new machine." The input text is sent to the terminal.

[1069] Step 2:

[1070] The device sends the received input text to the emotion engine, which analyzes the emotional data. The emotion engine analyzes the text data and extracts emotional data (e.g., notification tone and emotional intensity). This generates an API request containing the emotional data and sends it to the server.

[1071] Step 3:

[1072] The server receives the API request and parses the request body. It extracts the input text and emotion data and passes them to the generative AI model. The server then uses the generative AI model to generate an explanation. Specifically, the following prompt sentence is used as input:

[1073] Here is the procedure for setting up a new manufacturing machine.

[1074] The generative AI model generates explanatory text based on input text and sentiment data.

[1075] Step 4:

[1076] The server converts the generated description into an appropriate format (e.g., JSON) and sends a response containing the description and emotion data back to the device.

[1077] Step 5:

[1078] The device analyzes the response data received from the server, extracts explanatory text data and passes it to the speech synthesis engine, and adjusts the tone and speed of the generated speech based on the emotional data.

[1079] Step 6:

[1080] The device uses a speech synthesis engine (e.g., gTTS) to convert the description into audio data, which is then ready to be played by the AI ​​narrator.

[1081] Step 7:

[1082] The AI ​​narrator plays audio data at specified times, and the robot uses this audio data to explain the work.The robot also visually demonstrates the work steps and provides specific instructions to the worker.

[1083] Through the above processing steps, the system reduces the burden on users, effectively communicates work procedures at the manufacturing site, and reduces operational errors.

[1084] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1085] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1086] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1087] [Fourth embodiment]

[1088] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1089] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1090] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1091] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1092] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1093] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1094] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1095] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1096] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1097] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1098] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1099] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1100] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1101] This invention relates to a system that aims to reduce the burden on users when narrating technical terms and background information during meetings and improve the audience's concentration and comprehension. The system implements a series of processes: the user inputs the content to be narrated, the content is converted into explanatory text using a generative AI model, and finally, an AI narrator reads it aloud.

[1102] An embodiment of this system is described in detail below.

[1103] 1. User operations

[1104] User: Enters the content to be narrated (e.g., technical terms and their background) through the conferencing software interface. By entering text such as "What is machine learning?" into the input field, the system prepares to generate an explanation for that content.

[1105] 2. From the user to the device

[1106] Terminal: Receives text data entered by the user, converts it into an API request format, and then sends the request to the specified server endpoint.

[1107] 3. From the device to the server

[1108] Server: Receives requests at the API endpoint, analyzes the request content, and passes the analyzed content to the generative AI model.

[1109] 4. Server Processing

[1110] Server: Provides the analyzed text to a generative AI model (e.g., a natural language processing model) as input data. The generative AI model outputs an explanation based on "What is machine learning?" An example of a generated explanation would be "Machine learning is a technology that uses data to enable computers to learn on their own and perform specific tasks."

[1111] 5. From the server to the device

[1112] Server: Converts the generated text data into JSON or XML format and returns it to the requesting device as a response.

[1113] Terminal: Analyzes the received response data and prepares to convert it into voice data.

[1114] 6. Terminal Processing

[1115] Terminal: The received description is converted from text to voice data using a text-to-speech engine.

[1116] 7. AI Narrator Reading

[1117] Device: Passes the generated audio data to the AI ​​narrator and gives instructions to read it out at appropriate times during the meeting.

[1118] AI Narrator: Plays back audio data and provides generated explanations to meeting participants by voice, allowing the presenter to do other tasks while the explanation is being given, and the audience can deepen their understanding by listening to the audio explanation.

[1119] Specific examples

[1120] Meeting Scenarios

[1121] User (Presenter): You have a meeting where you need to explain a new technology called "machine learning" to the leaders of each department.

[1122] Narration content: The presenter enters the narration content "What is machine learning?" into the conference software.

[1123] System Operation

[1124] 1. A user types "What is machine learning?" into the conferencing software.

[1125] 2. The device sends the text data to the server as an API request.

[1126] 3. The server receives the request and generates a description using a generative AI model.

[1127] 4. The server returns the generated description to the device.

[1128] 5. The device receives the description and converts it into audio data.

[1129] 6. The AI ​​narrator plays the audio and reads, "Machine learning is a technology that uses data to teach computers how to perform specific tasks."

[1130] In this way, the present invention provides a system that reduces the burden on the presenter, promotes audience understanding, and helps them maintain their concentration.

[1131] The processing flow will be explained below.

[1132] Step 1:

[1133] A user types what they want to narrate into the conferencing software interface, for example, by typing "What is machine learning" into a text field.

[1134] Step 2:

[1135] The device receives the user's input and converts the text data into an API request, which includes details about the narration and any required explanations.

[1136] Step 3:

[1137] The device sends an API request to the server at the specified endpoint.

[1138] Step 4:

[1139] The server receives the API request, parses the request body, and prepares the parsed text data.

[1140] Step 5:

[1141] The server passes the parsed text data to a generative AI model, which then generates explanatory text based on the input data.

[1142] Step 6:

[1143] The server converts the description returned from the generative AI model into JSON or XML format.

[1144] Step 7:

[1145] The server returns the converted description data to the device as an API response.

[1146] Step 8:

[1147] The device receives the API response and parses the description data.

[1148] Step 9:

[1149] The device passes the analyzed description to a Text-to-Speech (TTS) engine and converts it into audio data.

[1150] Step 10:

[1151] The device passes the generated audio data to the AI ​​narrator, which is instructed on the timing of audio playback.

[1152] Step 11:

[1153] The AI ​​narrator plays back the audio data and provides audible explanations to meeting participants, allowing the presenter to focus on other tasks while the audience listens to the explanations to deepen their understanding.

[1154] Example 1

[1155] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1156] The present invention aims to reduce the burden on users when explaining technical terms and background information during meetings, thereby improving the audience's concentration and comprehension. In such situations, presenters are required to convey specialized information in an easy-to-understand manner, but doing so requires a great deal of effort and time. Currently, there is no effective system for reducing this burden, so the present invention provides a system that solves this problem.

[1157] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1158] In this invention, the server includes a means for a user to input content to be narrated, a means for transmitting the input text to a communication device, a means for the communication device to transmit the input text to an endpoint of the server in the form of an API request, a means for the server to analyze the request and generate explanatory text using a generative AI model, a means for the server to return the generated explanatory text to the communication device, and a means for the communication device to convert the explanatory text into audio data and play the audio. This allows presenters to easily deliver expert explanations and enables audiences to deepen their understanding through the audio explanations.

[1159] A "user" is someone who uses the system to input narration content and initiates the process of generating the description and playing it back.

[1160] A "communication device" refers to equipment or software for sending and receiving data between a user and a server, and is responsible for sending text entered by the user to the server and receiving responses from the server.

[1161] A "server" is a device or program that analyzes requests received from users, generates explanatory text using a generative AI model, and returns the generated explanatory text to the communication device.

[1162] A "generative AI model" is an artificial intelligence model that uses natural language processing algorithms to generate relevant explanatory text from input text.

[1163] An "API request format" refers to a standardized data format used by a communications device to send data to a server, typically including JSON or XML formats.

[1164] A "speech synthesis engine" is software or hardware that converts text data into voice data and has text-to-speech functionality.

[1165] "Explanation" is text data generated by a generative AI model that contains detailed explanations of technical terms and background information entered by the user.

[1166] "Audio data" is a data format that can reproduce explanatory text as audio, generated by a speech synthesis engine.

[1167] "Audio playback" refers to the act of an AI narrator or other audio output device using the generated audio data to play audio.

[1168] The present invention provides a system that aims to reduce the burden on users when narrating technical terms and background information during a meeting, thereby improving the audience's concentration and comprehension. This system allows users to easily deliver technical explanations, and allows the audience to deepen their understanding through the audio explanations. An embodiment of the present invention is described in detail below.

[1169] User operations

[1170] The user inputs the content to be narrated through the conferencing software interface. For example, the user enters the text "What is machine learning?" into the conferencing software's text input field and clicks the submit button. This prepares the system to convert the content into explanatory text.

[1171] Transmitting text data to a communication device

[1172] The communications device receives the text data entered by the user and converts it into an API request format. For example, it converts it into JSON format data. This API request is then sent as an HTTP request to the server's endpoint. An HTTP POST request is used, and the URL is "https: / / api.example.com / generate-explanation."

[1173] Sending the request to the server and parsing it

[1174] The server analyzes the request received at the API endpoint, checks the JSON data of the request, and obtains the value of the "query" key. The obtained text, "What is machine learning?", is passed to the generative AI model.

[1175] Generating explanatory text using a generative AI model

[1176] The server generates an explanation by providing the text analyzed using the generative AI model as input data. The generative AI model (e.g., a natural language processing model) is invoked to generate an explanation based on the prompt "What is machine learning?" The generated explanation is "Machine learning is a technology that uses data to enable computers to learn on their own and perform specific tasks."

[1177] Returning generated data to communication device and converting it into voice

[1178] The server converts the generated explanation into JSON format and returns it to the communication device. The communication device analyzes the received response data and obtains the value of the "explanation" key. A text-to-speech engine is used to convert the obtained text data into audio data. For example, Google's TTS API is used to convert the text into an audio file.

[1179] Audio playback by AI narrator

[1180] The communication device passes the generated audio data to the AI ​​narrator, which then issues instructions to read aloud, at an appropriate time during the meeting, "Machine learning is a technology that uses data to enable computers to learn on their own and perform specific tasks." The AI ​​narrator then plays back the audio data and provides the generated explanation to the meeting participants.

[1181] This process reduces the burden on the user (presenter) to explain technical information in an easy-to-understand manner, and allows the audience to understand the content through their eyes and ears.

[1182] Specific examples

[1183] Meeting Scenarios

[1184] The user (presenter) holds a meeting to explain the new technology "machine learning" to the leaders of each department. The presenter enters the narration content "What is machine learning?" into the conference software.

[1185] System Operation

[1186] During a meeting, a user enters "What is machine learning?" into a text input field and presses the send button. The communication device sends this text data to the server as an API request. The server uses a generative AI model to generate an explanation, which it returns to the communication device: "Machine learning is a technology that uses data to enable computers to learn on their own and perform specific tasks." The communication device then converts this explanation into audio data, which is then played back by an AI narrator.

[1187] In this way, the present invention provides a system that reduces the burden on users and promotes audience understanding.

[1188] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1189] Step 1:

[1190] The user enters the content to be narrated through the conferencing software interface. The user enters "What is machine learning?" in the text input field and presses the send button. Input (text): "What is machine learning?" Output (text data): "What is machine learning?" text data.

[1191] Step 2:

[1192] The device receives text data sent by the user. Since the server cannot understand the received text data as it is, the device converts it into an API request format. Specifically, it converts it into JSON format and creates the data structure "{"query": "What is machine learning"}". Input (text data): Text entered by the user, Output (API request): JSON format data.

[1193] Step 3:

[1194] The terminal sends the converted API request as an HTTP request to the server's API endpoint. The URL used is "https: / / api.example.com / generate-explanation". Input (API request): JSON format data, Output (request to server): HTTP POST request.

[1195] Step 4:

[1196] The server analyzes the request received at the API endpoint. The server parses the JSON data of the request and extracts the value of the "query" key. The extracted text, "What is machine learning?", is passed to the generative AI model. Input (HTTP request): API request data, Output (text data): Analyzed text data.

[1197] Step 5:

[1198] The server uses a generative AI model to generate an explanation based on the analyzed text, "What is machine learning?" The generative AI model uses a natural language processing algorithm to generate an explanation based on the prompt. An example output is the explanation, "Machine learning is a technology that uses data to enable computers to learn on their own and perform specific tasks." Input (text data): Analyzed text data, Output (explanation): Generated explanation.

[1199] Step 6:

[1200] The server converts the generated explanation into JSON format and returns it to the requesting device as an HTTP response. For example, the format is "{"explanation": "Machine learning is a technology that uses data to enable computers to learn on their own and perform specific tasks."}". Input (explanation): Generated explanation, Output (HTTP response): JSON format data.

[1201] Step 7:

[1202] The device analyzes the JSON format response data received from the server. It obtains the value of the "explanation" key and uses a speech synthesis engine to convert the text data into audio data. The synthesized speech engine uses Google's TTS API or similar to generate an audio file. Input (HTTP response): JSON format data, Output (audio data): Generated audio file.

[1203] Step 8:

[1204] The device passes the generated audio data to the AI ​​narrator, who then plays the audio data at the appropriate time during the meeting, reading aloud, "Machine learning is a technology that uses data to enable computers to learn on their own and perform specific tasks." Input (audio data): Generated audio file; Output (audio playback): Audio provided to meeting participants.

[1205] This series of processes reduces the burden of explanation on the presenter and allows the audience to deepen their understanding.

[1206] (Application example 1)

[1207] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1208] In recent years, with the spread of virtual stores, users have been demanding that they be able to easily obtain detailed information about products. However, many current virtual store systems have difficulty quickly and accurately providing the detailed explanations and background information that users require. Furthermore, when users ask questions about products, it is also difficult to provide appropriate answers and explanations in real time. This makes it difficult to increase user satisfaction and purchasing motivation.

[1209] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1210] In this invention, the server includes means for a user to input content to be narrated, means for transmitting the input text to the server, means for generating a description using a generative AI model, means for returning the generated description to the terminal, means for the terminal to convert the description into audio data and have an AI narrator read it aloud, means for the terminal to provide detailed descriptions of products in the virtual store in real time using the generative AI model, and means for customers to input questions through the virtual store interface. This allows users to simply input a question to obtain a detailed description using the generative AI model in real time and to listen to that description in audio.

[1211] A "user" is a user of the system who inputs narration content and questions.

[1212] "Narration" is the act of explaining information orally in a meeting or presentation.

[1213] "Input means" refers to an interface through which a user provides text data to the system.

[1214] "Input text" is character string data such as narration content or questions that a user inputs into the system.

[1215] A "server" is a computing system that processes input data and generates new data using generative AI models.

[1216] A "generative AI model" is an algorithm that uses natural language processing technology and generates explanations and answers based on input text.

[1217] An "explanation" is a document created by a generative AI model that contains detailed information related to the input text.

[1218] A "terminal" is a device operated by a user, and is a device that communicates with a server to send and receive data.

[1219] "Audio data" is data obtained by converting text data into audio, and is data for providing information to the user by audio.

[1220] "AI Narrator" is a virtual assistant that plays audio data and reads the generated explanation aloud.

[1221] A "virtual store" is a virtual store that operates on the Internet and is a platform where users can browse and purchase products online.

[1222] A "detailed description" is a description that includes detailed information about a product or service.

[1223] An "interface" is a part that includes an operation screen and means for exchanging information between the user and the system.

[1224] A "question" is a short sentence or document that allows a user to enter information they want to know about a product or service.

[1225] The present invention relates to a system that automatically generates detailed explanations and provides voice guidance when a user inputs a question about a product in a virtual store. Specific embodiments for carrying out the present invention will be described below.

[1226] First, a user enters a question through a virtual store interface, which could be a web browser or a mobile application. Suppose a user enters a question such as, "What are the features of this smartwatch?"

[1227] The terminal (client device) converts the text data entered by the user into an API request format and sends it to the specified server. This process uses an HTTP request, and the request content is encoded in JSON format or similar.

[1228] The server receives requests at an API endpoint. The received request content is passed to a generative AI model. The generative AI model uses natural language processing technology (e.g., BERT or GPT-3). The model generates an appropriate explanation for the input question.

[1229] The generated description is sent back to the device as a response from the server. The device then analyzes the received description and converts it into audio data. This process uses a speech synthesis engine such as the Google Text-to-Speech (gTTS) engine.

[1230] Once the audio data is generated, an AI narrator on the device plays it back and provides the generated explanation to the user, allowing users to simply input their question, receive a detailed explanation in real time using the generative AI model, and listen to that explanation in audio.

[1231] For example, if a user wants to know more about a new product, a "smartwatch," in a virtual store, they can type, "What are the features of this smartwatch?" The system sends this question to the server, and using a generative AI model, generates a description such as, "This smartwatch is a gadget with a long battery life, waterproof functionality, and comprehensive health tracking capabilities." The description is then converted into audio data and provided to the user by an AI narrator.

[1232] This allows the user to quickly and accurately obtain detailed information about products in the virtual store and use it as a basis for making a purchase decision.

[1233] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1234] Step 1:

[1235] The user inputs a question through the virtual store interface. The user inputs a question into the interface, such as "What are the features of this smartwatch?" The input data is sent to the terminal as a text string. In this step, the input data is the user's question text, and the output data is the text string received by the terminal.

[1236] Step 2:

[1237] The device converts the question text received from the user into an API request format and sends it to the specified server. Specifically, it encodes the text data into JSON format and sends it as an HTTP POST request. In this step, the input data is the user's question text, and the output data is the request converted into JSON format.

[1238] Step 3:

[1239] The server receives requests at the API endpoint. It analyzes the request content and passes it to the generative AI model. This process uses the Python requests library, etc. Specifically, it extracts the necessary text from the JSON-formatted data and passes it to the generative AI model. In this step, the input data is the JSON-formatted request, and the output data is the text data passed to the generative AI model.

[1240] Step 4:

[1241] A generative AI model generates a detailed explanation based on the input question text. For example, a natural language processing model such as GPT-3 or BERT is used. Specifically, it processes the input text as a prompt and outputs a generated explanation. In this step, the input data is the question text, and the output data is the generated explanation.

[1242] Step 5:

[1243] The server returns the generated description to the requesting terminal as a response. The response data is encoded in JSON format and sent as an HTTP response. In this step, the input data is the generated description, and the output data is a JSON-formatted response.

[1244] Step 6:

[1245] The device analyzes the received response data and converts it into voice data. Specifically, it uses a speech synthesis engine such as the Google Text-to-Speech (gTTS) engine to convert the text data into voice data. In this step, the input data is a JSON-formatted response, and the output data is voice data.

[1246] Step 7:

[1247] The AI ​​narrator plays the generated audio data and provides the user with audio information. Specifically, the audio is played through the device's speaker. In this step, the input data is the audio data, and the output data is the audio information provided to the user.

[1248] In this way, the user can quickly and accurately obtain detailed information about products in the virtual store and hear their explanations in audio.

[1249] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1250] This invention relates to a system that aims to reduce the burden on users when narrating technical terms and background information during meetings, thereby improving the audience's concentration and comprehension. The system implements a series of processes: the user inputs the content to be narrated, the content is converted into explanatory text using a generative AI model, and finally, an AI narrator reads it aloud. It also combines an emotion engine to analyze the user's emotions and dynamically adjust the explanatory text and audio data accordingly.

[1251] Specific embodiments will be described below.

[1252] 1. User operations

[1253] User: Enters the content to be narrated through the conferencing software interface. For example, the user enters the text "What is machine learning?" into the input field.

[1254] 2. From the user to the device

[1255] Terminal: Receives text data entered by the user and passes it to the emotion engine. The emotion engine analyzes the user's emotions from the text and voice data. It converts the data containing the analysis results into an API request format and sends it to the server endpoint.

[1256] 3. From the device to the server

[1257] Server: Receives the API request, analyzes the request body, and prepares the analyzed text data and emotion data.

[1258] 4. Server Processing

[1259] Server: Passes the analyzed text data and emotion data to the generative AI model, which generates a description based on the input data and adjusts the tone and content of the description based on the emotion data as needed.

[1260] 5. From the server to the device

[1261] Server: Converts the generated description into an appropriate format (for example, JSON or XML) and sends it back to the requesting device as a response.

[1262] Terminal: Analyzes the received response data and obtains the explanatory data.

[1263] 6. Terminal Processing

[1264] Device: The acquired description is passed to a Text-to-Speech (TTS) engine and converted into audio data. It is also possible to adjust the tone and speed of the audio based on emotional data.

[1265] 7. AI Narrator Reading

[1266] Device: Passes the generated voice data to the AI ​​narrator and issues instructions to read it aloud at the appropriate time.

[1267] AI Narrator: Plays back audio data and provides audio explanations to meeting participants, allowing the presenter to focus on other tasks while the audience listens to the explanations to deepen their understanding.

[1268] Specific examples

[1269] Meeting Scenarios

[1270] User (Presenter): You have a meeting where you need to explain a new technology called "machine learning" to the leaders of each department.

[1271] Narration content: The presenter enters the narration content "What is machine learning?" into the conference software.

[1272] System Operation

[1273] 1. A user types "What is machine learning?" into the conferencing software.

[1274] 2. The device receives the text data and sends an API request to the server, including the analysis results from the emotion engine.

[1275] 3. The server receives the request and generates a description using a generative AI model, adjusting the tone and content of the description as needed based on the sentiment data.

[1276] 4. The server returns the generated description to the device.

[1277] 5. The device receives the description and converts it into audio data, adjusting the tone and speed of the audio based on the emotional data.

[1278] 6. The AI ​​narrator plays the audio and reads, "Machine learning is a technology that uses data to teach computers how to perform specific tasks."

[1279] In this way, the presenter can reduce the burden on the presenter, promote the audience's understanding, and maintain their concentration. Furthermore, by combining it with an emotion engine, the presenter can provide flexible narration that responds to the user's emotions.

[1280] The processing flow will be explained below.

[1281] Step 1:

[1282] A user types what they want to narrate into the conferencing software interface, for example, by typing "What is machine learning" into a text field.

[1283] Step 2:

[1284] The terminal receives the user's input text and passes the data to the emotion engine, which analyzes the user's emotions from the text data and voice data and generates an analysis result.

[1285] Step 3:

[1286] The device converts the data, including the analysis results of the emotion engine, into an API request format, which includes the narration content and emotion data.

[1287] Step 4:

[1288] The device sends an API request to the server at the specified endpoint.

[1289] Step 5:

[1290] The server receives the API request, parses the request body, and prepares the parsed text data and emotion data.

[1291] Step 6:

[1292] The server passes the analyzed text data and emotion data to a generative AI model, which generates a description based on the user's input and adjusts the tone and content of the description based on the emotion data as needed.

[1293] Step 7:

[1294] The server converts the generated description into JSON or XML format and sends it back to the requesting device as a response.

[1295] Step 8:

[1296] The device receives the API response and parses and extracts the description data.

[1297] Step 9:

[1298] The device passes the extracted description to a Text-to-Speech (TTS) engine and converts it into audio data, which can adjust the tone and speed of the voice based on emotional data.

[1299] Step 10:

[1300] The device passes the generated audio data to the AI ​​narrator and instructs it on the timing of audio playback.

[1301] Step 11:

[1302] The AI ​​narrator plays back the audio data and provides audible explanations to meeting participants, allowing the presenter to focus on other tasks while the audience listens to the explanations to deepen their understanding.

[1303] Specific examples

[1304] A user types "What is machine learning?" into the conferencing software.

[1305] The device receives the user's input, and the emotion engine analyzes the user's emotion. For example, if the user is nervous, it analyzes it as "nervous."

[1306] The device sends an API request containing text data and emotion data to the server.

[1307] The server receives the request and uses a generative AI model to generate a description such as, "Machine learning is a technique that uses data to enable computers to teach themselves and perform specific tasks."

[1308] The server returns the generated explanation to the terminal.

[1309] The device receives the text and passes it to a TTS engine to convert it into speech data. Based on the emotional data, the tone and speed of the speech are adjusted to create a tone that alleviates tension.

[1310] The AI ​​narrator plays the audio and reads, "Machine learning is a technique that uses data to enable computers to teach themselves and perform specific tasks."

[1311] Example 2

[1312] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1313] Currently, narrating technical terms and background information during meetings can be a significant burden for presenters, potentially reducing the audience's concentration and comprehension. Furthermore, it is difficult to adapt flexibly to the user's emotions, limiting the effectiveness of narration. There is a need for a system that can resolve these issues and promote effective communication and understanding.

[1314] The specification process by the specification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes a means for a user to input content to be narrated, a means for receiving the input text and analyzing emotions using an analysis engine, a means for converting the analyzed data into an API request format and sending it to the server, a means for the server to generate explanatory text using a generation AI model, a means for adjusting the tone and content of the generated explanatory text based on the emotional data, a means for returning the generated explanatory text to the terminal, and a means for the terminal to convert the explanatory text into audio data and have an AI narrator read it aloud. This reduces the burden on the presenter in narrating and improves the audience's concentration and comprehension.

[1315] "User" refers to the person who uses this system to provide narration during a meeting.

[1316] "Narration content" refers to text data such as technical terms and background information that a user wants to explain during a conference.

[1317] "Terminal" refers to the computer or mobile device that a user uses to enter narration content.

[1318] An "analysis engine" refers to software or hardware that analyzes a user's emotions from input text data or voice data.

[1319] "Emotion data" refers to information about the user's emotions analyzed by the analysis engine, and includes emotion labels such as "neutral" and "excited."

[1320] "API request" refers to the protocol used by the device to send data to the server, including parsed text data and emotion data.

[1321] "Server" refers to the computational resources on the network that are responsible for processing the analyzed data and generating narratives using generative AI models.

[1322] "Generative AI model" refers to the artificial intelligence technology used by the server to generate the description, including, for example, a natural language processing model.

[1323] "Explanatory text" refers to text generated by a generative AI model that clearly explains the content of the user's narration.

[1324] "Text-to-Speech (TTS) engine" refers to software for converting text data into voice data.

[1325] "AI narrator" refers to software or hardware that plays back generated audio data and provides audio narration.

[1326] "Audio Data" refers to the audio files generated by the Text-to-Speech engine and played by the AI ​​narrator.

[1327] "Tone" refers to the emotional resonance or tone of a voice or text, and is adjusted based on emotional data.

[1328] This invention relates to a system that reduces the burden on users when narrating technical terms and background information during meetings, improving audience concentration and comprehension. The system implements a series of processes: the user inputs the content to be narrated, the content is converted into explanatory text using a generative AI model, and finally, an AI narrator reads it aloud. It also incorporates an emotion engine to analyze the user's emotions and dynamically adjust the explanatory text and audio data accordingly.

[1329] Specifically, a user enters the content to be narrated into a text field through the conferencing software interface. For example, consider a user entering "What is machine learning?" This input is received by the device and then analyzed by an emotion engine (e.g., IBM Watson Tone Analyzer). The emotion engine analyzes the user's emotions and generates emotion data such as "neutral" or "excited."

[1330] The analysis results are converted into an API request format and sent to a server. The server receives this API request and generates a description using a generative AI model (e.g., OpenAI GPT-3). The tone and content of the generated description may be adjusted based on the emotional data. For example, a description might be generated that reads, "Machine learning is a technology that uses data to enable computers to learn on their own and perform specific tasks."

[1331] The generated description is converted into an appropriate format, such as JSON, by the server and sent back to the device. The device receives and analyzes this data, then passes it to a Text-to-Speech (TTS) engine (e.g., Google Cloud Text-to-Speech) to convert it into audio data. The tone and speed of the voice are also adjusted based on the emotion data. For example, if a "neutral" emotion label is added, the voice will be generated in a calm and composed tone.

[1332] Finally, the device passes the generated audio data to the AI ​​narrator, which then issues instructions to read it aloud at the appropriate time. For example, the user can manually instruct the narrator to read it aloud during the presentation. The AI ​​narrator plays back the audio data and reads out loud, "Machine learning is a technology that uses data to enable computers to learn on their own and perform specific tasks," making it easier for the audience to understand the content.

[1333] Specific examples

[1334] Meeting Scenarios

[1335] A meeting is held in which the user (presenter) needs to explain a new technology called "machine learning" to the leaders of each department.

[1336] Narration content: The presenter enters the narration content "What is machine learning?" into the conference software.

[1337] Prompt Sentence Examples

[1338] The user types the following into the conferencing software:

[1339] What is machine learning?

[1340] This system reduces the burden on presenters, promotes audience understanding, and helps maintain concentration. In addition, by combining it with an emotion engine, it enables flexible narration that responds to the user's emotions.

[1341] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1342] System processing flow

[1343] Step 1:

[1344] The user enters the content to be narrated into a text field through the conferencing software interface. For example, they might enter "What is machine learning?" This input is then sent to the terminal as text data required for the next process.

[1345] Input: User-entered text "What is machine learning?"

[1346] Output: Text data "What is machine learning?"

[1347] Step 2:

[1348] The device receives the text data entered by the user. The received text data is then passed to an emotion engine (such as IBM Watson Tone Analyzer or a similar analysis engine) for analysis. The emotion engine analyzes the text data and generates an emotion label, such as "neutral" or "excited."

[1349] Input: Text data "What is machine learning?"

[1350] Output: Emotion data (e.g. "Neutral")

[1351] Step 3:

[1352] The device converts the analyzed text data and emotion data into an API request format. This API request format data is in a format for sending to the server. The converted data is then sent to the server.

[1353] Input: Text data "What is machine learning?", emotion data "Neutral"

[1354] Output: Data in API request format

[1355] Step 4:

[1356] The server receives the API request and parses the request body, taking the parsed text and sentiment data and feeding it into a generative AI model (such as OpenAI GPT-3 or a similar natural language processing model). This model combines the original text and sentiment data to generate a description.

[1357] Input: Data in API request format (text data "What is machine learning?", sentiment data "Neutral")

[1358] Output: Generated description (e.g., "Machine learning is a technique that uses data to teach computers how to perform specific tasks.")

[1359] Step 5:

[1360] The server converts the generated description into an appropriate format (e.g., JSON or XML) and sends it back to the requesting device as a response.

[1361] Input: Generated description

[1362] Output: JSON format data returned to the terminal

[1363] Step 6:

[1364] The device analyzes the response data received and obtains the generated explanation. This explanation is then passed to a Text-to-Speech (TTS) engine (such as Google Cloud Text-to-Speech or a similar voice synthesis engine) to convert it into voice data. At this time, the tone and speed of the voice are also adjusted based on the emotional data.

[1365] Input: JSON format data (generated description)

[1366] Output: Audio data

[1367] Step 7:

[1368] The device then passes the generated audio data to the AI ​​narrator and issues instructions to read it aloud at the appropriate time. For example, the user can manually instruct the AI ​​narrator to read it aloud during a presentation.

[1369] The AI ​​narrator plays the audio and provides audible explanations to meeting participants, such as, "Machine learning is a technology that uses data to enable computers to teach themselves and perform specific tasks."

[1370] Input: Audio data

[1371] Output: Audio instructions are provided to meeting participants

[1372] This series of steps allows users to provide easy-to-understand explanations to the audience while reducing the burden of narration. In addition, the introduction of an emotion engine enables flexible narration that responds to the user's emotions.

[1373] (Application example 2)

[1374] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1375] In manufacturing sites, there is a need for robots to efficiently and reliably provide instructions on work tasks and procedures. Detailed, easy-to-understand explanations are necessary for workers to understand the setup and operation procedures of new machines, but conventional systems have not been able to adequately provide this. There is also a need for technology that can reduce the burden on workers and reduce errors.

[1376] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a means for a user to input the content to be narrated, a means for transmitting the input text to the server, a means for the server to generate explanatory text using a generative AI model, a means for the server to return the generated explanatory text to the terminal, a means for the terminal to convert the explanatory text into audio data and for an AI narrator to read it aloud, and a means for the robot to provide audio and visual work instructions. This provides detailed and easy-to-understand explanations of work procedures at the manufacturing site, making it easier for workers to understand and reducing operational errors.

[1377] "User" refers to the person or operator who operates the system and inputs the narration content.

[1378] "Narration content" refers to the text information of explanations and comments that the user inputs to the system.

[1379] "Input text" refers to text data of the narration content input by the user to the system.

[1380] A "server" refers to a computer on a network that receives requests from users, uses a generative AI model to generate explanatory text, and returns it to the device.

[1381] A "generative AI model" refers to an artificial intelligence model that uses natural language processing technology to convert input text into explanatory text.

[1382] "Explanation" refers to the text information of explanations or descriptions generated by a generative AI model from input text.

[1383] "Terminal" refers to a device that receives user input and communicates with a server to convert generated descriptions into audio data.

[1384] "Audio data" refers to data in audio format that has been converted from explanatory text using a speech synthesis engine.

[1385] "AI Narrator" refers to a speech synthesis system that uses generated voice data to read explanations aloud on behalf of the user.

[1386] A "speech synthesis engine" refers to software or hardware for converting input text data into voice data.

[1387] "Robot" refers to an automated mechanical device that is placed on a manufacturing floor and is capable of providing audio and visual instructions for work.

[1388] A specific system for implementing this invention includes an interface where a user inputs narration content, a terminal for processing text data, a server that generates explanatory text using a generative AI model, and a robot that provides audio and visual explanations.

[1389] System configuration

[1390] 1. User Interface:

[1391] An interface for users to input narration content (e.g., input devices such as PCs and tablets).

[1392] The interface has the function of sending input text to the server.

[1393] 2. Terminal:

[1394] Receives input text and sends it to the server.

[1395] It has the function of receiving the description data returned from the server and converting it into audio data (example of use: Python library gTTS, etc.).

[1396] 3. Server:

[1397] Use a generative AI model (e.g., GPT-3) to generate a description from the input text.

[1398] A response including the description and associated emotion data is sent back to the device.

[1399] The server uses natural language processing technology to analyze the user's input and generate effective explanations.

[1400] 4. Speech synthesis engine:

[1401] It is responsible for converting explanatory text into audio data (e.g. gTTS library).

[1402] By adjusting the tone and speed of the generated voice data, the narration is performed according to the user's emotions.

[1403] 5. Robots:

[1404] An automated mechanical device that provides audio and visual instructions for work.

[1405] It has the function of playing back audio data and visually displaying specific work procedures.

[1406] System operation example

[1407] Consider a scenario where a user is explaining the setup procedure for a new manufacturing machine. The system works as follows:

[1408] 1. User Input:

[1409] The user enters the following into the interface: "I'd like to explain the setup procedure for a new machine."

[1410] 2. Sending input text:

[1411] The device sends the input text to the server, which also obtains emotional data via an emotion analysis engine.

[1412] 3. Description generation:

[1413] The server generates an explanation using a generative AI model, for example, using the following prompt:

[1414] Here is the procedure for setting up a new manufacturing machine.

[1415] 4. Return of description:

[1416] The server sends the generated instructions back to the terminal. For example, it generates instructions like, "First, connect the machine to the power source. Next, select "Settings" from the main menu to perform the initial setup and follow the instructions. After that, install each part in the correct position. Check that all parts are installed correctly."

[1417] 5. Generate audio data:

[1418] The device converts the received text into audio data, which the robot reads aloud. A speech synthesis engine narrates in a tone and speed that matches the user's emotions.

[1419] 6. Robot explanation:

[1420] The robot provides audio and visual instructions, allowing the worker to carry out the setup procedure with clear guidance.

[1421] In this way, work procedures at the manufacturing site can be explained in detail and in an easy-to-understand manner, promoting understanding among workers and reducing operational errors.

[1422] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1423] Step 1:

[1424] The user enters the narration content into the interface, for example, "I will explain the setup procedure for a new machine." The input text is sent to the terminal.

[1425] Step 2:

[1426] The device sends the received input text to the emotion engine, which analyzes the emotional data. The emotion engine analyzes the text data and extracts emotional data (e.g., notification tone and emotional intensity). This generates an API request containing the emotional data and sends it to the server.

[1427] Step 3:

[1428] The server receives the API request and parses the request body. It extracts the input text and emotion data and passes them to the generative AI model. The server then uses the generative AI model to generate an explanation. Specifically, the following prompt sentence is used as input:

[1429] Here is the procedure for setting up a new manufacturing machine.

[1430] The generative AI model generates explanatory text based on input text and sentiment data.

[1431] Step 4:

[1432] The server converts the generated description into an appropriate format (e.g., JSON) and sends a response containing the description and emotion data back to the device.

[1433] Step 5:

[1434] The device analyzes the response data received from the server, extracts explanatory text data and passes it to the speech synthesis engine, and adjusts the tone and speed of the generated speech based on the emotional data.

[1435] Step 6:

[1436] The device uses a speech synthesis engine (e.g., gTTS) to convert the description into audio data, which is then ready to be played by the AI ​​narrator.

[1437] Step 7:

[1438] The AI ​​narrator plays audio data at specified times, and the robot uses this audio data to explain the work.The robot also visually demonstrates the work steps and provides specific instructions to the worker.

[1439] Through the above processing steps, the system reduces the burden on users, effectively communicates work procedures at the manufacturing site, and reduces operational errors.

[1440] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1441] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1442] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1443] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1444] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1445] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1446] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1447] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1448] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1449] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1450] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1451] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1452] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1453] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1454] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1455] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1456] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1457] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1458] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1459] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1460] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1461] The following is further disclosed regarding the above embodiment.

[1462] (Claim 1)

[1463] a means for a user to input content to be narrated;

[1464] means for transmitting input text to a server;

[1465] A means for the server to generate an explanatory text using the generative AI model;

[1466] a means for the server to return the generated description to the terminal;

[1467] The device converts the description into audio data and an AI narrator reads it aloud.

[1468] A system including:

[1469] (Claim 2)

[1470] The system of claim 1, wherein the generative AI model used to generate the description is a natural language processing model.

[1471] (Claim 3)

[1472] 2. The system of claim 1, wherein the means used to generate the audio data is a speech synthesis engine.

[1473] "Example 1"

[1474] (Claim 1)

[1475] a means for a user to input content to be narrated;

[1476] means for transmitting the input text to a communication device;

[1477] means for the communication device to transmit input text to a server endpoint in the form of an API request;

[1478] a means for the server to parse the request and generate a description using a generative AI model;

[1479] means for the server to return the generated description to the communication device;

[1480] a means for converting the explanatory text into audio data and playing the audio in the communication device;

[1481] A system including:

[1482] (Claim 2)

[1483] The system of claim 1, wherein the generative AI model used to generate the description is a natural language processing model.

[1484] (Claim 3)

[1485] 2. The system of claim 1, wherein the means used to generate the audio data is a synthetic speech engine.

[1486] "Application Example 1"

[1487] (Claim 1)

[1488] a means for a user to input content to be narrated;

[1489] means for transmitting input text to a server;

[1490] A means for the server to generate an explanatory text using the generative AI model;

[1491] a means for the server to return the generated description to the terminal;

[1492] The device converts the description into audio data and an AI narrator reads it aloud.

[1493] a means for the terminal to provide detailed descriptions of products in the virtual store in real time using a generative AI model;

[1494] a means for customers to input questions through the virtual store interface;

[1495] A system including:

[1496] (Claim 2)

[1497] The system of claim 1, wherein the generative AI model used to generate the description is a natural language processing model.

[1498] (Claim 3)

[1499] 2. The system of claim 1, wherein the means used to generate the audio data is a speech synthesis engine.

[1500] "Example 2: Combining Emotion Engines"

[1501] (Claim 1)

[1502] a means for a user to input content to be narrated;

[1503] means for receiving input text and analyzing emotion using an analysis engine;

[1504] A means for converting the analyzed data into an API request format and sending it to the server;

[1505] A means for the server to generate an explanatory text using the generative AI model;

[1506] a means for adjusting the tone and content of the generated description based on the emotion data;

[1507] a means for the server to return the generated description to the terminal;

[1508] The device converts the description into audio data and an AI narrator reads it aloud.

[1509] A system including:

[1510] (Claim 2)

[1511] The system of claim 1, wherein the generative AI model used to generate the description is a natural language processing model.

[1512] (Claim 3)

[1513] 2. The system of claim 1, wherein the means used to generate the audio data is a speech synthesis engine.

[1514] "Application example 2 when combining emotion engines"

[1515] (Claim 1)

[1516] a means for a user to input content to be narrated;

[1517] means for transmitting input text to a server;

[1518] A means for the server to generate an explanatory text using the generative AI model;

[1519] a means for the server to return the generated description to the terminal;

[1520] The device converts the description into audio data and an AI narrator reads it aloud.

[1521] A means for the robot to provide audio and visual instructions for the work;

[1522] A system including:

[1523] (Claim 2)

[1524] 2. The system of claim 1, wherein the generative AI model is a natural language processing model.

[1525] (Claim 3)

[1526] 2. The system of claim 1, wherein the means used to generate the audio data is a speech synthesis engine. [Explanation of symbols]

[1527] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. a means for a user to input content to be narrated; means for transmitting input text to a server; A means for the server to generate an explanatory text using the generative AI model; a means for the server to return the generated description to the terminal; The device converts the description into audio data and an AI narrator reads it aloud. A system including:

2. The system of claim 1, wherein the generative AI model used to generate the narrative is a natural language processing model.

3. 2. The system of claim 1, wherein the means used to generate the audio data is a speech synthesis engine.

Images