system

Abstract

Provide a system. A method for receiving information from a user regarding history, personality, occupation, weather, and locality; data processing means for providing the generated speech transcript to a user; means for receiving feedback from a user and optimizing said generating means; a means of translation and diversity check; A speech assistance system including:

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional speechwriting services rely on human labor, which not only takes time and money, but also makes it difficult to maintain consistent speech quality. Furthermore, they lack multilingual support and consideration for diversity, making them necessary for international business situations and multicultural work environments. Furthermore, when speeches need to be written daily, there is a risk of repeating the same content. This invention aims to solve these problems and dramatically improve the quality and efficiency of speeches. [Means for solving the problem]

[0005] The present invention solves the above problems by using the following means. Specifically, it provides a speech support system including: means for receiving information from a user regarding background, personality, occupation, weather, and locality; data processing means for providing a generated speech script to the user; means for receiving user feedback and optimizing the generation means; and means for performing translation and diversity checks. Furthermore, by providing a means for adjusting the speech script based on the speech length and date and time specified by the user, it provides consistent, high-quality speeches. Furthermore, by including a means for delivering speech scripts translated into multiple languages ​​to users via QR codes (registered trademark), it can be adapted to international business situations and multicultural work environments. This improves the efficiency and quality of speechwriting.

[0006] "History" refers to information about a user's past activities and history, such as their career history, educational background, and work experience.

[0007] "Personality" is information that indicates the characteristics and features of an individual, such as a user's hobbies, personality, and values.

[0008] "Occupation" refers to information related to professional activities, such as the job, duties, and position of the user.

[0009] "Weather" refers to information about the weather conditions in the area where the user is currently located, such as sunny, rainy, or snowy.

[0010] "Region" refers to information about the geographical characteristics, culture, customs, etc. of the location where the user is currently located.

[0011] The "data processing means" is a means for generating a speech manuscript based on information received from a user and performing processing to provide the speech manuscript to the user in an appropriate format.

[0012] "Feedback" refers to the opinions and reactions users provide after a speech, particularly information about what was well-received and what was problematic.

[0013] The "generation means" is a means for automatically creating a new speech manuscript based on previously input information and feedback.

[0014] The "translation means" is a means for converting the generated speech manuscript into multiple languages.

[0015] A "diversity check" is a means of checking whether the content of a speech is discriminatory from a diversity perspective.

[0016] "QR code" refers to a form of two-dimensional barcode used to easily share speech manuscripts and translation results.

[0017] The "speech support system" is a system that automatically generates speech manuscripts based on user information, reflects feedback, and performs multilingual support and diversity checks. [Brief explanation of the drawings]

[0018] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8]FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0019] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0020] First, the terms used in the following description will be explained.

[0021] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0022] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0023] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0024] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0025] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0026] [First embodiment]

[0027] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0028] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0029] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0030] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0031] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0032] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0033] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0034] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0035] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0036] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0037] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0038] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0039] The present invention relates to a system for automating speechwriting functions and generating efficient, high-quality speech manuscripts. A specific method for implementing the present invention will be described below with reference to the following embodiments.

[0040] System Overview

[0041] The system is mainly composed of three elements: the user, the server, and the terminal. Specifically, the server generates a speech manuscript based on the information provided by the user and provides it to the user via the terminal.

[0042] User operations

[0043] Enter information:

[0044] The user launches the "Asarekun" app and enters necessary information such as background (e.g., career, educational background), personality (e.g., hobbies, character), occupation (e.g., engineer, teacher), current weather (e.g., sunny, rainy), and region (e.g., New York, Tokyo). In addition, the user specifies the length and date and time of the speech, and submits this information within the app.

[0045] Server Operation

[0046] Generate a speech transcript:

[0047] The server receives the information sent by the user. Based on the received information, it uses a generative AI model to generate a speech script. During this process, it also references past speech scripts and feedback information stored in a database to select the optimal content.

[0048] Reflecting feedback:

[0049] The server receives feedback provided by users after they have used the speech. The feedback includes phrases that were well-received and parts that received poor responses. The server then incorporates this feedback into the generative AI model and uses it when generating the next speech manuscript.

[0050] Translation and Diversity Checks:

[0051] If necessary, the server translates the speech manuscript into multiple languages ​​as specified. It also checks the translation to ensure it does not contain discriminatory language from the perspective of diversity and inclusion. This checking function provides speech manuscripts that can be used safely in global business situations.

[0052] Device behavior

[0053] Manuscript provided by:

[0054] The generated speech transcript is sent to the user's device and displayed at the specified date and time. The user can review the displayed speech transcript and edit it as needed. A QR code can be generated on the user's device for the translated speech transcript, which can be easily shared with other participants.

[0055] Specific examples

[0056] For example, consider a company manager preparing a morning assembly speech for employees. The manager launches the "Asa-Kun" app and enters information such as his or her background (career as a manager), personality (emphasis on leadership and teamwork), occupation (manager at an IT company), current weather (sunny in Tokyo), and regional characteristics (Japanese culture). The manager also specifies that the speech will last approximately five minutes.

[0057] Based on this information, the server uses a generative AI model to create an optimal speech script. For example, if the current weather is sunny, the script might include phrases such as, "I'm happy to be able to start a wonderful day with you all in this sunny weather today." Regional differences are also taken into consideration, and seasonal greetings and content that touches on unique Japanese customs are included.

[0058] After the speech, the manager can submit feedback through the "Asarekun" app. For example, the manager might say, "The part introducing the new project was particularly well-received." The server then reflects this feedback and adjusts the AI ​​model to provide more appropriate content the next time the speech is generated.

[0059] Furthermore, when giving a speech at an international conference, a manager can request that their speech manuscript be translated into English, Chinese, or other languages. The server translates the speech into the specified language and checks whether the content meets the requirements from a diversity and inclusion perspective. The translation result is generated as a QR code, which the manager can easily share with other participants.

[0060] The above is a specific example of speech support using the "Asare-kun" system. This system combines the functions of user input, speech generation, feedback reflection, translation, and checking to provide efficient, high-quality speech support.

[0061] The processing flow will be explained below.

[0062] Step 1: The user launches the "Asarekun" app.

[0063] When a user launches the "Asare-kun" app on their smartphone or computer, user authentication is performed and the user ID is sent to the server.

[0064] Step 2: The user enters information.

[0065] Users fill out a form in the app with information about their background, personality, occupation, current weather, and location, as well as the length and date and time of the speech.

[0066] Step 3: The server receives the data.

[0067] The server receives the information sent by the user and temporarily stores it in a database, at which point the user ID and the information are linked.

[0068] Step 4: The server generates the speech script.

[0069] The server then activates a generative AI model based on the received information to automatically generate a speech manuscript, using past speeches and feedback information stored in a database.

[0070] Step 5: The server sends the manuscript to the user.

[0071] The generated speech manuscript is sent from the server to the user's terminal, where the user can view the speech manuscript on the terminal at the specified date and time.

[0072] Step 6: The user reviews and corrects the speech.

[0073] The user can review the speech manuscript they receive and make corrections within the app if necessary, and the corrections will be resubmitted to the server.

[0074] Step 7: The user gives a speech.

[0075] The user gives a speech based on the provided speech script, either using the generated speech script as is or modifying it as desired.

[0076] Step 8: User submits feedback.

[0077] After the speech, the user can enter feedback on the speech through the "Asarekun" app, detailing phrases that were particularly well-received and parts that received a poor response.

[0078] Step 9: The server receives the feedback.

[0079] The server receives feedback from users and stores this feedback information in a database.

[0080] Step 10: The server optimizes the generated AI model.

[0081] The server updates and optimizes the generative AI model based on the received feedback, so that the feedback is reflected in the next speech generation.

[0082] Step 11: User requests translation and checking.

[0083] If necessary, users can specify the translation and language of the speech manuscript, as well as the nationality and religious leanings of the audience, within the app.

[0084] Step 12: The server performs the translation and diversity check.

[0085] The server translates the speech manuscript into the specified language and checks the content from a diversity and inclusion perspective to ensure it does not contain discriminatory language.

[0086] Step 13: The server sends the translation result.

[0087] The server sends the translated speech manuscript to the user's terminal, where the translation result is displayed for the user to check.

[0088] Step 14: The user shares the translation manuscript.

[0089] The translated speech manuscript is generated as a QR code on the device, which the user can share with other participants.

[0090] The above is a specific flow of processing in the "Mr. Asakai" system of the present invention, which allows users to use efficient and high-quality speech manuscripts.

[0091] Example 1

[0092] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0093] Conventional speech support systems have the drawback of being time-consuming and labor-intensive, as the creation of speech manuscripts, translation, and feedback integration are all done manually. Furthermore, they lacked checking functions to accommodate different languages ​​and cultures, making them difficult to use in global business settings.

[0094] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0095] In this invention, the server includes means for receiving information on user history, personality, occupation, weather, and locality from the user, means for generating prompts using a generative AI model and creating a speech manuscript, data processing means for providing the generated speech manuscript to the user, means for receiving feedback from the user and optimizing the generation means, means for translating the speech manuscript into multiple languages ​​and performing a diversity check, and means for generating the translated speech manuscript as a QR code. This automates the process from generating the speech manuscript to providing it, incorporating feedback, and translating and checking it, enabling the efficient provision of high-quality speech manuscripts.

[0096] "User" refers to an individual or organization that intends to use the system to generate a speech manuscript.

[0097] "History" refers to information indicating a user's past experiences, such as career history, educational background, and work history.

[0098] "Personality" refers to information that indicates personal characteristics such as a user's hobbies and character.

[0099] "Occupation" refers to information indicating the job or specialty in which the user is engaged.

[0100] "Weather" refers to information indicating the weather conditions in the area where the speech will be given on the day.

[0101] "Regionality" refers to information that indicates the characteristics of the culture, customs, etc. of the place where the speech is given.

[0102] "Generative AI model" refers to an artificial intelligence model that automatically generates a speech manuscript based on information provided by the user.

[0103] A "prompt sentence" is an input sentence given to a generative AI model, and refers to text that guides the generation of a speech manuscript.

[0104] "Data processing means" refers to an information processing system for providing the generated speech manuscript to the user.

[0105] "Feedback" refers to information indicating reactions such as evaluations and impressions provided by users after they give a speech.

[0106] "Optimization methods" refer to techniques for receiving user feedback and improving the performance of generative AI models.

[0107] "Translation means" refers to a method or system for converting a speech transcript into a different language.

[0108] "Diversity check measures" refer to methods used to check whether translated speech manuscripts contain discriminatory language.

[0109] A "QR code" is a type of two-dimensional code, a graphic used to easily share speech manuscripts.

[0110] The present invention relates to a system for automating speechwriting functions and efficiently generating high-quality speech manuscripts. A specific method for implementing the present invention will be described below with reference to the following embodiments.

[0111] System Overview

[0112] This system is mainly composed of three elements: the user, the server, and the terminal. The user provides information, and the server generates a speech manuscript and provides it to the user through the terminal.

[0113] User operations

[0114] Enter information:

[0115] The user launches the "Speech Assistant" app and inputs necessary information such as background (e.g., career, educational background), personality (e.g., hobbies, character), occupation (e.g., engineer, teacher), current weather (e.g., sunny, rainy), and region (e.g., New York, Tokyo). In addition, the user specifies the length and date and time of the speech, and submits this information within the app.

[0116] Server Operation

[0117] Generate a speech transcript:

[0118] The server receives the information sent by the user. Based on the received information, it uses a generative AI model to generate a speech script. During this process, it also references past speech scripts and feedback information stored in a database to select the optimal content.

[0119] Examples:

[0120] For example, consider a company manager preparing a morning assembly speech for employees. The manager launches the "Speech Assistant" app and inputs information such as his or her background (career as a manager), personality (emphasis on leadership and teamwork), occupation (manager at an IT company), current weather (sunny in Tokyo), and regional characteristics (Japanese culture). The manager also specifies that the speech will last approximately five minutes.

[0121] Based on this information, the server uses a generative AI model to create an optimal speech script. For example, if the current weather is sunny, the script might include phrases such as, "I'm happy to be able to start a wonderful day with you all in this sunny weather today." Regional differences are also taken into consideration, and seasonal greetings and content that touches on unique Japanese customs are included.

[0122] Example prompt sentence:

[0123] "His background is 10 years in the IT industry, the weather is fine, and the speech is 5 minutes long."

[0124] Reflecting feedback:

[0125] The server receives feedback provided by users after they have used the speech. This feedback includes phrases that were well-received and parts that received poor responses. The server then incorporates this feedback into the generative AI model and uses it when generating the next speech manuscript.

[0126] Examples:

[0127] After the speech, the manager can provide feedback through the Speech Assistant app, such as, "The part introducing the new project was particularly well-received." The server then reflects this feedback and adjusts the AI ​​model to provide more appropriate content the next time the speech is generated.

[0128] Translation and Diversity Checks:

[0129] If necessary, the server translates the speech manuscript into multiple languages ​​as specified, and also checks the translation from the perspective of diversity and inclusion to ensure it does not contain discriminatory language.

[0130] Examples:

[0131] When giving a speech at an international conference, a manager can request that their speech manuscript be translated into English, Chinese, or other languages. The server translates the speech into the specified language and checks whether the content is appropriate from a diversity and inclusion perspective. The translation result is generated as a QR code, which the manager can easily share with other participants.

[0132] Device behavior

[0133] Manuscript provided by:

[0134] The generated speech transcript is sent to the user's device and displayed at the specified date and time. The user can review the displayed speech transcript and edit it as necessary. A QR code can be generated on the user's device for the translated speech transcript, allowing it to be easily shared with other participants.

[0135] Examples:

[0136] Users can check the generated speech manuscript through the "Speech Assistant" app and edit it as necessary. For example, when giving a speech at an international conference, the translated speech manuscript can be generated as a QR code and distributed to other participants, enabling smooth communication.

[0137] The above is a concrete example of how the present invention can be implemented. This system allows users to create high-quality speech manuscripts in a short amount of time, and can be used safely in global business situations.

[0138] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0139] Step 1:

[0140] Input and Action:

[0141] The user launches the "Speech Assistant" app and enters the necessary information, such as background, personality, occupation, weather, region, speech length, date and time, etc. The information is sent within the app by pressing the send button.

[0142] Data processing and output:

[0143] The information entered by the user is sent to the server in JSON format. The server receives this JSON data, analyzes the information stored in each field, and saves it as basic data for generating a speech manuscript.

[0144] Step 2:

[0145] Input and Action:

[0146] The server receives the JSON-formatted information sent by the user and parses it to extract information such as history, personality, occupation, weather, region, speech length, and date and time.

[0147] Data processing and output:

[0148] Based on the extracted information, a prompt sentence is generated for the generative AI model. The prompt sentence is input to the generative AI model in the form of "history is 10 years of experience in the IT industry, the weather is sunny, and the speech length is 5 minutes."

[0149] Step 3:

[0150] Input and Action:

[0151] The server inputs the generated prompt sentences into the generative AI model, which analyzes the prompt sentences and automatically generates a speech script.

[0152] Data processing and output:

[0153] The generative AI model also references past speech scripts and feedback information stored in a database to generate the optimal speech script, which is then temporarily stored on a server.

[0154] Step 4:

[0155] Input and Action:

[0156] When submitting feedback, the user can enter and submit their feedback after the speech through the "Speech Assistant" app. For example, they could enter feedback such as "The introduction to the new project was particularly well-received."

[0157] Data processing and output:

[0158] The server analyzes the feedback received from the user and reflects it in the generative AI model. The generative AI model is then readjusted based on the feedback information and used for generating the next speech manuscript.

[0159] Step 5:

[0160] Input and Action:

[0161] If the user requests multilingual translation, the server translates the speech into the specified language, for example, if it needs to be translated into English or Chinese.

[0162] Data processing and output:

[0163] The server translates the generated speech manuscript into the specified language, checks it from a diversity and inclusion perspective, verifies that the content is not discriminatory, and saves the translation results.

[0164] Step 6:

[0165] Input and Action:

[0166] Once the translation and checking process is complete on the server, the speech manuscript is sent to the user's device, where the user can check the displayed speech manuscript through the "Speech Assistant" app.

[0167] Data processing and output:

[0168] The speech manuscript displayed on the user's device can be viewed and edited by the user. In addition, a QR code can be generated on the user's device for the translated speech manuscript, allowing it to be easily shared with other participants.

[0169] The above is a specific flow of program processing for the system based on the claims. This system allows users to create high-quality speech manuscripts in a short amount of time, and to obtain deliverables that are efficient and can be used in multiple languages.

[0170] (Application example 1)

[0171] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0172] In today's world, opportunities for speeches and presentations are increasing, creating a demand for systems that can improve the quality and efficiency of speech drafts. Furthermore, with the advancement of globalization, multilingual support and cultural sensitivity are required, as well as a means for easily distributing generated speech drafts. However, existing systems lack the ability to provide individualized support based on the user's history and personality, and optimizing the system to reflect feedback is difficult. The objective of this invention is to solve these problems and provide high-quality speech support.

[0173] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0174] In this invention, the server includes means for receiving information from the user regarding background, personality, occupation, weather, and locality, data processing means for providing the generated speech transcript to the user, means for receiving user feedback and optimizing the generating means, means for performing translation and diversity checks, means for generating content translated into multiple languages ​​and taking social and cultural considerations into account, and means for outputting and distributing the generated speech transcript as a video or audio file, thereby enabling the generation of high-quality speech transcripts based on the user's specific information and the provision of content that is multilingual and culturally sensitive.

[0175] "User" refers to a person or organization that uses the system to provide information for generating a speech manuscript.

[0176] "Provenance" refers to information about a user's past experience, work and academic history, and other career information.

[0177] "Personality" refers to information about a user's personality, interests, hobbies, and personal characteristics.

[0178] "Occupation" refers to information about the job or work the user is currently doing.

[0179] "Weather" refers to information about the local weather conditions when the user delivers a speech.

[0180] "Regionality" refers to information about the cultural and social characteristics of the region where the user gives speech.

[0181] "Data processing means" refers to a computer processing device or software that generates a speech manuscript based on information provided by the user and provides it to the user.

[0182] A "generative AI model" is a model for generating speech manuscripts using artificial intelligence, and refers to an algorithm that generates optimal content based on past data and user-specified information.

[0183] "Feedback" refers to information such as reactions, evaluations, and areas for improvement provided by a user after they give a speech.

[0184] "Optimization means" refers to methods and algorithms for receiving feedback information and reflecting that information in the next speech script generation to improve the generative AI model.

[0185] "Translation and diversity check" refers to the process of converting the generated speech manuscript into multiple languages ​​and verifying that the content does not contain discriminatory language.

[0186] "Content" refers to the generated speech manuscript and multimedia information such as video and audio files created based on it.

[0187] A "video or audio file" is an audiovisual material created based on the generated speech manuscript, and is in a format for providing the speech content to users or viewers.

[0188] The present invention is a system for automatically generating a speech manuscript based on user information and providing the speech in various formats. The invention is specifically embodied in the following embodiments.

[0189] System Configuration

[0190] The system mainly consists of users, servers, and terminals.

[0191] User operations

[0192] The user starts the application using the smartphone and enters the following information:

[0193] Background (career, academic history, etc.)

[0194] Personality (hobbies, character, etc.)

[0195] Occupation (engineer, teacher, etc.)

[0196] Current weather (sunny, rainy, etc.)

[0197] Regional characteristics (region, culture, etc.)

[0198] Length and date of speech

[0199] This information is submitted by the user within the application.

[0200] Server Operation

[0201] The server has the following functions:

[0202] 1. Information reception: Receives information sent by the user.

[0203] 2. Speech generation: Based on the received information, a speech script is generated using a generative AI model (e.g., OpenAI® GPT-4®). During this process, past speech data and feedback information are referenced to select the optimal content.

[0204] 3. Feedback reflection: The system receives feedback provided by the user after the speech and reflects that information in the generative AI model when generating the next speech manuscript.

[0205] 4. Translation and Diversity Check: Speech manuscripts will be translated into multiple languages ​​as needed and checked from a diversity and inclusion perspective to ensure they do not contain discriminatory language.

[0206] 5. Content generation: Based on the generated speech manuscript, video and audio files are created and made available for distribution by users.

[0207] Device behavior

[0208] The generated speech manuscript is sent to the user's device and can be viewed within the application. The user can edit it and check the final speech content. Furthermore, the generated content (video and audio files) can be easily distributed via social media, email, etc.

[0209] Program processing

[0210] The server-side program is built using Node.js and Express and handles all processes from receiving user information, generating speech scripts, incorporating feedback, translating and checking, and generating content.

[0211] Processing of received user information: The information is converted into an appropriate format and input into the generative AI model.

[0212] Speech generation: The generative AI model generates a speech based on the provided information. OpenAI's GPT-4 is used as the generative model.

[0213] Feedback Processing: After each speech, user feedback is received. This feedback is used in the next generation to improve the accuracy and adaptability of the model.

[0214] Translation and diversity check: The generated speech transcripts are translated into multiple languages ​​and checked for discriminatory content, making them available for global use.

[0215] Content generation: Video and audio files are generated based on the speech manuscript and made available to users.

[0216] Specific examples

[0217] For example, if a user provides the following information on the topic of "wedding speeches," the following prompt sentences will be used for the generative AI model:

[0218] Prompt statement:

[0219] "Generate speech under the following conditions:

[0220] Experience: Over 10 years of experience in the IT industry

[0221] Personality: Team player

[0222] Occupation: Systems Engineer

[0223] Weather: Sunny

[0224] Region: Tokyo

[0225] Length: 5 minutes

[0226] Please prepare the speech."

[0227] The generative AI model uses this information to generate a speech transcript and provides it to the user, who can then edit it and distribute it as a video or audio file.

[0228] The above is a specific embodiment of the present invention, which allows users to efficiently generate and provide highly personalized speech manuscripts.

[0229] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0230] Step 1:

[0231] The user starts the application on their smartphone and inputs information about their background, personality, occupation, weather, and locality. They also specify the length of the speech and the date and time. This information becomes the input data.

[0232] Step 2:

[0233] The terminal receives the information entered by the user and sends it to the server. Specifically, the data in the input form is converted to JSON format and sent to the server as an HTTP request. Once this data transmission is complete, the terminal displays a successful transmission status to the user.

[0234] Step 3:

[0235] The server generates a prompt based on the received user information. This prompt is used as input to the generative AI model. A speech script is generated based on the generated prompt. The prompt is generated by embedding user data in a template, which is then sent to the API of the generative AI model (e.g., OpenAI GPT-4).

[0236] Step 4:

[0237] The server receives the speech transcript data returned by the generative AI model and processes it into an appropriate format (e.g., text file, JSON). At this time, it also performs diversity checks and translation processing on the transcript content to generate a multilingual speech transcript. The processed data is stored in the DB system.

[0238] Step 5:

[0239] The generated speech script is sent to the terminal, which displays the received speech script to the user and provides an interface for the user to edit it as needed, allowing the user to review the speech content and provide feedback.

[0240] Step 6:

[0241] After the user finishes the speech, they input their feedback. The device then sends the feedback data to the server. This feedback includes phrases that were well-received and points for improvement.

[0242] Step 7:

[0243] The server processes the received feedback and refines the generative AI model. This refinement step uses machine learning algorithms to highlight the most effective parts and incorporate them into the next speech generation, thereby improving the quality of the generated speech transcript.

[0244] Step 8:

[0245] The server generates the final speech manuscript as video and audio files. These files are then sent back to the device so that the user can distribute them via social media, email, etc. The device provides a distribution interface, allowing the user to share the content with other users.

[0246] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0247] The present invention relates to a speech support system that combines an emotion engine with a speech support system to generate a speech manuscript that reflects the user's emotional state, thereby supporting more effective speeches. A specific method for implementing the present invention will be described below with reference to the following embodiments.

[0248] System Overview

[0249] The system is mainly composed of three main components: the user, the server, and the terminal. An emotion engine is also added to analyze the user's emotional information and reflect it in the speech manuscript.

[0250] User operations

[0251] Enter information:

[0252] The user launches the "Asarekun" app and enters necessary information such as background (e.g., career, educational background), personality (e.g., hobbies, character), occupation (e.g., engineer, teacher), current weather (e.g., sunny, rainy), and region (e.g., New York, Tokyo). In addition, the user specifies the length and date and time of the speech, and submits this information within the app.

[0253] Server Operation

[0254] Generate a speech transcript:

[0255] The server receives the information sent by the user and temporarily stores it in a database. It then activates a generative AI model based on the received information to automatically generate a speech manuscript. This process also references past speeches and feedback information stored in the database.

[0256] Leveraging the Emotion Engine:

[0257] The emotion engine analyzes the user's emotional information and adjusts the content of the generated speech. For example, if the user is nervous, it can include a message to relax them.

[0258] Reflecting feedback:

[0259] The server receives feedback provided by users after they have used the speech. The feedback includes phrases that were well-received and parts that received poor responses. The server then applies this feedback to the generative AI model and emotion engine, and uses it when generating the next speech manuscript.

[0260] Translation and Diversity Checks:

[0261] If necessary, the server translates the speech manuscript into multiple languages ​​as specified. It also checks the translation to ensure it does not contain discriminatory language from the perspective of diversity and inclusion. This checking function provides speech manuscripts that can be used safely in global business situations.

[0262] Device behavior

[0263] Manuscript provided by:

[0264] The generated speech transcript is sent to the user's device and displayed at the specified date and time. The user can review the displayed speech transcript and make any necessary corrections. A QR code can be generated on the user's device for the translated speech transcript, which can be easily shared with other participants.

[0265] Specific examples

[0266] For example, consider a situation where a teacher at a school is giving a speech at the end of the semester. The teacher launches the "Asarekun" app and enters information such as their background (career as a teacher), personality (how they interact with students), occupation (school teacher), current weather (sunny), and regional characteristics (Japanese school culture). They also specify that the speech will last approximately 10 minutes.

[0267] The server generates a speech script based on this information. For example, it includes a phrase such as, "I am happy to be able to conclude this wonderful semester with you all today in this beautiful sunny weather," taking into account the weather. It also includes content that takes into account the local area and school culture.

[0268] Additionally, an emotion engine analyzes the teacher's emotions. For example, if the teacher is emotional, the speech will be tailored to include a warm message that reflects that emotion.

[0269] After the speech, the teacher can input feedback through the "Assembling-kun" app. For example, the teacher might say, "There was a part that the students particularly enjoyed." The server then applies this feedback to the generative AI model and emotion engine, providing more appropriate content for the next speech draft.

[0270] Furthermore, when giving a speech at an international conference, teachers can specify the translation and language of the manuscript, as well as the nationality and religious leanings of the audience. The server translates the speech into the specified language and checks whether the content complies with diversity and inclusion standards. The translation result is generated as a QR code, which teachers can easily share with other participants.

[0271] The above is a specific embodiment of the speech support system that combines the emotion engine of the "Mr. Asarai" system of the present invention. This system combines the functions of user input, speech generation, emotion analysis, feedback reflection, translation, and checking to achieve efficient, high-quality speech support.

[0272] The processing flow will be explained below.

[0273] Step 1:

[0274] The user launches the "Asare-kun" app.

[0275] When a user launches the "Asare-kun" app on their smartphone or computer, user authentication is performed and the user ID is sent to the server.

[0276] Step 2:

[0277] The user enters the information.

[0278] Users enter information such as their background (e.g., career, educational background), personality (e.g., hobbies, character), occupation (e.g., engineer, teacher), current weather (e.g., sunny, rainy), and location (e.g., New York, Tokyo) into the app's input form. They also specify the length and date and time of the speech.

[0279] Step 3:

[0280] The server receives the data.

[0281] The server receives the information sent by the user and temporarily stores it in a database, at which point the user ID and the information are linked.

[0282] Step 4:

[0283] The device activates the emotion engine.

[0284] The device activates an emotion engine to recognize the user's emotions, analyzing the user's facial expressions and tone of voice to obtain emotional information.

[0285] Step 5:

[0286] The server generates a speech script.

[0287] The server then activates a generative AI model based on the received information and the emotional information obtained from the emotion engine, and automatically generates a speech manuscript. This process also utilizes past speeches and feedback information stored in a database.

[0288] Step 6:

[0289] The server sends the manuscript to the user.

[0290] The generated speech manuscript is sent from the server to the user's terminal, where the user can view the speech manuscript on the terminal at the specified date and time.

[0291] Step 7:

[0292] The user reviews and corrects the speech.

[0293] The user can review the speech manuscript they receive and make corrections within the app if necessary, and the corrections will be resubmitted to the server.

[0294] Step 8:

[0295] The user gives a speech.

[0296] The user gives a speech based on the provided speech script, either using the generated speech script as is or modifying it as desired.

[0297] Step 9:

[0298] The device analyzes the emotions expressed by the user during speech.

[0299] During speech, the device uses an emotion engine to analyze the user's facial expressions and tone of voice and obtains the results in real time.

[0300] Step 10:

[0301] The user submits feedback.

[0302] After the speech, the user can enter feedback on the speech through the "Asarekun" app, detailing phrases that were particularly well-received and parts that received a poor response.

[0303] Step 11:

[0304] The server receives the feedback.

[0305] The server receives feedback from users and stores this feedback information in a database.

[0306] Step 12:

[0307] The server optimizes the generative AI model and emotion engine.

[0308] The server updates and optimizes the generation AI model and emotion engine based on the received feedback and emotion analysis results, so that the feedback content and emotion information are reflected in the next speech generation.

[0309] Step 13:

[0310] The user requests translation and review.

[0311] If necessary, users can specify the translation and language of the speech manuscript, as well as the nationality and religious leanings of the audience, within the app.

[0312] Step 14:

[0313] The server performs the translation and diversity check.

[0314] The server translates the speech manuscript into the specified language and checks the content from a diversity and inclusion perspective to ensure it does not contain discriminatory language.

[0315] Step 15:

[0316] The server sends the translation results.

[0317] The server sends the translated speech manuscript to the user's terminal, where the translation result is displayed for the user to check.

[0318] Step 16:

[0319] Users share translation manuscripts.

[0320] The translated speech manuscript is generated as a QR code on the device, which the user can share with other participants.

[0321] The above is a specific processing flow in an embodiment of the present invention in which the "Morning Assembly" system is combined with an emotion engine, allowing users to use efficient and high-quality speech manuscripts.

[0322] Example 2

[0323] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0324] Conventional speech support systems do not adequately consider the user's emotions or feedback and can only provide uniform content, making it difficult to provide effective speech manuscripts that meet the user's needs.In addition, they do not perform multilingual translation or diversity checks, which limits their use in international settings.

[0325] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0326] In this invention, the server includes means for receiving information from a user regarding their background, personality, occupation, weather, and regional characteristics, means for generating a speech manuscript using a generative AI model based on the received information, means for analyzing the user's emotional information with an emotion engine for the generated speech manuscript and adjusting the manuscript content, means for receiving feedback from the user and optimizing the generation means, means for performing translation and diversity checks, and means for delivering speech manuscripts translated into multiple languages ​​to users via QR codes. This makes it possible to provide effective speech manuscripts that reflect the user's emotions and feedback, and the multilingual support and diversity checks realize a speech support system that can be used in international settings.

[0327] A "user" is a person or organization that provides the information necessary to generate a speech transcript using the system.

[0328] "History" refers to all information about a user's career and background.

[0329] "Personality" refers to personal characteristics such as a user's character and hobbies.

[0330] "Occupation" refers to information about the user's job or area of ​​expertise.

[0331] "Weather" refers to the weather conditions on the day the speech is given and in the area.

[0332] "Regionality" refers to information about the location and cultural background of the user's speech.

[0333] A "generative AI model" refers to an artificial intelligence model that automatically creates a speech manuscript based on information entered by the user.

[0334] An "emotion engine" refers to software or a system that analyzes a user's emotions and reflects the results in a speech manuscript.

[0335] "Feedback" refers to information based on evaluations and reactions provided by users after they give a speech.

[0336] A "diversity check" refers to the process of checking whether a speech manuscript contains discriminatory language.

[0337] "QR code" refers to a two-dimensional barcode that allows you to easily share your speech manuscript with other users.

[0338] The present invention relates to a system for supporting more effective speeches by combining a speech support system with an emotion engine to generate a speech manuscript that reflects the user's emotional state. A specific method for implementing the present invention will be described below.

[0339] The system is mainly composed of three main components: the user, the server, and the terminal. An emotion engine is also added to analyze the user's emotional information and reflect it in the speech manuscript.

[0340] User operations

[0341] The user launches the "Asarekun" app and enters information about themselves through the interface, such as their background (career history), personality (characteristics and hobbies), occupation (job and specialty), current weather, and locality. They also specify the length and date and time of the speech, and submit this information.

[0342] Server Operation

[0343] 1. Receiving and storing information:

[0344] The server receives the information sent by the user and temporarily stores it in a database. The specific database used is a relational database management system (RDBMS), such as MySQL (registered trademark) or PostgreSQL.

[0345] 2. Generate speech transcript:

[0346] Based on the received information, the server runs a generative AI model, such as OpenAI's GPT-4, to automatically generate a speech script. During this process, the following prompt is used:

[0347] Generate your speech. Consider the following information:

[0348] Biography: Teaching career

[0349] Personality: How you interact with students

[0350] Occupation: School teacher

[0351] Weather: Sunny

[0352] Regional Characteristics: Japanese School Culture

[0353] Speech length: 10 minutes

[0354] 3. Leveraging the Emotion Engine:

[0355] The generated speech script is then adjusted based on the user's emotional information using an emotion engine, such as IBM Watson's Tone Analyzer. For example, if the teacher is emotional, a warm message reflecting that emotion can be added.

[0356] 4. Receiving and storing feedback:

[0357] The system receives feedback provided by users after they have used the speech and stores it in a database. The feedback information is reflected in the next speech generation to further improve the quality of the speech manuscript.

[0358] 5. Translation and Diversity Check:

[0359] If necessary, the server translates the speech manuscript into multiple languages ​​as specified. The translation software used may be Google Translate. It also incorporates an algorithm to check whether the translation is acceptable from a diversity and inclusion perspective.

[0360] Device behavior

[0361] 1. Submission of manuscript:

[0362] The generated speech manuscript is sent to the user's terminal and displayed at the specified date and time. The user can check the displayed speech manuscript and make corrections as necessary.

[0363] 2. Generate and share your QR code:

[0364] A QR code is generated on the user's device for the translated speech manuscript, which can be easily shared with other participants. This QR code can be generated using a general QR code generation library.

[0365] Specific examples

[0366] For example, consider a situation where a teacher at a school is giving a speech at the end of the semester. The teacher launches the "Mr. Asareikun" app and enters information such as "Career as a teacher," "Interaction with students," "Occupation: School teacher," "Weather: Sunny," and "Region: Japanese school culture." The teacher also specifies that the speech will last approximately 10 minutes.

[0367] The server uses this information to generate a speech using a generative AI model, including phrases like, "I'm happy to be able to conclude this wonderful semester with you all today in this beautiful, sunny weather," and adds a warm, moving message using an emotion engine.

[0368] After the speech is completed, the teacher can send feedback through the app, such as, "There was a part that the students particularly enjoyed." This feedback information is stored in the database and reflected the next time the speech is generated.

[0369] When giving a speech at an international conference, teachers can specify the translation language of the manuscript and the nationality and religious orientation of the audience. The server translates the speech into the specified language and checks whether the content is appropriate from a diversity and inclusion perspective. The translation result is generated as a QR code that can be shared with other participants.

[0370] The speech support system of the present invention realizes efficient and high-quality speech support by linking the functions of user input, speech generation, emotion analysis, feedback reflection, translation, and checking.

[0371] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0372] Step 1:

[0373] Users launch the Asarekun app and enter information about their background, personality, occupation, weather, and locale through the interface.

[0374] Input: History, personality, occupation, weather, region, length and date of speech

[0375] Output: Sending input information

[0376] Specifically, the user enters each item into the app's form and presses the submit button to send the information to the server.

[0377] Step 2:

[0378] The server receives the information sent by the user and temporarily stores it in a database, such as a relational database management system like MySQL or PostgreSQL.

[0379] Input: Information submitted by the user

[0380] Output: Save to database

[0381] Specifically, the server receives the HTTP request, analyzes the information in the request body, and inserts it into a database.

[0382] Step 3:

[0383] Based on the information received, the server launches a generative AI model (such as GPT-4) to automatically generate a speech manuscript.

[0384] Input: User information stored in the database

[0385] Output: Generated speech transcript

[0386] Specifically, the following prompt sentence is input to the generative AI model:

[0387] Generate your speech. Consider the following information:

[0388] Biography: Teaching career

[0389] Personality: How you interact with students

[0390] Occupation: School teacher

[0391] Weather: Sunny

[0392] Regional Characteristics: Japanese School Culture

[0393] Speech length: 10 minutes

[0394] A generative AI model generates a speech script based on this prompt.

[0395] Step 4:

[0396] The server uses an emotion engine (such as IBM Watson's Tone Analyzer) to analyze the user's emotional information and adjust the content of the generated speech manuscript.

[0397] Input: Generated speech script, user's emotional information

[0398] Output: Adjusted speech transcript

[0399] Specifically, the emotion engine analyzes the emotional tone of the speech manuscript and makes adjustments based on the user's emotions (e.g., adding a warm message).

[0400] Step 5:

[0401] The server receives the feedback provided by the user after they give their speech and stores it in a database.

[0402] Input: User feedback

[0403] Output: Save feedback information to a database

[0404] Specifically, the information entered by the user into the feedback form through the app is sent to the server, which then inserts the information into a database.

[0405] Step 6:

[0406] If necessary, the server translates the speech manuscript into multiple languages ​​and performs a diversity check.

[0407] Input: Speech manuscript, specified language

[0408] Output: Translated speech manuscript, diversity checked content

[0409] Specifically, the server translates the speech manuscript using the Google Translate API or similar, and then applies an algorithm to check the content from a diversity and inclusion perspective.

[0410] Step 7:

[0411] The generated speech manuscript is sent to the user's device and displayed at the specified date and time.

[0412] Input: Adjusted and translated speech transcript

[0413] Output: Delivery to user terminal

[0414] Specifically, the server sends the speech manuscript to the user's terminal via an HTTP request or WebSocket, and configures the terminal to display it at the appropriate time.

[0415] Step 8:

[0416] A QR code is generated on the user's device for the translated speech manuscript, allowing it to be easily shared with other participants.

[0417] Input: Translated speech

[0418] Output: QR code

[0419] Specifically, the translated speech manuscript is encoded on the device using a QR code generation library (e.g., qrcode.js) to generate a QR code.

[0420] (Application example 2)

[0421] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0422] Conventional speech support systems simply generate speech scripts, but do not provide personalized support that reflects the user's emotional state. This makes it difficult to provide appropriate sales pitches in real time based on the user's emotions and the other person's reactions. In particular, in sales situations in physical stores, appropriate dialogue that reflects the customer's emotions is required, but conventional technology has had difficulty meeting this requirement.

[0423] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for receiving information on the user's history, personality, occupation, weather, and locality from the user; data processing means for providing the generated speech manuscript to the user; means for receiving feedback from the user and optimizing the generation means; means for reflecting the user's emotional information in the speech manuscript generated by the data processing means; means for providing the generated speech manuscript to the user in real time to support the user during the conversation; and means for performing translation and diversity checks. This makes it possible to generate a speech manuscript that takes the user's emotional state and feedback into consideration, thereby improving the quality of conversations in physical stores.

[0424] "History" is information about the user's career history, educational background, and past experiences.

[0425] "Personality" is information about a user's hobbies, character, and personal characteristics.

[0426] "Occupation" is information about the work or job the user is engaged in.

[0427] "Weather" is information about the weather conditions at the user's current location.

[0428] "Regionality" is information about the culture and customs specific to the area where the user lives or where the speech is given.

[0429] "Data processing means" refers to a processing device or program for generating a speech manuscript based on input information.

[0430] "Generation means" refers to an algorithm or system that creates a speech manuscript based on input information from the user.

[0431] "Means for reflecting emotional information" refers to a processing method or device for analyzing the user's emotional state and reflecting the results in the speech manuscript.

[0432] "Means for providing in real time" refers to a device or program that instantly displays the generated speech manuscript on the user's terminal and provides support during the conversation.

[0433] "Translation means" refers to a process or device for translating a generated speech transcript into a different language.

[0434] "Diversity check means" refers to a method or device for checking whether a generated speech manuscript is appropriate for people of diverse cultures and backgrounds.

[0435] A specific embodiment of the present invention will be described. This invention is realized by a speech support system that combines an emotion engine. The system is composed of a user, a server, and a terminal, and by adding an emotion engine, the system analyzes the user's emotion information and reflects it in the speech manuscript.

[0436] The server has a means to receive information provided by users about their background, personality, occupation, weather, and locality. The user inputs this information into a dedicated smartphone application. For example, when a salesperson introduces a new product on the shop floor, they input information about their background, hobbies, occupation, current weather, and local culture.

[0437] After receiving the information, the server uses data processing means to generate a speech manuscript based on the data. In this case, a generative AI model (e.g., GPT-4) is used to automatically generate the optimal speech manuscript, taking into account past data and feedback. The generated manuscript will initially be a speech with basic information.

[0438] The emotion engine then analyzes the speech and adapts it to reflect the user's emotional information. For example, if a salesperson is nervous, the system will adjust the speech script to include a message that will help them relax. The system uses the Affectiva SDK and Google Cloud Natural Language API to perform emotion analysis of speech and text.

[0439] The generated speech script is provided to the user's device (smartphone or tablet) in real time. This allows the salesperson to always present the most appropriate sales pitch while interacting with the customer. Below is an example of a prompt:

[0440] Is this product really right for me?

[0441] A feedback mechanism is also provided, allowing users to input their reactions to the speech into the application. For example, they can provide feedback such as, "This sales pitch was very effective." The server uses this feedback to optimize the generative AI model and emotion engine and reflect it in the next speech generation.

[0442] Additionally, the system also provides translation and diversity checks, which allow speeches to be translated into different languages ​​and checked for cultural and religious appropriateness. For example, if a speech is presented to an international audience, it can be translated and checked for discriminatory language.

[0443] By using this system, users can receive optimal conversation support in real time, improving the quality of conversations in physical stores.

[0444] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0445] Step 1:

[0446] The user launches a dedicated smartphone application and inputs information about their background, personality, occupation, weather, and locality. Based on this, initial customer information data is generated. This information is then sent to the server.

[0447] Input: history, personality, occupation, weather, region

[0448] Output: Initial customer information data

[0449] Specifically, the user enters the necessary information into the input form displayed on the application screen and presses the send button.

[0450] Step 2:

[0451] The server processes the initial customer information data received from the user and stores it in a database, which also stores past speech transcripts and feedback information.

[0452] Input: Initial customer information data

[0453] Output: Entries saved in the database

[0454] Specifically, the server analyzes the received data and stores it in a database in an appropriate format.

[0455] Step 3:

[0456] The server then runs a generative AI model (e.g., GPT-4) based on the stored data to automatically generate a speech manuscript, referencing past speeches and feedback information stored in a database during this process.

[0457] Input: Customer information data and historical data extracted from the database

[0458] Output: Initial speech manuscript

[0459] Specifically, the server inputs a prompt to the AI ​​model and retrieves the generated speech. For example, the prompt might be in the following format:

[0460] "Is this product really right for me?"

[0461] Step 4:

[0462] The server uses an emotion engine (e.g., Affectiva SDK or Google Cloud Natural Language API) to adapt the initial speech transcript to reflect the user's emotional information, extracted from additional information entered by the user and real-time speech analysis.

[0463] Input: Initial speech script and user's emotional information

[0464] Output: Emotionally-reflected speech transcript

[0465] Specifically, the emotion engine analyzes the user's voice data, extracts emotional elements (e.g., tension, excitement, relaxation), and reflects them in the content of the speech manuscript.

[0466] Step 5:

[0467] The speech manuscript reflecting the generated emotions is sent to the user's terminal in real time, and the user can use this speech manuscript to have a conversation with the customer.

[0468] Input: Speech manuscript reflecting emotions

[0469] Output: Speech manuscript displayed on the user's device

[0470] Specifically, the server sends the generated speech manuscript to the user's smartphone and displays it on the application screen.

[0471] Step 6:

[0472] After finishing their speech, users can input feedback through the application, which the server receives and uses to optimize the generative AI model and emotion engine when generating the next speech manuscript.

[0473] Input: User feedback

[0474] Output: Feedback information added to the database

[0475] Specifically, the user enters their thoughts and suggestions for improvement into the application's feedback form and presses the submit button. The server receives this and adds it to the database.

[0476] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0477] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0478] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0479] [Second embodiment]

[0480] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0481] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0482] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0483] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0484] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0485] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0486] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0487] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0488] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0489] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0490] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0491] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0492] The present invention relates to a system for automating speechwriting functions and generating efficient, high-quality speech manuscripts. A specific method for implementing the present invention will be described below with reference to the following embodiments.

[0493] System Overview

[0494] The system is mainly composed of three elements: the user, the server, and the terminal. Specifically, the server generates a speech manuscript based on the information provided by the user and provides it to the user via the terminal.

[0495] User operations

[0496] Enter information:

[0497] The user launches the "Asarekun" app and enters necessary information such as background (e.g., career, educational background), personality (e.g., hobbies, character), occupation (e.g., engineer, teacher), current weather (e.g., sunny, rainy), and region (e.g., New York, Tokyo). In addition, the user specifies the length and date and time of the speech, and submits this information within the app.

[0498] Server Operation

[0499] Generate a speech transcript:

[0500] The server receives the information sent by the user. Based on the received information, it uses a generative AI model to generate a speech script. During this process, it also references past speech scripts and feedback information stored in a database to select the optimal content.

[0501] Reflecting feedback:

[0502] The server receives feedback provided by users after they have used the speech. The feedback includes phrases that were well-received and parts that received poor responses. The server then incorporates this feedback into the generative AI model and uses it when generating the next speech manuscript.

[0503] Translation and Diversity Checks:

[0504] If necessary, the server translates the speech manuscript into multiple languages ​​as specified. It also checks the translation to ensure it does not contain discriminatory language from the perspective of diversity and inclusion. This checking function provides speech manuscripts that can be used safely in global business situations.

[0505] Device behavior

[0506] Manuscript provided by:

[0507] The generated speech transcript is sent to the user's device and displayed at the specified date and time. The user can review the displayed speech transcript and edit it as needed. A QR code can be generated on the user's device for the translated speech transcript, which can be easily shared with other participants.

[0508] Specific examples

[0509] For example, consider a company manager preparing a morning assembly speech for employees. The manager launches the "Asa-Kun" app and enters information such as his or her background (career as a manager), personality (emphasis on leadership and teamwork), occupation (manager at an IT company), current weather (sunny in Tokyo), and regional characteristics (Japanese culture). The manager also specifies that the speech will last approximately five minutes.

[0510] Based on this information, the server uses a generative AI model to create an optimal speech script. For example, if the current weather is sunny, the script might include phrases such as, "I'm happy to be able to start a wonderful day with you all in this sunny weather today." Regional differences are also taken into consideration, and seasonal greetings and content that touches on unique Japanese customs are included.

[0511] After the speech, the manager can submit feedback through the "Asarekun" app. For example, the manager might say, "The part introducing the new project was particularly well-received." The server then reflects this feedback and adjusts the AI ​​model to provide more appropriate content the next time the speech is generated.

[0512] Furthermore, when giving a speech at an international conference, a manager can request that their speech manuscript be translated into English, Chinese, or other languages. The server translates the speech into the specified language and checks whether the content meets the requirements from a diversity and inclusion perspective. The translation result is generated as a QR code, which the manager can easily share with other participants.

[0513] The above is a specific example of speech support using the "Asare-kun" system. This system combines the functions of user input, speech generation, feedback reflection, translation, and checking to provide efficient, high-quality speech support.

[0514] The processing flow will be explained below.

[0515] Step 1: The user launches the "Asarekun" app.

[0516] When a user launches the "Asare-kun" app on their smartphone or computer, user authentication is performed and the user ID is sent to the server.

[0517] Step 2: The user enters information.

[0518] Users fill out a form in the app with information about their background, personality, occupation, current weather, and location, as well as the length and date and time of the speech.

[0519] Step 3: The server receives the data.

[0520] The server receives the information sent by the user and temporarily stores it in a database, at which point the user ID and the information are linked.

[0521] Step 4: The server generates the speech script.

[0522] The server then activates a generative AI model based on the received information to automatically generate a speech manuscript, using past speeches and feedback information stored in a database.

[0523] Step 5: The server sends the manuscript to the user.

[0524] The generated speech manuscript is sent from the server to the user's terminal, where the user can view the speech manuscript on the terminal at the specified date and time.

[0525] Step 6: The user reviews and corrects the speech.

[0526] The user can review the speech manuscript they receive and make corrections within the app if necessary, and the corrections will be resubmitted to the server.

[0527] Step 7: The user gives a speech.

[0528] The user gives a speech based on the provided speech script, either using the generated speech script as is or modifying it as desired.

[0529] Step 8: User submits feedback.

[0530] After the speech, the user can enter feedback on the speech through the "Asarekun" app, detailing phrases that were particularly well-received and parts that received a poor response.

[0531] Step 9: The server receives the feedback.

[0532] The server receives feedback from users and stores this feedback information in a database.

[0533] Step 10: The server optimizes the generated AI model.

[0534] The server updates and optimizes the generative AI model based on the received feedback, so that the feedback is reflected in the next speech generation.

[0535] Step 11: User requests translation and checking.

[0536] If necessary, users can specify the translation and language of the speech manuscript, as well as the nationality and religious leanings of the audience, within the app.

[0537] Step 12: The server performs the translation and diversity check.

[0538] The server translates the speech manuscript into the specified language and checks the content from a diversity and inclusion perspective to ensure it does not contain discriminatory language.

[0539] Step 13: The server sends the translation result.

[0540] The server sends the translated speech manuscript to the user's terminal, where the translation result is displayed for the user to check.

[0541] Step 14: The user shares the translation manuscript.

[0542] The translated speech manuscript is generated as a QR code on the device, which the user can share with other participants.

[0543] The above is a specific flow of processing in the "Mr. Asakai" system of the present invention, which allows users to use efficient and high-quality speech manuscripts.

[0544] Example 1

[0545] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0546] Conventional speech support systems have the drawback of being time-consuming and labor-intensive, as the creation of speech manuscripts, translation, and feedback integration are all done manually. Furthermore, they lacked checking functions to accommodate different languages ​​and cultures, making them difficult to use in global business settings.

[0547] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0548] In this invention, the server includes means for receiving information on user history, personality, occupation, weather, and locality from the user, means for generating prompts using a generative AI model and creating a speech manuscript, data processing means for providing the generated speech manuscript to the user, means for receiving feedback from the user and optimizing the generation means, means for translating the speech manuscript into multiple languages ​​and performing a diversity check, and means for generating the translated speech manuscript as a QR code. This automates the process from generating the speech manuscript to providing it, incorporating feedback, and translating and checking it, enabling the efficient provision of high-quality speech manuscripts.

[0549] "User" refers to an individual or organization that intends to use the system to generate a speech manuscript.

[0550] "History" refers to information indicating a user's past experiences, such as career history, educational background, and work history.

[0551] "Personality" refers to information that indicates personal characteristics such as a user's hobbies and character.

[0552] "Occupation" refers to information indicating the job or specialty in which the user is engaged.

[0553] "Weather" refers to information indicating the weather conditions in the area where the speech will be given on the day.

[0554] "Regionality" refers to information that indicates the characteristics of the culture, customs, etc. of the place where the speech is given.

[0555] "Generative AI model" refers to an artificial intelligence model that automatically generates a speech manuscript based on information provided by the user.

[0556] A "prompt sentence" is an input sentence given to a generative AI model, and refers to text that guides the generation of a speech manuscript.

[0557] "Data processing means" refers to an information processing system for providing the generated speech manuscript to the user.

[0558] "Feedback" refers to information indicating reactions such as evaluations and impressions provided by users after they give a speech.

[0559] "Optimization methods" refer to techniques for receiving user feedback and improving the performance of generative AI models.

[0560] "Translation means" refers to a method or system for converting a speech transcript into a different language.

[0561] "Diversity check measures" refer to methods used to check whether translated speech manuscripts contain discriminatory language.

[0562] A "QR code" is a type of two-dimensional code, a graphic used to easily share speech manuscripts.

[0563] The present invention relates to a system for automating speechwriting functions and efficiently generating high-quality speech manuscripts. A specific method for implementing the present invention will be described below with reference to the following embodiments.

[0564] System Overview

[0565] This system is mainly composed of three elements: the user, the server, and the terminal. The user provides information, and the server generates a speech manuscript and provides it to the user through the terminal.

[0566] User operations

[0567] Enter information:

[0568] The user launches the "Speech Assistant" app and inputs necessary information such as background (e.g., career, educational background), personality (e.g., hobbies, character), occupation (e.g., engineer, teacher), current weather (e.g., sunny, rainy), and region (e.g., New York, Tokyo). In addition, the user specifies the length and date and time of the speech, and submits this information within the app.

[0569] Server Operation

[0570] Generate a speech transcript:

[0571] The server receives the information sent by the user. Based on the received information, it uses a generative AI model to generate a speech script. During this process, it also references past speech scripts and feedback information stored in a database to select the optimal content.

[0572] Examples:

[0573] For example, consider a company manager preparing a morning assembly speech for employees. The manager launches the "Speech Assistant" app and inputs information such as his or her background (career as a manager), personality (emphasis on leadership and teamwork), occupation (manager at an IT company), current weather (sunny in Tokyo), and regional characteristics (Japanese culture). The manager also specifies that the speech will last approximately five minutes.

[0574] Based on this information, the server uses a generative AI model to create an optimal speech script. For example, if the current weather is sunny, the script might include phrases such as, "I'm happy to be able to start a wonderful day with you all in this sunny weather today." Regional differences are also taken into consideration, and seasonal greetings and content that touches on unique Japanese customs are included.

[0575] Example prompt sentence:

[0576] "His background is 10 years in the IT industry, the weather is fine, and the speech is 5 minutes long."

[0577] Reflecting feedback:

[0578] The server receives feedback provided by users after they have used the speech. This feedback includes phrases that were well-received and parts that received poor responses. The server then incorporates this feedback into the generative AI model and uses it when generating the next speech manuscript.

[0579] Examples:

[0580] After the speech, the manager can provide feedback through the Speech Assistant app, such as, "The part introducing the new project was particularly well-received." The server then reflects this feedback and adjusts the AI ​​model to provide more appropriate content the next time the speech is generated.

[0581] Translation and Diversity Checks:

[0582] If necessary, the server translates the speech manuscript into multiple languages ​​as specified, and also checks the translation from the perspective of diversity and inclusion to ensure it does not contain discriminatory language.

[0583] Examples:

[0584] When giving a speech at an international conference, a manager can request that their speech manuscript be translated into English, Chinese, or other languages. The server translates the speech into the specified language and checks whether the content is appropriate from a diversity and inclusion perspective. The translation result is generated as a QR code, which the manager can easily share with other participants.

[0585] Device behavior

[0586] Manuscript provided by:

[0587] The generated speech transcript is sent to the user's device and displayed at the specified date and time. The user can review the displayed speech transcript and edit it as necessary. A QR code can be generated on the user's device for the translated speech transcript, allowing it to be easily shared with other participants.

[0588] Examples:

[0589] Users can check the generated speech manuscript through the "Speech Assistant" app and edit it as necessary. For example, when giving a speech at an international conference, the translated speech manuscript can be generated as a QR code and distributed to other participants, enabling smooth communication.

[0590] The above is a concrete example of how the present invention can be implemented. This system allows users to create high-quality speech manuscripts in a short amount of time, and can be used safely in global business situations.

[0591] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0592] Step 1:

[0593] Input and Action:

[0594] The user launches the "Speech Assistant" app and enters the necessary information, such as background, personality, occupation, weather, region, speech length, date and time, etc. The information is sent within the app by pressing the send button.

[0595] Data processing and output:

[0596] The information entered by the user is sent to the server in JSON format. The server receives this JSON data, analyzes the information stored in each field, and saves it as basic data for generating a speech manuscript.

[0597] Step 2:

[0598] Input and Action:

[0599] The server receives the JSON-formatted information sent by the user and parses it to extract information such as history, personality, occupation, weather, region, speech length, and date and time.

[0600] Data processing and output:

[0601] Based on the extracted information, a prompt sentence is generated for the generative AI model. The prompt sentence is input to the generative AI model in the form of "history is 10 years of experience in the IT industry, the weather is sunny, and the speech length is 5 minutes."

[0602] Step 3:

[0603] Input and Action:

[0604] The server inputs the generated prompt sentences into the generative AI model, which analyzes the prompt sentences and automatically generates a speech script.

[0605] Data processing and output:

[0606] The generative AI model also references past speech scripts and feedback information stored in a database to generate the optimal speech script, which is then temporarily stored on a server.

[0607] Step 4:

[0608] Input and Action:

[0609] When submitting feedback, the user can enter and submit their feedback after the speech through the "Speech Assistant" app. For example, they could enter feedback such as "The introduction to the new project was particularly well-received."

[0610] Data processing and output:

[0611] The server analyzes the feedback received from the user and reflects it in the generative AI model. The generative AI model is then readjusted based on the feedback information and used for generating the next speech manuscript.

[0612] Step 5:

[0613] Input and Action:

[0614] If the user requests multilingual translation, the server translates the speech into the specified language, for example, if it needs to be translated into English or Chinese.

[0615] Data processing and output:

[0616] The server translates the generated speech manuscript into the specified language, checks it from a diversity and inclusion perspective, verifies that the content is not discriminatory, and saves the translation results.

[0617] Step 6:

[0618] Input and Action:

[0619] Once the translation and checking process is complete on the server, the speech manuscript is sent to the user's device, where the user can check the displayed speech manuscript through the "Speech Assistant" app.

[0620] Data processing and output:

[0621] The speech manuscript displayed on the user's device can be viewed and edited by the user. In addition, a QR code can be generated on the user's device for the translated speech manuscript, allowing it to be easily shared with other participants.

[0622] The above is a specific flow of program processing for the system based on the claims. This system allows users to create high-quality speech manuscripts in a short amount of time, and to obtain deliverables that are efficient and can be used in multiple languages.

[0623] (Application example 1)

[0624] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0625] In today's world, opportunities for speeches and presentations are increasing, creating a demand for systems that can improve the quality and efficiency of speech drafts. Furthermore, with the advancement of globalization, multilingual support and cultural sensitivity are required, as well as a means for easily distributing generated speech drafts. However, existing systems lack the ability to provide individualized support based on the user's history and personality, and optimizing the system to reflect feedback is difficult. The objective of this invention is to solve these problems and provide high-quality speech support.

[0626] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0627] In this invention, the server includes means for receiving information from the user regarding background, personality, occupation, weather, and locality, data processing means for providing the generated speech transcript to the user, means for receiving user feedback and optimizing the generating means, means for performing translation and diversity checks, means for generating content translated into multiple languages ​​and taking social and cultural considerations into account, and means for outputting and distributing the generated speech transcript as a video or audio file, thereby enabling the generation of high-quality speech transcripts based on the user's specific information and the provision of content that is multilingual and culturally sensitive.

[0628] "User" refers to a person or organization that uses the system to provide information for generating a speech manuscript.

[0629] "Provenance" refers to information about a user's past experience, work and academic history, and other career information.

[0630] "Personality" refers to information about a user's personality, interests, hobbies, and personal characteristics.

[0631] "Occupation" refers to information about the job or work the user is currently doing.

[0632] "Weather" refers to information about the local weather conditions when the user delivers a speech.

[0633] "Regionality" refers to information about the cultural and social characteristics of the region where the user gives speech.

[0634] "Data processing means" refers to a computer processing device or software that generates a speech manuscript based on information provided by the user and provides it to the user.

[0635] A "generative AI model" is a model for generating speech manuscripts using artificial intelligence, and refers to an algorithm that generates optimal content based on past data and user-specified information.

[0636] "Feedback" refers to information such as reactions, evaluations, and areas for improvement provided by a user after they give a speech.

[0637] "Optimization means" refers to methods and algorithms for receiving feedback information and reflecting that information in the next speech script generation to improve the generative AI model.

[0638] "Translation and diversity check" refers to the process of converting the generated speech manuscript into multiple languages ​​and verifying that the content does not contain discriminatory language.

[0639] "Content" refers to the generated speech manuscript and multimedia information such as video and audio files created based on it.

[0640] A "video or audio file" is an audiovisual material created based on the generated speech manuscript, and is in a format for providing the speech content to users or viewers.

[0641] The present invention is a system for automatically generating a speech manuscript based on user information and providing the speech in various formats. The invention is specifically embodied in the following embodiments.

[0642] System Configuration

[0643] The system mainly consists of users, servers, and terminals.

[0644] User operations

[0645] The user starts the application using the smartphone and enters the following information:

[0646] Background (career, academic history, etc.)

[0647] Personality (hobbies, character, etc.)

[0648] Occupation (engineer, teacher, etc.)

[0649] Current weather (sunny, rainy, etc.)

[0650] Regional characteristics (region, culture, etc.)

[0651] Length and date of speech

[0652] This information is submitted by the user within the application.

[0653] Server Operation

[0654] The server has the following functions:

[0655] 1. Information reception: Receives information sent by the user.

[0656] 2. Speech generation: Based on the received information, a speech script is generated using a generative AI model (e.g., OpenAI GPT-4). During this process, past speech data and feedback information are referenced to select the optimal content.

[0657] 3. Feedback reflection: The system receives feedback provided by the user after the speech and reflects that information in the generative AI model when generating the next speech manuscript.

[0658] 4. Translation and Diversity Check: Speech manuscripts will be translated into multiple languages ​​as needed and checked from a diversity and inclusion perspective to ensure they do not contain discriminatory language.

[0659] 5. Content generation: Based on the generated speech manuscript, video and audio files are created and made available for distribution by users.

[0660] Device behavior

[0661] The generated speech manuscript is sent to the user's device and can be viewed within the application. The user can edit it and check the final speech content. Furthermore, the generated content (video and audio files) can be easily distributed via social media, email, etc.

[0662] Program processing

[0663] The server-side program is built using Node.js and Express and handles all processes from receiving user information, generating speech scripts, incorporating feedback, translating and checking, and generating content.

[0664] Processing of received user information: The information is converted into an appropriate format and input into the generative AI model.

[0665] Speech generation: The generative AI model generates a speech based on the provided information. OpenAI's GPT-4 is used as the generative model.

[0666] Feedback Processing: After each speech, user feedback is received. This feedback is used in the next generation to improve the accuracy and adaptability of the model.

[0667] Translation and diversity check: The generated speech transcripts are translated into multiple languages ​​and checked for discriminatory content, making them available for global use.

[0668] Content generation: Video and audio files are generated based on the speech manuscript and made available to users.

[0669] Specific examples

[0670] For example, if a user provides the following information on the topic of "wedding speeches," the following prompt sentences will be used for the generative AI model:

[0671] Prompt statement:

[0672] "Generate speech under the following conditions:

[0673] Experience: Over 10 years of experience in the IT industry

[0674] Personality: Team player

[0675] Occupation: Systems Engineer

[0676] Weather: Sunny

[0677] Region: Tokyo

[0678] Length: 5 minutes

[0679] Please prepare the speech."

[0680] The generative AI model uses this information to generate a speech transcript and provides it to the user, who can then edit it and distribute it as a video or audio file.

[0681] The above is a specific embodiment of the present invention, which allows users to efficiently generate and provide highly personalized speech manuscripts.

[0682] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0683] Step 1:

[0684] The user starts the application on their smartphone and inputs information about their background, personality, occupation, weather, and locality. They also specify the length of the speech and the date and time. This information becomes the input data.

[0685] Step 2:

[0686] The terminal receives the information entered by the user and sends it to the server. Specifically, the data in the input form is converted to JSON format and sent to the server as an HTTP request. Once this data transmission is complete, the terminal displays a successful transmission status to the user.

[0687] Step 3:

[0688] The server generates a prompt based on the received user information. This prompt is used as input to the generative AI model. A speech script is generated based on the generated prompt. The prompt is generated by embedding user data in a template, which is then sent to the API of the generative AI model (e.g., OpenAI GPT-4).

[0689] Step 4:

[0690] The server receives the speech transcript data returned by the generative AI model and processes it into an appropriate format (e.g., text file, JSON). At this time, it also performs diversity checks and translation processing on the transcript content to generate a multilingual speech transcript. The processed data is stored in the DB system.

[0691] Step 5:

[0692] The generated speech script is sent to the terminal, which displays the received speech script to the user and provides an interface for the user to edit it as needed, allowing the user to review the speech content and provide feedback.

[0693] Step 6:

[0694] After the user finishes the speech, they input their feedback. The device then sends the feedback data to the server. This feedback includes phrases that were well-received and points for improvement.

[0695] Step 7:

[0696] The server processes the received feedback and refines the generative AI model. This refinement step uses machine learning algorithms to highlight the most effective parts and incorporate them into the next speech generation, thereby improving the quality of the generated speech transcript.

[0697] Step 8:

[0698] The server generates the final speech manuscript as video and audio files. These files are then sent back to the device so that the user can distribute them via social media, email, etc. The device provides a distribution interface, allowing the user to share the content with other users.

[0699] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0700] The present invention relates to a system for supporting more effective speeches by combining a speech support system with an emotion engine to generate a speech manuscript that reflects the emotional state of a user. A specific method for implementing the present invention will be described below with reference to the following embodiments.

[0701] System Overview

[0702] The system is mainly composed of three main components: the user, the server, and the terminal. An emotion engine is also added to analyze the user's emotional information and reflect it in the speech manuscript.

[0703] User operations

[0704] Enter information:

[0705] The user launches the "Asarekun" app and enters necessary information such as background (e.g., career, educational background), personality (e.g., hobbies, character), occupation (e.g., engineer, teacher), current weather (e.g., sunny, rainy), and region (e.g., New York, Tokyo). In addition, the user specifies the length and date and time of the speech, and submits this information within the app.

[0706] Server Operation

[0707] Generate a speech transcript:

[0708] The server receives the information sent by the user and temporarily stores it in a database. It then activates a generative AI model based on the received information to automatically generate a speech manuscript. This process also references past speeches and feedback information stored in the database.

[0709] Leveraging the Emotion Engine:

[0710] The emotion engine analyzes the user's emotional information and adjusts the content of the generated speech. For example, if the user is nervous, it can include a message to relax them.

[0711] Reflecting feedback:

[0712] The server receives feedback provided by users after they have used the speech. The feedback includes phrases that were well-received and parts that received poor responses. The server then applies this feedback to the generative AI model and emotion engine, and uses it when generating the next speech manuscript.

[0713] Translation and Diversity Checks:

[0714] If necessary, the server translates the speech manuscript into multiple languages ​​as specified. It also checks the translation to ensure it does not contain discriminatory language from the perspective of diversity and inclusion. This checking function provides speech manuscripts that can be used safely in global business situations.

[0715] Device behavior

[0716] Manuscript provided by:

[0717] The generated speech transcript is sent to the user's device and displayed at the specified date and time. The user can review the displayed speech transcript and make any necessary corrections. A QR code can be generated on the user's device for the translated speech transcript, which can be easily shared with other participants.

[0718] Specific examples

[0719] For example, consider a situation where a teacher at a school is giving a speech at the end of the semester. The teacher launches the "Asarekun" app and enters information such as their background (career as a teacher), personality (how they interact with students), occupation (school teacher), current weather (sunny), and regional characteristics (Japanese school culture). They also specify that the speech will last approximately 10 minutes.

[0720] The server generates a speech script based on this information. For example, it includes a phrase such as, "I am happy to be able to conclude this wonderful semester with you all today in this beautiful sunny weather," taking into account the weather. It also includes content that takes into account the local area and school culture.

[0721] Additionally, an emotion engine analyzes the teacher's emotions. For example, if the teacher is emotional, the speech will be tailored to include a warm message that reflects that emotion.

[0722] After the speech, the teacher can input feedback through the "Assembling-kun" app. For example, the teacher might say, "There was a part that the students particularly enjoyed." The server then applies this feedback to the generative AI model and emotion engine, providing more appropriate content for the next speech draft.

[0723] Furthermore, when giving a speech at an international conference, teachers can specify the translation and language of the manuscript, as well as the nationality and religious leanings of the audience. The server translates the speech into the specified language and checks whether the content complies with diversity and inclusion standards. The translation result is generated as a QR code, which teachers can easily share with other participants.

[0724] The above is a specific embodiment of the speech support system that combines the emotion engine of the "Mr. Asarai" system of the present invention. This system combines the functions of user input, speech generation, emotion analysis, feedback reflection, translation, and checking to achieve efficient, high-quality speech support.

[0725] The processing flow will be explained below.

[0726] Step 1:

[0727] The user launches the "Asare-kun" app.

[0728] When a user launches the "Asare-kun" app on their smartphone or computer, user authentication is performed and the user ID is sent to the server.

[0729] Step 2:

[0730] The user enters the information.

[0731] Users enter information such as their background (e.g., career, educational background), personality (e.g., hobbies, character), occupation (e.g., engineer, teacher), current weather (e.g., sunny, rainy), and location (e.g., New York, Tokyo) into the app's input form. They also specify the length and date and time of the speech.

[0732] Step 3:

[0733] The server receives the data.

[0734] The server receives the information sent by the user and temporarily stores it in a database, at which point the user ID and the information are linked.

[0735] Step 4:

[0736] The device activates the emotion engine.

[0737] The device activates an emotion engine to recognize the user's emotions, analyzing the user's facial expressions and tone of voice to obtain emotional information.

[0738] Step 5:

[0739] The server generates a speech script.

[0740] The server then activates a generative AI model based on the received information and the emotional information obtained from the emotion engine, and automatically generates a speech manuscript. This process also utilizes past speeches and feedback information stored in a database.

[0741] Step 6:

[0742] The server sends the manuscript to the user.

[0743] The generated speech manuscript is sent from the server to the user's terminal, where the user can view the speech manuscript on the terminal at the specified date and time.

[0744] Step 7:

[0745] The user reviews and corrects the speech.

[0746] The user can review the speech manuscript they receive and make corrections within the app if necessary, and the corrections will be resubmitted to the server.

[0747] Step 8:

[0748] The user gives a speech.

[0749] The user gives a speech based on the provided speech script, either using the generated speech script as is or modifying it as desired.

[0750] Step 9:

[0751] The device analyzes the emotions expressed by the user during speech.

[0752] During speech, the device uses an emotion engine to analyze the user's facial expressions and tone of voice and obtains the results in real time.

[0753] Step 10:

[0754] The user submits feedback.

[0755] After the speech, the user can enter feedback on the speech through the "Asarekun" app, detailing phrases that were particularly well-received and parts that received a poor response.

[0756] Step 11:

[0757] The server receives the feedback.

[0758] The server receives feedback from users and stores this feedback information in a database.

[0759] Step 12:

[0760] The server optimizes the generative AI model and emotion engine.

[0761] The server updates and optimizes the generation AI model and emotion engine based on the received feedback and emotion analysis results, so that the feedback content and emotion information are reflected in the next speech generation.

[0762] Step 13:

[0763] The user requests translation and review.

[0764] If necessary, users can specify the translation and language of the speech manuscript, as well as the nationality and religious leanings of the audience, within the app.

[0765] Step 14:

[0766] The server performs the translation and diversity check.

[0767] The server translates the speech manuscript into the specified language and checks the content from a diversity and inclusion perspective to ensure it does not contain discriminatory language.

[0768] Step 15:

[0769] The server sends the translation results.

[0770] The server sends the translated speech manuscript to the user's terminal, where the translation result is displayed for the user to check.

[0771] Step 16:

[0772] Users share translation manuscripts.

[0773] The translated speech manuscript is generated as a QR code on the device, which the user can share with other participants.

[0774] The above is a specific processing flow in an embodiment of the present invention in which the "Morning Assembly" system is combined with an emotion engine, allowing users to use efficient and high-quality speech manuscripts.

[0775] Example 2

[0776] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0777] Conventional speech support systems do not adequately consider the user's emotions or feedback and can only provide uniform content, making it difficult to provide effective speech manuscripts that meet the user's needs.In addition, they do not perform multilingual translation or diversity checks, which limits their use in international settings.

[0778] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0779] In this invention, the server includes means for receiving information from a user regarding their background, personality, occupation, weather, and regional characteristics, means for generating a speech manuscript using a generative AI model based on the received information, means for analyzing the user's emotional information with an emotion engine for the generated speech manuscript and adjusting the manuscript content, means for receiving feedback from the user and optimizing the generation means, means for performing translation and diversity checks, and means for delivering speech manuscripts translated into multiple languages ​​to users via QR codes. This makes it possible to provide effective speech manuscripts that reflect the user's emotions and feedback, and the multilingual support and diversity checks realize a speech support system that can be used in international settings.

[0780] A "user" is a person or organization that provides the information necessary to generate a speech transcript using the system.

[0781] "History" refers to all information about a user's career and background.

[0782] "Personality" refers to personal characteristics such as a user's character and hobbies.

[0783] "Occupation" refers to information about the user's job or area of ​​expertise.

[0784] "Weather" refers to the weather conditions on the day the speech is given and in the area.

[0785] "Regionality" refers to information about the location and cultural background of the user's speech.

[0786] A "generative AI model" refers to an artificial intelligence model that automatically creates a speech manuscript based on information entered by the user.

[0787] An "emotion engine" refers to software or a system that analyzes a user's emotions and reflects the results in a speech manuscript.

[0788] "Feedback" refers to information based on evaluations and reactions provided by users after they give a speech.

[0789] A "diversity check" refers to the process of checking whether a speech manuscript contains discriminatory language.

[0790] "QR code" refers to a two-dimensional barcode that allows you to easily share your speech manuscript with other users.

[0791] The present invention relates to a system for supporting more effective speeches by combining a speech support system with an emotion engine to generate a speech manuscript that reflects the user's emotional state. A specific method for implementing the present invention will be described below.

[0792] The system is mainly composed of three main components: the user, the server, and the terminal. An emotion engine is also added to analyze the user's emotional information and reflect it in the speech manuscript.

[0793] User operations

[0794] The user launches the "Asarekun" app and enters information about themselves through the interface, such as their background (career history), personality (characteristics and hobbies), occupation (job and specialty), current weather, and locality. They also specify the length and date and time of the speech, and submit this information.

[0795] Server Operation

[0796] 1. Receiving and storing information:

[0797] The server receives the information sent by the user and temporarily stores it in a database. The specific database used is a relational database management system (RDBMS), such as MySQL or PostgreSQL.

[0798] 2. Generate speech transcript:

[0799] Based on the received information, the server runs a generative AI model, such as OpenAI's GPT-4, to automatically generate a speech script. During this process, the following prompt is used:

[0800] Generate your speech. Consider the following information:

[0801] Biography: Teaching career

[0802] Personality: How you interact with students

[0803] Occupation: School teacher

[0804] Weather: Sunny

[0805] Regional Characteristics: Japanese School Culture

[0806] Speech length: 10 minutes

[0807] 3. Leveraging the Emotion Engine:

[0808] The generated speech is then analyzed using an emotion engine, such as IBM Watson's Tone Analyzer, to analyze the user's emotional information and adjust the content of the speech. For example, if the teacher is moved, a warm message reflecting that emotion is added.

[0809] 4. Receiving and storing feedback:

[0810] The system receives feedback provided by users after they have used the speech and stores it in a database. The feedback information is reflected in the next speech generation to further improve the quality of the speech manuscript.

[0811] 5. Translation and Diversity Check:

[0812] If necessary, the server translates the speech manuscript into multiple languages, possibly using translation software such as Google Translate. It also incorporates an algorithm to check whether the translation is acceptable from a diversity and inclusion perspective.

[0813] Device behavior

[0814] 1. Submission of manuscript:

[0815] The generated speech manuscript is sent to the user's terminal and displayed at the specified date and time. The user can check the displayed speech manuscript and make corrections as necessary.

[0816] 2. Generate and share your QR code:

[0817] A QR code is generated on the user's device for the translated speech manuscript, which can be easily shared with other participants. This QR code can be generated using a general QR code generation library.

[0818] Specific examples

[0819] For example, consider a situation where a teacher at a school is giving a speech at the end of the semester. The teacher launches the "Mr. Asareikun" app and enters information such as "Career as a teacher," "Interaction with students," "Occupation: School teacher," "Weather: Sunny," and "Region: Japanese school culture." The teacher also specifies that the speech will last approximately 10 minutes.

[0820] The server uses this information to generate a speech using a generative AI model, including phrases like, "I'm happy to be able to conclude this wonderful semester with you all today in this beautiful, sunny weather," and adds a warm, moving message using an emotion engine.

[0821] After the speech is completed, the teacher can send feedback through the app, such as, "There was a part that the students particularly enjoyed." This feedback information is stored in the database and reflected the next time the speech is generated.

[0822] When giving a speech at an international conference, teachers can specify the translation language of the manuscript and the nationality and religious orientation of the audience. The server translates the speech into the specified language and checks whether the content is appropriate from a diversity and inclusion perspective. The translation result is generated as a QR code that can be shared with other participants.

[0823] The speech support system of the present invention realizes efficient and high-quality speech support by linking the functions of user input, speech generation, emotion analysis, feedback reflection, translation, and checking.

[0824] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0825] Step 1:

[0826] Users launch the Asarekun app and enter information about their background, personality, occupation, weather, and locale through the interface.

[0827] Input: History, personality, occupation, weather, region, length and date of speech

[0828] Output: Sending input information

[0829] Specifically, the user enters each item into the app's form and presses the submit button to send the information to the server.

[0830] Step 2:

[0831] The server receives the information sent by the user and temporarily stores it in a database, such as a relational database management system like MySQL or PostgreSQL.

[0832] Input: Information submitted by the user

[0833] Output: Save to database

[0834] Specifically, the server receives the HTTP request, analyzes the information in the request body, and inserts it into a database.

[0835] Step 3:

[0836] Based on the information received, the server launches a generative AI model (such as GPT-4) to automatically generate a speech manuscript.

[0837] Input: User information stored in the database

[0838] Output: Generated speech transcript

[0839] Specifically, the following prompt sentence is input to the generative AI model:

[0840] Generate your speech. Consider the following information:

[0841] Biography: Teaching career

[0842] Personality: How you interact with students

[0843] Occupation: School teacher

[0844] Weather: Sunny

[0845] Regional Characteristics: Japanese School Culture

[0846] Speech length: 10 minutes

[0847] A generative AI model generates a speech script based on this prompt.

[0848] Step 4:

[0849] The server uses an emotion engine (such as IBM Watson's Tone Analyzer) to analyze the user's emotional information and adjust the content of the generated speech manuscript.

[0850] Input: Generated speech script, user's emotional information

[0851] Output: Adjusted speech transcript

[0852] Specifically, the emotion engine analyzes the emotional tone of the speech manuscript and makes adjustments based on the user's emotions (e.g., adding a warm message).

[0853] Step 5:

[0854] The server receives the feedback provided by the user after they give their speech and stores it in a database.

[0855] Input: User feedback

[0856] Output: Save feedback information to a database

[0857] Specifically, the information entered by the user into the feedback form through the app is sent to the server, which then inserts the information into a database.

[0858] Step 6:

[0859] If necessary, the server translates the speech manuscript into multiple languages ​​and performs a diversity check.

[0860] Input: Speech manuscript, specified language

[0861] Output: Translated speech manuscript, diversity checked content

[0862] Specifically, the server translates the speech manuscript using the Google Translate API or similar, and then applies an algorithm to check the content from a diversity and inclusion perspective.

[0863] Step 7:

[0864] The generated speech manuscript is sent to the user's device and displayed at the specified date and time.

[0865] Input: Adjusted and translated speech transcript

[0866] Output: Delivery to user terminal

[0867] Specifically, the server sends the speech manuscript to the user's terminal via an HTTP request or WebSocket, and configures the terminal to display it at the appropriate time.

[0868] Step 8:

[0869] A QR code is generated on the user's device for the translated speech manuscript, allowing it to be easily shared with other participants.

[0870] Input: Translated speech

[0871] Output: QR code

[0872] Specifically, the translated speech manuscript is encoded on the device using a QR code generation library (e.g., qrcode.js) to generate a QR code.

[0873] (Application example 2)

[0874] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0875] Conventional speech support systems simply generate speech scripts, but do not provide personalized support that reflects the user's emotional state. This makes it difficult to provide appropriate sales pitches in real time based on the user's emotions and the other person's reactions. In particular, in sales situations in physical stores, appropriate dialogue that reflects the customer's emotions is required, but conventional technology has had difficulty meeting this requirement.

[0876] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for receiving information on the user's history, personality, occupation, weather, and locality from the user; data processing means for providing the generated speech manuscript to the user; means for receiving feedback from the user and optimizing the generation means; means for reflecting the user's emotional information in the speech manuscript generated by the data processing means; means for providing the generated speech manuscript to the user in real time to support the user during the conversation; and means for performing translation and diversity checks. This makes it possible to generate a speech manuscript that takes the user's emotional state and feedback into consideration, thereby improving the quality of conversations in physical stores.

[0877] "History" is information about the user's career history, educational background, and past experiences.

[0878] "Personality" is information about a user's hobbies, character, and personal characteristics.

[0879] "Occupation" is information about the work or job the user is engaged in.

[0880] "Weather" is information about the weather conditions at the user's current location.

[0881] "Regionality" is information about the culture and customs specific to the area where the user lives or where the speech is given.

[0882] "Data processing means" refers to a processing device or program for generating a speech manuscript based on input information.

[0883] "Generation means" refers to an algorithm or system that creates a speech manuscript based on input information from the user.

[0884] "Means for reflecting emotional information" refers to a processing method or device for analyzing the user's emotional state and reflecting the results in the speech manuscript.

[0885] "Means for providing in real time" refers to a device or program that instantly displays the generated speech manuscript on the user's terminal and provides support during the conversation.

[0886] "Translation means" refers to a process or device for translating a generated speech transcript into a different language.

[0887] "Diversity check means" refers to a method or device for checking whether a generated speech manuscript is appropriate for people of diverse cultures and backgrounds.

[0888] A specific embodiment of the present invention will be described. This invention is realized by a speech support system that combines an emotion engine. The system is composed of a user, a server, and a terminal, and by adding an emotion engine, the system analyzes the user's emotion information and reflects it in the speech manuscript.

[0889] The server has a means to receive information provided by users about their background, personality, occupation, weather, and locality. The user inputs this information into a dedicated smartphone application. For example, when a salesperson introduces a new product on the shop floor, they input information about their background, hobbies, occupation, current weather, and local culture.

[0890] After receiving the information, the server uses data processing means to generate a speech manuscript based on the data. In this case, a generative AI model (e.g., GPT-4) is used to automatically generate the optimal speech manuscript, taking into account past data and feedback. The generated manuscript will initially be a speech with basic information.

[0891] The emotion engine then analyzes the speech and adapts it to reflect the user's emotional information. For example, if a salesperson is nervous, the system will adjust the speech script to include a message that will help them relax. The system uses the Affectiva SDK and Google Cloud Natural Language API to perform emotion analysis of speech and text.

[0892] The generated speech script is provided to the user's device (smartphone or tablet) in real time. This allows the salesperson to always present the most appropriate sales pitch while interacting with the customer. Below is an example of a prompt:

[0893] Is this product really right for me?

[0894] A feedback mechanism is also provided, allowing users to input their reactions to the speech into the application. For example, they can provide feedback such as, "This sales pitch was very effective." The server uses this feedback to optimize the generative AI model and emotion engine and reflect it in the next speech generation.

[0895] Additionally, the system also provides translation and diversity checks, which allow speeches to be translated into different languages ​​and checked for cultural and religious appropriateness. For example, if a speech is presented to an international audience, it can be translated and checked for discriminatory language.

[0896] By using this system, users can receive optimal conversation support in real time, improving the quality of conversations in physical stores.

[0897] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0898] Step 1:

[0899] The user launches a dedicated smartphone application and inputs information about their background, personality, occupation, weather, and locality. Based on this, initial customer information data is generated. This information is then sent to the server.

[0900] Input: history, personality, occupation, weather, region

[0901] Output: Initial customer information data

[0902] Specifically, the user enters the necessary information into the input form displayed on the application screen and presses the send button.

[0903] Step 2:

[0904] The server processes the initial customer information data received from the user and stores it in a database, which also stores past speech transcripts and feedback information.

[0905] Input: Initial customer information data

[0906] Output: Entries saved in the database

[0907] Specifically, the server analyzes the received data and stores it in a database in an appropriate format.

[0908] Step 3:

[0909] The server then runs a generative AI model (e.g., GPT-4) based on the stored data to automatically generate a speech manuscript, referencing past speeches and feedback information stored in a database during this process.

[0910] Input: Customer information data and historical data extracted from the database

[0911] Output: Initial speech manuscript

[0912] Specifically, the server inputs a prompt to the AI ​​model and retrieves the generated speech. For example, the prompt might be in the following format:

[0913] "Is this product really right for me?"

[0914] Step 4:

[0915] The server uses an emotion engine (e.g., Affectiva SDK or Google Cloud Natural Language API) to adapt the initial speech transcript to reflect the user's emotional information, extracted from additional information entered by the user and real-time speech analysis.

[0916] Input: Initial speech script and user's emotional information

[0917] Output: Emotionally-reflected speech transcript

[0918] Specifically, the emotion engine analyzes the user's voice data, extracts emotional elements (e.g., tension, excitement, relaxation), and reflects them in the content of the speech manuscript.

[0919] Step 5:

[0920] The speech manuscript reflecting the generated emotions is sent to the user's terminal in real time, and the user can use this speech manuscript to have a conversation with the customer.

[0921] Input: Speech manuscript reflecting emotions

[0922] Output: Speech manuscript displayed on the user's device

[0923] Specifically, the server sends the generated speech manuscript to the user's smartphone and displays it on the application screen.

[0924] Step 6:

[0925] After finishing their speech, users can input feedback through the application, which the server receives and uses to optimize the generative AI model and emotion engine when generating the next speech manuscript.

[0926] Input: User feedback

[0927] Output: Feedback information added to the database

[0928] Specifically, the user enters their thoughts and suggestions for improvement into the application's feedback form and presses the submit button. The server receives this and adds it to the database.

[0929] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0930] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0931] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0932] [Third embodiment]

[0933] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0934] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0935] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0936] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0937] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0938] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0939] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0940] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0941] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0942] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0943] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0944] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0945] The present invention relates to a system for automating speechwriting functions and generating efficient, high-quality speech manuscripts. A specific method for implementing the present invention will be described below with reference to the following embodiments.

[0946] System Overview

[0947] The system is mainly composed of three elements: the user, the server, and the terminal. Specifically, the server generates a speech manuscript based on the information provided by the user and provides it to the user via the terminal.

[0948] User operations

[0949] Enter information:

[0950] The user launches the "Asarekun" app and enters necessary information such as background (e.g., career, educational background), personality (e.g., hobbies, character), occupation (e.g., engineer, teacher), current weather (e.g., sunny, rainy), and region (e.g., New York, Tokyo). In addition, the user specifies the length and date and time of the speech, and submits this information within the app.

[0951] Server Operation

[0952] Generate a speech transcript:

[0953] The server receives the information sent by the user. Based on the received information, it uses a generative AI model to generate a speech script. During this process, it also references past speech scripts and feedback information stored in a database to select the optimal content.

[0954] Reflecting feedback:

[0955] The server receives feedback provided by users after they have used the speech. The feedback includes phrases that were well-received and parts that received poor responses. The server then incorporates this feedback into the generative AI model and uses it when generating the next speech manuscript.

[0956] Translation and Diversity Checks:

[0957] If necessary, the server translates the speech manuscript into multiple languages ​​as specified. It also checks the translation to ensure it does not contain discriminatory language from the perspective of diversity and inclusion. This checking function provides speech manuscripts that can be used safely in global business situations.

[0958] Device behavior

[0959] Manuscript provided by:

[0960] The generated speech transcript is sent to the user's device and displayed at the specified date and time. The user can review the displayed speech transcript and edit it as needed. A QR code can be generated on the user's device for the translated speech transcript, which can be easily shared with other participants.

[0961] Specific examples

[0962] For example, consider a company manager preparing a morning assembly speech for employees. The manager launches the "Asa-Kun" app and enters information such as his or her background (career as a manager), personality (emphasis on leadership and teamwork), occupation (manager at an IT company), current weather (sunny in Tokyo), and regional characteristics (Japanese culture). The manager also specifies that the speech will last approximately five minutes.

[0963] Based on this information, the server uses a generative AI model to create an optimal speech script. For example, if the current weather is sunny, the script might include phrases such as, "I'm happy to be able to start a wonderful day with you all in this sunny weather today." Regional differences are also taken into consideration, and seasonal greetings and content that touches on unique Japanese customs are included.

[0964] After the speech, the manager can submit feedback through the "Asarekun" app. For example, the manager might say, "The part introducing the new project was particularly well-received." The server then reflects this feedback and adjusts the AI ​​model to provide more appropriate content the next time the speech is generated.

[0965] Furthermore, when giving a speech at an international conference, a manager can request that their speech manuscript be translated into English, Chinese, or other languages. The server translates the speech into the specified language and checks whether the content meets the requirements from a diversity and inclusion perspective. The translation result is generated as a QR code, which the manager can easily share with other participants.

[0966] The above is a specific example of speech support using the "Asare-kun" system. This system combines the functions of user input, speech generation, feedback reflection, translation, and checking to provide efficient, high-quality speech support.

[0967] The processing flow will be explained below.

[0968] Step 1: The user launches the "Asarekun" app.

[0969] When a user launches the "Asare-kun" app on their smartphone or computer, user authentication is performed and the user ID is sent to the server.

[0970] Step 2: The user enters information.

[0971] Users fill out a form in the app with information about their background, personality, occupation, current weather, and location, as well as the length and date and time of the speech.

[0972] Step 3: The server receives the data.

[0973] The server receives the information sent by the user and temporarily stores it in a database, at which point the user ID and the information are linked.

[0974] Step 4: The server generates the speech script.

[0975] The server then activates a generative AI model based on the received information to automatically generate a speech manuscript, using past speeches and feedback information stored in a database.

[0976] Step 5: The server sends the manuscript to the user.

[0977] The generated speech manuscript is sent from the server to the user's terminal, where the user can view the speech manuscript on the terminal at the specified date and time.

[0978] Step 6: The user reviews and corrects the speech.

[0979] The user can review the speech manuscript they receive and make corrections within the app if necessary, and the corrections will be resubmitted to the server.

[0980] Step 7: The user gives a speech.

[0981] The user gives a speech based on the provided speech script, either using the generated speech script as is or modifying it as desired.

[0982] Step 8: User submits feedback.

[0983] After the speech, the user can enter feedback on the speech through the "Asarekun" app, detailing phrases that were particularly well-received and parts that received a poor response.

[0984] Step 9: The server receives the feedback.

[0985] The server receives feedback from users and stores this feedback information in a database.

[0986] Step 10: The server optimizes the generated AI model.

[0987] The server updates and optimizes the generative AI model based on the received feedback, so that the feedback is reflected in the next speech generation.

[0988] Step 11: User requests translation and checking.

[0989] If necessary, users can specify the translation and language of the speech manuscript, as well as the nationality and religious leanings of the audience, within the app.

[0990] Step 12: The server performs the translation and diversity check.

[0991] The server translates the speech manuscript into the specified language and checks the content from a diversity and inclusion perspective to ensure it does not contain discriminatory language.

[0992] Step 13: The server sends the translation result.

[0993] The server sends the translated speech manuscript to the user's terminal, where the translation result is displayed for the user to check.

[0994] Step 14: The user shares the translation manuscript.

[0995] The translated speech manuscript is generated as a QR code on the device, which the user can share with other participants.

[0996] The above is a specific flow of processing in the "Mr. Asakai" system of the present invention, which allows users to use efficient and high-quality speech manuscripts.

[0997] Example 1

[0998] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0999] Conventional speech support systems have the drawback of being time-consuming and labor-intensive, as the creation of speech manuscripts, translation, and feedback integration are all done manually. Furthermore, they lacked checking functions to accommodate different languages ​​and cultures, making them difficult to use in global business settings.

[1000] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1001] In this invention, the server includes means for receiving information on user history, personality, occupation, weather, and locality from the user, means for generating prompts using a generative AI model and creating a speech manuscript, data processing means for providing the generated speech manuscript to the user, means for receiving feedback from the user and optimizing the generation means, means for translating the speech manuscript into multiple languages ​​and performing a diversity check, and means for generating the translated speech manuscript as a QR code. This automates the process from generating the speech manuscript to providing it, incorporating feedback, and translating and checking it, enabling the efficient provision of high-quality speech manuscripts.

[1002] "User" refers to an individual or organization that intends to use the system to generate a speech manuscript.

[1003] "History" refers to information indicating a user's past experiences, such as career history, educational background, and work history.

[1004] "Personality" refers to information that indicates personal characteristics such as a user's hobbies and character.

[1005] "Occupation" refers to information indicating the job or specialty in which the user is engaged.

[1006] "Weather" refers to information indicating the weather conditions in the area where the speech will be given on the day.

[1007] "Regionality" refers to information that indicates the characteristics of the culture, customs, etc. of the place where the speech is given.

[1008] "Generative AI model" refers to an artificial intelligence model that automatically generates a speech manuscript based on information provided by the user.

[1009] A "prompt sentence" is an input sentence given to a generative AI model, and refers to text that guides the generation of a speech manuscript.

[1010] "Data processing means" refers to an information processing system for providing the generated speech manuscript to the user.

[1011] "Feedback" refers to information indicating reactions such as evaluations and impressions provided by users after they give a speech.

[1012] "Optimization methods" refer to techniques for receiving user feedback and improving the performance of generative AI models.

[1013] "Translation means" refers to a method or system for converting a speech transcript into a different language.

[1014] "Diversity check measures" refer to methods used to check whether translated speech manuscripts contain discriminatory language.

[1015] A "QR code" is a type of two-dimensional code, a graphic used to easily share speech manuscripts.

[1016] The present invention relates to a system for automating speechwriting functions and efficiently generating high-quality speech manuscripts. A specific method for implementing the present invention will be described below with reference to the following embodiments.

[1017] System Overview

[1018] This system is mainly composed of three elements: the user, the server, and the terminal. The user provides information, and the server generates a speech manuscript and provides it to the user through the terminal.

[1019] User operations

[1020] Enter information:

[1021] The user launches the "Speech Assistant" app and inputs necessary information such as background (e.g., career, educational background), personality (e.g., hobbies, character), occupation (e.g., engineer, teacher), current weather (e.g., sunny, rainy), and region (e.g., New York, Tokyo). In addition, the user specifies the length and date and time of the speech, and submits this information within the app.

[1022] Server Operation

[1023] Generate a speech transcript:

[1024] The server receives the information sent by the user. Based on the received information, it uses a generative AI model to generate a speech script. During this process, it also references past speech scripts and feedback information stored in a database to select the optimal content.

[1025] Examples:

[1026] For example, consider a company manager preparing a morning assembly speech for employees. The manager launches the "Speech Assistant" app and inputs information such as his or her background (career as a manager), personality (emphasis on leadership and teamwork), occupation (manager at an IT company), current weather (sunny in Tokyo), and regional characteristics (Japanese culture). The manager also specifies that the speech will last approximately five minutes.

[1027] Based on this information, the server uses a generative AI model to create an optimal speech script. For example, if the current weather is sunny, the script might include phrases such as, "I'm happy to be able to start a wonderful day with you all in this sunny weather today." Regional differences are also taken into consideration, and seasonal greetings and content that touches on unique Japanese customs are included.

[1028] Example prompt sentence:

[1029] "His background is 10 years in the IT industry, the weather is fine, and the speech is 5 minutes long."

[1030] Reflecting feedback:

[1031] The server receives feedback provided by users after they have used the speech. This feedback includes phrases that were well-received and parts that received poor responses. The server then incorporates this feedback into the generative AI model and uses it when generating the next speech manuscript.

[1032] Examples:

[1033] After the speech, the manager can provide feedback through the Speech Assistant app, such as, "The part introducing the new project was particularly well-received." The server then reflects this feedback and adjusts the AI ​​model to provide more appropriate content the next time the speech is generated.

[1034] Translation and Diversity Checks:

[1035] If necessary, the server translates the speech manuscript into multiple languages ​​as specified, and also checks the translation from the perspective of diversity and inclusion to ensure it does not contain discriminatory language.

[1036] Examples:

[1037] When giving a speech at an international conference, a manager can request that their speech manuscript be translated into English, Chinese, or other languages. The server translates the speech into the specified language and checks whether the content is appropriate from a diversity and inclusion perspective. The translation result is generated as a QR code, which the manager can easily share with other participants.

[1038] Device behavior

[1039] Manuscript provided by:

[1040] The generated speech transcript is sent to the user's device and displayed at the specified date and time. The user can review the displayed speech transcript and edit it as necessary. A QR code can be generated on the user's device for the translated speech transcript, allowing it to be easily shared with other participants.

[1041] Examples:

[1042] Users can check the generated speech manuscript through the "Speech Assistant" app and edit it as necessary. For example, when giving a speech at an international conference, the translated speech manuscript can be generated as a QR code and distributed to other participants, enabling smooth communication.

[1043] The above is a concrete example of how the present invention can be implemented. This system allows users to create high-quality speech manuscripts in a short amount of time, and can be used safely in global business situations.

[1044] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1045] Step 1:

[1046] Input and Action:

[1047] The user launches the "Speech Assistant" app and enters the necessary information, such as background, personality, occupation, weather, region, speech length, date and time, etc. The information is sent within the app by pressing the send button.

[1048] Data processing and output:

[1049] The information entered by the user is sent to the server in JSON format. The server receives this JSON data, analyzes the information stored in each field, and saves it as basic data for generating a speech manuscript.

[1050] Step 2:

[1051] Input and Action:

[1052] The server receives the JSON-formatted information sent by the user and parses it to extract information such as history, personality, occupation, weather, region, speech length, and date and time.

[1053] Data processing and output:

[1054] Based on the extracted information, a prompt sentence is generated for the generative AI model. The prompt sentence is input to the generative AI model in the form of "history is 10 years of experience in the IT industry, the weather is sunny, and the speech length is 5 minutes."

[1055] Step 3:

[1056] Input and Action:

[1057] The server inputs the generated prompt sentences into the generative AI model, which analyzes the prompt sentences and automatically generates a speech script.

[1058] Data processing and output:

[1059] The generative AI model also references past speech scripts and feedback information stored in a database to generate the optimal speech script, which is then temporarily stored on a server.

[1060] Step 4:

[1061] Input and Action:

[1062] When submitting feedback, the user can enter and submit their feedback after the speech through the "Speech Assistant" app. For example, they could enter feedback such as "The introduction to the new project was particularly well-received."

[1063] Data processing and output:

[1064] The server analyzes the feedback received from the user and reflects it in the generative AI model. The generative AI model is then readjusted based on the feedback information and used for generating the next speech manuscript.

[1065] Step 5:

[1066] Input and Action:

[1067] If the user requests multilingual translation, the server translates the speech into the specified language, for example, if it needs to be translated into English or Chinese.

[1068] Data processing and output:

[1069] The server translates the generated speech manuscript into the specified language, checks it from a diversity and inclusion perspective, verifies that the content is not discriminatory, and saves the translation results.

[1070] Step 6:

[1071] Input and Action:

[1072] Once the translation and checking process is complete on the server, the speech manuscript is sent to the user's device, where the user can check the displayed speech manuscript through the "Speech Assistant" app.

[1073] Data processing and output:

[1074] The speech manuscript displayed on the user's device can be viewed and edited by the user. In addition, a QR code can be generated on the user's device for the translated speech manuscript, allowing it to be easily shared with other participants.

[1075] The above is a specific flow of program processing for the system based on the claims. This system allows users to create high-quality speech manuscripts in a short amount of time, and to obtain deliverables that are efficient and can be used in multiple languages.

[1076] (Application example 1)

[1077] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1078] In today's world, opportunities for speeches and presentations are increasing, creating a demand for systems that can improve the quality and efficiency of speech drafts. Furthermore, with the advancement of globalization, multilingual support and cultural sensitivity are required, as well as a means for easily distributing generated speech drafts. However, existing systems lack the ability to provide individualized support based on the user's history and personality, and optimizing the system to reflect feedback is difficult. The objective of this invention is to solve these problems and provide high-quality speech support.

[1079] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1080] In this invention, the server includes means for receiving information from the user regarding background, personality, occupation, weather, and locality, data processing means for providing the generated speech transcript to the user, means for receiving user feedback and optimizing the generating means, means for performing translation and diversity checks, means for generating content translated into multiple languages ​​and taking social and cultural considerations into account, and means for outputting and distributing the generated speech transcript as a video or audio file, thereby enabling the generation of high-quality speech transcripts based on the user's specific information and the provision of content that is multilingual and culturally sensitive.

[1081] "User" refers to a person or organization that provides information to generate a speech manuscript using the system.

[1082] "Provenance" refers to information about a user's past experience, work and academic history, and other career information.

[1083] "Personality" refers to information about a user's personality, interests, hobbies, and personal characteristics.

[1084] "Occupation" refers to information about the job or work the user is currently doing.

[1085] "Weather" refers to information about the local weather conditions when the user delivers a speech.

[1086] "Regionality" refers to information about the cultural and social characteristics of the region where the user gives speech.

[1087] "Data processing means" refers to a computer processing device or software that generates a speech manuscript based on information provided by the user and provides it to the user.

[1088] A "generative AI model" is a model for generating speech manuscripts using artificial intelligence, and refers to an algorithm that generates optimal content based on past data and user-specified information.

[1089] "Feedback" refers to information such as reactions, evaluations, and areas for improvement provided by a user after they give a speech.

[1090] "Optimization means" refers to methods and algorithms for receiving feedback information and reflecting that information in the next speech script generation to improve the generative AI model.

[1091] "Translation and diversity check" refers to the process of converting the generated speech manuscript into multiple languages ​​and verifying that the content does not contain discriminatory language.

[1092] "Content" refers to the generated speech manuscript and multimedia information such as video and audio files created based on it.

[1093] A "video or audio file" is an audiovisual material created based on the generated speech manuscript, and is in a format for providing the speech content to users or viewers.

[1094] The present invention is a system for automatically generating a speech manuscript based on user information and providing the speech in various formats. The invention is specifically embodied in the following embodiments.

[1095] System Configuration

[1096] The system mainly consists of users, servers, and terminals.

[1097] User operations

[1098] The user starts the application using the smartphone and enters the following information:

[1099] Background (career, academic history, etc.)

[1100] Personality (hobbies, character, etc.)

[1101] Occupation (engineer, teacher, etc.)

[1102] Current weather (sunny, rainy, etc.)

[1103] Regional characteristics (region, culture, etc.)

[1104] Length and date of speech

[1105] This information is submitted by the user within the application.

[1106] Server Operation

[1107] The server has the following functions:

[1108] 1. Information reception: Receives information sent by the user.

[1109] 2. Speech generation: Based on the received information, a speech script is generated using a generative AI model (e.g., OpenAI GPT-4). During this process, past speech data and feedback information are referenced to select the optimal content.

[1110] 3. Feedback reflection: The system receives feedback provided by the user after the speech and reflects that information in the generative AI model when generating the next speech manuscript.

[1111] 4. Translation and Diversity Check: Speech manuscripts will be translated into multiple languages ​​as needed and checked from a diversity and inclusion perspective to ensure they do not contain discriminatory language.

[1112] 5. Content generation: Based on the generated speech manuscript, video and audio files are created and made available for distribution by users.

[1113] Device behavior

[1114] The generated speech manuscript is sent to the user's device and can be viewed within the application. The user can edit it and check the final speech content. Furthermore, the generated content (video and audio files) can be easily distributed via social media, email, etc.

[1115] Program processing

[1116] The server-side program is built using Node.js and Express and handles all processes from receiving user information, generating speech scripts, incorporating feedback, translating and checking, and generating content.

[1117] Processing of received user information: The information is converted into an appropriate format and input into the generative AI model.

[1118] Speech generation: The generative AI model generates a speech based on the provided information. OpenAI's GPT-4 is used as the generative model.

[1119] Feedback Processing: After each speech, user feedback is received. This feedback is used in the next generation to improve the accuracy and adaptability of the model.

[1120] Translation and diversity check: The generated speech transcripts are translated into multiple languages ​​and checked for discriminatory content, making them available for global use.

[1121] Content generation: Video and audio files are generated based on the speech manuscript and made available to users.

[1122] Specific examples

[1123] For example, if a user provides the following information on the topic of "wedding speeches," the following prompt sentences will be used for the generative AI model:

[1124] Prompt statement:

[1125] "Generate speech under the following conditions:

[1126] Experience: Over 10 years of experience in the IT industry

[1127] Personality: Team player

[1128] Occupation: Systems Engineer

[1129] Weather: Sunny

[1130] Region: Tokyo

[1131] Length: 5 minutes

[1132] Please prepare the speech."

[1133] The generative AI model uses this information to generate a speech transcript and provides it to the user, who can then edit it and distribute it as a video or audio file.

[1134] The above is a specific embodiment of the present invention, which allows users to efficiently generate and provide highly personalized speech manuscripts.

[1135] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1136] Step 1:

[1137] The user starts the application on their smartphone and inputs information about their background, personality, occupation, weather, and locality. They also specify the length of the speech and the date and time. This information becomes the input data.

[1138] Step 2:

[1139] The terminal receives the information entered by the user and sends it to the server. Specifically, the data in the input form is converted to JSON format and sent to the server as an HTTP request. Once this data transmission is complete, the terminal displays a successful transmission status to the user.

[1140] Step 3:

[1141] The server generates a prompt based on the received user information. This prompt is used as input to the generative AI model. A speech script is generated based on the generated prompt. The prompt is generated by embedding user data in a template, which is then sent to the API of the generative AI model (e.g., OpenAI GPT-4).

[1142] Step 4:

[1143] The server receives the speech transcript data returned by the generative AI model and processes it into an appropriate format (e.g., text file, JSON). At this time, it also performs diversity checks and translation processing on the transcript content to generate a multilingual speech transcript. The processed data is stored in the DB system.

[1144] Step 5:

[1145] The generated speech script is sent to the terminal, which displays the received speech script to the user and provides an interface for the user to edit it as needed, allowing the user to review the speech content and provide feedback.

[1146] Step 6:

[1147] After the user finishes the speech, they input their feedback. The device then sends the feedback data to the server. This feedback includes phrases that were well-received and points for improvement.

[1148] Step 7:

[1149] The server processes the received feedback and refines the generative AI model. This refinement step uses machine learning algorithms to highlight the most effective parts and incorporate them into the next speech generation, thereby improving the quality of the generated speech transcript.

[1150] Step 8:

[1151] The server generates the final speech manuscript as video and audio files. These files are then sent back to the device so that the user can distribute them via social media, email, etc. The device provides a distribution interface, allowing the user to share the content with other users.

[1152] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1153] The present invention relates to a system for supporting more effective speeches by combining a speech support system with an emotion engine to generate a speech manuscript that reflects the emotional state of a user. A specific method for implementing the present invention will be described below with reference to the following embodiments.

[1154] System Overview

[1155] The system is mainly composed of three main components: the user, the server, and the terminal. An emotion engine is also added to analyze the user's emotional information and reflect it in the speech manuscript.

[1156] User operations

[1157] Enter information:

[1158] The user launches the "Asarekun" app and enters necessary information such as background (e.g., career, educational background), personality (e.g., hobbies, character), occupation (e.g., engineer, teacher), current weather (e.g., sunny, rainy), and region (e.g., New York, Tokyo). In addition, the user specifies the length and date and time of the speech, and submits this information within the app.

[1159] Server Operation

[1160] Generate a speech transcript:

[1161] The server receives the information sent by the user and temporarily stores it in a database. It then activates a generative AI model based on the received information to automatically generate a speech manuscript. This process also references past speeches and feedback information stored in the database.

[1162] Leveraging the Emotion Engine:

[1163] The emotion engine analyzes the user's emotional information and adjusts the content of the generated speech. For example, if the user is nervous, it can include a message to relax them.

[1164] Reflecting feedback:

[1165] The server receives feedback provided by users after they have used the speech. The feedback includes phrases that were well-received and parts that received poor responses. The server then applies this feedback to the generative AI model and emotion engine, and uses it when generating the next speech manuscript.

[1166] Translation and Diversity Checks:

[1167] If necessary, the server translates the speech manuscript into multiple languages ​​as specified. It also checks the translation to ensure it does not contain discriminatory language from the perspective of diversity and inclusion. This checking function provides speech manuscripts that can be used safely in global business situations.

[1168] Device behavior

[1169] Manuscript provided by:

[1170] The generated speech transcript is sent to the user's device and displayed at the specified date and time. The user can review the displayed speech transcript and make any necessary corrections. A QR code can be generated on the user's device for the translated speech transcript, which can be easily shared with other participants.

[1171] Specific examples

[1172] For example, consider a situation where a teacher at a school is giving a speech at the end of the semester. The teacher launches the "Asarekun" app and enters information such as their background (career as a teacher), personality (how they interact with students), occupation (school teacher), current weather (sunny), and regional characteristics (Japanese school culture). They also specify that the speech will last approximately 10 minutes.

[1173] The server generates a speech script based on this information. For example, it includes a phrase such as, "I am happy to be able to conclude this wonderful semester with you all today in this beautiful sunny weather," taking into account the weather. It also includes content that takes into account the local area and school culture.

[1174] Additionally, an emotion engine analyzes the teacher's emotions. For example, if the teacher is emotional, the speech will be tailored to include a warm message that reflects that emotion.

[1175] After the speech, the teacher can input feedback through the "Assembling-kun" app. For example, the teacher might say, "There was a part that the students particularly enjoyed." The server then applies this feedback to the generative AI model and emotion engine, providing more appropriate content for the next speech draft.

[1176] Furthermore, when giving a speech at an international conference, teachers can specify the translation and language of the manuscript, as well as the nationality and religious leanings of the audience. The server translates the speech into the specified language and checks whether the content complies with diversity and inclusion standards. The translation result is generated as a QR code, which teachers can easily share with other participants.

[1177] The above is a specific embodiment of the speech support system that combines the emotion engine of the "Mr. Asarai" system of the present invention. This system combines the functions of user input, speech generation, emotion analysis, feedback reflection, translation, and checking to achieve efficient, high-quality speech support.

[1178] The processing flow will be explained below.

[1179] Step 1:

[1180] The user launches the "Asare-kun" app.

[1181] When a user launches the "Asare-kun" app on their smartphone or computer, user authentication is performed and the user ID is sent to the server.

[1182] Step 2:

[1183] The user enters the information.

[1184] Users enter information such as their background (e.g., career, educational background), personality (e.g., hobbies, character), occupation (e.g., engineer, teacher), current weather (e.g., sunny, rainy), and location (e.g., New York, Tokyo) into the app's input form. They also specify the length and date and time of the speech.

[1185] Step 3:

[1186] The server receives the data.

[1187] The server receives the information sent by the user and temporarily stores it in a database, at which point the user ID and the information are linked.

[1188] Step 4:

[1189] The device activates the emotion engine.

[1190] The device activates an emotion engine to recognize the user's emotions, analyzing the user's facial expressions and tone of voice to obtain emotional information.

[1191] Step 5:

[1192] The server generates a speech script.

[1193] The server then activates a generative AI model based on the received information and the emotional information obtained from the emotion engine, and automatically generates a speech manuscript. This process also utilizes past speeches and feedback information stored in a database.

[1194] Step 6:

[1195] The server sends the manuscript to the user.

[1196] The generated speech manuscript is sent from the server to the user's terminal, where the user can view the speech manuscript on the terminal at the specified date and time.

[1197] Step 7:

[1198] The user reviews and corrects the speech.

[1199] The user can review the speech manuscript they receive and make corrections within the app if necessary, and the corrections will be resubmitted to the server.

[1200] Step 8:

[1201] The user gives a speech.

[1202] The user gives a speech based on the provided speech script, either using the generated speech script as is or modifying it as desired.

[1203] Step 9:

[1204] The device analyzes the emotions expressed by the user during speech.

[1205] During speech, the device uses an emotion engine to analyze the user's facial expressions and tone of voice and obtains the results in real time.

[1206] Step 10:

[1207] The user submits feedback.

[1208] After the speech, the user can enter feedback on the speech through the "Asarekun" app, detailing phrases that were particularly well-received and parts that received a poor response.

[1209] Step 11:

[1210] The server receives the feedback.

[1211] The server receives feedback from users and stores this feedback information in a database.

[1212] Step 12:

[1213] The server optimizes the generative AI model and emotion engine.

[1214] The server updates and optimizes the generation AI model and emotion engine based on the received feedback and emotion analysis results, so that the feedback content and emotion information are reflected in the next speech generation.

[1215] Step 13:

[1216] The user requests translation and review.

[1217] If necessary, users can specify the translation and language of the speech manuscript, as well as the nationality and religious leanings of the audience, within the app.

[1218] Step 14:

[1219] The server performs the translation and diversity check.

[1220] The server translates the speech manuscript into the specified language and checks the content from a diversity and inclusion perspective to ensure it does not contain discriminatory language.

[1221] Step 15:

[1222] The server sends the translation results.

[1223] The server sends the translated speech manuscript to the user's terminal, where the translation result is displayed for the user to check.

[1224] Step 16:

[1225] Users share translation manuscripts.

[1226] The translated speech manuscript is generated as a QR code on the device, which the user can share with other participants.

[1227] The above is a specific processing flow in an embodiment of the present invention in which the "Morning Assembly" system is combined with an emotion engine, allowing users to use efficient and high-quality speech manuscripts.

[1228] Example 2

[1229] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1230] Conventional speech support systems do not adequately consider the user's emotions or feedback and can only provide uniform content, making it difficult to provide effective speech manuscripts that meet the user's needs.In addition, they do not perform multilingual translation or diversity checks, which limits their use in international settings.

[1231] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1232] In this invention, the server includes means for receiving information from a user regarding their background, personality, occupation, weather, and regional characteristics, means for generating a speech manuscript using a generative AI model based on the received information, means for analyzing the user's emotional information with an emotion engine for the generated speech manuscript and adjusting the manuscript content, means for receiving feedback from the user and optimizing the generation means, means for performing translation and diversity checks, and means for delivering speech manuscripts translated into multiple languages ​​to users via QR codes. This makes it possible to provide effective speech manuscripts that reflect the user's emotions and feedback, and the multilingual support and diversity checks realize a speech support system that can be used in international settings.

[1233] A "user" is a person or organization that provides the information necessary to generate a speech transcript using the system.

[1234] "History" refers to all information about a user's career and background.

[1235] "Personality" refers to personal characteristics such as a user's character and hobbies.

[1236] "Occupation" refers to information about the user's job or area of ​​expertise.

[1237] "Weather" refers to the weather conditions on the day the speech is given and in the area.

[1238] "Regionality" refers to information about the location and cultural background of the user's speech.

[1239] A "generative AI model" refers to an artificial intelligence model that automatically creates a speech manuscript based on information entered by the user.

[1240] An "emotion engine" refers to software or a system that analyzes a user's emotions and reflects the results in a speech manuscript.

[1241] "Feedback" refers to information based on evaluations and reactions provided by users after they give a speech.

[1242] A "diversity check" refers to the process of checking whether a speech manuscript contains discriminatory language.

[1243] "QR code" refers to a two-dimensional barcode that allows you to easily share your speech manuscript with other users.

[1244] The present invention relates to a system for supporting more effective speeches by combining a speech support system with an emotion engine to generate a speech manuscript that reflects the user's emotional state. A specific method for implementing the present invention will be described below.

[1245] The system is mainly composed of three main components: the user, the server, and the terminal. An emotion engine is also added to analyze the user's emotional information and reflect it in the speech manuscript.

[1246] User operations

[1247] The user launches the "Asarekun" app and enters information about themselves through the interface, such as their background (career history), personality (characteristics and hobbies), occupation (job and specialty), current weather, and locality. They also specify the length and date and time of the speech, and submit this information.

[1248] Server Operation

[1249] 1. Receiving and storing information:

[1250] The server receives the information sent by the user and temporarily stores it in a database. The specific database used is a relational database management system (RDBMS), such as MySQL or PostgreSQL.

[1251] 2. Generate speech transcript:

[1252] Based on the received information, the server runs a generative AI model, such as OpenAI's GPT-4, to automatically generate a speech script. During this process, the following prompt is used:

[1253] Generate your speech. Consider the following information:

[1254] Biography: Teaching career

[1255] Personality: How you interact with students

[1256] Occupation: School teacher

[1257] Weather: Sunny

[1258] Regional Characteristics: Japanese School Culture

[1259] Speech length: 10 minutes

[1260] 3. Leveraging the Emotion Engine:

[1261] The generated speech is then analyzed using an emotion engine, such as IBM Watson's Tone Analyzer, to analyze the user's emotional information and adjust the content of the speech. For example, if the teacher is moved, a warm message reflecting that emotion is added.

[1262] 4. Receiving and storing feedback:

[1263] The system receives feedback provided by users after they have used the speech and stores it in a database. The feedback information is reflected in the next speech generation to further improve the quality of the speech manuscript.

[1264] 5. Translation and Diversity Check:

[1265] If necessary, the server translates the speech manuscript into multiple languages, possibly using translation software such as Google Translate. It also incorporates an algorithm to check whether the translation is acceptable from a diversity and inclusion perspective.

[1266] Device behavior

[1267] 1. Submission of manuscript:

[1268] The generated speech manuscript is sent to the user's terminal and displayed at the specified date and time. The user can check the displayed speech manuscript and make corrections as necessary.

[1269] 2. Generate and share your QR code:

[1270] A QR code is generated on the user's device for the translated speech manuscript, which can be easily shared with other participants. This QR code can be generated using a general QR code generation library.

[1271] Specific examples

[1272] For example, consider a situation where a teacher at a school is giving a speech at the end of the semester. The teacher launches the "Mr. Asareikun" app and enters information such as "Career as a teacher," "Interaction with students," "Occupation: School teacher," "Weather: Sunny," and "Region: Japanese school culture." The teacher also specifies that the speech will last approximately 10 minutes.

[1273] The server uses this information to generate a speech using a generative AI model, including phrases like, "I'm happy to be able to conclude this wonderful semester with you all today in this beautiful, sunny weather," and adds a warm, moving message using an emotion engine.

[1274] After the speech is completed, the teacher can send feedback through the app, such as, "There was a part that the students particularly enjoyed." This feedback information is stored in the database and reflected the next time the speech is generated.

[1275] When giving a speech at an international conference, teachers can specify the translation language of the manuscript and the nationality and religious orientation of the audience. The server translates the speech into the specified language and checks whether the content is appropriate from a diversity and inclusion perspective. The translation result is generated as a QR code that can be shared with other participants.

[1276] The speech support system of the present invention realizes efficient and high-quality speech support by linking the functions of user input, speech generation, emotion analysis, feedback reflection, translation, and checking.

[1277] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1278] Step 1:

[1279] Users launch the Asarekun app and enter information about their background, personality, occupation, weather, and locale through the interface.

[1280] Input: History, personality, occupation, weather, region, length and date of speech

[1281] Output: Sending input information

[1282] Specifically, the user enters each item into the app's form and presses the submit button to send the information to the server.

[1283] Step 2:

[1284] The server receives the information sent by the user and temporarily stores it in a database, such as a relational database management system like MySQL or PostgreSQL.

[1285] Input: Information submitted by the user

[1286] Output: Save to database

[1287] Specifically, the server receives the HTTP request, analyzes the information in the request body, and inserts it into a database.

[1288] Step 3:

[1289] Based on the information received, the server launches a generative AI model (such as GPT-4) to automatically generate a speech manuscript.

[1290] Input: User information stored in the database

[1291] Output: Generated speech transcript

[1292] Specifically, the following prompt sentence is input to the generative AI model:

[1293] Generate your speech. Consider the following information:

[1294] Biography: Teaching career

[1295] Personality: How you interact with students

[1296] Occupation: School teacher

[1297] Weather: Sunny

[1298] Regional Characteristics: Japanese School Culture

[1299] Speech length: 10 minutes

[1300] A generative AI model generates a speech script based on this prompt.

[1301] Step 4:

[1302] The server uses an emotion engine (such as IBM Watson's Tone Analyzer) to analyze the user's emotional information and adjust the content of the generated speech manuscript.

[1303] Input: Generated speech script, user's emotional information

[1304] Output: Adjusted speech transcript

[1305] Specifically, the emotion engine analyzes the emotional tone of the speech manuscript and makes adjustments based on the user's emotions (e.g., adding a warm message).

[1306] Step 5:

[1307] The server receives the feedback provided by the user after they give their speech and stores it in a database.

[1308] Input: User feedback

[1309] Output: Save feedback information to a database

[1310] Specifically, the information entered by the user into the feedback form through the app is sent to the server, which then inserts the information into a database.

[1311] Step 6:

[1312] If necessary, the server translates the speech manuscript into multiple languages ​​and performs a diversity check.

[1313] Input: Speech manuscript, specified language

[1314] Output: Translated speech manuscript, diversity checked content

[1315] Specifically, the server translates the speech manuscript using the Google Translate API or similar, and then applies an algorithm to check the content from a diversity and inclusion perspective.

[1316] Step 7:

[1317] The generated speech manuscript is sent to the user's device and displayed at the specified date and time.

[1318] Input: Adjusted and translated speech transcript

[1319] Output: Delivery to user terminal

[1320] Specifically, the server sends the speech manuscript to the user's terminal via an HTTP request or WebSocket, and configures the terminal to display it at the appropriate time.

[1321] Step 8:

[1322] A QR code is generated on the user's device for the translated speech manuscript, allowing it to be easily shared with other participants.

[1323] Input: Translated speech

[1324] Output: QR code

[1325] Specifically, the translated speech manuscript is encoded on the device using a QR code generation library (e.g., qrcode.js) to generate a QR code.

[1326] (Application example 2)

[1327] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1328] Conventional speech support systems simply generate speech scripts, but do not provide personalized support that reflects the user's emotional state. This makes it difficult to provide appropriate sales pitches in real time based on the user's emotions and the other person's reactions. In particular, in sales situations in physical stores, appropriate dialogue that reflects the customer's emotions is required, but conventional technology has had difficulty meeting this requirement.

[1329] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for receiving information on the user's history, personality, occupation, weather, and locality from the user; data processing means for providing the generated speech manuscript to the user; means for receiving feedback from the user and optimizing the generation means; means for reflecting the user's emotional information in the speech manuscript generated by the data processing means; means for providing the generated speech manuscript to the user in real time to support the user during the conversation; and means for performing translation and diversity checks. This makes it possible to generate a speech manuscript that takes the user's emotional state and feedback into consideration, thereby improving the quality of conversations in physical stores.

[1330] "History" is information about the user's career history, educational background, and past experiences.

[1331] "Personality" is information about a user's hobbies, character, and personal characteristics.

[1332] "Occupation" is information about the work or job the user is engaged in.

[1333] "Weather" is information about the weather conditions at the user's current location.

[1334] "Regionality" is information about the culture and customs specific to the area where the user lives or where the speech is given.

[1335] "Data processing means" refers to a processing device or program for generating a speech manuscript based on input information.

[1336] "Generation means" refers to an algorithm or system that creates a speech manuscript based on input information from the user.

[1337] "Means for reflecting emotional information" refers to a processing method or device for analyzing the user's emotional state and reflecting the results in the speech manuscript.

[1338] "Means for providing in real time" refers to a device or program that instantly displays the generated speech manuscript on the user's terminal and provides support during the conversation.

[1339] "Translation means" refers to a process or device for translating a generated speech transcript into a different language.

[1340] "Diversity check means" refers to a method or device for checking whether a generated speech manuscript is appropriate for people of diverse cultures and backgrounds.

[1341] A specific embodiment of the present invention will be described. This invention is realized by a speech support system that combines an emotion engine. The system is composed of a user, a server, and a terminal, and by adding an emotion engine, the system analyzes the user's emotion information and reflects it in the speech manuscript.

[1342] The server has a means to receive information provided by users about their background, personality, occupation, weather, and locality. The user inputs this information into a dedicated smartphone application. For example, when a salesperson introduces a new product on the shop floor, they input information about their background, hobbies, occupation, current weather, and local culture.

[1343] After receiving the information, the server uses data processing means to generate a speech manuscript based on the data. In this case, a generative AI model (e.g., GPT-4) is used to automatically generate the optimal speech manuscript, taking into account past data and feedback. The generated manuscript will initially be a speech with basic information.

[1344] The emotion engine then analyzes the speech and adapts it to reflect the user's emotional information. For example, if a salesperson is nervous, the system will adjust the speech script to include a message that will help them relax. The system uses the Affectiva SDK and Google Cloud Natural Language API to perform emotion analysis of speech and text.

[1345] The generated speech script is provided to the user's device (smartphone or tablet) in real time. This allows the salesperson to always present the most appropriate sales pitch while interacting with the customer. Below is an example of a prompt:

[1346] Is this product really right for me?

[1347] A feedback mechanism is also provided, allowing users to input their reactions to the speech into the application. For example, they can provide feedback such as, "This sales pitch was very effective." The server uses this feedback to optimize the generative AI model and emotion engine and reflect it in the next speech generation.

[1348] Additionally, the system also provides translation and diversity checks, which allow speeches to be translated into different languages ​​and checked for cultural and religious appropriateness. For example, if a speech is presented to an international audience, it can be translated and checked for discriminatory language.

[1349] By using this system, users can receive optimal conversation support in real time, improving the quality of conversations in physical stores.

[1350] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1351] Step 1:

[1352] The user launches a dedicated smartphone application and inputs information about their background, personality, occupation, weather, and locality. Based on this, initial customer information data is generated. This information is then sent to the server.

[1353] Input: history, personality, occupation, weather, region

[1354] Output: Initial customer information data

[1355] Specifically, the user enters the necessary information into the input form displayed on the application screen and presses the send button.

[1356] Step 2:

[1357] The server processes the initial customer information data received from the user and stores it in a database, which also stores past speech transcripts and feedback information.

[1358] Input: Initial customer information data

[1359] Output: Entries saved in the database

[1360] Specifically, the server analyzes the received data and stores it in a database in an appropriate format.

[1361] Step 3:

[1362] The server then runs a generative AI model (e.g., GPT-4) based on the stored data to automatically generate a speech manuscript, referencing past speeches and feedback information stored in a database during this process.

[1363] Input: Customer information data and historical data extracted from the database

[1364] Output: Initial speech manuscript

[1365] Specifically, the server inputs a prompt to the AI ​​model and retrieves the generated speech. For example, the prompt might be in the following format:

[1366] "Is this product really right for me?"

[1367] Step 4:

[1368] The server uses an emotion engine (e.g., Affectiva SDK or Google Cloud Natural Language API) to adapt the initial speech transcript to reflect the user's emotional information, extracted from additional information entered by the user and real-time speech analysis.

[1369] Input: Initial speech script and user's emotional information

[1370] Output: Emotionally-reflected speech transcript

[1371] Specifically, the emotion engine analyzes the user's voice data, extracts emotional elements (e.g., tension, excitement, relaxation), and reflects them in the content of the speech manuscript.

[1372] Step 5:

[1373] The speech manuscript reflecting the generated emotions is sent to the user's terminal in real time, and the user can use this speech manuscript to have a conversation with the customer.

[1374] Input: Speech manuscript reflecting emotions

[1375] Output: Speech manuscript displayed on the user's device

[1376] Specifically, the server sends the generated speech manuscript to the user's smartphone and displays it on the application screen.

[1377] Step 6:

[1378] After finishing their speech, users can input feedback through the application, which the server receives and uses to optimize the generative AI model and emotion engine when generating the next speech manuscript.

[1379] Input: User feedback

[1380] Output: Feedback information added to the database

[1381] Specifically, the user enters their thoughts and suggestions for improvement into the application's feedback form and presses the submit button. The server receives this and adds it to the database.

[1382] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1383] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1384] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1385] [Fourth embodiment]

[1386] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1387] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1388] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1389] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1390] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1391] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1392] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1393] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1394] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1395] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1396] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1397] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1398] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1399] The present invention relates to a system for automating speechwriting functions and generating efficient, high-quality speech manuscripts. A specific method for implementing the present invention will be described below with reference to the following embodiments.

[1400] System Overview

[1401] The system is mainly composed of three elements: the user, the server, and the terminal. Specifically, the server generates a speech manuscript based on the information provided by the user and provides it to the user via the terminal.

[1402] User operations

[1403] Enter information:

[1404] The user launches the "Asarekun" app and enters necessary information such as background (e.g., career, educational background), personality (e.g., hobbies, character), occupation (e.g., engineer, teacher), current weather (e.g., sunny, rainy), and region (e.g., New York, Tokyo). In addition, the user specifies the length and date and time of the speech, and submits this information within the app.

[1405] Server Operation

[1406] Generate a speech transcript:

[1407] The server receives the information sent by the user. Based on the received information, it uses a generative AI model to generate a speech script. During this process, it also references past speech scripts and feedback information stored in a database to select the optimal content.

[1408] Reflecting feedback:

[1409] The server receives feedback provided by users after they have used the speech. The feedback includes phrases that were well-received and parts that received poor responses. The server then incorporates this feedback into the generative AI model and uses it when generating the next speech manuscript.

[1410] Translation and Diversity Checks:

[1411] If necessary, the server translates the speech manuscript into multiple languages ​​as specified. It also checks the translation to ensure it does not contain discriminatory language from the perspective of diversity and inclusion. This checking function provides speech manuscripts that can be used safely in global business situations.

[1412] Device behavior

[1413] Manuscript provided by:

[1414] The generated speech transcript is sent to the user's device and displayed at the specified date and time. The user can review the displayed speech transcript and edit it as needed. A QR code can be generated on the user's device for the translated speech transcript, which can be easily shared with other participants.

[1415] Specific examples

[1416] For example, consider a company manager preparing a morning assembly speech for employees. The manager launches the "Asa-Kun" app and enters information such as his or her background (career as a manager), personality (emphasis on leadership and teamwork), occupation (manager at an IT company), current weather (sunny in Tokyo), and regional characteristics (Japanese culture). The manager also specifies that the speech will last approximately five minutes.

[1417] Based on this information, the server uses a generative AI model to create an optimal speech script. For example, if the current weather is sunny, the script might include phrases such as, "I'm happy to be able to start a wonderful day with you all in this sunny weather today." Regional differences are also taken into consideration, and seasonal greetings and content that touches on unique Japanese customs are included.

[1418] After the speech, the manager can submit feedback through the "Asarekun" app. For example, the manager might say, "The part introducing the new project was particularly well-received." The server then reflects this feedback and adjusts the AI ​​model to provide more appropriate content the next time the speech is generated.

[1419] Furthermore, when giving a speech at an international conference, a manager can request that their speech manuscript be translated into English, Chinese, or other languages. The server translates the speech into the specified language and checks whether the content meets the requirements from a diversity and inclusion perspective. The translation result is generated as a QR code, which the manager can easily share with other participants.

[1420] The above is a specific example of speech support using the "Asare-kun" system. This system combines the functions of user input, speech generation, feedback reflection, translation, and checking to provide efficient, high-quality speech support.

[1421] The processing flow will be explained below.

[1422] Step 1: The user launches the "Asarekun" app.

[1423] When a user launches the "Asare-kun" app on their smartphone or computer, user authentication is performed and the user ID is sent to the server.

[1424] Step 2: The user enters information.

[1425] Users fill out a form in the app with information about their background, personality, occupation, current weather, and location, as well as the length and date and time of the speech.

[1426] Step 3: The server receives the data.

[1427] The server receives the information sent by the user and temporarily stores it in a database, at which point the user ID and the information are linked.

[1428] Step 4: The server generates the speech script.

[1429] The server then activates a generative AI model based on the received information to automatically generate a speech manuscript, using past speeches and feedback information stored in a database.

[1430] Step 5: The server sends the manuscript to the user.

[1431] The generated speech manuscript is sent from the server to the user's terminal, where the user can view the speech manuscript on the terminal at the specified date and time.

[1432] Step 6: The user reviews and corrects the speech.

[1433] The user can review the speech manuscript they receive and make corrections within the app if necessary, and the corrections will be resubmitted to the server.

[1434] Step 7: The user gives a speech.

[1435] The user gives a speech based on the provided speech script, either using the generated speech script as is or modifying it as desired.

[1436] Step 8: User submits feedback.

[1437] After the speech, the user can enter feedback on the speech through the "Asarekun" app, detailing phrases that were particularly well-received and parts that received a poor response.

[1438] Step 9: The server receives the feedback.

[1439] The server receives feedback from users and stores this feedback information in a database.

[1440] Step 10: The server optimizes the generated AI model.

[1441] The server updates and optimizes the generative AI model based on the received feedback, so that the feedback is reflected in the next speech generation.

[1442] Step 11: User requests translation and checking.

[1443] If necessary, users can specify the translation and language of the speech manuscript, as well as the nationality and religious leanings of the audience, within the app.

[1444] Step 12: The server performs the translation and diversity check.

[1445] The server translates the speech manuscript into the specified language and checks the content from a diversity and inclusion perspective to ensure it does not contain discriminatory language.

[1446] Step 13: The server sends the translation result.

[1447] The server sends the translated speech manuscript to the user's terminal, where the translation result is displayed for the user to check.

[1448] Step 14: The user shares the translation manuscript.

[1449] The translated speech manuscript is generated as a QR code on the device, which the user can share with other participants.

[1450] The above is a specific flow of processing in the "Mr. Asakai" system of the present invention, which allows users to use efficient and high-quality speech manuscripts.

[1451] Example 1

[1452] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1453] Conventional speech support systems have the drawback of being time-consuming and labor-intensive, as the creation of speech manuscripts, translation, and feedback integration are all done manually. Furthermore, they lacked checking functions to accommodate different languages ​​and cultures, making them difficult to use in global business settings.

[1454] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1455] In this invention, the server includes means for receiving information on user history, personality, occupation, weather, and locality from the user, means for generating prompts using a generative AI model and creating a speech manuscript, data processing means for providing the generated speech manuscript to the user, means for receiving feedback from the user and optimizing the generation means, means for translating the speech manuscript into multiple languages ​​and performing a diversity check, and means for generating the translated speech manuscript as a QR code. This automates the process from generating the speech manuscript to providing it, incorporating feedback, and translating and checking it, enabling the efficient provision of high-quality speech manuscripts.

[1456] "User" refers to an individual or organization that intends to use the system to generate a speech manuscript.

[1457] "History" refers to information indicating a user's past experiences, such as career history, educational background, and work history.

[1458] "Personality" refers to information that indicates personal characteristics such as a user's hobbies and character.

[1459] "Occupation" refers to information indicating the job or specialty in which the user is engaged.

[1460] "Weather" refers to information indicating the weather conditions in the area where the speech will be given on the day.

[1461] "Regionality" refers to information that indicates the characteristics of the culture, customs, etc. of the place where the speech is given.

[1462] "Generative AI model" refers to an artificial intelligence model that automatically generates a speech manuscript based on information provided by the user.

[1463] A "prompt sentence" is an input sentence given to a generative AI model, and refers to text that guides the generation of a speech manuscript.

[1464] "Data processing means" refers to an information processing system for providing the generated speech manuscript to the user.

[1465] "Feedback" refers to information indicating reactions such as evaluations and impressions provided by users after they give a speech.

[1466] "Optimization methods" refer to techniques for receiving user feedback and improving the performance of generative AI models.

[1467] "Translation means" refers to a method or system for converting a speech transcript into a different language.

[1468] "Diversity check measures" refer to methods used to check whether translated speech manuscripts contain discriminatory language.

[1469] A "QR code" is a type of two-dimensional code, a graphic used to easily share speech manuscripts.

[1470] The present invention relates to a system for automating speechwriting functions and efficiently generating high-quality speech manuscripts. A specific method for implementing the present invention will be described below with reference to the following embodiments.

[1471] System Overview

[1472] This system is mainly composed of three elements: the user, the server, and the terminal. The user provides information, and the server generates a speech manuscript and provides it to the user through the terminal.

[1473] User operations

[1474] Enter information:

[1475] The user launches the "Speech Assistant" app and inputs necessary information such as background (e.g., career, educational background), personality (e.g., hobbies, character), occupation (e.g., engineer, teacher), current weather (e.g., sunny, rainy), and region (e.g., New York, Tokyo). In addition, the user specifies the length and date and time of the speech, and submits this information within the app.

[1476] Server Operation

[1477] Generate a speech transcript:

[1478] The server receives the information sent by the user. Based on the received information, it uses a generative AI model to generate a speech script. During this process, it also references past speech scripts and feedback information stored in a database to select the optimal content.

[1479] Examples:

[1480] For example, consider a company manager preparing a morning assembly speech for employees. The manager launches the "Speech Assistant" app and inputs information such as his or her background (career as a manager), personality (emphasis on leadership and teamwork), occupation (manager at an IT company), current weather (sunny in Tokyo), and regional characteristics (Japanese culture). The manager also specifies that the speech will last approximately five minutes.

[1481] Based on this information, the server uses a generative AI model to create an optimal speech script. For example, if the current weather is sunny, the script might include phrases such as, "I'm happy to be able to start a wonderful day with you all in this sunny weather today." Regional differences are also taken into consideration, and seasonal greetings and content that touches on unique Japanese customs are included.

[1482] Example prompt sentence:

[1483] "His background is 10 years in the IT industry, the weather is fine, and the speech is 5 minutes long."

[1484] Reflecting feedback:

[1485] The server receives feedback provided by users after they have used the speech. This feedback includes phrases that were well-received and parts that received poor responses. The server then incorporates this feedback into the generative AI model and uses it when generating the next speech manuscript.

[1486] Examples:

[1487] After the speech, the manager can provide feedback through the Speech Assistant app, such as, "The part introducing the new project was particularly well-received." The server then reflects this feedback and adjusts the AI ​​model to provide more appropriate content the next time the speech is generated.

[1488] Translation and Diversity Checks:

[1489] If necessary, the server translates the speech manuscript into multiple languages ​​as specified, and also checks the translation from the perspective of diversity and inclusion to ensure it does not contain discriminatory language.

[1490] Examples:

[1491] When giving a speech at an international conference, a manager can request that their speech manuscript be translated into English, Chinese, or other languages. The server translates the speech into the specified language and checks whether the content is appropriate from a diversity and inclusion perspective. The translation result is generated as a QR code, which the manager can easily share with other participants.

[1492] Device behavior

[1493] Manuscript provided by:

[1494] The generated speech transcript is sent to the user's device and displayed at the specified date and time. The user can review the displayed speech transcript and edit it as necessary. A QR code can be generated on the user's device for the translated speech transcript, allowing it to be easily shared with other participants.

[1495] Examples:

[1496] Users can check the generated speech manuscript through the "Speech Assistant" app and edit it as necessary. For example, when giving a speech at an international conference, the translated speech manuscript can be generated as a QR code and distributed to other participants, enabling smooth communication.

[1497] The above is a concrete example of how the present invention can be implemented. This system allows users to create high-quality speech manuscripts in a short amount of time, and can be used safely in global business situations.

[1498] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1499] Step 1:

[1500] Input and Action:

[1501] The user launches the "Speech Assistant" app and enters the necessary information, such as background, personality, occupation, weather, region, speech length, date and time, etc. The information is sent within the app by pressing the send button.

[1502] Data processing and output:

[1503] The information entered by the user is sent to the server in JSON format. The server receives this JSON data, analyzes the information stored in each field, and saves it as basic data for generating a speech manuscript.

[1504] Step 2:

[1505] Input and Action:

[1506] The server receives the JSON-formatted information sent by the user and parses it to extract information such as history, personality, occupation, weather, region, speech length, and date and time.

[1507] Data processing and output:

[1508] Based on the extracted information, a prompt sentence is generated for the generative AI model. The prompt sentence is input to the generative AI model in the form of "history is 10 years of experience in the IT industry, the weather is sunny, and the speech length is 5 minutes."

[1509] Step 3:

[1510] Input and Action:

[1511] The server inputs the generated prompt sentences into the generative AI model, which analyzes the prompt sentences and automatically generates a speech script.

[1512] Data processing and output:

[1513] The generative AI model also references past speech scripts and feedback information stored in a database to generate the optimal speech script, which is then temporarily stored on a server.

[1514] Step 4:

[1515] Input and Action:

[1516] When submitting feedback, the user can enter and submit their feedback after the speech through the "Speech Assistant" app. For example, they could enter feedback such as "The introduction to the new project was particularly well-received."

[1517] Data processing and output:

[1518] The server analyzes the feedback received from the user and reflects it in the generative AI model. The generative AI model is then readjusted based on the feedback information and used for generating the next speech manuscript.

[1519] Step 5:

[1520] Input and Action:

[1521] If the user requests multilingual translation, the server translates the speech into the specified language, for example, if it needs to be translated into English or Chinese.

[1522] Data processing and output:

[1523] The server translates the generated speech manuscript into the specified language, checks it from a diversity and inclusion perspective, verifies that the content is not discriminatory, and saves the translation results.

[1524] Step 6:

[1525] Input and Action:

[1526] Once the translation and checking process is complete on the server, the speech manuscript is sent to the user's device, where the user can check the displayed speech manuscript through the "Speech Assistant" app.

[1527] Data processing and output:

[1528] The speech manuscript displayed on the user's device can be viewed and edited by the user. In addition, a QR code can be generated on the user's device for the translated speech manuscript, allowing it to be easily shared with other participants.

[1529] The above is a specific flow of program processing for the system based on the claims. This system allows users to create high-quality speech manuscripts in a short amount of time, and to obtain deliverables that are efficient and can be used in multiple languages.

[1530] (Application example 1)

[1531] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1532] In today's world, opportunities for speeches and presentations are increasing, creating a demand for systems that can improve the quality and efficiency of speech drafts. Furthermore, with the advancement of globalization, multilingual support and cultural sensitivity are required, as well as a means for easily distributing generated speech drafts. However, existing systems lack the ability to provide individualized support based on the user's history and personality, and optimizing the system to reflect feedback is difficult. The objective of this invention is to solve these problems and provide high-quality speech support.

[1533] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1534] In this invention, the server includes means for receiving information from the user regarding background, personality, occupation, weather, and locality, data processing means for providing the generated speech transcript to the user, means for receiving user feedback and optimizing the generating means, means for performing translation and diversity checks, means for generating content translated into multiple languages ​​and taking social and cultural considerations into account, and means for outputting and distributing the generated speech transcript as a video or audio file, thereby enabling the generation of high-quality speech transcripts based on the user's specific information and the provision of content that is multilingual and culturally sensitive.

[1535] "User" refers to a person or organization that uses the system to provide information for generating a speech manuscript.

[1536] "Provenance" refers to information about a user's past experience, work and academic history, and other career information.

[1537] "Personality" refers to information about a user's personality, interests, hobbies, and personal characteristics.

[1538] "Occupation" refers to information about the job or work the user is currently doing.

[1539] "Weather" refers to information about the local weather conditions when the user delivers a speech.

[1540] "Regionality" refers to information about the cultural and social characteristics of the region where the user gives speech.

[1541] "Data processing means" refers to a computer processing device or software that generates a speech manuscript based on information provided by the user and provides it to the user.

[1542] A "generative AI model" is a model for generating speech manuscripts using artificial intelligence, and refers to an algorithm that generates optimal content based on past data and user-specified information.

[1543] "Feedback" refers to information such as reactions, evaluations, and areas for improvement provided by a user after they give a speech.

[1544] "Optimization means" refers to methods and algorithms for receiving feedback information and reflecting that information in the next speech script generation to improve the generative AI model.

[1545] "Translation and diversity check" refers to the process of converting the generated speech manuscript into multiple languages ​​and verifying that the content does not contain discriminatory language.

[1546] "Content" refers to the generated speech manuscript and multimedia information such as video and audio files created based on it.

[1547] A "video or audio file" is an audiovisual material created based on the generated speech manuscript, and is in a format for providing the speech content to users or viewers.

[1548] The present invention is a system for automatically generating a speech manuscript based on user information and providing the speech in various formats. The invention is specifically embodied in the following embodiments.

[1549] System Configuration

[1550] The system mainly consists of users, servers, and terminals.

[1551] User operations

[1552] The user starts the application using the smartphone and enters the following information:

[1553] Background (career, academic history, etc.)

[1554] Personality (hobbies, character, etc.)

[1555] Occupation (engineer, teacher, etc.)

[1556] Current weather (sunny, rainy, etc.)

[1557] Regional characteristics (region, culture, etc.)

[1558] Length and date of speech

[1559] This information is submitted by the user within the application.

[1560] Server Operation

[1561] The server has the following functions:

[1562] 1. Information reception: Receives information sent by the user.

[1563] 2. Speech generation: Based on the received information, a speech script is generated using a generative AI model (e.g., OpenAI GPT-4). During this process, past speech data and feedback information are referenced to select the optimal content.

[1564] 3. Reflecting feedback: The system receives feedback provided by the user after the speech and reflects that information in the generative AI model when generating the next speech manuscript.

[1565] 4. Translation and Diversity Check: Speech manuscripts will be translated into multiple languages ​​as needed and checked from a diversity and inclusion perspective to ensure they do not contain discriminatory language.

[1566] 5. Content generation: Based on the generated speech manuscript, video and audio files are created and made available for distribution by users.

[1567] Device behavior

[1568] The generated speech manuscript is sent to the user's device and can be viewed within the application. The user can edit it and check the final speech content. Furthermore, the generated content (video and audio files) can be easily distributed via social media, email, etc.

[1569] Program processing

[1570] The server-side program is built using Node.js and Express and handles all processes from receiving user information, generating speech scripts, incorporating feedback, translating and checking, and generating content.

[1571] Processing of received user information: The information is converted into an appropriate format and input into the generative AI model.

[1572] Speech generation: The generative AI model generates a speech based on the provided information. OpenAI's GPT-4 is used as the generative model.

[1573] Feedback Processing: After each speech, user feedback is received. This feedback is used in the next generation to improve the accuracy and adaptability of the model.

[1574] Translation and diversity check: The generated speech transcripts are translated into multiple languages ​​and checked for discriminatory content, making them available for global use.

[1575] Content generation: Video and audio files are generated based on the speech manuscript and made available to users.

[1576] Specific examples

[1577] For example, if a user provides the following information on the topic of "wedding speeches," the following prompt sentences will be used for the generative AI model:

[1578] Prompt statement:

[1579] "Generate speech under the following conditions:

[1580] Experience: Over 10 years of experience in the IT industry

[1581] Personality: Team player

[1582] Occupation: Systems Engineer

[1583] Weather: Sunny

[1584] Region: Tokyo

[1585] Length: 5 minutes

[1586] Please prepare the speech."

[1587] The generative AI model uses this information to generate a speech transcript and provides it to the user, who can then edit it and distribute it as a video or audio file.

[1588] The above is a specific embodiment of the present invention, which allows users to efficiently generate and provide highly personalized speech manuscripts.

[1589] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1590] Step 1:

[1591] The user starts the application on their smartphone and inputs information about their background, personality, occupation, weather, and locality. They also specify the length of the speech and the date and time. This information becomes the input data.

[1592] Step 2:

[1593] The terminal receives the information entered by the user and sends it to the server. Specifically, the data in the input form is converted to JSON format and sent to the server as an HTTP request. Once this data transmission is complete, the terminal displays a successful transmission status to the user.

[1594] Step 3:

[1595] The server generates a prompt based on the received user information. This prompt is used as input to the generative AI model. A speech script is generated based on the generated prompt. The prompt is generated by embedding user data in a template, which is then sent to the API of the generative AI model (e.g., OpenAI GPT-4).

[1596] Step 4:

[1597] The server receives the speech transcript data returned by the generative AI model and processes it into an appropriate format (e.g., text file, JSON). At this time, it also performs diversity checks and translation processing on the transcript content to generate a multilingual speech transcript. The processed data is stored in the DB system.

[1598] Step 5:

[1599] The generated speech script is sent to the terminal, which displays the received speech script to the user and provides an interface for the user to edit it as needed, allowing the user to review the speech content and provide feedback.

[1600] Step 6:

[1601] After the user finishes the speech, they input their feedback. The device then sends the feedback data to the server. This feedback includes phrases that were well-received and points for improvement.

[1602] Step 7:

[1603] The server processes the received feedback and refines the generative AI model. This refinement step uses machine learning algorithms to highlight the most effective parts and incorporate them into the next speech generation, thereby improving the quality of the generated speech transcript.

[1604] Step 8:

[1605] The server generates the final speech manuscript as video and audio files. These files are then sent back to the device so that the user can distribute them via social media, email, etc. The device provides a distribution interface, allowing the user to share the content with other users.

[1606] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1607] The present invention relates to a system for supporting more effective speeches by combining a speech support system with an emotion engine to generate a speech manuscript that reflects the emotional state of a user. A specific method for implementing the present invention will be described below with reference to the following embodiments.

[1608] System Overview

[1609] The system is mainly composed of three main components: the user, the server, and the terminal. An emotion engine is also added to analyze the user's emotional information and reflect it in the speech manuscript.

[1610] User operations

[1611] Enter information:

[1612] The user launches the "Asarekun" app and enters necessary information such as background (e.g., career, educational background), personality (e.g., hobbies, character), occupation (e.g., engineer, teacher), current weather (e.g., sunny, rainy), and region (e.g., New York, Tokyo). In addition, the user specifies the length and date and time of the speech, and submits this information within the app.

[1613] Server Operation

[1614] Generate a speech transcript:

[1615] The server receives the information sent by the user and temporarily stores it in a database. It then activates a generative AI model based on the received information to automatically generate a speech manuscript. This process also references past speeches and feedback information stored in the database.

[1616] Leveraging the Emotion Engine:

[1617] The emotion engine analyzes the user's emotional information and adjusts the content of the generated speech. For example, if the user is nervous, it can include a message to relax them.

[1618] Reflecting feedback:

[1619] The server receives feedback provided by users after they have used the speech. The feedback includes phrases that were well-received and parts that received poor responses. The server then applies this feedback to the generative AI model and emotion engine, and uses it when generating the next speech manuscript.

[1620] Translation and Diversity Checks:

[1621] If necessary, the server translates the speech manuscript into multiple languages ​​as specified. It also checks the translation to ensure it does not contain discriminatory language from the perspective of diversity and inclusion. This checking function provides speech manuscripts that can be used safely in global business situations.

[1622] Device behavior

[1623] Manuscript provided by:

[1624] The generated speech transcript is sent to the user's device and displayed at the specified date and time. The user can review the displayed speech transcript and make any necessary corrections. A QR code can be generated on the user's device for the translated speech transcript, which can be easily shared with other participants.

[1625] Specific examples

[1626] For example, consider a situation where a teacher at a school is giving a speech at the end of the semester. The teacher launches the "Asarekun" app and enters information such as their background (career as a teacher), personality (how they interact with students), occupation (school teacher), current weather (sunny), and regional characteristics (Japanese school culture). They also specify that the speech will last approximately 10 minutes.

[1627] The server generates a speech script based on this information. For example, it includes a phrase such as, "I am happy to be able to conclude this wonderful semester with you all today in this beautiful sunny weather," taking into account the weather. It also includes content that takes into account the local area and school culture.

[1628] Additionally, an emotion engine analyzes the teacher's emotions. For example, if the teacher is emotional, the speech will be tailored to include a warm message that reflects that emotion.

[1629] After the speech, the teacher can input feedback through the "Assembling-kun" app. For example, the teacher might say, "There was a part that the students particularly enjoyed." The server then applies this feedback to the generative AI model and emotion engine, providing more appropriate content for the next speech draft.

[1630] Furthermore, when giving a speech at an international conference, teachers can specify the translation and language of the manuscript, as well as the nationality and religious leanings of the audience. The server translates the speech into the specified language and checks whether the content complies with diversity and inclusion standards. The translation result is generated as a QR code, which teachers can easily share with other participants.

[1631] The above is a specific embodiment of the speech support system that combines the emotion engine of the "Mr. Asarai" system of the present invention. This system combines the functions of user input, speech generation, emotion analysis, feedback reflection, translation, and checking to achieve efficient, high-quality speech support.

[1632] The processing flow will be explained below.

[1633] Step 1:

[1634] The user launches the "Asare-kun" app.

[1635] When a user launches the "Asare-kun" app on their smartphone or computer, user authentication is performed and the user ID is sent to the server.

[1636] Step 2:

[1637] The user enters the information.

[1638] Users enter information such as their background (e.g., career, educational background), personality (e.g., hobbies, character), occupation (e.g., engineer, teacher), current weather (e.g., sunny, rainy), and location (e.g., New York, Tokyo) into the app's input form. They also specify the length and date and time of the speech.

[1639] Step 3:

[1640] The server receives the data.

[1641] The server receives the information sent by the user and temporarily stores it in a database, at which point the user ID and the information are linked.

[1642] Step 4:

[1643] The device activates the emotion engine.

[1644] The device activates an emotion engine to recognize the user's emotions, analyzing the user's facial expressions and tone of voice to obtain emotional information.

[1645] Step 5:

[1646] The server generates a speech script.

[1647] The server then activates a generative AI model based on the received information and the emotional information obtained from the emotion engine, and automatically generates a speech manuscript. This process also utilizes past speeches and feedback information stored in a database.

[1648] Step 6:

[1649] The server sends the manuscript to the user.

[1650] The generated speech manuscript is sent from the server to the user's terminal, where the user can view the speech manuscript on the terminal at the specified date and time.

[1651] Step 7:

[1652] The user reviews and corrects the speech.

[1653] The user can review the speech manuscript they receive and make corrections within the app if necessary, and the corrections will be resubmitted to the server.

[1654] Step 8:

[1655] The user gives a speech.

[1656] The user gives a speech based on the provided speech script, either using the generated speech script as is or modifying it as desired.

[1657] Step 9:

[1658] The device analyzes the emotions expressed by the user during speech.

[1659] During speech, the device uses an emotion engine to analyze the user's facial expressions and tone of voice and obtains the results in real time.

[1660] Step 10:

[1661] The user submits feedback.

[1662] After the speech, the user can enter feedback on the speech through the "Asarekun" app, detailing phrases that were particularly well-received and parts that received a poor response.

[1663] Step 11:

[1664] The server receives the feedback.

[1665] The server receives feedback from users and stores this feedback information in a database.

[1666] Step 12:

[1667] The server optimizes the generative AI model and emotion engine.

[1668] The server updates and optimizes the generation AI model and emotion engine based on the received feedback and emotion analysis results, so that the feedback content and emotion information are reflected in the next speech generation.

[1669] Step 13:

[1670] The user requests translation and review.

[1671] If necessary, users can specify the translation and language of the speech manuscript, as well as the nationality and religious leanings of the audience, within the app.

[1672] Step 14:

[1673] The server performs the translation and diversity check.

[1674] The server translates the speech manuscript into the specified language and checks the content from a diversity and inclusion perspective to ensure it does not contain discriminatory language.

[1675] Step 15:

[1676] The server sends the translation results.

[1677] The server sends the translated speech manuscript to the user's terminal, where the translation result is displayed for the user to check.

[1678] Step 16:

[1679] Users share translation manuscripts.

[1680] The translated speech manuscript is generated as a QR code on the device, which the user can share with other participants.

[1681] The above is a specific processing flow in an embodiment of the present invention in which the "Morning Assembly" system is combined with an emotion engine, allowing users to use efficient and high-quality speech manuscripts.

[1682] Example 2

[1683] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1684] Conventional speech support systems do not adequately consider the user's emotions or feedback and can only provide uniform content, making it difficult to provide effective speech manuscripts that meet the user's needs.In addition, they do not perform multilingual translation or diversity checks, which limits their use in international settings.

[1685] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1686] In this invention, the server includes means for receiving information from a user regarding their background, personality, occupation, weather, and regional characteristics, means for generating a speech manuscript using a generative AI model based on the received information, means for analyzing the user's emotional information with an emotion engine for the generated speech manuscript and adjusting the manuscript content, means for receiving feedback from the user and optimizing the generation means, means for performing translation and diversity checks, and means for delivering speech manuscripts translated into multiple languages ​​to users via QR codes. This makes it possible to provide effective speech manuscripts that reflect the user's emotions and feedback, and the multilingual support and diversity checks realize a speech support system that can be used in international settings.

[1687] A "user" is a person or organization that provides the information necessary to generate a speech transcript using the system.

[1688] "History" refers to all information about a user's career and background.

[1689] "Personality" refers to personal characteristics such as a user's character and hobbies.

[1690] "Occupation" refers to information about the user's job or area of ​​expertise.

[1691] "Weather" refers to the weather conditions on the day the speech is given and in the area.

[1692] "Regionality" refers to information about the location and cultural background of the user's speech.

[1693] A "generative AI model" refers to an artificial intelligence model that automatically creates a speech manuscript based on information entered by the user.

[1694] An "emotion engine" refers to software or a system that analyzes a user's emotions and reflects the results in a speech manuscript.

[1695] "Feedback" refers to information based on evaluations and reactions provided by users after they give a speech.

[1696] A "diversity check" refers to the process of checking whether a speech manuscript contains discriminatory language.

[1697] "QR code" refers to a two-dimensional barcode that allows you to easily share your speech manuscript with other users.

[1698] The present invention relates to a system for supporting more effective speeches by combining a speech support system with an emotion engine to generate a speech manuscript that reflects the user's emotional state. A specific method for implementing the present invention will be described below.

[1699] The system is mainly composed of three main components: the user, the server, and the terminal. An emotion engine is also added to analyze the user's emotional information and reflect it in the speech manuscript.

[1700] User operations

[1701] The user launches the "Asarekun" app and enters information about themselves through the interface, such as their background (career history), personality (characteristics and hobbies), occupation (job and specialty), current weather, and locality. They also specify the length and date and time of the speech, and submit this information.

[1702] Server Operation

[1703] 1. Receiving and storing information:

[1704] The server receives the information sent by the user and temporarily stores it in a database. The specific database used is a relational database management system (RDBMS), such as MySQL or PostgreSQL.

[1705] 2. Generate speech transcript:

[1706] Based on the received information, the server runs a generative AI model, such as OpenAI's GPT-4, to automatically generate a speech script. During this process, the following prompt is used:

[1707] Generate your speech. Consider the following information:

[1708] Biography: Teaching career

[1709] Personality: How you interact with students

[1710] Occupation: School teacher

[1711] Weather: Sunny

[1712] Regional Characteristics: Japanese School Culture

[1713] Speech length: 10 minutes

[1714] 3. Leveraging the Emotion Engine:

[1715] The generated speech is then analyzed using an emotion engine, such as IBM Watson's Tone Analyzer, to analyze the user's emotional information and adjust the content of the speech. For example, if the teacher is moved, a warm message reflecting that emotion is added.

[1716] 4. Receiving and storing feedback:

[1717] The system receives feedback provided by users after they have used the speech and stores it in a database. The feedback information is reflected in the next speech generation to further improve the quality of the speech manuscript.

[1718] 5. Translation and Diversity Check:

[1719] If necessary, the server translates the speech manuscript into multiple languages, possibly using translation software such as Google Translate. It also incorporates an algorithm to check whether the translation is acceptable from a diversity and inclusion perspective.

[1720] Device behavior

[1721] 1. Submission of manuscript:

[1722] The generated speech manuscript is sent to the user's terminal and displayed at the specified date and time. The user can check the displayed speech manuscript and make corrections as necessary.

[1723] 2. Generate and share your QR code:

[1724] A QR code is generated on the user's device for the translated speech manuscript, which can be easily shared with other participants. This QR code can be generated using a general QR code generation library.

[1725] Specific examples

[1726] For example, consider a situation where a teacher at a school is giving a speech at the end of the semester. The teacher launches the "Mr. Asareikun" app and enters information such as "Career as a teacher," "Interaction with students," "Occupation: School teacher," "Weather: Sunny," and "Region: Japanese school culture." The teacher also specifies that the speech will last approximately 10 minutes.

[1727] The server uses this information to generate a speech using a generative AI model, including phrases like, "I'm happy to be able to conclude this wonderful semester with you all today in this beautiful, sunny weather," and adds a warm, moving message using an emotion engine.

[1728] After the speech is completed, the teacher can send feedback through the app, such as, "There was a part that the students particularly enjoyed." This feedback information is stored in the database and reflected the next time the speech is generated.

[1729] When giving a speech at an international conference, teachers can specify the translation language of the manuscript and the nationality and religious orientation of the audience. The server translates the speech into the specified language and checks whether the content is appropriate from a diversity and inclusion perspective. The translation result is generated as a QR code that can be shared with other participants.

[1730] The speech support system of the present invention realizes efficient and high-quality speech support by linking the functions of user input, speech generation, emotion analysis, feedback reflection, translation, and checking.

[1731] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1732] Step 1:

[1733] Users launch the Asarekun app and enter information about their background, personality, occupation, weather, and locale through the interface.

[1734] Input: History, personality, occupation, weather, region, length and date of speech

[1735] Output: Sending input information

[1736] Specifically, the user enters each item into the app's form and presses the submit button to send the information to the server.

[1737] Step 2:

[1738] The server receives the information sent by the user and temporarily stores it in a database, such as a relational database management system like MySQL or PostgreSQL.

[1739] Input: Information submitted by the user

[1740] Output: Save to database

[1741] Specifically, the server receives the HTTP request, analyzes the information in the request body, and inserts it into a database.

[1742] Step 3:

[1743] Based on the information received, the server launches a generative AI model (such as GPT-4) to automatically generate a speech manuscript.

[1744] Input: User information stored in the database

[1745] Output: Generated speech transcript

[1746] Specifically, the following prompt sentence is input to the generative AI model:

[1747] Generate your speech. Consider the following information:

[1748] Biography: Teaching career

[1749] Personality: How you interact with students

[1750] Occupation: School teacher

[1751] Weather: Sunny

[1752] Regional Characteristics: Japanese School Culture

[1753] Speech length: 10 minutes

[1754] A generative AI model generates a speech script based on this prompt.

[1755] Step 4:

[1756] The server uses an emotion engine (such as IBM Watson's Tone Analyzer) to analyze the user's emotional information and adjust the content of the generated speech manuscript.

[1757] Input: Generated speech script, user's emotional information

[1758] Output: Adjusted speech transcript

[1759] Specifically, the emotion engine analyzes the emotional tone of the speech manuscript and makes adjustments based on the user's emotions (e.g., adding a warm message).

[1760] Step 5:

[1761] The server receives the feedback provided by the user after they give their speech and stores it in a database.

[1762] Input: User feedback

[1763] Output: Save feedback information to a database

[1764] Specifically, the information entered by the user into the feedback form through the app is sent to the server, which then inserts the information into a database.

[1765] Step 6:

[1766] If necessary, the server translates the speech manuscript into multiple languages ​​and performs a diversity check.

[1767] Input: Speech manuscript, specified language

[1768] Output: Translated speech manuscript, diversity checked content

[1769] Specifically, the server translates the speech manuscript using the Google Translate API or similar, and then applies an algorithm to check the content from a diversity and inclusion perspective.

[1770] Step 7:

[1771] The generated speech manuscript is sent to the user's device and displayed at the specified date and time.

[1772] Input: Adjusted and translated speech transcript

[1773] Output: Delivery to user terminal

[1774] Specifically, the server sends the speech manuscript to the user's terminal via an HTTP request or WebSocket, and configures the terminal to display it at the appropriate time.

[1775] Step 8:

[1776] A QR code is generated on the user's device for the translated speech manuscript, allowing it to be easily shared with other participants.

[1777] Input: Translated speech

[1778] Output: QR code

[1779] Specifically, the translated speech manuscript is encoded on the device using a QR code generation library (e.g., qrcode.js) to generate a QR code.

[1780] (Application example 2)

[1781] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1782] Conventional speech support systems simply generate speech scripts, but do not provide personalized support that reflects the user's emotional state. This makes it difficult to provide appropriate sales pitches in real time based on the user's emotions and the other person's reactions. In particular, in sales situations in physical stores, appropriate dialogue that reflects the customer's emotions is required, but conventional technology has had difficulty meeting this requirement.

[1783] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for receiving information on the user's history, personality, occupation, weather, and locality from the user; data processing means for providing the generated speech manuscript to the user; means for receiving feedback from the user and optimizing the generation means; means for reflecting the user's emotional information in the speech manuscript generated by the data processing means; means for providing the generated speech manuscript to the user in real time to support the user during the conversation; and means for performing translation and diversity checks. This makes it possible to generate a speech manuscript that takes the user's emotional state and feedback into consideration, thereby improving the quality of conversations in physical stores.

[1784] "History" is information about the user's career history, educational background, and past experiences.

[1785] "Personality" is information about a user's hobbies, character, and personal characteristics.

[1786] "Occupation" is information about the work or job the user is engaged in.

[1787] "Weather" is information about the weather conditions at the user's current location.

[1788] "Regionality" is information about the culture and customs specific to the area where the user lives or where the speech is given.

[1789] "Data processing means" refers to a processing device or program for generating a speech manuscript based on input information.

[1790] "Generation means" refers to an algorithm or system that creates a speech manuscript based on input information from the user.

[1791] "Means for reflecting emotional information" refers to a processing method or device for analyzing the user's emotional state and reflecting the results in the speech manuscript.

[1792] "Means for providing in real time" refers to a device or program that instantly displays the generated speech manuscript on the user's terminal and provides support during the conversation.

[1793] "Translation means" refers to a process or device for translating a generated speech transcript into a different language.

[1794] "Diversity check means" refers to a method or device for checking whether a generated speech manuscript is appropriate for people of diverse cultures and backgrounds.

[1795] A specific embodiment of the present invention will be described. This invention is realized by a speech support system that combines an emotion engine. The system is composed of a user, a server, and a terminal, and by adding an emotion engine, the system analyzes the user's emotion information and reflects it in the speech manuscript.

[1796] The server has a means to receive information provided by users about their background, personality, occupation, weather, and locality. The user inputs this information into a dedicated smartphone application. For example, when a salesperson introduces a new product on the shop floor, they input information about their background, hobbies, occupation, current weather, and local culture.

[1797] After receiving the information, the server uses data processing means to generate a speech manuscript based on the data. In this case, a generative AI model (e.g., GPT-4) is used to automatically generate the optimal speech manuscript, taking into account past data and feedback. The generated manuscript will initially be a speech with basic information.

[1798] The emotion engine then analyzes the speech and adapts it to reflect the user's emotional information. For example, if a salesperson is nervous, the system will adjust the speech script to include a message that will help them relax. The system uses the Affectiva SDK and Google Cloud Natural Language API to perform emotion analysis of speech and text.

[1799] The generated speech script is provided to the user's device (smartphone or tablet) in real time. This allows the salesperson to always present the most appropriate sales pitch while interacting with the customer. Below is an example of a prompt:

[1800] Is this product really right for me?

[1801] A feedback mechanism is also provided, allowing users to input their reactions to the speech into the application. For example, they can provide feedback such as, "This sales pitch was very effective." The server uses this feedback to optimize the generative AI model and emotion engine and reflect it in the next speech generation.

[1802] Additionally, the system also provides translation and diversity checks, which allow speeches to be translated into different languages ​​and checked for cultural and religious appropriateness. For example, if a speech is presented to an international audience, it can be translated and checked for discriminatory language.

[1803] By using this system, users can receive optimal conversation support in real time, improving the quality of conversations in physical stores.

[1804] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1805] Step 1:

[1806] The user launches a dedicated smartphone application and inputs information about their background, personality, occupation, weather, and locality. Based on this, initial customer information data is generated. This information is then sent to the server.

[1807] Input: history, personality, occupation, weather, region

[1808] Output: Initial customer information data

[1809] Specifically, the user enters the necessary information into the input form displayed on the application screen and presses the send button.

[1810] Step 2:

[1811] The server processes the initial customer information data received from the user and stores it in a database, which also stores past speech transcripts and feedback information.

[1812] Input: Initial customer information data

[1813] Output: Entries saved in the database

[1814] Specifically, the server analyzes the received data and stores it in a database in an appropriate format.

[1815] Step 3:

[1816] The server then runs a generative AI model (e.g., GPT-4) based on the stored data to automatically generate a speech manuscript, referencing past speeches and feedback information stored in a database during this process.

[1817] Input: Customer information data and historical data extracted from the database

[1818] Output: Initial speech manuscript

[1819] Specifically, the server inputs a prompt to the AI ​​model and retrieves the generated speech. For example, the prompt might be in the following format:

[1820] "Is this product really right for me?"

[1821] Step 4:

[1822] The server uses an emotion engine (e.g., Affectiva SDK or Google Cloud Natural Language API) to adapt the initial speech transcript to reflect the user's emotional information, extracted from additional information entered by the user and real-time speech analysis.

[1823] Input: Initial speech script and user's emotional information

[1824] Output: Emotionally-reflected speech transcript

[1825] Specifically, the emotion engine analyzes the user's voice data, extracts emotional elements (e.g., tension, excitement, relaxation), and reflects them in the content of the speech manuscript.

[1826] Step 5:

[1827] The speech manuscript reflecting the generated emotions is sent to the user's terminal in real time, and the user can use this speech manuscript to have a conversation with the customer.

[1828] Input: Speech manuscript reflecting emotions

[1829] Output: Speech manuscript displayed on the user's device

[1830] Specifically, the server sends the generated speech manuscript to the user's smartphone and displays it on the application screen.

[1831] Step 6:

[1832] After finishing their speech, users can input feedback through the application, which the server receives and uses to optimize the generative AI model and emotion engine when generating the next speech manuscript.

[1833] Input: User feedback

[1834] Output: Feedback information added to the database

[1835] Specifically, the user enters their thoughts and suggestions for improvement into the application's feedback form and presses the submit button. The server receives this and adds it to the database.

[1836] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1837] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1838] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1839] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1840] FIG. 9 illustrates an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and behaviors arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1841] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1842] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1843] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1844] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1845] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1846] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1847] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1848] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1849] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1850] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1851] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1852] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1853] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1854] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1855] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1856] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1857] The following is further disclosed regarding the above embodiment.

[1858] (Claim 1)

[1859] means for receiving biographical, personality, occupation, weather, and locality information from a user;

[1860] data processing means for providing the generated speech transcript to a user;

[1861] means for receiving feedback from a user and optimizing said generating means;

[1862] a means of translation and diversity check;

[1863] A speech assistance system including:

[1864] (Claim 2)

[1865] 10. The speech support system of claim 1, further comprising means for adjusting the speech manuscript based on a length and date and time of the speech specified by the user.

[1866] (Claim 3)

[1867] 10. The speech support system according to claim 1, further comprising means for delivering a speech manuscript translated into a plurality of languages ​​to the user by a QR code.

[1868] "Example 1"

[1869] (Claim 1)

[1870] means for receiving biographical, personality, occupation, weather, and locality information from a user;

[1871] A means for generating prompt sentences using a generative AI model and creating a speech manuscript;

[1872] data processing means for providing the generated speech transcript to a user;

[1873] means for receiving feedback from a user and optimizing said generating means;

[1874] A means of translating speech manuscripts into multiple languages ​​and conducting diversity checks;

[1875] A means for generating a QR code of the translated speech transcript;

[1876] A system including:

[1877] (Claim 2)

[1878] 10. The system of claim 1, further comprising means for adjusting the speech transcript based on a user-specified length and date and time of the speech.

[1879] (Claim 3)

[1880] 10. The system of claim 1, further comprising means for delivering the speech transcript translated into multiple languages ​​to the user via a QR code.

[1881] "Application Example 1"

[1882] (Claim 1)

[1883] means for receiving biographical, personality, occupation, weather, and locality information from a user;

[1884] data processing means for providing the generated speech transcript to a user;

[1885] means for receiving feedback from a user and optimizing said generating means;

[1886] a means of translation and diversity check;

[1887] A means to generate content translated into multiple languages ​​and take into account social and cultural considerations;

[1888] a means for outputting and distributing the generated speech manuscript as a video or audio file;

[1889] A system including:

[1890] (Claim 2)

[1891] 10. The system of claim 1, further comprising means for adjusting the speech transcript based on a user-specified length and date and time of the speech.

[1892] (Claim 3)

[1893] 10. The system of claim 1, further comprising means for delivering the speech transcript translated into multiple languages ​​to the user via a QR code.

[1894] "Example 2: Combining Emotion Engines"

[1895] (Claim 1)

[1896] means for receiving biographical, personality, occupation, weather, and locality information from a user;

[1897] means for generating a speech manuscript using a generative AI model based on the received information;

[1898] a means for analyzing the user's emotional information about the generated speech manuscript by an emotion engine and adjusting the manuscript content;

[1899] means for receiving feedback from a user and optimizing said generating means;

[1900] a means of translation and diversity check;

[1901] A means for distributing speech manuscripts translated into multiple languages ​​to users via QR codes;

[1902] A system including:

[1903] (Claim 2)

[1904] 10. The system of claim 1, further comprising means for adjusting the speech transcript based on a user-specified length and date and time of the speech.

[1905] (Claim 3)

[1906] 10. The system of claim 1, further comprising means for storing the feedback information in a database and reflecting the feedback information in the next speech draft generation.

[1907] "Application example 2 when combining emotion engines"

[1908] (Claim 1)

[1909] means for receiving biographical, personality, occupation, weather, and locality information from a user;

[1910] data processing...

Claims

1. means for receiving biographical, personality, occupation, weather, and locality information from a user; data processing means for providing the generated speech transcript to a user; means for receiving feedback from a user and optimizing said generating means; a means of translation and diversity check; A speech assistance system including:

2. 2. The speech support system of claim 1, further comprising means for adjusting the speech manuscript based on the length and date and time of the speech specified by the user.

3. 2. The speech support system according to claim 1, further comprising means for delivering a speech manuscript translated into a plurality of languages ​​to the user by a QR code.

Images