System

Abstract

To provide a system.SOLUTION: The system includes: means for converting meeting recording data to text data by AI; summarizing the converted text data by a generation system AI; recognizing the emotion state of the user from the recording data; and means for generating an animation video on the basis of the summarized text data and the recognized emotion state of the user.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In traditional internal meetings, to accurately record the contents of a meeting and allow for subsequent reference and sharing, humans had to transcribe and summarize the recorded data. However, this process required time and effort, and because it was done by humans, there was a risk of subjectivity and misunderstanding. Furthermore, in order to communicate the summarized content to other members, it had to be explained again verbally or in writing, which made the communication of information inefficient. [Means for solving the problem]

[0005] This invention uses artificial intelligence to convert recorded data from internal meetings into text and then uses generative artificial intelligence to summarize it. Furthermore, the summarized text is automatically generated into an animated video, which presents the content of the meeting in a visually easy-to-understand format. This makes it possible to efficiently record and share the content of meetings and to communicate information clearly and quickly. [Brief explanation of the drawings]

[0006] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 2 is a sequence diagram showing a flow of processing in the data processing system according to the first embodiment of the first form example. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Embodiment 1. [Figure 13]FIG. 10 is a sequence diagram showing a processing flow of a data processing system in a second embodiment of the second form example. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 of Embodiment Example 2. [Figure 15] FIG. 10 is a sequence diagram showing the flow of processing in a data processing system according to a third embodiment of the third embodiment. [Figure 16] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 3 of Embodiment 3. [Figure 17] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the first embodiment of the first form example when an emotion engine is combined. [Figure 18] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Form Example 1 when an emotion engine is combined. [Figure 19] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the second embodiment of the second form example when an emotion engine is combined. [Figure 20] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 of Form Example 2 when an emotion engine is combined. [Figure 21] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the third embodiment of the third form example when an emotion engine is combined. [Figure 22] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 3 of Form Example 3 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0007] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0008] First, the terms used in the following description will be explained.

[0009] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, the processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (TENSOR PROCESSING UNIT (registered trademark)).

[0010] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0011] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0012] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0013] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0014] [First embodiment]

[0015] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0016] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0017] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0018] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0019] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0020] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0021] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0022] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0023] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0024] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0025] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0026] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0027] "Example 1"

[0028] This embodiment of the present invention implements a system that uses artificial intelligence to convert recorded data from an internal meeting into text. Specifically, the system receives the recorded data of the meeting as input and converts it into text using speech recognition technology. This speech recognition technology can use a speech recognition model based on deep learning.

[0029] "Example 2"

[0030] Next, the textual data is summarized using generative artificial intelligence. Specifically, the textual data is received as input and summarized using natural language processing technology. This natural language processing technology can use a text generation model based on deep learning.

[0031] "Example 3"

[0032] Finally, the summarized text is automatically converted into an animated video. Specifically, the summarized text is received as input and converted into an animated video using video generation technology. This video generation technology can use a video generation model based on deep learning. As a specific example, the summarized text "The project is progressing smoothly" can be converted into an animated video depicting project members working.

[0033] The processing flow of each embodiment will be described below.

[0034] "Example 1"

[0035] Step 1: Receive the recording data of the meeting as input. This recording data is the audio data of an internal meeting.

[0036] Step 2: The recorded data is converted into text using speech recognition technology. This speech recognition technology uses a deep learning-based speech recognition model. This model is trained from a large amount of audio data and its associated text data.

[0037] "Example 2"

[0038] Step 1: Receive textual data as input. This data is the content of a meeting that has been converted into text using speech recognition technology.

[0039] Step 2: Summarize the text data using natural language processing technology. This natural language processing technology uses a text generation model based on deep learning. This model is trained from large amounts of text data and has the ability to generate summaries of the text.

[0040] "Example 3"

[0041] Step 1: Receive a summarized text as input. This text is the content of a meeting summarized using natural language processing technology.

[0042] Step 2: Using video generation technology, the summarized text is converted into an animated video. This video generation technology uses a deep learning-based video generation model. This model is trained from a large amount of video data and its associated text data, and has the ability to generate corresponding videos from text.

[0043] Step 3: For example, convert the summary sentence "The project is progressing smoothly" into an animated video showing project members working. This video is intended to present the content of the meeting in a visually easy-to-understand way.

[0044] Example 1

[0045] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0046] There is a need to efficiently convert recordings of internal meetings into text, summarize the text, and automatically generate animated videos that are easy to understand visually. However, in conventional systems, the entire process from receiving and saving the recordings, calling up the speech recognition model, generating and saving the text data, and providing access to the user was not automated, and manual operations were required. This resulted in issues such as a lack of efficiency and accuracy.

[0047] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0048] In this invention, the server includes means for converting recorded data of an internal meeting into text using artificial intelligence, means for summarizing the textual data using generative artificial intelligence, means for automatically generating an animated video from the summarized text, means for receiving and saving the recorded data, means for calling a speech recognition model and inputting the recorded data to generate text data, and means for saving the generated text data and making it accessible to users. This automates the entire process from receiving the recorded data to generating, saving, and providing the text data to users, enabling efficient and accurate processing.

[0049] "Internal meeting recording data" refers to digital data that records the audio of meetings held within a company.

[0050] "Artificial intelligence" refers to the technology of data processing and decision-making that enables computer systems to mimic human intelligence.

[0051] "Means for converting text" refers to technology or devices for converting voice data into text data.

[0052] "Generative AI" is AI that has the ability to generate new data from existing data.

[0053] A "summarization tool" is a technique or device that extracts important information from a long piece of text and summarizes it in a short form.

[0054] "Means for automatically generating animated videos" refers to technology or devices that automatically create videos that are visually easy to understand based on text data.

[0055] "Means for receiving and storing recorded data" refers to technology or devices that receive audio data transmitted from an external source and store it in digital form.

[0056] A "speech recognition model" is an algorithm or program for analyzing speech data and generating corresponding text data.

[0057] The "means for generating text data" refers to a technique or device that creates text data from voice data using a voice recognition model.

[0058] "Means for storing the generated text data and making it accessible to users" refers to technologies and devices that store the generated text data in digital form and allow users to view or retrieve it.

[0059] This invention relates to a system that uses artificial intelligence to convert recorded data from an internal meeting into text, summarizes the text, and automatically generates an animated video that is visually easy to understand. A specific embodiment of this system is described below.

[0060] System configuration

[0061] Hardware and software used

[0062] Hardware: High-performance server (e.g., server equipped with GPU)

[0063] Software: speech recognition models (e.g., Google® Speech-to-Text API, DeepSpeech), generative artificial intelligence, animation generation software

[0064] Program processing

[0065] Subject: Server

[0066] 1. Receiving and storing recording data

[0067] The server receives the recording data of the internal meeting uploaded by the user. The received data is saved in the specified directory. For example, it is saved in the path " / uploads / meeting_recording_20231001.wav".

[0068] 2. Calling the speech recognition model

[0069] The server prepares an API to call the speech recognition model. For example, if you use the Google Speech-to-Text API, you set the API key and endpoint.

[0070] 3. Inputting the recorded data into the speech recognition model

[0071] The server inputs the saved recording data into the speech recognition model. Specifically, it sends an API request specifying the path of the recording data. For example, it sends " / uploads / MeetingRecordings_20231001.wav" to the Google Speech-to-Text API.

[0072] 4. Generating Text Data

[0073] The speech recognition model analyzes the recorded data and generates corresponding text data. For example, if the recorded data contains the sentence, "In today's meeting, we discussed the development schedule for a new product," the model generates text data with the same content.

[0074] 5. Saving the generated text data

[0075] The server stores the generated text data, for example, in a file named " / texts / meetingrecording_20231001.txt."

[0076] 6. Providing Access to Users

[0077] The server makes the generated text data accessible to the user. The user accesses " / texts / meetingrecording_20231001.txt" using a web browser to view the text data.

[0078] Specific examples

[0079] For example, if a recording of an internal meeting is uploaded to a server with the file name "Meeting_Recording_20231001.wav," the server inputs this file into a speech recognition model. The speech recognition model analyzes the recording and generates text data like the following:

[0080] Recording contents:

[0081] "At today's meeting, we discussed the development schedule for new products."

[0082] Generated text data:

[0083] "At today's meeting, we discussed the development schedule for new products."

[0084] Prompt Sentence Examples

[0085] Examples of prompts to be input to a generative AI model include:

[0086] "Please convert the following recording to text: Meeting Recording_20231001.wav"

[0087] Using this prompt, the server inputs the recorded data into a speech recognition model and begins the process of generating text data.

[0088] The above is a specific embodiment of the present invention. This system automates the entire process from receiving recorded data to generating text data, saving it, and providing it to the user, making it possible to process the data efficiently and accurately.

[0089] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0090] Step 1:

[0091] Users upload recordings

[0092] The user uploads the recording data of an internal meeting from their device to the server. Specifically, the user opens a web browser and accesses the system's upload page. They click the file selection button and select "Meeting Recording_20231001.wav" from their local disk. When they click the upload button, the recording data is sent to the server.

[0093] Input: Recording data file selected by the user (e.g. "Meeting Recording_20231001.wav")

[0094] Output: Recording data is sent to the server

[0095] Step 2:

[0096] The server receives and stores the recording data.

[0097] The server receives the recording data uploaded by the user. The received data is saved in the specified directory. For example, it is saved in the path " / uploads / meetingrecording_20231001.wav".

[0098] Input: Recording data sent by the user

[0099] Output: Recording data saved in the specified directory on the server

[0100] Step 3:

[0101] The server calls the speech recognition model

[0102] The server prepares an API to call the speech recognition model. For example, if you use the Google Speech-to-Text API, you set the API key and endpoint.

[0103] Input: Path to the recording data (e.g. " / uploads / MeetingRecording_20231001.wav")

[0104] Output: The speech recognition model is ready to be invoked

[0105] Step 4:

[0106] The server inputs the recording into a speech recognition model.

[0107] The server inputs the saved recording data into the speech recognition model. Specifically, it sends an API request specifying the path of the recording data. For example, it sends " / uploads / MeetingRecordings_20231001.wav" to the Google Speech-to-Text API.

[0108] Input: Path to the recording data (e.g. " / uploads / MeetingRecording_20231001.wav")

[0109] Output: The recording sent to the speech recognition model

[0110] Step 5:

[0111] The speech recognition model analyzes the recorded data and generates text data.

[0112] The speech recognition model analyzes the recorded data and generates corresponding text data. For example, if the recorded data contains the sentence, "In today's meeting, we discussed the development schedule for a new product," the model generates text data with the same content.

[0113] Input: The recording sent to the speech recognition model

[0114] Output: Generated text data

[0115] Step 6:

[0116] The server stores the generated text data.

[0117] The server stores the generated text data, for example, in a file named " / texts / meetingrecording_20231001.txt."

[0118] Input: Generated text data

[0119] Output: Text data saved in the specified directory on the server

[0120] Step 7:

[0121] The user checks the generated text data

[0122] The user accesses the server and checks the generated text data. Specifically, the user uses a web browser to access " / texts / MeetingRecording_20231001.txt" and view the text data.

[0123] Input: Path of the text data stored on the server

[0124] Output: Text data viewed by the user

[0125] (Application example 1)

[0126] Next, a description will be given of Application Example 1 of Embodiment Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0127] Conventional systems for converting recorded data from internal meetings into text can summarize the contents of meetings and generate animated videos to aid visual understanding, but they lack the functionality to convert recorded data from work instructions and maintenance meetings within factories into text in real time and provide it to workers and managers. This poses the problem of not being able to fully improve work efficiency or reduce errors within factories.

[0128] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0129] In this invention, the server includes means for converting recorded data of in-house meetings into text using artificial intelligence, means for summarizing the textual data using generative artificial intelligence, means for automatically generating the summarized text into an animated video, means for converting recorded data of work instructions and maintenance meetings within a factory into text in real time, and means for providing the real-time textual data to workers and managers. This makes it possible to convert the contents of work instructions and maintenance meetings within a factory into text in real time and provide it to workers and managers.

[0130] An "internal meeting" is a meeting held within a company or organization, where business progress, problems, future policies, etc. are discussed.

[0131] "Recorded data" means data in which sound is recorded in digital or analog format and stored in a format that allows for later playback and analysis.

[0132] "Artificial intelligence" refers to a computer system that mimics human intelligence and has capabilities such as learning, reasoning, and recognition.

[0133] "Texting" is the process of converting non-text data, such as audio or video, into text form.

[0134] "Generative AI" is AI that has the ability to generate new information and content from given data.

[0135] "Summarization" is the process of extracting important information from long text or data and summarizing it in a concise manner.

[0136] "Animation video" is a video format that expresses movement by displaying a series of consecutive images in sequence.

[0137] "Inside a factory" refers to the facility where products are manufactured or processed in the manufacturing or production industries.

[0138] "Work instructions" are specific instructions and procedures for carrying out a specific task.

[0139] A "maintenance meeting" is a meeting where discussions and plans are made regarding the maintenance and inspection of equipment and machinery.

[0140] "Real-time" refers to data processing and information provision occurring immediately, without delay.

[0141] "Workers" are workers who actually perform work in factories or on-site.

[0142] An "administrator" is a person responsible for running and supervising an organization or project.

[0143] A system for implementing this invention is configured as follows: First, a server acquires recorded data of internal meetings, work instructions in a factory, and maintenance meetings. Next, the server converts this recorded data into text using speech recognition technology. This speech recognition technology uses Wav2Vec2, a speech recognition model based on deep learning.

[0144] The server summarizes the text data using a generative AI. This generative AI is implemented using the Transformers library. The summarized text is then automatically generated into an animated video. This animated video is intended to present the summarized text in a visually easy-to-understand format.

[0145] The server also converts work instructions and recordings of maintenance meetings into text in real time and provides it to workers and managers. This real-time text conversion contributes to improving work efficiency and reducing errors.

[0146] The specific hardware used is a microphone and a computer, and the software used is Python (registered trademark), a speech recognition library, a Transformers library, and a Wav2Vec2 model.

[0147] For example, if you use recorded data from a maintenance meeting held in a factory, when you input this data into the server, the following text will be output.

[0148] "At today's maintenance meeting, it was confirmed that Machine A requires a routine inspection. The next inspection is scheduled for next Monday."

[0149] An example of a prompt sentence to input to the generative AI model is as follows:

[0150] "Please convert the following audio data to text: [path to audio data]"

[0151] This prompt sentence can be used to efficiently convert voice data into text.

[0152] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0153] Step 1:

[0154] The server acquires recorded data from internal meetings, work instructions in factories, and maintenance meetings. The input is the recorded data, and the output is an audio data file. Specifically, audio is recorded using a microphone and the data is saved in digital format.

[0155] Step 2:

[0156] The server converts the acquired recorded data into text using speech recognition technology. The input is a voice data file, and the output is text data. Specifically, a speech recognition library is used to convert the voice data into text. In this process, speech recognition is performed using the Wav2Vec2 model.

[0157] Step 3:

[0158] The server summarizes the text data using generative artificial intelligence. The input is text data, and the output is summarized text data. Specifically, it uses the Transformers library to extract important information from the text data and generate a summary.

[0159] Step 4:

[0160] The server automatically generates an animated video from the summarized text. The input is the summarized text data, and the output is an animated video file. Specifically, animation generation software is used to express the summarized text as a video in a visually easy-to-understand format.

[0161] Step 5:

[0162] The server converts factory work instructions and recordings of maintenance meetings into text in real time. The input is real-time voice data, and the output is real-time text data. Specifically, it performs real-time voice recognition and instantly converts it into text.

[0163] Step 6:

[0164] The server provides documented data to workers and managers in real time. The input is real-time text data, and the output is text information displayed on the terminals of the workers and managers. Specifically, the text data is sent via the network and displayed on the terminals of the workers and managers.

[0165] Example 2

[0166] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0167] In conventional systems, the process of transcribing recordings of internal meetings and summarizing them was often done manually, which was time-consuming and labor-intensive. Furthermore, there was a lack of a way to present the summarized text in a visually easy-to-understand format. Furthermore, there was no system that could efficiently summarize text entered by users.

[0168] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0169] In this invention, the server includes means for converting recorded data of an internal meeting into text using artificial intelligence, means for summarizing the textual data using generative artificial intelligence, means for automatically generating an animated video from the summarized text, means for a terminal to send text entered by a user to the server, means for the server to preprocess the received text and input a prompt sentence to the generative artificial intelligence, means for the generative artificial intelligence to generate a summary and the server to send the summary result to the terminal, and means for the terminal to display the summary result to the user. This automates the process from converting recorded data into text to summarizing and visual representation, enabling efficient summarization of text entered by a user.

[0170] "Internal meeting recording data" refers to data in which the content of statements made during a meeting is recorded in audio format.

[0171] "Artificial intelligence" refers to computer systems that mimic human intelligence and can learn and reason.

[0172] "Texting" is the process of converting audio data or other forms of data into text form.

[0173] "Generative AI" is AI that has the ability to generate new text or information based on input data.

[0174] "Summarization" is the process of shortening long sentences or data and extracting only the important information.

[0175] "Animation video" is a video format that uses a series of images or videos to express movement.

[0176] A "terminal" is a device such as a computer or smartphone that is operated by a user.

[0177] A "server" is a computer system that processes and stores data on a network.

[0178] "Preprocessing" is the initial stage of processing to convert data into a format that is easier to analyze and process.

[0179] A "prompt sentence" is an input sentence that causes a generative artificial intelligence to perform a specific task.

[0180] The "summarization result" is the shortened text that is generated through the summarization process.

[0181] "User" refers to a person who uses the system.

[0182] This invention is a system that uses artificial intelligence to convert recorded data from internal meetings into text, then uses generative artificial intelligence to summarize the text, and then automatically generates an animated video from the summarized text. It also includes a function to efficiently summarize text entered by the user.

[0183] Hardware and software used

[0184] Server: A server equipped with a high-performance GPU

[0185] Software: OpenAI (registered trademark) GPT-3 (registered trademark), BERT, Python, Flask (web framework)

[0186] Device: A computer or smartphone that the user operates.

[0187] Data processing and calculation

[0188] Server Processing

[0189] The server receives text data provided by the user. This data is text stored in a text file or database. The server then uses a deep learning-based text generation model to summarize the received data using natural language processing techniques. Examples of such models include OpenAI's GPT-3 and BERT.

[0190] The server preprocesses the received text, tokenizing and normalizing it. For example, it uses NLTK or spaCy to tokenize the text and remove unnecessary whitespace and special characters. The preprocessed text is then converted into a format that can be input to the generative artificial intelligence.

[0191] Terminal handling

[0192] The terminal is responsible for transmitting the text data entered by the user to the server. When the user enters text through the terminal interface and presses the send button, the terminal transmits the data to the server. The terminal also displays the summary results returned from the server to the user.

[0193] User Action

[0194] The user inputs text data using the terminal. For example, the user inputs "Please summarize this text" and provides the specific text. The user can check the summary results displayed on the terminal and request another summary if necessary.

[0195] Specific examples

[0196] Prompt Sentence Examples

[0197] Examples of prompts that users can input into a generative AI model include:

[0198] "Please summarize the following sentence: 'This system is designed to summarize sentences entered by users. Specifically, it uses a deep learning-based text generation model to generate short summaries of the entered sentences.'"

[0199] In this way, the user can utilize a system that allows them to easily summarize text.

[0200] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0201] Step 1:

[0202] The user inputs a sentence.

[0203] A user inputs a sentence to be summarized through the terminal interface. For example, the user might input, "This system is designed to summarize the sentence entered by the user. Specifically, it uses a text generation model based on deep learning to summarize the entered sentence in a short form." The input data is in text format.

[0204] Step 2:

[0205] The terminal sends the input text to the server.

[0206] The terminal sends the text entered by the user to the server. Specifically, it uses an HTTP request to send the text data to the server. For example, if you are using Flask, you can send a POST request using the requests library. The input is the text data entered by the user, and the output is what is sent to the server.

[0207] Step 3:

[0208] The server receives the text and performs preprocessing.

[0209] The server receives text sent from the terminal. The received text is tokenized and normalized as preprocessing. For example, NLTK or spaCy is used to tokenize the text and remove unnecessary whitespace and special characters. The input is the text data received from the terminal, and the output is the preprocessed text data.

[0210] Step 4:

[0211] The server inputs a prompt sentence into the generative AI model.

[0212] The server converts the preprocessed text into a format that can be input to the generative AI model. Specifically, it creates a prompt sentence and inputs it to the generative AI model. For example, it creates a prompt sentence such as, "Please summarize the following sentence: 'This system is designed to summarize sentences entered by users. Specifically, it uses a text generation model based on deep learning to summarize the input sentence in a concise manner.'" The input is the preprocessed text data, and the output is the prompt sentence.

[0213] Step 5:

[0214] A generative AI model generates the summary.

[0215] The generative AI model analyzes the input prompt sentence and generates a summary. For example, if OpenAI's GPT-3 is used, an API is called to generate the summary. The generated summary will look something like, "This system summarizes the sentence entered by the user using a deep learning model." The input is the prompt sentence, and the output is the summarized text data.

[0216] Step 6:

[0217] The server sends the summary results to the terminal.

[0218] The server sends the generated summary to the terminal. Specifically, it returns the summary result using an HTTP response. For example, if you are using Flask, you can use the jsonify function to return the summary result in JSON format. The input is the summarized text data, and the output is what is sent to the terminal.

[0219] Step 7:

[0220] The terminal displays the summary results to the user.

[0221] The terminal displays the summary results received from the server to the user. Specifically, HTML and JavaScript (registered trademark) are used to display the summary results on the user interface. For example, the summary results are displayed in a text area or a dialog box. The input is the summary results received from the server, and the output is what is displayed to the user.

[0222] (Application example 2)

[0223] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0224] In today's information society, a huge amount of information is generated every day. In particular, text data such as internal meetings and news articles contain a large amount of information, making it difficult to quickly grasp important information. In addition, there is a lack of means to present this information in a visually easy-to-understand format, so efficient information processing and understanding are required.

[0225] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for converting recorded data of an internal meeting into text using artificial intelligence, means for summarizing the textual data using generative artificial intelligence, means for visually displaying the summarized text, and means for providing the summarized text to the user. This makes it possible to quickly extract important information from huge amounts of text data and provide it in a visually easy-to-understand format.

[0226] "Internal meeting recording data" refers to data that records the audio of meetings held within a company or organization.

[0227] "Artificial intelligence" refers to computer systems that mimic human intelligence and can learn and reason.

[0228] "Means for converting text" refers to technology or devices for converting voice data into text data.

[0229] "Generative AI" is AI that has the ability to generate new text based on input data.

[0230] A "summarization method" is a technique or device for extracting important information from long text data and summarizing it in a concise manner.

[0231] A "visual display means" is any technique or device for displaying text data in a graphical format.

[0232] "Means for providing to the user" refers to the technology or device used to deliver the generated summary or visual display to the user.

[0233] The system for implementing this invention uses artificial intelligence to convert recorded data from internal meetings into text, then uses generative artificial intelligence to summarize the text, visually displaying the summarized text and providing it to the user.

[0234] System configuration

[0235] The system consists of the following main components:

[0236] 1. Means of converting recorded data into text: Speech recognition software to convert recorded data into text data. Specifically, we use Google Speech-to-Text API, etc.

[0237] 2. Generative AI: A generative AI model for summarizing textual data. Specifically, we use a model such as the BERT model using the Transformers library.

[0238] 3. Visual display means: A graphical user interface (GUI) for visually displaying the summarized text. Specifically, a web application using HTML / CSS and JavaScript.

[0239] 4. User delivery method: A notification system to provide users with summarized information, specifically via a smartphone app or web browser.

[0240] Program processing explanation

[0241] Transcription of recorded data

[0242] The server receives recordings of internal meetings and converts them into text using speech recognition software, using the Google Speech-to-Text API.

[0243] Documented data summary

[0244] Generative AI takes text data as input, extracts important information, and generates a summary. This process is done using the Transformers library.

[0245] Visual display of summarized data

[0246] The summarized text data is displayed in a visually understandable format, specifically through a web application using HTML / CSS and JavaScript.

[0247] Provision to users

[0248] Finally, the summarized information is provided to the user via a smartphone app or web browser.

[0249] Specific examples

[0250] For example, if a user selects "articles about Japan's economic growth," the system operates as follows:

[0251] 1. Transcription of audio recording: "Japan's economy continues to grow in 2023. Innovation in technology, in particular, is driving economic growth. The government has introduced new policies to support corporate innovation. This has led to the creation of many start-ups and the emergence of new business models. Furthermore, international cooperation is progressing, and Japanese technology is gaining recognition around the world."

[0252] 2. Summary of the written data: "Japan's economy will continue to grow in 2023, driven in particular by the technology sector. New government policies are encouraging the creation of new startups and promoting international collaboration."

[0253] 3. Visual display of summarized data: The summarized text is displayed on the web application.

[0254] 4. Provision to users: Users will be notified via a smartphone app.

[0255] Prompt Sentence Examples

[0256] "Summarize the following news article: Japan's economy continues to grow in 2023. Innovation in technology, in particular, is driving economic growth. The government has introduced new policies to support corporate innovation. This has led to the creation of many start-ups and the emergence of new business models. Furthermore, international cooperation is progressing, and Japanese technology is gaining recognition around the world."

[0257] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0258] Step 1:

[0259] The server receives the recording data of the internal meeting. The input is audio data, and the output is the raw audio data. This audio data is converted into text data in the next step.

[0260] Step 2:

[0261] The server uses speech recognition software (e.g., Google Speech-to-Text API) to convert the recorded data into text data. The input is voice data, and the output is text data. This conversion process turns the voice data into text.

[0262] Step 3:

[0263] The server uses generative artificial intelligence (e.g., the BERT model using the Transformers library) to summarize the text data. The input is text data, and the output is summarized text data. This summarization process extracts important information and summarizes it in a concise manner.

[0264] Step 4:

[0265] The server generates a graphical user interface (GUI) to visually display the summarized text data. The input is the summarized text data, and the output is visually displayable data (e.g., a web page using HTML / CSS and JavaScript). This display process makes it easier for users to visually understand the summarized information.

[0266] Step 5:

[0267] The server provides the user with visually displayed summary data. The input is visually displayable data, and the output is summary information displayed on the user's device. This provision process allows the user to receive the summary information through a smartphone app or web browser.

[0268] Step 6:

[0269] The user checks the provided summary information and obtains more detailed information as needed. The input is the user's operation, and the output is the display of detailed information. This checking process allows the user to efficiently grasp important information and obtain more detailed information as needed.

[0270] Example 3

[0271] Next, a description will be given of a third embodiment of the third embodiment. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0272] In the conventional method of managing meeting recording data, the recording data is converted into text and then summarized, which is time-consuming and laborious. Furthermore, there is a lack of a way to present the summarized text in a visually easy-to-understand format, which reduces the efficiency of information transmission. This makes it difficult to quickly and effectively share the contents of the meeting.

[0273] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[0274] In this invention, the server includes means for converting recorded data of an internal meeting into text using artificial intelligence, means for summarizing the textual data using generative artificial intelligence, means for automatically generating an animated video from the summarized text using a deep learning model, means for transmitting the generated animated video to a terminal, and means for displaying the received animated video to a user, thereby enabling the content of the meeting to be shared quickly and effectively.

[0275] "Internal meeting recording data" refers to data in which the content of statements made during a meeting is recorded in audio format.

[0276] "Artificial intelligence" is a technology that mimics human intelligence and automatically analyzes and processes data.

[0277] "Texting means" refers to techniques or methods for converting audio data into text format.

[0278] "Generative AI" is AI that has the ability to generate new information or summaries from input data.

[0279] "Methods of summarization" are techniques or methods for extracting important information from long pieces of text or data and summarizing it in a concise manner.

[0280] A "deep learning model" is a technology that uses multi-layered neural networks to analyze data and learn patterns.

[0281] "Animation video" is a video format that expresses movement by displaying a series of images.

[0282] "Means for automatic generation" refers to techniques or methods for automatically generating output data based on specific input data.

[0283] A "terminal" is a device operated by a user, and includes a computer, a smartphone, etc.

[0284] A "transmission medium" is a technique or method for transferring data from one place to another.

[0285] "Display means" refers to the techniques and methods used to visually present data to the user.

[0286] The present invention provides a system for efficiently managing recorded data of in-house meetings and providing summarized information in a visually easy-to-understand format. Specific embodiments of this system are described below.

[0287] Hardware and software used

[0288] This system is mainly composed of a server, terminals, and user operations. The specific hardware and software used are as follows:

[0289] Server: We use powerful computer servers to process and store data.

[0290] Terminal: A device operated by a user, including a PC or smartphone.

[0291] Artificial Intelligence: Artificial intelligence with speech recognition technology to convert recorded data into text.

[0292] Generative AI: AI that uses natural language processing techniques to summarize written data.

[0293] Deep learning model: A model for converting summarized text into animated videos.

[0294] System processing flow

[0295] 1. The user inputs the summarized text into the terminal.

[0296] The user inputs the summarized sentence into an input field on the terminal, for example, "The project is progressing smoothly."

[0297] 2. The device sends the input text to the server

[0298] The device sends the text entered by the user to the server as an HTTP request, converting the text into JSON format and sending it to the server's API endpoint.

[0299] 3. The server inputs the received sentences into the deep learning model

[0300] The server analyzes the JSON-formatted text received from the terminal and inputs it into the deep learning model.

[0301] 4. Deep learning models analyze text and generate animated videos

[0302] The deep learning model analyzes input sentences and generates animated videos based on their content. For example, based on the sentence "The project is progressing smoothly," it generates an animated video depicting project members working.

[0303] 5. The server sends the generated animation to the device.

[0304] The server receives the animation video generated by the deep learning model and sends it to the device using an HTTP response.

[0305] 6. The device displays the animation video to the user.

[0306] The terminal displays the animation received from the server to the user, who can then view the generated animation on the screen of the terminal.

[0307] Specific examples

[0308] The user inputs a summarized sentence, "The project is progressing smoothly," into the device. The device sends this sentence to the server. The server uses a deep learning-based video generation model to generate an animated video depicting the project members working. The generated video is then sent from the server to the device, where the user can view it.

[0309] Prompt Sentence Examples

[0310] Take the condensed sentence "The project is progressing smoothly" and turn it into an animated video.

[0311] In this way, the contents of the conference can be shared quickly and effectively. The flow of the identification process in the third embodiment will be described with reference to FIG.

[0312] Step 1:

[0313] The user inputs the summarized text into the terminal.

[0314] As a specific operation, the user enters "The project is progressing smoothly" into the input field of the terminal and clicks the send button.

[0315] Input: A summary of the sentence (e.g., "The project is progressing smoothly.")

[0316] Output: Text typed into the terminal

[0317] Step 2:

[0318] The terminal sends the input text to the server.

[0319] Specifically, the device converts the input text into JSON format and sends an HTTP POST request to the server's API endpoint.

[0320] Input: Text typed into the terminal

[0321] Output: JSON formatted document sent to the server

[0322] Step 3:

[0323] The server inputs the received sentences into the deep learning model.

[0324] Specifically, the server analyzes the received JSON data, extracts sentences, and inputs them into the deep learning model.

[0325] Input: JSON formatted text sent to the server

[0326] Output: The sentences fed into the deep learning model

[0327] Step 4:

[0328] A deep learning model analyzes text and generates animated videos.

[0329] Specifically, the deep learning model analyzes input sentences and generates animated scenes based on their content. For example, based on the sentence "The project is progressing smoothly," it generates a scene of project members working at their desks.

[0330] Input: The sentence fed into the deep learning model

[0331] Output: Generated animation video

[0332] Step 5:

[0333] The server transmits the generated animation to the terminal.

[0334] Specifically, the server encodes the generated animation video in MP4 format and sends it to the terminal as an HTTP response.

[0335] Input: Generated animation video

[0336] Output: Animated video in MP4 format sent to your device

[0337] Step 6:

[0338] The terminal displays the received animation video to the user.

[0339] Specifically, the device displays the received MP4 animation video in a player, and the user can play the video in a browser to view it.

[0340] Input: Animated video in MP4 format sent to the device

[0341] Output: Animated video displayed to the user

[0342] (Application example 3)

[0343] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0344] In conventional advertising production, creating visually appealing animated videos based on advertising copy requires a great deal of time and effort. Furthermore, because specialized knowledge and skills are required, it is difficult for advertisers to easily create videos themselves. This has led to issues such as reduced efficiency in advertising production and increased costs.

[0345] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[0346] In this invention, the server includes means for converting recorded data of an internal meeting into text using artificial intelligence, means for summarizing the text data using generative artificial intelligence, means for automatically generating an animated video from the summarized text, means for receiving the summarized advertising copy as input and generating an animated advertising video using an animation generation model based on deep learning, and means for displaying the generated animated advertising video. This makes it possible to automatically generate visually appealing animated advertising videos based on the summarized advertising copy provided by the advertiser.

[0347] "Internal meeting recording data" refers to audio recordings of meetings held within a company or organization.

[0348] "Artificial intelligence" is an information processing technology in which computer systems imitate human intelligence.

[0349] "Texting" is the process of converting audio data into text data.

[0350] "Generative AI" is AI that has the ability to generate new information and content from given data.

[0351] "Summarization" is the process of extracting important information from long text or data and summarizing it in a concise manner.

[0352] "Animation video" is a video format that uses a series of images to express movement.

[0353] "Deep learning" is a branch of machine learning that uses multi-layered neural networks to analyze data and learn patterns.

[0354] An "video generation model" is an algorithm or framework for generating video based on input data.

[0355] "Ad copy" is text created to promote a product or service.

[0356] An "animated advertising video" is a visually appealing animated advertisement generated based on advertising copy.

[0357] "Display" is the process of visually presenting generated content to a user.

[0358] A system for implementing the present invention has the following configuration.

[0359] System Configuration

[0360] 1. Hardware:

[0361] Server: A server equipped with a high-performance GPU

[0362] Devices: Smartphones, smart glasses

[0363] 2. Software:

[0364] Natural Language Processing (NLP) libraries (e.g., spaCy, NLTK)

[0365] Deep learning frameworks (e.g., TENSORFLOW (registered trademark), PyTorch)

[0366] Application Interface

[0367] Processing flow

[0368] 1. Receiving input data:

[0369] The user inputs the advertisement copy. For example, the advertisement copy is "The new smartphone is equipped with a high-performance camera."

[0370] 2. Data preprocessing:

[0371] The server uses a natural language processing library to tokenize the input ad copy and perform grammatical analysis.

[0372] 3. Application of video generation model:

[0373] The server passes the tokenized data to a video generation model pre-trained using a deep learning framework to generate animated advertising videos.

[0374] 4. Video output:

[0375] The generated animated advertising video is displayed to the user through the application interface of the terminal.

[0376] Specific examples

[0377] For example, if an advertiser enters the ad copy, "Our new smartphone is equipped with a high-performance camera," the system operates as follows:

[0378] 1. Receiving input data:

[0379] The user enters the ad copy through a smartphone application.

[0380] 2. Data preprocessing:

[0381] The server uses a natural language processing library to tokenize and parse the ad copy.

[0382] 3. Application of video generation model:

[0383] The server then inputs the tokenized data into a video generation model using a deep learning framework to generate animated advertising videos that highlight the smartphone's high-performance camera.

[0384] 4. Video output:

[0385] The generated animated advertising video is displayed to the user through the smartphone application interface.

[0386] In this way, it is possible to automatically generate visually appealing animated advertising videos based on the summarized advertising copy provided by the advertiser.

[0387] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[0388] Step 1:

[0389] The user inputs the advertisement copy through the smartphone application. For example, the advertisement copy might be, "Our new smartphone is equipped with a high-performance camera." This advertisement copy is then sent to the system.

[0390] Step 2:

[0391] The server receives the input ad copy and tokenizes it using a natural language processing library (e.g., spaCy, NLTK). The tokenized data is divided into each word and phrase of the sentence. Next, grammatical analysis is performed to analyze the structure of the sentence. This allows the meaning and important elements of the ad copy to be extracted.

[0392] Step 3:

[0393] The server then inputs the tokenized and parsed data into a pre-trained video generation model using a deep learning framework (e.g., TensorFlow, PyTorch). The video generation model generates animated ad videos based on the input data. The model creates appropriate visual effects and animations based on the content of the ad copy.

[0394] Step 4:

[0395] The generated animated advertising video is sent from the server to the terminal and displayed to the user through the terminal's application interface. The user can check the generated animated advertising video and modify or regenerate it as necessary.

[0396] In this way, it is possible to automatically generate visually appealing animated advertising videos based on the summarized advertising copy provided by the advertiser.

[0397] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0398] "Example 1"

[0399] One embodiment of the present invention is a system that combines an emotion engine. This system includes an emotion engine that recognizes emotions from a user's voice data. The emotion engine analyzes the acoustic characteristics and speaking patterns of the voice data and infers the user's emotional state from the results. Specifically, it analyzes characteristics of the voice data, such as pitch, volume, and speaking rate, and recognizes the emotional state indicated by these characteristics.

[0400] "Example 2"

[0401] The emotion engine also generates summaries based on the recognized emotional state. Specifically, if the user is recognized as being in a joyful or excited state, a positive summary reflecting that emotional state is generated. Conversely, if the user is recognized as being in a sad or angry state, a negative summary reflecting that emotional state is generated.

[0402] "Example 3"

[0403] Furthermore, the animation visually expresses the user's emotions. Specifically, if the user recognizes that they are in a happy emotional state, the animation is generated with bright colors and active movements that reflect that emotional state. Conversely, if the user recognizes that they are in a sad emotional state, the animation is generated with dark colors and quiet movements that reflect that emotional state.

[0404] The processing flow of each embodiment will be described below.

[0405] "Example 1"

[0406] Step 1: Collect voice data from the user.

[0407] Step 2: Analyze the acoustic characteristics and speaking patterns of the collected voice data.

[0408] Step 3: Estimate the user's emotional state from the analysis results.

[0409] "Example 2"

[0410] Step 1: Recognize the user's emotional state.

[0411] Step 2: Generate a summary based on the recognized emotional state.

[0412] Step 3: Present the generated summary to the user.

[0413] "Example 3"

[0414] Step 1: Recognize the user's emotional state.

[0415] Step 2: Generate an animated video that reflects the recognized emotional state.

[0416] Step 3: Present the generated animation video to the user.

[0417] Example 1

[0418] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0419] Conventional systems for transcribing recordings of internal meetings only provide a summary or visual representation of the meeting content, and are unable to grasp the emotional state of the speakers. This makes it difficult to understand the atmosphere of the meeting and the emotions of the speakers, making it difficult to accurately grasp the content of the meeting. Furthermore, because the generated report does not include the emotional state, there is a risk of missing important nuances in the meeting.

[0420] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0421] In this invention, the server includes means for converting recorded data of an internal meeting into text using artificial intelligence, means for summarizing the textual data using generative artificial intelligence, means for automatically generating animated videos from the summarized text, means for analyzing the acoustic characteristics of the audio data and speaking patterns to estimate emotional states, means for generating a report including the meeting content and emotional states, and means for providing the generated report to a user, thereby enabling an accurate understanding of the meeting content and the emotional states of speakers.

[0422] "Internal meeting recording data" refers to data that records the audio of an internal meeting.

[0423] "Artificial intelligence" is a technology that refers to the processing and judgment that computer systems perform by imitating human intelligence.

[0424] "Means of converting text" refers to techniques and methods for converting voice data into text data.

[0425] "Generative AI" is an AI technology that extracts important information from text data and generates summaries and new sentences.

[0426] "Methods of summarization" are techniques or methods for extracting important parts from long pieces of text or data and summarizing them in a concise manner.

[0427] "Means for automatically generating animated videos" refers to techniques and methods for automatically creating visually easy-to-understand animated videos based on text data.

[0428] "Acoustic characteristics of voice data" refers to acoustic features such as pitch, volume, and speaking rate contained in the voice data.

[0429] "Speaking patterns" refer to the characteristic rhythm, intonation, and word usage that can be seen in a speaker's speech.

[0430] "Means for estimating emotional state" refers to techniques or methods for estimating the speaker's emotions by analyzing the acoustic characteristics of speech data and speaking patterns.

[0431] The "means for generating a report" refers to a technique or method for creating a report including the meeting content and emotional state based on the results of speech recognition and emotion analysis.

[0432] The "means for providing to the user" refers to the technology or method for delivering the generated report to the user and making it viewable.

[0433] This invention is a system that uses artificial intelligence to convert recorded data from an internal meeting into text and analyze the emotional state of the participants. Specific embodiments of this system are described below.

[0434] First, the user uploads the recording of the internal meeting from their device to the server. The recording data is in audio file format such as WAV or MP3. The server receives the recording data uploaded by the user.

[0435] The server then converts the recording into text using speech recognition technology, such as Google Cloud Speech-to-Text API or IBM Watson® Speech to Text, which uses deep learning-based speech recognition models to convert the audio data into text with high accuracy.

[0436] The text data is then passed to an emotion engine. The server uses an emotion engine, such as the Emotion API from Microsoft® Azure® Cognitive Services, to analyze the user's emotional state from the text data. The emotion engine analyzes the acoustic characteristics of the voice data (such as pitch, volume, and speaking rate) and speaking patterns, and recognizes the emotional state indicated by these characteristics.

[0437] The server integrates the results of speech recognition and emotion analysis to generate a report containing the meeting content and the emotional state of each speaker. This report is generated in text or PDF format and provided to the user. The user can download the report via their device and check the meeting content and emotional state.

[0438] For example, the following prompts can be fed to a generative AI model:

[0439] "Upload recordings of internal meetings. The server uses speech recognition technology to convert the recordings into text, and then uses an emotion engine to analyze the emotional state of each speaker. Finally, it generates and provides a report showing the content and emotional state of the meeting."

[0440] In this way, the system processes while clarifying the roles of the server, terminal, and user. This system enables accurate understanding of the content of the meeting and the emotional state of the speaker, ensuring effective meeting management without overlooking important nuances.

[0441] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0442] Step 1:

[0443] A user uploads the recording data of an internal meeting from their device to the server.

[0444] Input: Recording data of internal meetings (audio file formats such as WAV and MP3)

[0445] Output: Audio file uploaded to the server

[0446] Specific operation: The user uses the file upload function of the device to send the recorded data to the server, and the server saves the received audio file in a specified directory.

[0447] Step 2:

[0448] The server receives the recorded data and begins the voice recognition process.

[0449] Input: Uploaded audio file

[0450] Output: Start signal for speech recognition processing

[0451] Specific operation: The server checks the existence of the received audio file and prepares to start the speech recognition process. Specifically, it checks the format of the audio file and converts it if necessary.

[0452] Step 3:

[0453] The server uses voice recognition technology to convert the recorded data into text.

[0454] Input: Audio file

[0455] Output: Text data

[0456] How it works: The server calls voice recognition software such as Google Cloud Speech-to-Text API or IBM Watson Speech to Text to convert the audio file into text, which is then temporarily stored on the server.

[0457] Step 4:

[0458] The server passes the converted data to the emotion engine.

[0459] Input: Text data

[0460] Output: Input data to the emotion engine

[0461] Specific operation: The server converts the text data into an appropriate format to be passed to the emotion engine. Specifically, the server converts the text data into JSON format or similar and sends it to the emotion engine's API.

[0462] Step 5:

[0463] The server uses an emotion engine to analyze the emotional state from the text data.

[0464] Input: Input data to the emotion engine

[0465] Output: Sentiment analysis results

[0466] How it works: The server uses an emotion engine, such as the Emotion API from Microsoft Azure Cognitive Services, to analyze the user's emotional state from the text data. The emotion engine analyzes the acoustic characteristics and speaking patterns of the audio data to estimate the user's emotional state. The analysis results are returned to the server.

[0467] Step 6:

[0468] The server generates a report containing the meeting content and emotional state.

[0469] Input: Speech recognition results, emotion analysis results

[0470] Output: Report (text or PDF format)

[0471] Specific operation: The server integrates the results of speech recognition and sentiment analysis to generate a report containing the meeting content and the emotional state of each speaker. The report is created in a specified format and saved on the server.

[0472] Step 7:

[0473] The server provides the generated report to the user.

[0474] Input: Generated report

[0475] Output: Report provided to user

[0476] Specific operation: The server generates a download link to provide the generated report to the user. The user can click the link through their device to download the report and check its contents.

[0477] (Application example 1)

[0478] Next, a description will be given of Application Example 1 of Embodiment Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0479] To efficiently issue work instructions and reports within a factory, it is necessary to convert voice instructions into text in real time and to recognize the emotional state of workers and provide appropriate feedback. However, current systems do not adequately convert voice instructions into text or recognize emotions, which creates issues with work efficiency and worker stress management.

[0480] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means. In this invention, the server includes means for converting recorded data of an internal meeting into text using artificial intelligence, means for summarizing the textual data using generative artificial intelligence, means for automatically generating animated videos from the summarized text, means for converting voice instructions into text in real time using a robot that issues work instructions and reports in the factory by voice, means including an emotion engine that recognizes emotional states from voice data, and means for providing appropriate feedback according to the emotional state. This enables work instructions and reports to be issued efficiently in the factory and appropriate feedback according to the emotional state of workers.

[0481] "Internal meeting recording data" refers to audio recordings of meetings held within a company.

[0482] "Artificial intelligence" is an information processing technology in which computer systems imitate human intelligence.

[0483] "Texting" is the process of converting audio data into text data.

[0484] "Generative AI" is AI that has the ability to generate new information and content from input data.

[0485] "Summarization" is the process of extracting important information from long text or data and summarizing it in a concise manner.

[0486] "Animation video" is a technology that generates moving images.

[0487] "Work instructions and reports within the factory" refers to instructions and progress reports regarding work carried out within the factory.

[0488] A "robot" is a mechanical device that performs work automatically.

[0489] "Real-time" refers to processing occurring immediately without delay.

[0490] An "emotion engine" is a technology that recognizes emotional states from voice data and other input data.

[0491] "Feedback" is the response or instruction that the system provides to the user.

[0492] The system for implementing this invention uses artificial intelligence to convert recorded data from in-house meetings into text, and then applies it to a robot that issues work instructions and reports in a factory by voice. A specific embodiment of this system will be described below.

[0493] System configuration

[0494] Hardware

[0495] 1. Microphone: Used to record the voice of the worker.

[0496] 2. Robot: Hardware for receiving voice instructions, transcribing them and recognizing emotions.

[0497] 3. Server: Processes and stores voice data.

[0498] software

[0499] 1. Google Cloud Speech-to-Text API: Used to convert speech to text.

[0500] 2. Microsoft Azure Emotion API: Used to recognize emotions from speech.

[0501] 3. Generative AI: Extracts important information from documented data and generates summaries.

[0502] 4. Emotion Engine: Analyzes the acoustic characteristics and speaking patterns of voice data to estimate emotional states.

[0503] Data processing and calculation

[0504] Recording audio data

[0505] When the user issues a voice command through the microphone, the robot records the voice data, which is then sent to the server.

[0506] Text conversion of voice data

[0507] The server converts the recording into text using the Google Cloud Speech-to-Text API, a process that transcribes voice commands in real time.

[0508] emotion recognition

[0509] The server uses the Microsoft Azure Emotion API to recognize emotional states from text data, and the emotion engine analyzes characteristics of the voice data, such as pitch, volume, and speaking rate, to estimate the user's emotional state.

[0510] Providing Feedback

[0511] The server provides appropriate feedback based on the user's emotional state. For example, if the user says, "I'm tired today," the emotion engine will interpret this as "fatigue" and provide feedback such as, "Please take a break."

[0512] Specific examples

[0513] When a worker says to the robot, "I'm tired today," the robot records the voice and translates it into a sentence in real time. It then uses emotion recognition to determine that the worker is "tired," and provides feedback such as, "Please take a break."

[0514] Prompt Sentence Examples

[0515] When a worker says to the robot, "I'm tired today," the robot records the voice and converts it into text in real time. It then uses emotion recognition to determine that the worker is "fatigued," and provides feedback such as, "Please take a break."

[0516] In this way, work instructions and reports can be given efficiently within the factory, and appropriate feedback can be given according to the emotional state of the workers.

[0517] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0518] Step 1:

[0519] The user issues voice commands through a microphone. The input is the user's voice data, and the output is the recorded voice data. In concrete terms, when the user says, "I'm tired today," the voice is recorded by the microphone.

[0520] Step 2:

[0521] The robot sends the recorded voice data to the server. The input is the recorded voice data, and the output is the voice data sent to the server. In concrete terms, the robot uploads the voice data to the server via the network.

[0522] Step 3:

[0523] The server converts the voice data into text data using the Google Cloud Speech-to-Text API. The input is the voice data sent to the server, and the output is text data. Specifically, the server sends the voice data to the API and receives the returned text data.

[0524] Step 4:

[0525] The server uses the Microsoft Azure Emotion API to recognize emotional states from text data. The input is text data, and the output is emotional state data. Specifically, the server sends text data to the API and receives the returned emotional state data.

[0526] Step 5:

[0527] The server generates appropriate feedback depending on the emotional state. The input is emotional state data, and the output is a feedback message. Specifically, the server analyzes the emotional state, and if it determines that the user is "fatigued," it generates a message saying, "Please take a break."

[0528] Step 6:

[0529] The server sends the generated feedback message to the robot. The input is the feedback message, and the output is the feedback message sent to the robot. In concrete terms, the server sends the feedback message to the robot via the network.

[0530] Step 7:

[0531] The robot conveys a feedback message to the user. The input is the feedback message sent to the robot, and the output is the feedback message conveyed to the user. As a specific action, the robot verbally tells the user, "Please take a break."

[0532] Example 2

[0533] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0534] Conventional systems for converting meeting recording data into text and summarizing it have the problem that the summarized text does not reflect the user's emotional state, making it difficult to provide appropriate information to the user. Furthermore, there is a lack of a way to present the summarized text in a visually easy-to-understand format, which reduces the efficiency of information transmission.

[0535] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0536] In this invention, the server includes means for converting recorded data of an internal meeting into text using artificial intelligence, means for summarizing the text using generative artificial intelligence, means for adjusting the summarized text based on the user's emotional state using an emotion recognition engine, and means for automatically generating an animated video from the adjusted summary. This makes it possible to provide an appropriate summary that reflects the user's emotional state and to communicate information in a form that is visually easy to understand.

[0537] An "internal meeting" is a meeting held within a company or organization, where work progress, issues, and solutions are discussed.

[0538] "Recorded data" refers to data in which audio is recorded in digital or analog format, and is used to play back and review the contents of a meeting at a later date.

[0539] "Artificial intelligence" refers to computer systems that mimic human intelligence and have capabilities such as learning, reasoning, and recognition.

[0540] "Transcription" refers to the process of converting audio data or other forms of data into text form.

[0541] "Generative AI" refers to AI that has the ability to generate new text or information based on input data.

[0542] "Summarization" is the process of shortening long sentences or data and extracting only the important information.

[0543] An "emotion recognition engine" is software that analyzes and recognizes a user's emotional state from text and voice data.

[0544] A "user's emotional state" refers to the emotion a user is feeling at a particular point in time, and may include joy, sadness, anger, etc.

[0545] "Reconciliation" is the process of changing or modifying data or information based on specific criteria or conditions.

[0546] An "animated video" is a video that uses a series of images or videos to express movement, and is a means of visually conveying information.

[0547] "Auto-generation" refers to the process by which a system or program generates data or information without human intervention.

[0548] MODE FOR CARRYING OUT THE INVENTION

[0549] This invention is a system that uses artificial intelligence to convert recorded data from an internal meeting into text, then uses generative artificial intelligence to summarize it, and then uses an emotion recognition engine to adjust the summary based on the user's emotional state, ultimately automatically generating an animated video.

[0550] Hardware and software used

[0551] 1. Server:

[0552] The server receives the recorded data, transcribes it into text, summarizes it, recognizes emotions, adjusts the summary, and generates animated videos.

[0553] Software used: Deep learning-based text generation models (e.g., GPT-3), emotion recognition engines (e.g., IBM Watson Tone Analyzer).

[0554] 2. Terminal:

[0555] The terminal is used by the user to input the recording data and display the final summary or animated video.

[0556] The terminal communicates with the server through an internet connection.

[0557] 3. User:

[0558] The user inputs the recorded data of the conference into the terminal and checks the generated summary and animated video.

[0559] What the program does

[0560] 1. Data Entry:

[0561] The user inputs the recording data of the conference into an input field on the terminal.

[0562] Example: A user types the following sentence: "In today's meeting, we discussed the progress of a new project. Many challenges were identified, but some solutions were also proposed."

[0563] 2. Data transmission:

[0564] The device makes an API request to send the entered data to the server.

[0565] Example: A device sends input data to a server using an HTTP POST request.

[0566] 3. Data reception and summary generation:

[0567] The server summarizes the received data using natural language processing techniques.

[0568] Software used: Deep learning-based text generation models (e.g., GPT-3).

[0569] Example: A server summarizes the sentence "At today's meeting, we discussed the progress of the new project. Many issues were identified, but some solutions were also proposed" as "At the meeting, the progress, issues, and solutions of the new project were discussed."

[0570] 4. Emotion recognition:

[0571] The server uses an emotion recognition engine to analyze the user's emotional state.

[0572] Software used: Emotion recognition engine (e.g. IBM Watson Tone Analyzer).

[0573] Example: The server recognizes the user's emotional state as "joy" from the input data.

[0574] 5. Adjusting the summary:

[0575] The server adjusts the summary based on the perceived emotional state.

[0576] Example: If the user is in a "joy" emotional state, add positive elements to the summary. For example, you could write, "The meeting discussed the progress, challenges, and solutions of the new project, and many positive opinions were expressed."

[0577] 6. Generate animated videos:

[0578] The server automatically generates an animated video based on the adjusted summary.

[0579] Example: The server generates an animated video that presents the summary in a visually easy-to-understand format.

[0580] 7. Data transmission and display:

[0581] The server sends the final summary and animation video to the terminal.

[0582] The terminal displays the received summary and animation video to the user.

[0583] Example: The device displays a summary on the screen: "At the meeting, progress on the new project, challenges, and solutions were discussed, and many positive opinions were expressed." along with a corresponding animated video.

[0584] Prompt Sentence Examples

[0585] "Summarize the following sentence. If the user's emotional state is joy, generate a positive summary. Sentence: 'In today's meeting, we discussed the progress of the new project. Many challenges were identified, but some solutions were also proposed.'"

[0586] In this way, a system is realized in which the server, terminal, and user work together to summarize text and adjust it based on emotions, ultimately generating animated videos that are visually easy to understand.

[0587] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0588] Step 1:

[0589] The user inputs textual data.

[0590] Input: The user inputs the meeting recording in text format.

[0591] What happens: A user types into a text field using a smartphone or computer keyboard: "In today's meeting, we discussed the progress of a new project. Many challenges were identified, but some solutions were also proposed."

[0592] Output: The input text data.

[0593] Step 2:

[0594] The terminal sends the input data to the server.

[0595] Input: Text data entered by the user.

[0596] Specific operation: The device creates an API request to send the entered data to the server, and sends the data to the server using an HTTP POST request.

[0597] Output: The text data sent to the server.

[0598] Step 3:

[0599] A server receives the data and generates a summary using natural language processing techniques.

[0600] Input: Text data sent from the terminal.

[0601] Specific operation: The server analyzes the received data and generates a summary by invoking a deep learning-based text generation model (e.g., GPT-3).

[0602] Output: Summarized text data. For example, the summary generated is "The progress, challenges, and solutions of the new project were discussed at the meeting."

[0603] Step 4:

[0604] The server uses an emotion recognition engine to recognize the user's emotional state.

[0605] Input: Summarized text data.

[0606] Specific operation: The server uses an emotion recognition engine (e.g., IBM Watson Tone Analyzer) to analyze the user's emotional state from the summarized text data.

[0607] Output: The user's emotional state (e.g., happy, sad, angry, etc.). For example, the user's emotional state is recognized as "happy."

[0608] Step 5:

[0609] The server adjusts the summary based on the emotional state.

[0610] Input: Summarized text data and the user's emotional state.

[0611] Specific behavior: The server adjusts the summary based on the perceived emotional state, for example, if the user is in the "joy" emotional state, it adds positive elements to the summary.

[0612] Output: Adjusted summary text. For example, the summary generated is "The progress, challenges, and solutions of the new project were discussed at the meeting, and many positive opinions were expressed."

[0613] Step 6:

[0614] The server automatically generates an animated video based on the adjusted summary.

[0615] Input: The adjusted summary text.

[0616] Specific operation: The server calls software for generating an animated video based on the adjusted summary, and generates an animated video that presents the summary in a visually easy-to-understand form.

[0617] Output: Animated video.

[0618] Step 7:

[0619] The server sends the final summary and animation video to the device.

[0620] Input: Adjusted summary text and animated video.

[0621] Specific operation: The server creates an API response to send the final summary and animation video to the device, and sends the data to the device using an HTTP response.

[0622] Output: Summary text and animated video sent to device.

[0623] Step 8:

[0624] The device displays a summary and an animated video to the user.

[0625] Input: Summary text and animation video sent from the server.

[0626] Specific operation: The device displays the received summary and animation video on the screen so that the user can review them.

[0627] Output: Summary text and animated video displayed on the screen. For example, the summary "At the meeting, the progress, challenges, and solutions of the new project were discussed, and many positive opinions were expressed." is displayed along with the corresponding animated video.

[0628] (Application example 2)

[0629] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0630] Conventional systems for summarizing recorded meeting data generate summaries without taking into account the user's emotional state, which means they are unable to provide appropriate information based on the user's emotions. Furthermore, there is a lack of a way to present the summarized text in a visually easy-to-understand format. This makes it difficult for users to quickly and accurately understand the information.

[0631] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0632] In this invention, the server includes means for converting recorded data of an internal meeting into text using artificial intelligence, means for summarizing the text data using generative artificial intelligence, means for automatically generating an animated video from the summarized text, means for detecting the user's emotional state, and means for generating a summary based on the detected emotional state. This makes it possible to provide an appropriate summary according to the user's emotional state and to provide information in a form that is visually easy to understand.

[0633] "Internal meeting recording data" refers to audio recordings of meetings held within a company or organization.

[0634] "Artificial intelligence" refers to computer systems that mimic human intelligence and can learn and reason.

[0635] "Transcription" refers to the process of converting audio or other forms of data into text form.

[0636] "Generative AI" refers to AI that has the ability to generate new text or information from given data.

[0637] "Summarization" refers to the process of extracting and summarizing long pieces of text or data into short, important pieces.

[0638] "Animation video" refers to a video format that uses a series of images or videos to express movement.

[0639] "Auto-generation" refers to the process by which a system automatically creates data or content without manual intervention.

[0640] "User's emotional state" refers to the emotions or moods that a user is feeling at a particular point in time.

[0641] "Detection" refers to the process of identifying and recognizing specific information or conditions.

[0642] "Means" refer to the methods or techniques used to achieve a particular goal.

[0643] A system for implementing this invention has the following configuration: A server includes means for converting recorded data of an internal meeting into text using artificial intelligence, means for summarizing the textual data using generative artificial intelligence, means for automatically generating an animated video from the summarized text, means for detecting the emotional state of a user, and means for generating a summary based on the detected emotional state.

[0644] Hardware and software used

[0645] Hardware: Servers, smartphones

[0646] Software: Python, OpenAI API (GPT-3), TextBlob

[0647] Data processing and calculation

[0648] 1. Transcription of recordings:

[0649] The server receives the recordings of internal meetings and converts them into text using speech recognition technology, such as the Google Cloud Speech-to-Text API.

[0650] 2. Summarizing the documented data:

[0651] The server uses generative artificial intelligence (such as OpenAI's GPT-3) to summarize the textual data, extracting key information and summarizing it in a concise format.

[0652] 3. Detecting the user's emotional state:

[0653] The server receives the user's input text (e.g., "I feel very happy today!") and performs sentiment analysis using TextBlob to classify the emotional state as positive, negative, or neutral.

[0654] 4. Generating summaries based on emotional states:

[0655] The server generates a summary based on the detected emotional state, for example, if the user is in a positive emotional state, it provides a prompt sentence to the generative artificial intelligence to generate a positive summary.

[0656] 5. Automatic generation of animated videos:

[0657] The server automatically generates animated videos to present the summarized text in a visually easy-to-understand format, using video editing software such as Adobe After Effects.

[0658] Specific examples

[0659] If a user types "I'm feeling very happy today!", the server receives this text and uses TextBlob to detect the emotional state as "positive." The server then provides the following prompt to the generative AI to generate a positive summary:

[0660] Prompt Sentence Examples

[0661] Summarize the following news article: The emotional state is positive.

[0662] Today's news reported that economic growth is stronger than expected, and experts predict this growth will continue.

[0663] summary:

[0664] The summaries generated in this way are then visually presented as animated videos and provided to users, allowing them to quickly and accurately understand appropriate information according to their emotional state.

[0665] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0666] Step 1:

[0667] The server receives recordings of internal meetings. The input is audio data and the output is text data. The server uses speech recognition technology (e.g., Google Cloud Speech-to-Text API) to convert the audio data into text data. This process involves analyzing the audio data and converting it into text.

[0668] Step 2:

[0669] The server uses generative artificial intelligence (e.g., OpenAI's GPT-3) to summarize the text data. The input is text data, and the output is a summary text. The server extracts important information from the text data and summarizes it in a concise form. In this process, natural language processing techniques are used to summarize the text.

[0670] Step 3:

[0671] The user inputs text that indicates their emotional state. The input is the text that indicates the user's emotional state, and the output is the classification result of the emotional state. The server uses TextBlob to perform sentiment analysis from the user's input text. In this process, the text is sentiment analyzed and classified as positive, negative, or neutral.

[0672] Step 4:

[0673] The server generates a summary based on the detected emotional state. The input is the summary text and the emotional state, and the output is the summary text according to the emotional state. The server provides the generative AI with a prompt that reflects the emotional state, and generates a summary according to the emotion. In this process, the prompt is generated based on the emotional state and the summary is regenerated.

[0674] Step 5:

[0675] The server automatically generates animated videos to present the summarized text in a visually easy-to-understand format. The input is summary text corresponding to the emotional state, and the output is an animated video. The server uses video editing software (e.g., Adobe After Effects) to create the animated video based on the summary text. This process involves visualizing text data and generating a video.

[0676] Step 6:

[0677] The user receives and watches the generated animation. The input is the animation, and the output is the user's understanding. The user quickly and accurately understands the information provided in a visually easy-to-understand format. This process involves receiving and understanding information.

[0678] Example 3

[0679] Next, a description will be given of a third embodiment of the third embodiment. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0680] In conventional methods for managing recorded meeting data, the process of converting the recorded data into text and then summarizing it is time-consuming and laborious. Furthermore, there is a lack of a way to present the summarized text in a visually easy-to-understand format, which reduces the efficiency of information transmission. Furthermore, technology for generating animated videos that reflect the user's emotional state is still underdeveloped, making it difficult to provide appropriate information according to the user's emotions.

[0681] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[0682] In this invention, the server includes means for converting recorded data of an internal meeting into text using artificial intelligence, means for summarizing the textual data using generative artificial intelligence, means for analyzing the summarized text using natural language processing technology, means for recognizing the user's emotional state, means for generating animated videos using a deep learning model based on the analysis results and the emotional state, and means for providing the generated animated videos to the user. This enables efficient management and summarization of recorded data, generation of visually easy-to-understand animated videos, and provision of appropriate information according to the user's emotional state.

[0683] "Internal meeting recording data" refers to data in which the content of statements made during a meeting is recorded in audio format.

[0684] "Artificial intelligence" refers to computer systems that mimic human intelligence and can learn and reason.

[0685] "Texting" is the process of converting audio data into text form.

[0686] "Generative AI" is AI that has the ability to extract important information from data and generate summaries and new content.

[0687] "Summarization" is the process of extracting important information from original data and putting it into a concise form.

[0688] "Natural language processing technology" is a technology that uses computers to understand, analyze, and generate human language.

[0689] "Analysis" is the process of examining data in detail to understand its structure and meaning.

[0690] "Emotional state" refers to the user's current feelings and mood.

[0691] "Recognition" is the process of identifying specific information or patterns in data.

[0692] A "deep learning model" is a model that uses a multi-layer neural network to learn data and make predictions or generate results.

[0693] "Animation video" is a video format that expresses movement by displaying a series of images.

[0694] "Serving" is the process of delivering generated content to a user.

[0695] This invention is a system for efficiently managing recording data of in-house meetings and converting summarized text into visually easy-to-understand animated videos. Specific embodiments of this system are described below.

[0696] Hardware and software used

[0697] The server uses the following hardware and software:

[0698] Hardware: A server with a powerful processor, sufficient memory, and storage

[0699] Software: Natural language processing libraries (e.g., spaCy, NLTK), emotion recognition models, deep learning models

[0700] A terminal is a device through which a user inputs data, and includes a personal computer, a tablet, a smartphone, and the like.

[0701] Data processing and calculation

[0702] 1. Transcription of recordings:

[0703] The server receives recordings of internal meetings and uses artificial intelligence to convert the audio data into text format, using voice recognition technology.

[0704] 2. Summarizing text data:

[0705] The server uses generative artificial intelligence to extract key information from the documented data and generate a summary. This summarization process succinctly summarizes the essence of the information.

[0706] 3. Analysis of the summarized text:

[0707] The server uses natural language processing technology to analyze the summarized sentences. For example, it analyzes the sentence "The project is progressing smoothly" and identifies scenes in which project members are working.

[0708] 4. User Emotion Recognition:

[0709] The server analyzes the user's input and facial expression data to recognize the user's emotional state. The user inputs the emotional state using the terminal or provides facial expression data through a camera.

[0710] 5. Generate animated videos:

[0711] The server uses a deep learning model to generate an animated video based on the analysis results and the user's emotional state. For example, the server inputs a prompt: "An animated video of project members actively working in bright colors."

[0712] 6. Providing generated videos:

[0713] The server provides the generated animation video to the user, who then uses the terminal to view the video via the provided link.

[0714] Specific examples

[0715] As a specific example, if the input is the summarized sentence "The project is progressing smoothly," the server will input the following prompt sentence into the generative AI model.

[0716] Example prompt sentence:

[0717] Summarized text: "The project is progressing smoothly."

[0718] Emotional state: "Joy"

[0719] Video to generate: "An animated video of project members actively working in bright colors."

[0720] Based on this prompt, the server uses a deep learning model to generate an animated video depicting project members working.

[0721] In this way, it is possible to efficiently manage and summarize recorded data, generate visually easy-to-understand animated videos, and provide appropriate information according to the user's emotional state. The flow of the identification process in the third embodiment will be described with reference to FIG.

[0722] Step 1:

[0723] Receiving and documenting the recordings

[0724] The server receives recordings of internal meetings from users. It receives audio data as input and converts it into text using speech recognition technology. Specifically, the server uses a speech recognition library (e.g., Google Speech-to-Text API) to analyze the audio data and convert it into text. The output is the meeting content in text format.

[0725] Step 2:

[0726] Summarizing text data

[0727] The server inputs the text data into a generative AI to perform summarization. It receives text data as input, extracts important information using natural language processing technology, and generates a concise summary. Specifically, the server uses a generative AI (e.g., BERT, GPT-3) to generate a summary. The output is a summarized sentence.

[0728] Step 3:

[0729] Analysis of summarized text

[0730] The server analyzes the summarized text using natural language processing technology. It receives the summary text as input and analyzes it to identify the content and scene of the text. Specifically, the server uses an NLP library (e.g., spaCy, NLTK) to analyze the meaning of the text. The output is scene information as the analysis result.

[0731] Step 4:

[0732] User Emotion Recognition

[0733] The server analyzes the user's input and facial expression data to recognize the user's emotional state. It receives the user's emotional data (text input and facial expression images) as input and analyzes it using an emotion recognition model. Specifically, the server uses the emotion recognition model to identify the user's emotional state. The output is the user's emotional state.

[0734] Step 5:

[0735] Generate animated videos

[0736] The server uses a deep learning model to generate an animated video based on the analysis results and the emotional state. It receives scene information and the emotional state as input, generates a prompt sentence, and inputs it into the deep learning model. Specifically, the server uses the generative AI model to input the prompt sentence: "An animated video of project members actively working in bright colors." The output is the generated animated video.

[0737] Step 6:

[0738] Serving generated videos

[0739] The server provides the generated animation video to the user. It receives the generated video as input and generates a link to provide it to the user. Specifically, the server saves the video in storage and sends a viewing link to the user. The output is a video link that the user can access.

[0740] (Application example 3)

[0741] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0742] Conventional systems for transcribing and summarizing meeting recordings lack the means to provide summarized information in a visually understandable format. Furthermore, content generation does not reflect the user's emotional state, making it difficult to obtain appropriate feedback based on the user's emotions.

[0743] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[0744] In this invention, the server includes means for converting recorded data of an internal meeting into text using artificial intelligence, means for summarizing the textual data using generative artificial intelligence, means for automatically generating an animated video from the summarized text, and means for analyzing the emotional state of a user and generating an animated video with colors and movements that reflect that emotional state. This makes it possible to provide summarized information in a visually easy-to-understand format and to obtain appropriate feedback according to the user's emotional state.

[0745] "Internal meeting recording data" refers to audio recordings of meetings held within a company or organization.

[0746] "Artificial intelligence" refers to computer systems that mimic human intelligence and can learn and reason.

[0747] "Texting" refers to the process of converting audio data into text data.

[0748] "Generative AI" refers to AI that has the ability to generate new information and content from data.

[0749] "Summarization" refers to the process of extracting important information from long text or data and summarizing it in a concise manner.

[0750] "Animation video" refers to a video format that uses a series of images to express movement.

[0751] "User's emotional state" refers to the emotions or moods that a user is feeling at a particular moment.

[0752] "Color tone" refers to the placement and balance of colors in an image or video.

[0753] "Movement" refers to the movement and changes of characters and objects in animated videos.

[0754] "Visually easy to understand" refers to a state in which information is presented in a way that makes it easy to understand visually.

[0755] "Feedback" refers to returning the results or reactions of a system or process to the user.

[0756] A system for implementing this invention has the following configuration: A server includes means for converting recorded data of an internal meeting into text using artificial intelligence, means for summarizing the textual data using generative artificial intelligence, means for automatically generating animated videos from the summarized text, and means for analyzing the emotional state of a user and generating animated videos with colors and movements that reflect that emotional state.

[0757] Hardware and software used

[0758] Hardware: Servers, smartphones

[0759] software:

[0760] Python: Programming Language

[0761] Sentiment Analysis Model

[0762] Video Generation API

[0763] Data processing and calculation

[0764] 1. Transcription of recordings:

[0765] The server receives the recordings of internal meetings and converts them into text using speech recognition technology, using a speech recognition model.

[0766] 2. Summarizing text data:

[0767] Generative AI extracts key information from written data and generates summaries, a process that uses natural language processing techniques.

[0768] 3. Generate animated videos:

[0769] The server receives the summarized text as input and generates an animated video using a deep learning-based video generation model. In doing so, it analyzes the user's emotional state and generates an animated video with colors and movements that reflect that emotional state.

[0770] Specific examples

[0771] For example, if a user types the sentence "The project is progressing smoothly," the server summarizes the sentence and performs a sentiment analysis. If the result of the sentiment analysis is positive, it generates an animated video with bright colors and lively movements.

[0772] Prompt Sentence Examples

[0773] Input sentence: "The project is progressing smoothly."

[0774] Outputs the URL of the generated animation video.

[0775] In this way, summarized information can be provided in a visually easy-to-understand format, and appropriate feedback can be obtained according to the user's emotional state.

[0776] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[0777] Step 1:

[0778] The server receives the recording data of the internal meeting and converts it into text data using speech recognition technology. The input is the recording data and the output is text data. Specifically, the server analyzes the speech data using a speech recognition model and generates corresponding text.

[0779] Step 2:

[0780] The server inputs the generated text data into a generative artificial intelligence (AI) system, extracts important information, and generates a summary. The input is text data, and the output is a summarized text. Specifically, the system uses natural language processing technology to analyze the text data and extract important keywords and phrases to create a summary.

[0781] Step 3:

[0782] The server inputs the summarized text into a sentiment analysis model to analyze the user's emotional state. The input is the summarized text, and the output is the emotional state (positive, negative, neutral). Specifically, the sentiment analysis model is used to analyze the sentiment of the text and obtain the results.

[0783] Step 4:

[0784] The server inputs the summarized text and emotional state into the video generation API to generate an animated video. The input is the summarized text and emotional state, and the output is the URL of the animated video. Specifically, the server sends the summarized text and emotional state to the video generation API and obtains the URL of the generated animated video.

[0785] Step 5:

[0786] The device provides the user with the URL of the animation video obtained from the server. The input is the URL of the animation video, and the output is an animation video that the user can watch. Specifically, the video is displayed to the user through the device's browser or application.

[0787] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0788] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0789] Another example of generative AI is Gemini (registered trademark) (Internet search engine). <url: https: gemini.google.com ?hl="ja">) are mentioned.

[0790] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0791] [Second embodiment]

[0792] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0793] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0794] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0795] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0796] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0797] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0798] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0799] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0800] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0801] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0802] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0803] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0804] "Example 1"

[0805] This embodiment of the present invention implements a system that uses artificial intelligence to convert recorded data from an internal meeting into text. Specifically, the system receives the recorded data of the meeting as input and converts it into text using speech recognition technology. This speech recognition technology can use a speech recognition model based on deep learning.

[0806] "Example 2"

[0807] Next, the textual data is summarized using generative artificial intelligence. Specifically, the textual data is received as input and summarized using natural language processing technology. This natural language processing technology can use a text generation model based on deep learning.

[0808] "Example 3"

[0809] Finally, the summarized text is automatically converted into an animated video. Specifically, the summarized text is received as input and converted into an animated video using video generation technology. This video generation technology can use a video generation model based on deep learning. As a specific example, the summarized text "The project is progressing smoothly" can be converted into an animated video depicting project members working.

[0810] The processing flow of each embodiment will be described below.

[0811] "Example 1"

[0812] Step 1: Receive the recording data of the meeting as input. This recording data is the audio data of an internal meeting.

[0813] Step 2: The recorded data is converted into text using speech recognition technology. This speech recognition technology uses a deep learning-based speech recognition model. This model is trained from a large amount of audio data and its associated text data.

[0814] "Example 2"

[0815] Step 1: Receive textual data as input. This data is the content of a meeting that has been converted into text using speech recognition technology.

[0816] Step 2: Summarize the text data using natural language processing technology. This natural language processing technology uses a text generation model based on deep learning. This model is trained from large amounts of text data and has the ability to generate summaries of the text.

[0817] "Example 3"

[0818] Step 1: Receive a summarized text as input. This text is the content of a meeting summarized using natural language processing technology.

[0819] Step 2: Using video generation technology, the summarized text is converted into an animated video. This video generation technology uses a deep learning-based video generation model. This model is trained from a large amount of video data and its associated text data, and has the ability to generate corresponding videos from text.

[0820] Step 3: For example, convert the summary sentence "The project is progressing smoothly" into an animated video showing project members working. This video is intended to present the content of the meeting in a visually easy-to-understand way.

[0821] Example 1

[0822] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0823] There is a need to efficiently convert recordings of internal meetings into text, summarize the text, and automatically generate animated videos that are easy to understand visually. However, in conventional systems, the entire process from receiving and saving the recordings, calling up the speech recognition model, generating and saving the text data, and providing access to the user was not automated, and manual operations were required. This resulted in issues such as a lack of efficiency and accuracy.

[0824] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0825] In this invention, the server includes means for converting recorded data of an internal meeting into text using artificial intelligence, means for summarizing the textual data using generative artificial intelligence, means for automatically generating an animated video from the summarized text, means for receiving and saving the recorded data, means for calling a speech recognition model and inputting the recorded data to generate text data, and means for saving the generated text data and making it accessible to users. This automates the entire process from receiving the recorded data to generating, saving, and providing the text data to users, enabling efficient and accurate processing.

[0826] "Internal meeting recording data" refers to digital data that records the audio of meetings held within a company.

[0827] "Artificial intelligence" refers to the technology of data processing and decision-making that enables computer systems to mimic human intelligence.

[0828] "Means for converting text" refers to technology or devices for converting voice data into text data.

[0829] "Generative AI" is AI that has the ability to generate new data from existing data.

[0830] A "summarization tool" is a technique or device that extracts important information from a long piece of text and summarizes it in a short form.

[0831] "Means for automatically generating animated videos" refers to technology or devices that automatically create videos that are visually easy to understand based on text data.

[0832] "Means for receiving and storing recorded data" refers to technology or devices that receive audio data transmitted from an external source and store it in digital form.

[0833] A "speech recognition model" is an algorithm or program for analyzing speech data and generating corresponding text data.

[0834] The "means for generating text data" refers to a technique or device that creates text data from voice data using a voice recognition model.

[0835] "Means for storing the generated text data and making it accessible to users" refers to technologies and devices that store the generated text data in digital form and allow users to view or retrieve it.

[0836] This invention relates to a system that uses artificial intelligence to convert recorded data from an internal meeting into text, summarizes the text, and automatically generates an animated video that is visually easy to understand. A specific embodiment of this system is described below.

[0837] System configuration

[0838] Hardware and software used

[0839] Hardware: High-performance server (e.g., server equipped with GPU)

[0840] Software: Speech recognition models (e.g., Google Speech-to-Text API, DeepSpeech), generative artificial intelligence, animation generation software

[0841] Program processing

[0842] Subject: Server

[0843] 1. Receiving and storing recording data

[0844] The server receives the recording data of the internal meeting uploaded by the user. The received data is saved in the specified directory. For example, it is saved in the path " / uploads / meeting_recording_20231001.wav".

[0845] 2. Calling the speech recognition model

[0846] The server prepares an API to call the speech recognition model. For example, if you use the Google Speech-to-Text API, you set the API key and endpoint.

[0847] 3. Inputting the recorded data into the speech recognition model

[0848] The server inputs the saved recording data into the speech recognition model. Specifically, it sends an API request specifying the path of the recording data. For example, it sends " / uploads / MeetingRecordings_20231001.wav" to the Google Speech-to-Text API.

[0849] 4. Generating Text Data

[0850] The speech recognition model analyzes the recorded data and generates corresponding text data. For example, if the recorded data contains the sentence, "In today's meeting, we discussed the development schedule for a new product," the model generates text data with the same content.

[0851] 5. Saving the generated text data

[0852] The server stores the generated text data, for example, in a file named " / texts / meetingrecording_20231001.txt."

[0853] 6. Providing Access to Users

[0854] The server makes the generated text data accessible to the user. The user accesses " / texts / meetingrecording_20231001.txt" using a web browser to view the text data.

[0855] Specific examples

[0856] For example, if a recording of an internal meeting is uploaded to a server with the file name "Meeting_Recording_20231001.wav," the server inputs this file into a speech recognition model. The speech recognition model analyzes the recording and generates text data like the following:

[0857] Recording contents:

[0858] "At today's meeting, we discussed the development schedule for new products."

[0859] Generated text data:

[0860] "At today's meeting, we discussed the development schedule for new products."

[0861] Prompt Sentence Examples

[0862] Examples of prompts to be input to a generative AI model include:

[0863] "Please convert the following recording to text: Meeting Recording_20231001.wav"

[0864] Using this prompt, the server inputs the recorded data into a speech recognition model and begins the process of generating text data.

[0865] The above is a specific embodiment of the present invention. This system automates the entire process from receiving recorded data to generating text data, saving it, and providing it to the user, making it possible to process the data efficiently and accurately.

[0866] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0867] Step 1:

[0868] Users upload recordings

[0869] The user uploads the recording data of an internal meeting from their device to the server. Specifically, the user opens a web browser and accesses the system's upload page. They click the file selection button and select "Meeting Recording_20231001.wav" from their local disk. When they click the upload button, the recording data is sent to the server.

[0870] Input: Recording data file selected by the user (e.g. "Meeting Recording_20231001.wav")

[0871] Output: Recording data is sent to the server

[0872] Step 2:

[0873] The server receives and stores the recording data.

[0874] The server receives the recording data uploaded by the user. The received data is saved in the specified directory. For example, it is saved in the path " / uploads / meetingrecording_20231001.wav".

[0875] Input: Recording data sent by the user

[0876] Output: Recording data saved in the specified directory on the server

[0877] Step 3:

[0878] The server calls the speech recognition model

[0879] The server prepares an API to call the speech recognition model. For example, if you use the Google Speech-to-Text API, you set the API key and endpoint.

[0880] Input: Path to the recording data (e.g. " / uploads / MeetingRecording_20231001.wav")

[0881] Output: The speech recognition model is ready to be invoked

[0882] Step 4:

[0883] The server inputs the recording into a speech recognition model.

[0884] The server inputs the saved recording data into the speech recognition model. Specifically, it sends an API request specifying the path of the recording data. For example, it sends " / uploads / MeetingRecordings_20231001.wav" to the Google Speech-to-Text API.

[0885] Input: Path to the recording data (e.g. " / uploads / MeetingRecording_20231001.wav")

[0886] Output: The recording sent to the speech recognition model

[0887] Step 5:

[0888] The speech recognition model analyzes the recorded data and generates text data.

[0889] The speech recognition model analyzes the recorded data and generates corresponding text data. For example, if the recorded data contains the sentence, "In today's meeting, we discussed the development schedule for a new product," the model generates text data with the same content.

[0890] Input: The recording sent to the speech recognition model

[0891] Output: Generated text data

[0892] Step 6:

[0893] The server stores the generated text data.

[0894] The server stores the generated text data, for example, in a file named " / texts / meetingrecording_20231001.txt."

[0895] Input: Generated text data

[0896] Output: Text data saved in the specified directory on the server

[0897] Step 7:

[0898] The user checks the generated text data

[0899] The user accesses the server and checks the generated text data. Specifically, the user uses a web browser to access " / texts / MeetingRecording_20231001.txt" and view the text data.

[0900] Input: Path of the text data stored on the server

[0901] Output: Text data viewed by the user

[0902] (Application example 1)

[0903] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0904] Conventional systems for converting recorded data from internal meetings into text can summarize the contents of meetings and generate animated videos to aid visual understanding, but they lack the functionality to convert recorded data from work instructions and maintenance meetings within factories into text in real time and provide it to workers and managers. This poses the problem of not being able to fully improve work efficiency or reduce errors within factories.

[0905] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0906] In this invention, the server includes means for converting recorded data of in-house meetings into text using artificial intelligence, means for summarizing the textual data using generative artificial intelligence, means for automatically generating the summarized text into an animated video, means for converting recorded data of work instructions and maintenance meetings within a factory into text in real time, and means for providing the real-time textual data to workers and managers. This makes it possible to convert the contents of work instructions and maintenance meetings within a factory into text in real time and provide it to workers and managers.

[0907] An "internal meeting" is a meeting held within a company or organization, where business progress, problems, future policies, etc. are discussed.

[0908] "Recorded data" means data in which sound is recorded in digital or analog format and stored in a format that allows for later playback and analysis.

[0909] "Artificial intelligence" refers to a computer system that mimics human intelligence and has capabilities such as learning, reasoning, and recognition.

[0910] "Texting" is the process of converting non-text data, such as audio or video, into text form.

[0911] "Generative AI" is AI that has the ability to generate new information and content from given data.

[0912] "Summarization" is the process of extracting important information from long text or data and summarizing it in a concise manner.

[0913] "Animation video" is a video format that expresses movement by displaying a series of consecutive images in sequence.

[0914] "Inside a factory" refers to the facility where products are manufactured or processed in the manufacturing or production industries.

[0915] "Work instructions" are specific instructions and procedures for carrying out a specific task.

[0916] A "maintenance meeting" is a meeting where discussions and plans are made regarding the maintenance and inspection of equipment and machinery.

[0917] "Real-time" refers to data processing and information provision occurring immediately, without delay.

[0918] "Workers" are workers who actually perform work in factories or on-site.

[0919] An "administrator" is a person responsible for running and supervising an organization or project.

[0920] A system for implementing this invention is configured as follows: First, a server acquires recorded data of internal meetings, work instructions in a factory, and maintenance meetings. Next, the server converts this recorded data into text using speech recognition technology. This speech recognition technology uses Wav2Vec2, a speech recognition model based on deep learning.

[0921] The server summarizes the text data using a generative AI. This generative AI is implemented using the Transformers library. The summarized text is then automatically generated into an animated video. This animated video is intended to present the summarized text in a visually easy-to-understand format.

[0922] The server also converts work instructions and recordings of maintenance meetings into text in real time and provides it to workers and managers. This real-time text conversion contributes to improving work efficiency and reducing errors.

[0923] The specific hardware used is a microphone and a computer, and the software used is Python, a speech recognition library, the Transformers library, and a Wav2Vec2 model.

[0924] For example, if you use recorded data from a maintenance meeting held in a factory, when you input this data into the server, the following text will be output.

[0925] "At today's maintenance meeting, it was confirmed that Machine A requires a routine inspection. The next inspection is scheduled for next Monday."

[0926] An example of a prompt sentence to input to the generative AI model is as follows:

[0927] "Please convert the following audio data to text: [path to audio data]"

[0928] This prompt sentence can be used to efficiently convert voice data into text.

[0929] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0930] Step 1:

[0931] The server acquires recorded data from internal meetings, work instructions in factories, and maintenance meetings. The input is the recorded data, and the output is an audio data file. Specifically, audio is recorded using a microphone and the data is saved in digital format.

[0932] Step 2:

[0933] The server converts the acquired recorded data into text using speech recognition technology. The input is a voice data file, and the output is text data. Specifically, a speech recognition library is used to convert the voice data into text. In this process, speech recognition is performed using the Wav2Vec2 model.

[0934] Step 3:

[0935] The server summarizes the text data using generative artificial intelligence. The input is text data, and the output is summarized text data. Specifically, it uses the Transformers library to extract important information from the text data and generate a summary.

[0936] Step 4:

[0937] The server automatically generates an animated video from the summarized text. The input is the summarized text data, and the output is an animated video file. Specifically, animation generation software is used to express the summarized text as a video in a visually easy-to-understand format.

[0938] Step 5:

[0939] The server converts factory work instructions and recordings of maintenance meetings into text in real time. The input is real-time voice data, and the output is real-time text data. Specifically, it performs real-time voice recognition and instantly converts it into text.

[0940] Step 6:

[0941] The server provides documented data to workers and managers in real time. The input is real-time text data, and the output is text information displayed on the terminals of the workers and managers. Specifically, the text data is sent via the network and displayed on the terminals of the workers and managers.

[0942] Example 2

[0943] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0944] In conventional systems, the process of transcribing recordings of internal meetings and summarizing them was often done manually, which was time-consuming and labor-intensive. Furthermore, there was a lack of a way to present the summarized text in a visually easy-to-understand format. Furthermore, there was no system that could efficiently summarize text entered by users.

[0945] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0946] In this invention, the server includes means for converting recorded data of an internal meeting into text using artificial intelligence, means for summarizing the textual data using generative artificial intelligence, means for automatically generating an animated video from the summarized text, means for a terminal to send text entered by a user to the server, means for the server to preprocess the received text and input a prompt sentence to the generative artificial intelligence, means for the generative artificial intelligence to generate a summary and the server to send the summary result to the terminal, and means for the terminal to display the summary result to the user. This automates the process from converting recorded data into text to summarizing and visual representation, enabling efficient summarization of text entered by a user.

[0947] "Internal meeting recording data" refers to data in which the content of statements made during a meeting is recorded in audio format.

[0948] "Artificial intelligence" refers to computer systems that mimic human intelligence and can learn and reason.

[0949] "Texting" is the process of converting audio data or other forms of data into text form.

[0950] "Generative AI" is AI that has the ability to generate new text or information based on input data.

[0951] "Summarization" is the process of shortening long sentences or data and extracting only the important information.

[0952] "Animation video" is a video format that uses a series of images or videos to express movement.

[0953] A "terminal" is a device such as a computer or smartphone that is operated by a user.

[0954] A "server" is a computer system that processes and stores data on a network.

[0955] "Preprocessing" is the initial stage of processing to convert data into a format that is easier to analyze and process.

[0956] A "prompt sentence" is an input sentence that causes a generative artificial intelligence to perform a specific task.

[0957] The "summarization result" is the shortened text that is generated through the summarization process.

[0958] "User" refers to a person who uses the system.

[0959] This invention is a system that uses artificial intelligence to convert recorded data from internal meetings into text, then uses generative artificial intelligence to summarize the text, and then automatically generates an animated video from the summarized text. It also includes a function to efficiently summarize text entered by the user.

[0960] Hardware and software used

[0961] Server: A server equipped with a high-performance GPU

[0962] Software: OpenAI GPT-3, BERT, Python, Flask (web framework)

[0963] Device: A computer or smartphone that the user operates.

[0964] Data processing and calculation

[0965] Server Processing

[0966] The server receives text data provided by the user. This data is text stored in a text file or database. The server then uses a deep learning-based text generation model to summarize the received data using natural language processing techniques. Examples of such models include OpenAI's GPT-3 and BERT.

[0967] The server preprocesses the received text, tokenizing and normalizing it. For example, it uses NLTK or spaCy to tokenize the text and remove unnecessary whitespace and special characters. The preprocessed text is then converted into a format that can be input to the generative artificial intelligence.

[0968] Terminal handling

[0969] The terminal is responsible for transmitting the text data entered by the user to the server. When the user enters text through the terminal interface and presses the send button, the terminal transmits the data to the server. The terminal also displays the summary results returned from the server to the user.

[0970] User Action

[0971] The user inputs text data using the terminal. For example, the user inputs "Please summarize this text" and provides the specific text. The user can check the summary results displayed on the terminal and request another summary if necessary.

[0972] Specific examples

[0973] Prompt Sentence Examples

[0974] Examples of prompts that users can input into a generative AI model include:

[0975] "Please summarize the following sentence: 'This system is designed to summarize sentences entered by users. Specifically, it uses a deep learning-based text generation model to generate short summaries of the entered sentences.'"

[0976] In this way, the user can utilize a system that allows them to easily summarize text.

[0977] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0978] Step 1:

[0979] The user inputs a sentence.

[0980] A user inputs a sentence to be summarized through the terminal interface. For example, the user might input, "This system is designed to summarize the sentence entered by the user. Specifically, it uses a text generation model based on deep learning to summarize the entered sentence in a short form." The input data is in text format.

[0981] Step 2:

[0982] The terminal sends the input text to the server.

[0983] The terminal sends the text entered by the user to the server. Specifically, it uses an HTTP request to send the text data to the server. For example, if you are using Flask, you can send a POST request using the requests library. The input is the text data entered by the user, and the output is what is sent to the server.

[0984] Step 3:

[0985] The server receives the text and performs preprocessing.

[0986] The server receives text sent from the terminal. The received text is tokenized and normalized as preprocessing. For example, NLTK or spaCy is used to tokenize the text and remove unnecessary whitespace and special characters. The input is the text data received from the terminal, and the output is the preprocessed text data.

[0987] Step 4:

[0988] The server inputs a prompt sentence into the generative AI model.

[0989] The server converts the preprocessed text into a format that can be input to the generative AI model. Specifically, it creates a prompt sentence and inputs it to the generative AI model. For example, it creates a prompt sentence such as, "Please summarize the following sentence: 'This system is designed to summarize sentences entered by users. Specifically, it uses a text generation model based on deep learning to summarize the input sentence in a concise manner.'" The input is the preprocessed text data, and the output is the prompt sentence.

[0990] Step 5:

[0991] A generative AI model generates the summary.

[0992] The generative AI model analyzes the input prompt sentence and generates a summary. For example, if OpenAI's GPT-3 is used, an API is called to generate the summary. The generated summary will look something like, "This system summarizes the sentence entered by the user using a deep learning model." The input is the prompt sentence, and the output is the summarized text data.

[0993] Step 6:

[0994] The server sends the summary results to the terminal.

[0995] The server sends the generated summary to the terminal. Specifically, it returns the summary result using an HTTP response. For example, if you are using Flask, you can use the jsonify function to return the summary result in JSON format. The input is the summarized text data, and the output is what is sent to the terminal.

[0996] Step 7:

[0997] The terminal displays the summary results to the user.

[0998] The terminal displays the summary results received from the server to the user. Specifically, it uses HTML and JavaScript to display the summary results in a user interface. For example, it displays the summary results in a text area or a dialog box. The input is the summary results received from the server, and the output is what is displayed to the user.

[0999] (Application example 2)

[1000] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1001] In today's information society, a huge amount of information is generated every day. In particular, text data such as internal meetings and news articles contain a large amount of information, making it difficult to quickly grasp important information. In addition, there is a lack of means to present this information in a visually easy-to-understand format, so efficient information processing and understanding are required.

[1002] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for converting recorded data of an internal meeting into text using artificial intelligence, means for summarizing the textual data using generative artificial intelligence, means for visually displaying the summarized text, and means for providing the summarized text to the user. This makes it possible to quickly extract important information from huge amounts of text data and provide it in a visually easy-to-understand format.

[1003] "Internal meeting recording data" refers to data that records the audio of meetings held within a company or organization.

[1004] "Artificial intelligence" refers to computer systems that mimic human intelligence and can learn and reason.

[1005] "Means for converting text" refers to technology or devices for converting voice data into text data.

[1006] "Generative AI" is AI that has the ability to generate new text based on input data.

[1007] A "summarization method" is a technique or device for extracting important information from long text data and summarizing it in a concise manner.

[1008] A "visual display means" is any technique or device for displaying text data in a graphical format.

[1009] "Means for providing to the user" refers to the technology or device used to deliver the generated summary or visual display to the user.

[1010] The system for implementing this invention uses artificial intelligence to convert recorded data from internal meetings into text, then uses generative artificial intelligence to summarize the text, visually displaying the summarized text and providing it to the user.

[1011] System configuration

[1012] The system consists of the following main components:

[1013] 1. Means of converting recorded data into text: Speech recognition software to convert recorded data into text data. Specifically, we use Google Speech-to-Text API, etc.

[1014] 2. Generative AI: A generative AI model for summarizing textual data. Specifically, we use a model such as the BERT model using the Transformers library.

[1015] 3. Visual display means: A graphical user interface (GUI) for visually displaying the summarized text. Specifically, a web application using HTML / CSS and JavaScript.

[1016] 4. User delivery method: A notification system to provide users with summarized information, specifically via a smartphone app or web browser.

[1017] Program processing explanation

[1018] Transcription of recorded data

[1019] The server receives recordings of internal meetings and converts them into text using speech recognition software, using the Google Speech-to-Text API.

[1020] Documented data summary

[1021] Generative AI takes text data as input, extracts important information, and generates a summary. This process is done using the Transformers library.

[1022] Visual display of summarized data

[1023] The summarized text data is displayed in a visually understandable format, specifically through a web application using HTML / CSS and JavaScript.

[1024] Provision to users

[1025] Finally, the summarized information is provided to the user via a smartphone app or web browser.

[1026] Specific examples

[1027] For example, if a user selects "articles about Japan's economic growth," the system operates as follows:

[1028] 1. Transcription of audio recording: "Japan's economy continues to grow in 2023. Innovation in technology, in particular, is driving economic growth. The government has introduced new policies to support corporate innovation. This has led to the creation of many start-ups and the emergence of new business models. Furthermore, international cooperation is progressing, and Japanese technology is gaining recognition around the world."

[1029] 2. Summary of the written data: "Japan's economy will continue to grow in 2023, driven in particular by the technology sector. New government policies are encouraging the creation of new startups and promoting international collaboration."

[1030] 3. Visual display of summarized data: The summarized text is displayed on the web application.

[1031] 4. Provision to users: Users will be notified via a smartphone app.

[1032] Prompt Sentence Examples

[1033] "Summarize the following news article: Japan's economy continues to grow in 2023. Innovation in technology, in particular, is driving economic growth. The government has introduced new policies to support corporate innovation. This has led to the creation of many start-ups and the emergence of new business models. Furthermore, international cooperation is progressing, and Japanese technology is gaining recognition around the world."

[1034] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1035] Step 1:

[1036] The server receives the recording data of the internal meeting. The input is audio data, and the output is the raw audio data. This audio data is converted into text data in the next step.

[1037] Step 2:

[1038] The server uses speech recognition software (e.g., Google Speech-to-Text API) to convert the recorded data into text data. The input is voice data, and the output is text data. This conversion process turns the voice data into text.

[1039] Step 3:

[1040] The server uses generative artificial intelligence (e.g., the BERT model using the Transformers library) to summarize the text data. The input is text data, and the output is summarized text data. This summarization process extracts important information and summarizes it in a concise manner.

[1041] Step 4:

[1042] The server generates a graphical user interface (GUI) to visually display the summarized text data. The input is the summarized text data, and the output is visually displayable data (e.g., a web page using HTML / CSS and JavaScript). This display process makes it easier for users to visually understand the summarized information.

[1043] Step 5:

[1044] The server provides the user with visually displayed summary data. The input is visually displayable data, and the output is summary information displayed on the user's device. This provision process allows the user to receive the summary information through a smartphone app or web browser.

[1045] Step 6:

[1046] The user checks the provided summary information and obtains more detailed information as needed. The input is the user's operation, and the output is the display of detailed information. This checking process allows the user to efficiently grasp important information and obtain more detailed information as needed.

[1047] Example 3

[1048] Next, a description will be given of Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1049] In the conventional method of managing meeting recording data, the recording data is converted into text and then summarized, which is time-consuming and laborious. Furthermore, there is a lack of a way to present the summarized text in a visually easy-to-understand format, which reduces the efficiency of information transmission. This makes it difficult to quickly and effectively share the contents of the meeting.

[1050] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[1051] In this invention, the server includes means for converting recorded data of an internal meeting into text using artificial intelligence, means for summarizing the textual data using generative artificial intelligence, means for automatically generating an animated video from the summarized text using a deep learning model, means for transmitting the generated animated video to a terminal, and means for displaying the received animated video to a user, thereby enabling the content of the meeting to be shared quickly and effectively.

[1052] "Internal meeting recording data" refers to data in which the content of statements made during a meeting is recorded in audio format.

[1053] "Artificial intelligence" is a technology that mimics human intelligence and automatically analyzes and processes data.

[1054] "Texting means" refers to techniques or methods for converting audio data into text format.

[1055] "Generative AI" is AI that has the ability to generate new information or summaries from input data.

[1056] "Methods of summarization" are techniques or methods for extracting important information from long pieces of text or data and summarizing it in a concise manner.

[1057] A "deep learning model" is a technology that uses multi-layered neural networks to analyze data and learn patterns.

[1058] "Animation video" is a video format that expresses movement by displaying a series of images.

[1059] "Means for automatic generation" refers to techniques or methods for automatically generating output data based on specific input data.

[1060] A "terminal" is a device operated by a user, and includes a computer, a smartphone, etc.

[1061] A "transmission medium" is a technique or method for transferring data from one place to another.

[1062] "Display means" refers to the techniques and methods used to visually present data to the user.

[1063] The present invention provides a system for efficiently managing recorded data of in-house meetings and providing summarized information in a visually easy-to-understand format. Specific embodiments of this system are described below.

[1064] Hardware and software used

[1065] This system is mainly composed of a server, terminals, and user operations. The specific hardware and software used are as follows:

[1066] Server: We use powerful computer servers to process and store data.

[1067] Terminal: A device operated by a user, including a PC or smartphone.

[1068] Artificial Intelligence: Artificial intelligence with speech recognition technology to convert recorded data into text.

[1069] Generative AI: AI that uses natural language processing techniques to summarize written data.

[1070] Deep learning model: A model for converting summarized text into animated videos.

[1071] System processing flow

[1072] 1. The user inputs the summarized text into the terminal.

[1073] The user inputs the summarized sentence into an input field on the terminal, for example, "The project is progressing smoothly."

[1074] 2. The device sends the input text to the server

[1075] The device sends the text entered by the user to the server as an HTTP request, converting the text into JSON format and sending it to the server's API endpoint.

[1076] 3. The server inputs the received sentences into the deep learning model

[1077] The server analyzes the JSON-formatted text received from the terminal and inputs it into the deep learning model.

[1078] 4. Deep learning models analyze text and generate animated videos

[1079] The deep learning model analyzes input sentences and generates animated videos based on their content. For example, based on the sentence "The project is progressing smoothly," it generates an animated video depicting project members working.

[1080] 5. The server sends the generated animation to the device.

[1081] The server receives the animation video generated by the deep learning model and sends it to the device using an HTTP response.

[1082] 6. The device displays the animation video to the user.

[1083] The terminal displays the animation received from the server to the user, who can then view the generated animation on the screen of the terminal.

[1084] Specific examples

[1085] The user inputs a summarized sentence, "The project is progressing smoothly," into the device. The device sends this sentence to the server. The server uses a deep learning-based video generation model to generate an animated video depicting the project members working. The generated video is then sent from the server to the device, where the user can view it.

[1086] Prompt Sentence Examples

[1087] Take the condensed sentence "The project is progressing smoothly" and turn it into an animated video.

[1088] In this way, the contents of the conference can be shared quickly and effectively. The flow of the identification process in the third embodiment will be described with reference to FIG.

[1089] Step 1:

[1090] The user inputs the summarized text into the terminal.

[1091] As a specific operation, the user enters "The project is progressing smoothly" into the input field of the terminal and clicks the send button.

[1092] Input: A summary of the sentence (e.g., "The project is progressing smoothly.")

[1093] Output: Text typed into the terminal

[1094] Step 2:

[1095] The terminal sends the input text to the server.

[1096] Specifically, the device converts the input text into JSON format and sends an HTTP POST request to the server's API endpoint.

[1097] Input: Text typed into the terminal

[1098] Output: JSON formatted document sent to the server

[1099] Step 3:

[1100] The server inputs the received sentences into the deep learning model.

[1101] Specifically, the server analyzes the received JSON data, extracts sentences, and inputs them into the deep learning model.

[1102] Input: JSON formatted text sent to the server

[1103] Output: The sentences fed into the deep learning model

[1104] Step 4:

[1105] A deep learning model analyzes text and generates animated videos.

[1106] Specifically, the deep learning model analyzes input sentences and generates animated scenes based on their content. For example, based on the sentence "The project is progressing smoothly," it generates a scene of project members working at their desks.

[1107] Input: The sentence fed into the deep learning model

[1108] Output: Generated animation video

[1109] Step 5:

[1110] The server transmits the generated animation to the terminal.

[1111] Specifically, the server encodes the generated animation video in MP4 format and sends it to the terminal as an HTTP response.

[1112] Input: Generated animation video

[1113] Output: Animated video in MP4 format sent to your device

[1114] Step 6:

[1115] The terminal displays the received animation video to the user.

[1116] Specifically, the device displays the received MP4 animation video in a player, and the user can play the video in a browser to view it.

[1117] Input: Animated video in MP4 format sent to the device

[1118] Output: Animated video displayed to the user

[1119] (Application example 3)

[1120] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1121] In conventional advertising production, creating visually appealing animated videos based on advertising copy requires a great deal of time and effort. Furthermore, because specialized knowledge and skills are required, it is difficult for advertisers to easily create videos themselves. This has led to issues such as reduced efficiency in advertising production and increased costs.

[1122] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[1123] In this invention, the server includes means for converting recorded data of an internal meeting into text using artificial intelligence, means for summarizing the text data using generative artificial intelligence, means for automatically generating an animated video from the summarized text, means for receiving the summarized advertising copy as input and generating an animated advertising video using an animation generation model based on deep learning, and means for displaying the generated animated advertising video. This makes it possible to automatically generate visually appealing animated advertising videos based on the summarized advertising copy provided by the advertiser.

[1124] "Internal meeting recording data" refers to audio recordings of meetings held within a company or organization.

[1125] "Artificial intelligence" is an information processing technology in which computer systems imitate human intelligence.

[1126] "Texting" is the process of converting audio data into text data.

[1127] "Generative AI" is AI that has the ability to generate new information and content from given data.

[1128] "Summarization" is the process of extracting important information from long text or data and summarizing it in a concise manner.

[1129] "Animation video" is a video format that uses a series of images to express movement.

[1130] "Deep learning" is a branch of machine learning that uses multi-layered neural networks to analyze data and learn patterns.

[1131] An "video generation model" is an algorithm or framework for generating video based on input data.

[1132] "Ad copy" is text created to promote a product or service.

[1133] An "animated advertising video" is a visually appealing animated advertisement generated based on advertising copy.

[1134] "Display" is the process of visually presenting generated content to a user.

[1135] A system for implementing the present invention has the following configuration.

[1136] System Configuration

[1137] 1. Hardware:

[1138] Server: A server equipped with a high-performance GPU

[1139] Devices: Smartphones, smart glasses

[1140] 2. Software:

[1141] Natural Language Processing (NLP) libraries (e.g., spaCy, NLTK)

[1142] Deep learning frameworks (e.g. TensorFlow, PyTorch)

[1143] Application Interface

[1144] Processing flow

[1145] 1. Receiving input data:

[1146] The user inputs the advertisement copy. For example, the advertisement copy is "The new smartphone is equipped with a high-performance camera."

[1147] 2. Data preprocessing:

[1148] The server uses a natural language processing library to tokenize the input ad copy and perform grammatical analysis.

[1149] 3. Application of video generation model:

[1150] The server passes the tokenized data to a video generation model pre-trained using a deep learning framework to generate animated advertising videos.

[1151] 4. Video output:

[1152] The generated animated advertising video is displayed to the user through the application interface of the terminal.

[1153] Specific examples

[1154] For example, if an advertiser enters the ad copy, "Our new smartphone is equipped with a high-performance camera," the system operates as follows:

[1155] 1. Receiving input data:

[1156] The user enters the ad copy through a smartphone application.

[1157] 2. Data preprocessing:

[1158] The server uses a natural language processing library to tokenize and parse the ad copy.

[1159] 3. Application of video generation model:

[1160] The server then inputs the tokenized data into a video generation model using a deep learning framework to generate animated advertising videos that highlight the smartphone's high-performance camera.

[1161] 4. Video output:

[1162] The generated animated advertising video is displayed to the user through the smartphone application interface.

[1163] In this way, it is possible to automatically generate visually appealing animated advertising videos based on the summarized advertising copy provided by the advertiser.

[1164] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[1165] Step 1:

[1166] The user inputs the advertisement copy through the smartphone application. For example, the advertisement copy might be, "Our new smartphone is equipped with a high-performance camera." This advertisement copy is then sent to the system.

[1167] Step 2:

[1168] The server receives the input ad copy and tokenizes it using a natural language processing library (e.g., spaCy, NLTK). The tokenized data is divided into each word and phrase of the sentence. Next, grammatical analysis is performed to analyze the structure of the sentence. This allows the meaning and important elements of the ad copy to be extracted.

[1169] Step 3:

[1170] The server then inputs the tokenized and parsed data into a pre-trained video generation model using a deep learning framework (e.g., TensorFlow, PyTorch). The video generation model generates animated ad videos based on the input data. The model creates appropriate visual effects and animations based on the content of the ad copy.

[1171] Step 4:

[1172] The generated animated advertising video is sent from the server to the terminal and displayed to the user through the terminal's application interface. The user can check the generated animated advertising video and modify or regenerate it as necessary.

[1173] In this way, it is possible to automatically generate visually appealing animated advertising videos based on the summarized advertising copy provided by the advertiser.

[1174] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1175] "Example 1"

[1176] One embodiment of the present invention is a system that combines an emotion engine. This system includes an emotion engine that recognizes emotions from a user's voice data. The emotion engine analyzes the acoustic characteristics and speaking patterns of the voice data and infers the user's emotional state from the results. Specifically, it analyzes characteristics of the voice data, such as pitch, volume, and speaking rate, and recognizes the emotional state indicated by these characteristics.

[1177] "Example 2"

[1178] The emotion engine also generates summaries based on the recognized emotional state. Specifically, if the user is recognized as being in a joyful or excited state, a positive summary reflecting that emotional state is generated. Conversely, if the user is recognized as being in a sad or angry state, a negative summary reflecting that emotional state is generated.

[1179] "Example 3"

[1180] Furthermore, the animation visually expresses the user's emotions. Specifically, if the user recognizes that they are in a happy emotional state, the animation is generated with bright colors and active movements that reflect that emotional state. Conversely, if the user recognizes that they are in a sad emotional state, the animation is generated with dark colors and quiet movements that reflect that emotional state.

[1181] The processing flow of each embodiment will be described below.

[1182] "Example 1"

[1183] Step 1: Collect voice data from the user.

[1184] Step 2: Analyze the acoustic characteristics and speaking patterns of the collected voice data.

[1185] Step 3: Estimate the user's emotional state from the analysis results.

[1186] "Example 2"

[1187] Step 1: Recognize the user's emotional state.

[1188] Step 2: Generate a summary based on the recognized emotional state.

[1189] Step 3: Present the generated summary to the user.

[1190] "Example 3"

[1191] Step 1: Recognize the user's emotional state.

[1192] Step 2: Generate an animated video that reflects the recognized emotional state.

[1193] Step 3: Present the generated animation video to the user.

[1194] Example 1

[1195] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1196] Conventional systems for transcribing recordings of internal meetings only provide a summary or visual representation of the meeting content, and are unable to grasp the emotional state of the speakers. This makes it difficult to understand the atmosphere of the meeting and the emotions of the speakers, making it difficult to accurately grasp the content of the meeting. Furthermore, because the generated report does not include the emotional state, there is a risk of missing important nuances in the meeting.

[1197] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1198] In this invention, the server includes means for converting recorded data of an internal meeting into text using artificial intelligence, means for summarizing the textual data using generative artificial intelligence, means for automatically generating animated videos from the summarized text, means for analyzing the acoustic characteristics of the audio data and speaking patterns to estimate emotional states, means for generating a report including the meeting content and emotional states, and means for providing the generated report to a user, thereby enabling an accurate understanding of the meeting content and the emotional states of speakers.

[1199] "Internal meeting recording data" refers to data that records the audio of an internal meeting.

[1200] "Artificial intelligence" is a technology that refers to the processing and judgment that computer systems perform by imitating human intelligence.

[1201] "Means of converting text" refers to techniques and methods for converting voice data into text data.

[1202] "Generative AI" is an AI technology that extracts important information from text data and generates summaries and new sentences.

[1203] "Methods of summarization" are techniques or methods for extracting important parts from long pieces of text or data and summarizing them in a concise manner.

[1204] "Means for automatically generating animated videos" refers to techniques and methods for automatically creating visually easy-to-understand animated videos based on text data.

[1205] "Acoustic characteristics of voice data" refers to acoustic features such as pitch, volume, and speaking rate contained in the voice data.

[1206] "Speaking patterns" refer to the characteristic rhythm, intonation, and word usage that can be seen in a speaker's speech.

[1207] "Means for estimating emotional state" refers to techniques or methods for estimating the speaker's emotions by analyzing the acoustic characteristics of speech data and speaking patterns.

[1208] The "means for generating a report" refers to a technique or method for creating a report including the meeting content and emotional state based on the results of speech recognition and emotion analysis.

[1209] The "means for providing to the user" refers to the technology or method for delivering the generated report to the user and making it viewable.

[1210] This invention is a system that uses artificial intelligence to convert recorded data from an internal meeting into text and analyze the emotional state of the participants. Specific embodiments of this system are described below.

[1211] First, the user uploads the recording of the internal meeting from their device to the server. The recording data is in audio file format such as WAV or MP3. The server receives the recording data uploaded by the user.

[1212] The server then converts the recording into text using speech recognition technology, using software such as Google Cloud Speech-to-Text API or IBM Watson Speech to Text, which uses deep learning-based speech recognition models to convert the audio data into text with high accuracy.

[1213] The text data is then passed to an emotion engine. The server uses an emotion engine, such as the Emotion API from Microsoft Azure Cognitive Services, to analyze the user's emotional state from the text data. The emotion engine analyzes the acoustic characteristics of the audio data (such as pitch, volume, and speaking rate) and speaking patterns to identify the emotional state indicated by these characteristics.

[1214] The server integrates the results of speech recognition and emotion analysis to generate a report containing the meeting content and the emotional state of each speaker. This report is generated in text or PDF format and provided to the user. The user can download the report via their device and check the meeting content and emotional state.

[1215] For example, the following prompts can be fed to a generative AI model:

[1216] "Upload recordings of internal meetings. The server uses speech recognition technology to convert the recordings into text, and then uses an emotion engine to analyze the emotional state of each speaker. Finally, it generates and provides a report showing the content and emotional state of the meeting."

[1217] In this way, the system processes while clarifying the roles of the server, terminal, and user. This system enables accurate understanding of the content of the meeting and the emotional state of the speaker, ensuring effective meeting management without overlooking important nuances.

[1218] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1219] Step 1:

[1220] A user uploads the recording data of an internal meeting from their device to the server.

[1221] Input: Recording data of internal meetings (audio file formats such as WAV and MP3)

[1222] Output: Audio file uploaded to the server

[1223] Specific operation: The user uses the file upload function of the device to send the recorded data to the server, and the server saves the received audio file in a specified directory.

[1224] Step 2:

[1225] The server receives the recorded data and begins the voice recognition process.

[1226] Input: Uploaded audio file

[1227] Output: Start signal for speech recognition processing

[1228] Specific operation: The server checks the existence of the received audio file and prepares to start the speech recognition process. Specifically, it checks the format of the audio file and converts it if necessary.

[1229] Step 3:

[1230] The server uses voice recognition technology to convert the recorded data into text.

[1231] Input: Audio file

[1232] Output: Text data

[1233] How it works: The server calls voice recognition software such as Google Cloud Speech-to-Text API or IBM Watson Speech to Text to convert the audio file into text, which is then temporarily stored on the server.

[1234] Step 4:

[1235] The server passes the converted data to the emotion engine.

[1236] Input: Text data

[1237] Output: Input data to the emotion engine

[1238] Specific operation: The server converts the text data into an appropriate format to be passed to the emotion engine. Specifically, the server converts the text data into JSON format or similar and sends it to the emotion engine's API.

[1239] Step 5:

[1240] The server uses an emotion engine to analyze the emotional state from the text data.

[1241] Input: Input data to the emotion engine

[1242] Output: Sentiment analysis results

[1243] How it works: The server uses an emotion engine, such as the Emotion API from Microsoft Azure Cognitive Services, to analyze the user's emotional state from the text data. The emotion engine analyzes the acoustic characteristics and speaking patterns of the audio data to estimate the user's emotional state. The analysis results are returned to the server.

[1244] Step 6:

[1245] The server generates a report containing the meeting content and emotional state.

[1246] Input: Speech recognition results, emotion analysis results

[1247] Output: Report (text or PDF format)

[1248] Specific operation: The server integrates the results of speech recognition and sentiment analysis to generate a report containing the meeting content and the emotional state of each speaker. The report is created in a specified format and saved on the server.

[1249] Step 7:

[1250] The server provides the generated report to the user.

[1251] Input: Generated report

[1252] Output: Report provided to user

[1253] Specific operation: The server generates a download link to provide the generated report to the user. The user can click the link through their device to download the report and check its contents.

[1254] (Application example 1)

[1255] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1256] To efficiently issue work instructions and reports within a factory, it is necessary to convert voice instructions into text in real time and to recognize the emotional state of workers and provide appropriate feedback. However, current systems do not adequately convert voice instructions into text or recognize emotions, which creates issues with work efficiency and worker stress management.

[1257] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means. In this invention, the server includes means for converting recorded data of an internal meeting into text using artificial intelligence, means for summarizing the textual data using generative artificial intelligence, means for automatically generating animated videos from the summarized text, means for converting voice instructions into text in real time using a robot that issues work instructions and reports in the factory by voice, means including an emotion engine that recognizes emotional states from voice data, and means for providing appropriate feedback according to the emotional state. This enables work instructions and reports to be issued efficiently in the factory and appropriate feedback according to the emotional state of workers.

[1258] "Internal meeting recording data" refers to audio recordings of meetings held within a company.

[1259] "Artificial intelligence" is an information processing technology in which computer systems imitate human intelligence.

[1260] "Texting" is the process of converting audio data into text data.

[1261] "Generative AI" is AI that has the ability to generate new information and content from input data.

[1262] "Summarization" is the process of extracting important information from long text or data and summarizing it in a concise manner.

[1263] "Animation video" is a technology that generates moving images.

[1264] "Work instructions and reports within the factory" refers to instructions and progress reports regarding work carried out within the factory.

[1265] A "robot" is a mechanical device that performs work automatically.

[1266] "Real-time" refers to processing occurring immediately without delay.

[1267] An "emotion engine" is a technology that recognizes emotional states from voice data and other input data.

[1268] "Feedback" is the response or instruction that the system provides to the user.

[1269] The system for implementing this invention uses artificial intelligence to convert recorded data from in-house meetings into text, and then applies it to a robot that issues work instructions and reports in a factory by voice. A specific embodiment of this system will be described below.

[1270] System configuration

[1271] Hardware

[1272] 1. Microphone: Used to record the voice of the worker.

[1273] 2. Robot: Hardware for receiving voice instructions, transcribing them and recognizing emotions.

[1274] 3. Server: Processes and stores voice data.

[1275] software

[1276] 1. Google Cloud Speech-to-Text API: Used to convert speech to text.

[1277] 2. Microsoft Azure Emotion API: Used to recognize emotions from speech.

[1278] 3. Generative AI: Extracts important information from documented data and generates summaries.

[1279] 4. Emotion Engine: Analyzes the acoustic characteristics and speaking patterns of voice data to estimate emotional states.

[1280] Data processing and calculation

[1281] Recording audio data

[1282] When the user issues a voice command through the microphone, the robot records the voice data, which is then sent to the server.

[1283] Text conversion of voice data

[1284] The server converts the recording into text using the Google Cloud Speech-to-Text API, a process that transcribes voice commands in real time.

[1285] emotion recognition

[1286] The server uses the Microsoft Azure Emotion API to recognize emotional states from text data, and the emotion engine analyzes characteristics of the voice data, such as pitch, volume, and speaking rate, to estimate the user's emotional state.

[1287] Providing Feedback

[1288] The server provides appropriate feedback based on the user's emotional state. For example, if the user says, "I'm tired today," the emotion engine will interpret this as "fatigue" and provide feedback such as, "Please take a break."

[1289] Specific examples

[1290] When a worker says to the robot, "I'm tired today," the robot records the voice and translates it into a sentence in real time. It then uses emotion recognition to determine that the worker is "tired," and provides feedback such as, "Please take a break."

[1291] Prompt Sentence Examples

[1292] When a worker says to the robot, "I'm tired today," the robot records the voice and converts it into text in real time. It then uses emotion recognition to determine that the worker is "fatigued," and provides feedback such as, "Please take a break."

[1293] In this way, work instructions and reports can be given efficiently within the factory, and appropriate feedback can be given according to the emotional state of the workers.

[1294] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1295] Step 1:

[1296] The user issues voice commands through a microphone. The input is the user's voice data, and the output is the recorded voice data. In concrete terms, when the user says, "I'm tired today," the voice is recorded by the microphone.

[1297] Step 2:

[1298] The robot sends the recorded voice data to the server. The input is the recorded voice data, and the output is the voice data sent to the server. In concrete terms, the robot uploads the voice data to the server via the network.

[1299] Step 3:

[1300] The server converts the voice data into text data using the Google Cloud Speech-to-Text API. The input is the voice data sent to the server, and the output is text data. Specifically, the server sends the voice data to the API and receives the returned text data.

[1301] Step 4:

[1302] The server uses the Microsoft Azure Emotion API to recognize emotional states from text data. The input is text data, and the output is emotional state data. Specifically, the server sends text data to the API and receives the returned emotional state data.

[1303] Step 5:

[1304] The server generates appropriate feedback depending on the emotional state. The input is emotional state data, and the output is a feedback message. Specifically, the server analyzes the emotional state, and if it determines that the user is "fatigued," it generates a message saying, "Please take a break."

[1305] Step 6:

[1306] The server sends the generated feedback message to the robot. The input is the feedback message, and the output is the feedback message sent to the robot. In concrete terms, the server sends the feedback message to the robot via the network.

[1307] Step 7:

[1308] The robot conveys a feedback message to the user. The input is the feedback message sent to the robot, and the output is the feedback message conveyed to the user. As a specific action, the robot verbally tells the user, "Please take a break."

[1309] Example 2

[1310] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1311] Conventional systems for converting meeting recording data into text and summarizing it have the problem that the summarized text does not reflect the user's emotional state, making it difficult to provide appropriate information to the user. Furthermore, there is a lack of a way to present the summarized text in a visually easy-to-understand format, which reduces the efficiency of information transmission.

[1312] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1313] In this invention, the server includes means for converting recorded data of an internal meeting into text using artificial intelligence, means for summarizing the text using generative artificial intelligence, means for adjusting the summarized text based on the user's emotional state using an emotion recognition engine, and means for automatically generating an animated video from the adjusted summary. This makes it possible to provide an appropriate summary that reflects the user's emotional state and to communicate information in a form that is visually easy to understand.

[1314] An "internal meeting" is a meeting held within a company or organization, where work progress, issues, and solutions are discussed.

[1315] "Recorded data" refers to data in which audio is recorded in digital or analog format, and is used to play back and review the contents of a meeting at a later date.

[1316] "Artificial intelligence" refers to computer systems that mimic human intelligence and have capabilities such as learning, reasoning, and recognition.

[1317] "Transcription" refers to the process of converting audio data or other forms of data into text form.

[1318] "Generative AI" refers to AI that has the ability to generate new text or information based on input data.

[1319] "Summarization" is the process of shortening long sentences or data and extracting only the important information.

[1320] An "emotion recognition engine" is software that analyzes and recognizes a user's emotional state from text and voice data.

[1321] A "user's emotional state" refers to the emotion a user is feeling at a particular point in time, and may include joy, sadness, anger, etc.

[1322] "Reconciliation" is the process of changing or modifying data or information based on specific criteria or conditions.

[1323] An "animated video" is a video that uses a series of images or videos to express movement, and is a means of visually conveying information.

[1324] "Auto-generation" refers to the process by which a system or program generates data or information without human intervention.

[1325] MODE FOR CARRYING OUT THE INVENTION

[1326] This invention is a system that uses artificial intelligence to convert recorded data from an internal meeting into text, then uses generative artificial intelligence to summarize it, and then uses an emotion recognition engine to adjust the summary based on the user's emotional state, ultimately automatically generating an animated video.

[1327] Hardware and software used

[1328] 1. Server:

[1329] The server receives the recorded data, transcribes it into text, summarizes it, recognizes emotions, adjusts the summary, and generates animated videos.

[1330] Software used: Deep learning-based text generation models (e.g., GPT-3), emotion recognition engines (e.g., IBM Watson Tone Analyzer).

[1331] 2. Terminal:

[1332] The terminal is used by the user to input the recording data and display the final summary or animated video.

[1333] The terminal communicates with the server through an internet connection.

[1334] 3. User:

[1335] The user inputs the recorded data of the conference into the terminal and checks the generated summary and animated video.

[1336] What the program does

[1337] 1. Data Entry:

[1338] The user inputs the recording data of the conference into an input field on the terminal.

[1339] Example: A user types the following sentence: "In today's meeting, we discussed the progress of a new project. Many challenges were identified, but some solutions were also proposed."

[1340] 2. Data transmission:

[1341] The device makes an API request to send the entered data to the server.

[1342] Example: A device sends input data to a server using an HTTP POST request.

[1343] 3. Data reception and summary generation:

[1344] The server summarizes the received data using natural language processing techniques.

[1345] Software used: Deep learning-based text generation models (e.g., GPT-3).

[1346] Example: A server summarizes the sentence "At today's meeting, we discussed the progress of the new project. Many issues were identified, but some solutions were also proposed" as "At the meeting, the progress, issues, and solutions of the new project were discussed."

[1347] 4. Emotion recognition:

[1348] The server uses an emotion recognition engine to analyze the user's emotional state.

[1349] Software used: Emotion recognition engine (e.g. IBM Watson Tone Analyzer).

[1350] Example: The server recognizes the user's emotional state as "joy" from the input data.

[1351] 5. Adjusting the summary:

[1352] The server adjusts the summary based on the perceived emotional state.

[1353] Example: If the user is in a "joy" emotional state, add positive elements to the summary. For example, you could write, "The meeting discussed the progress, challenges, and solutions of the new project, and many positive opinions were expressed."

[1354] 6. Generate animated videos:

[1355] The server automatically generates an animated video based on the adjusted summary.

[1356] Example: The server generates an animated video that presents the summary in a visually easy-to-understand format.

[1357] 7. Data transmission and display:

[1358] The server sends the final summary and animation video to the terminal.

[1359] The terminal displays the received summary and animation video to the user.

[1360] Example: The device displays a summary on the screen: "At the meeting, progress on the new project, challenges, and solutions were discussed, and many positive opinions were expressed." along with a corresponding animated video.

[1361] Prompt Sentence Examples

[1362] "Summarize the following sentence. If the user's emotional state is joy, generate a positive summary. Sentence: 'In today's meeting, we discussed the progress of the new project. Many challenges were identified, but some solutions were also proposed.'"

[1363] In this way, a system is realized in which the server, terminal, and user work together to summarize text and adjust it based on emotions, ultimately generating animated videos that are visually easy to understand.

[1364] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1365] Step 1:

[1366] The user inputs textual data.

[1367] Input: The user inputs the meeting recording in text format.

[1368] What happens: A user types into a text field using a smartphone or computer keyboard: "In today's meeting, we discussed the progress of a new project. Many challenges were identified, but some solutions were also proposed."

[1369] Output: The input text data.

[1370] Step 2:

[1371] The terminal sends the input data to the server.

[1372] Input: Text data entered by the user.

[1373] Specific operation: The device creates an API request to send the entered data to the server, and sends the data to the server using an HTTP POST request.

[1374] Output: The text data sent to the server.

[1375] Step 3:

[1376] A server receives the data and generates a summary using natural language processing techniques.

[1377] Input: Text data sent from the terminal.

[1378] Specific operation: The server analyzes the received data and generates a summary by invoking a deep learning-based text generation model (e.g., GPT-3).

[1379] Output: Summarized text data. For example, the summary generated is "The progress, challenges, and solutions of the new project were discussed at the meeting."

[1380] Step 4:

[1381] The server uses an emotion recognition engine to recognize the user's emotional state.

[1382] Input: Summarized text data.

[1383] Specific operation: The server uses an emotion recognition engine (e.g., IBM Watson Tone Analyzer) to analyze the user's emotional state from the summarized text data.

[1384] Output: The user's emotional state (e.g., happy, sad, angry, etc.). For example, the user's emotional state is recognized as "happy."

[1385] Step 5:

[1386] The server adjusts the summary based on the emotional state.

[1387] Input: Summarized text data and the user's emotional state.

[1388] Specific behavior: The server adjusts the summary based on the perceived emotional state, for example, if the user is in the "joy" emotional state, it adds positive elements to the summary.

[1389] Output: Adjusted summary text. For example, the summary generated is "The progress, challenges, and solutions of the new project were discussed at the meeting, and many positive opinions were expressed."

[1390] Step 6:

[1391] The server automatically generates an animated video based on the adjusted summary.

[1392] Input: The adjusted summary text.

[1393] Specific operation: The server calls software for generating an animated video based on the adjusted summary, and generates an animated video that presents the summary in a visually easy-to-understand form.

[1394] Output: Animated video.

[1395] Step 7:

[1396] The server sends the final summary and animation video to the device.

[1397] Input: Adjusted summary text and animated video.

[1398] Specific operation: The server creates an API response to send the final summary and animation video to the device, and sends the data to the device using an HTTP response.

[1399] Output: Summary text and animated video sent to device.

[1400] Step 8:

[1401] The device displays a summary and an animated video to the user.

[1402] Input: Summary text and animation video sent from the server.

[1403] Specific operation: The device displays the received summary and animation video on the screen so that the user can review them.

[1404] Output: Summary text and animated video displayed on the screen. For example, the summary "At the meeting, the progress, challenges, and solutions of the new project were discussed, and many positive opinions were expressed." is displayed along with the corresponding animated video.

[1405] (Application example 2)

[1406] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1407] Conventional systems for summarizing recorded meeting data generate summaries without taking into account the user's emotional state, which means they are unable to provide appropriate information based on the user's emotions. Furthermore, there is a lack of a way to present the summarized text in a visually easy-to-understand format. This makes it difficult for users to quickly and accurately understand the information.

[1408] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1409] In this invention, the server includes means for converting recorded data of an internal meeting into text using artificial intelligence, means for summarizing the text data using generative artificial intelligence, means for automatically generating an animated video from the summarized text, means for detecting the user's emotional state, and means for generating a summary based on the detected emotional state. This makes it possible to provide an appropriate summary according to the user's emotional state and to provide information in a form that is visually easy to understand.

[1410] "Internal meeting recording data" refers to audio recordings of meetings held within a company or organization.

[1411] "Artificial intelligence" refers to computer systems that mimic human intelligence and can learn and reason.

[1412] "Transcription" refers to the process of converting audio or other forms of data into text form.

[1413] "Generative AI" refers to AI that has the ability to generate new text or information from given data.

[1414] "Summarization" refers to the process of extracting and summarizing long pieces of text or data into short, important pieces.

[1415] "Animation video" refers to a video format that uses a series of images or videos to express movement.

[1416] "Auto-generation" refers to the process by which a system automatically creates data or content without manual intervention.

[1417] "User's emotional state" refers to the emotions or moods that a user is feeling at a particular point in time.

[1418] "Detection" refers to the process of identifying and recognizing specific information or conditions.

[1419] "Means" refer to the methods or techniques used to achieve a particular goal.

[1420] A system for implementing this invention has the following configuration: A server includes means for converting recorded data of an internal meeting into text using artificial intelligence, means for summarizing the textual data using generative artificial intelligence, means for automatically generating an animated video from the summarized text, means for detecting the emotional state of a user, and means for generating a summary based on the detected emotional state.

[1421] Hardware and software used

[1422] Hardware: Servers, smartphones

[1423] Software: Python, OpenAI API (GPT-3), TextBlob

[1424] Data processing and calculation

[1425] 1. Transcription of recordings:

[1426] The server receives the recordings of internal meetings and converts them into text using speech recognition technology, such as the Google Cloud Speech-to-Text API.

[1427] 2. Summarizing the documented data:

[1428] The server uses generative artificial intelligence (such as OpenAI's GPT-3) to summarize the textual data, extracting key information and summarizing it in a concise format.

[1429] 3. Detecting the user's emotional state:

[1430] The server receives the user's input text (e.g., "I feel very happy today!") and performs sentiment analysis using TextBlob to classify the emotional state as positive, negative, or neutral.

[1431] 4. Generating summaries based on emotional states:

[1432] The server generates a summary based on the detected emotional state, for example, if the user is in a positive emotional state, it provides a prompt sentence to the generative artificial intelligence to generate a positive summary.

[1433] 5. Automatic generation of animated videos:

[1434] The server automatically generates animated videos to present the summarized text in a visually easy-to-understand format, using video editing software such as Adobe After Effects.

[1435] Specific examples

[1436] If a user types "I'm feeling very happy today!", the server receives this text and uses TextBlob to detect the emotional state as "positive." The server then provides the following prompt to the generative AI to generate a positive summary:

[1437] Prompt Sentence Examples

[1438] Summarize the following news article: The emotional state is positive.

[1439] Today's news reported that economic growth is stronger than expected, and experts predict this growth will continue.

[1440] summary:

[1441] The summaries generated in this way are then visually presented as animated videos and provided to users, allowing them to quickly and accurately understand appropriate information according to their emotional state.

[1442] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1443] Step 1:

[1444] The server receives recordings of internal meetings. The input is audio data and the output is text data. The server uses speech recognition technology (e.g., Google Cloud Speech-to-Text API) to convert the audio data into text data. This process involves analyzing the audio data and converting it into text.

[1445] Step 2:

[1446] The server uses generative artificial intelligence (e.g., OpenAI's GPT-3) to summarize the text data. The input is text data, and the output is a summary text. The server extracts important information from the text data and summarizes it in a concise form. In this process, natural language processing techniques are used to summarize the text.

[1447] Step 3:

[1448] The user inputs text that indicates their emotional state. The input is the text that indicates the user's emotional state, and the output is the classification result of the emotional state. The server uses TextBlob to perform sentiment analysis from the user's input text. In this process, the text is sentiment analyzed and classified as positive, negative, or neutral.

[1449] Step 4:

[1450] The server generates a summary based on the detected emotional state. The input is the summary text and the emotional state, and the output is the summary text according to the emotional state. The server provides the generative AI with a prompt that reflects the emotional state, and generates a summary according to the emotion. In this process, the prompt is generated based on the emotional state and the summary is regenerated.

[1451] Step 5:

[1452] The server automatically generates animated videos to present the summarized text in a visually easy-to-understand format. The input is summary text corresponding to the emotional state, and the output is an animated video. The server uses video editing software (e.g., Adobe After Effects) to create the animated video based on the summary text. This process involves visualizing text data and generating a video.

[1453] Step 6:

[1454] The user receives and watches the generated animation. The input is the animation, and the output is the user's understanding. The user quickly and accurately understands the information provided in a visually easy-to-understand format. This process involves receiving and understanding information.

[1455] Example 3

[1456] Next, a description will be given of Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1457] In conventional methods for managing recorded meeting data, the process of converting the recorded data into text and then summarizing it is time-consuming and laborious. Furthermore, there is a lack of a way to present the summarized text in a visually easy-to-understand format, which reduces the efficiency of information transmission. Furthermore, technology for generating animated videos that reflect the user's emotional state is still underdeveloped, making it difficult to provide appropriate information according to the user's emotions.

[1458] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[1459] In this invention, the server includes means for converting recorded data of an internal meeting into text using artificial intelligence, means for summarizing the textual data using generative artificial intelligence, means for analyzing the summarized text using natural language processing technology, means for recognizing the user's emotional state, means for generating animated videos using a deep learning model based on the analysis results and the emotional state, and means for providing the generated animated videos to the user. This enables efficient management and summarization of recorded data, generation of visually easy-to-understand animated videos, and provision of appropriate information according to the user's emotional state.

[1460] "Internal meeting recording data" refers to data in which the content of statements made during a meeting is recorded in audio format.

[1461] "Artificial intelligence" refers to computer systems that mimic human intelligence and can learn and reason.

[1462] "Texting" is the process of converting audio data into text form.

[1463] "Generative AI" is AI that has the ability to extract important information from data and generate summaries and new content.

[1464] "Summarization" is the process of extracting important information from original data and putting it into a concise form.

[1465] "Natural language processing technology" is a technology that uses computers to understand, analyze, and generate human language.

[1466] "Analysis" is the process of examining data in detail to understand its structure and meaning.

[1467] "Emotional state" refers to the user's current feelings and mood.

[1468] "Recognition" is the process of identifying specific information or patterns in data.

[1469] A "deep learning model" is a model that uses a multi-layer neural network to learn data and make predictions or generate results.

[1470] "Animation video" is a video format that expresses movement by displaying a series of images.

[1471] "Serving" is the process of delivering generated content to a user.

[1472] This invention is a system for efficiently managing recording data of in-house meetings and converting summarized text into visually easy-to-understand animated videos. Specific embodiments of this system are described below.

[1473] Hardware and software used

[1474] The server uses the following hardware and software:

[1475] Hardware: A server with a powerful processor, sufficient memory, and storage

[1476] Software: Natural language processing libraries (e.g., spaCy, NLTK), emotion recognition models, deep learning models

[1477] A terminal is a device through which a user inputs data, and includes a personal computer, a tablet, a smartphone, and the like.

[1478] Data processing and calculation

[1479] 1. Transcription of recordings:

[1480] The server receives recordings of internal meetings and uses artificial intelligence to convert the audio data into text format, using voice recognition technology.

[1481] 2. Summarizing text data:

[1482] The server uses generative artificial intelligence to extract key information from the documented data and generate a summary. This summarization process succinctly summarizes the essence of the information.

[1483] 3. Analysis of the summarized text:

[1484] The server uses natural language processing technology to analyze the summarized sentences. For example, it analyzes the sentence "The project is progressing smoothly" and identifies scenes in which project members are working.

[1485] 4. User Emotion Recognition:

[1486] The server analyzes the user's input and facial expression data to recognize the user's emotional state. The user inputs the emotional state using the terminal or provides facial expression data through a camera.

[1487] 5. Generate animated videos:

[1488] The server uses a deep learning model to generate an animated video based on the analysis results and the user's emotional state. For example, the server inputs a prompt: "An animated video of project members actively working in bright colors."

[1489] 6. Providing generated videos:

[1490] The server provides the generated animation video to the user, who then uses the terminal to view the video via the provided link.

[1491] Specific examples

[1492] As a specific example, if the input is the summarized sentence "The project is progressing smoothly," the server will input the following prompt sentence into the generative AI model.

[1493] Example prompt sentence:

[1494] Summarized text: "The project is progressing smoothly."

[1495] Emotional state: "Joy"

[1496] Video to generate: "An animated video of project members actively working in bright colors."

[1497] Based on this prompt, the server uses a deep learning model to generate an animated video depicting project members working.

[1498] In this way, it is possible to efficiently manage and summarize recorded data, generate visually easy-to-understand animated videos, and provide appropriate information according to the user's emotional state. The flow of the identification process in the third embodiment will be described with reference to FIG.

[1499] Step 1:

[1500] Receiving and documenting the recordings

[1501] The server receives recordings of internal meetings from users. It receives audio data as input and converts it into text using speech recognition technology. Specifically, the server uses a speech recognition library (e.g., Google Speech-to-Text API) to analyze the audio data and convert it into text. The output is the meeting content in text format.

[1502] Step 2:

[1503] Summarizing text data

[1504] The server inputs the text data into a generative AI to perform summarization. It receives text data as input, extracts important information using natural language processing technology, and generates a concise summary. Specifically, the server uses a generative AI (e.g., BERT, GPT-3) to generate a summary. The output is a summarized sentence.

[1505] Step 3:

[1506] Analysis of summarized text

[1507] The server analyzes the summarized text using natural language processing technology. It receives the summary text as input and analyzes it to identify the content and scene of the text. Specifically, the server uses an NLP library (e.g., spaCy, NLTK) to analyze the meaning of the text. The output is scene information as the analysis result.

[1508] Step 4:

[1509] User Emotion Recognition

[1510] The server analyzes the user's input and facial expression data to recognize the user's emotional state. It receives the user's emotional data (text input and facial expression images) as input and analyzes it using an emotion recognition model. Specifically, the server uses the emotion recognition model to identify the user's emotional state. The output is the user's emotional state.

[1511] Step 5:

[1512] Generate animated videos

[1513] The server uses a deep learning model to generate an animated video based on the analysis results and the emotional state. It receives scene information and the emotional state as input, generates a prompt sentence, and inputs it into the deep learning model. Specifically, the server uses the generative AI model to input the prompt sentence: "An animated video of project members actively working in bright colors." The output is the generated animated video.

[1514] Step 6:

[1515] Serving generated videos

[1516] The server provides the generated animation video to the user. It receives the generated video as input and generates a link to provide it to the user. Specifically, the server saves the video in storage and sends a viewing link to the user. The output is a video link that the user can access.

[1517] (Application example 3)

[1518] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1519] Conventional systems for transcribing and summarizing meeting recordings lack the means to provide summarized information in a visually understandable format. Furthermore, content generation does not reflect the user's emotional state, making it difficult to obtain appropriate feedback based on the user's emotions.

[1520] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[1521] In this invention, the server includes means for converting recorded data of an internal meeting into text using artificial intelligence, means for summarizing the textual data using generative artificial intelligence, means for automatically generating an animated video from the summarized text, and means for analyzing the emotional state of a user and generating an animated video with colors and movements that reflect that emotional state. This makes it possible to provide summarized information in a visually easy-to-understand format and to obtain appropriate feedback according to the user's emotional state.

[1522] "Internal meeting recording data" refers to audio recordings of meetings held within a company or organization.

[1523] "Artificial intelligence" refers to computer systems that mimic human intelligence and can learn and reason.

[1524] "Texting" refers to the process of converting audio data into text data.

[1525] "Generative AI" refers to AI that has the ability to generate new information and content from data.

[1526] "Summarization" refers to the process of extracting important information from long text or data and summarizing it in a concise manner.

[1527] "Animation video" refers to a video format that uses a series of images to express movement.

[1528] "User's emotional state" refers to the emotions or moods that a user is feeling at a particular moment.

[1529] "Color tone" refers to the placement and balance of colors in an image or video.

[1530] "Movement" refers to the movement and changes of characters and objects in animated videos.

[1531] "Visually easy to understand" refers to a state in which information is presented in a way that makes it easy to understand visually.

[1532] "Feedback" refers to returning the results or reactions of a system or process to the user.

[1533] A system for implementing this invention has the following configuration: A server includes means for converting recorded data of an internal meeting into text using artificial intelligence, means for summarizing the textual data using generative artificial intelligence, means for automatically generating animated videos from the summarized text, and means for analyzing the emotional state of a user and generating animated videos with colors and movements that reflect that emotional state.

[1534] Hardware and software used

[1535] Hardware: Servers, smartphones

[1536] software:

[1537] Python: Programming Language

[1538] Sentiment Analysis Model

[1539] Video Generation API

[1540] Data processing and calculation

[1541] 1. Transcription of recordings:

[1542] The server receives the recordings of internal meetings and converts them into text using speech recognition technology, using a speech recognition model.

[1543] 2. Summarizing text data:

[1544] Generative AI extracts key information from written data and generates summaries, a process that uses natural language processing techniques.

[1545] 3. Generate animated videos:

[1546] The server receives the summarized text as input and generates an animated video using a deep learning-based video generation model. In doing so, it analyzes the user's emotional state and generates an animated video with colors and movements that reflect that emotional state.

[1547] Specific examples

[1548] For example, if a user types the sentence "The project is progressing smoothly," the server summarizes the sentence and performs a sentiment analysis. If the result of the sentiment analysis is positive, it generates an animated video with bright colors and lively movements.

[1549] Prompt Sentence Examples

[1550] Input sentence: "The project is progressing smoothly."

[1551] Outputs the URL of the generated animation video.

[1552] In this way, summarized information can be provided in a visually easy-to-understand format, and appropriate feedback can be obtained according to the user's emotional state.

[1553] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[1554] Step 1:

[1555] The server receives the recording data of the internal meeting and converts it into text data using speech recognition technology. The input is the recording data and the output is text data. Specifically, the server analyzes the speech data using a speech recognition model and generates corresponding text.

[1556] Step 2:

[1557] The server inputs the generated text data into a generative artificial intelligence (AI) system, extracts important information, and generates a summary. The input is text data, and the output is a summarized text. Specifically, the system uses natural language processing technology to analyze the text data and extract important keywords and phrases to create a summary.

[1558] Step 3:

[1559] The server inputs the summarized text into a sentiment analysis model to analyze the user's emotional state. The input is the summarized text, and the output is the emotional state (positive, negative, neutral). Specifically, the sentiment analysis model is used to analyze the sentiment of the text and obtain the results.

[1560] Step 4:

[1561] The server inputs the summarized text and emotional state into the video generation API to generate an animated video. The input is the summarized text and emotional state, and the output is the URL of the animated video. Specifically, the server sends the summarized text and emotional state to the video generation API and obtains the URL of the generated animated video.

[1562] Step 5:

[1563] The device provides the user with the URL of the animation video obtained from the server. The input is the URL of the animation video, and the output is an animation video that the user can watch. Specifically, the video is displayed to the user through the device's browser or application.

[1564] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1565] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1566] Another example of generative AI is Gemini (internet search engine). <url: https: gemini.google.com ?hl="ja">) are mentioned.

[1567] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1568] [Third embodiment]

[1569] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1570] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1571] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1572] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1573] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1574] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1575] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1576] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1577] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1578] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1579] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1580] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[1581] "Example 1"

[1582] This embodiment of the present invention implements a system that uses artificial intelligence to convert recorded data from an internal meeting into text. Specifically, the system receives the recorded data of the meeting as input and converts it into text using speech recognition technology. This speech recognition technology can use a speech recognition model based on deep learning.

[1583] "Example 2"

[1584] Next, the textual data is summarized using generative artificial intelligence. Specifically, the textual data is received as input and summarized using natural language processing technology. This natural language processing technology can use a text generation model based on deep learning.

[1585] "Example 3"

[1586] Finally, the summarized text is automatically converted into an animated video. Specifically, the summarized text is received as input and converted into an animated video using video generation technology. This video generation technology can use a video generation model based on deep learning. As a specific example, the summarized text "The project is progressing smoothly" can be converted into an animated video depicting project members working.

[1587] The processing flow of each embodiment will be described below.

[1588] "Example 1"

[1589] Step 1: Receive the recording data of the meeting as input. This recording data is the audio data of an internal meeting.

[1590] Step 2: The recorded data is converted into text using speech recognition technology. This speech recognition technology uses a deep learning-based speech recognition model. This model is trained from a large amount of audio data and its associated text data.

[1591] "Example 2"

[1592] Step 1: Receive textual data as input. This data is the content of a meeting that has been converted into text using speech recognition technology.

[1593] Step 2: Summarize the text data using natural language processing technology. This natural language processing technology uses a text generation model based on deep learning. This model is trained from large amounts of text data and has the ability to generate summaries of the text.

[1594] "Example 3"

[1595] Step 1: Receive a summarized text as input. This text is the content of a meeting summarized using natural language processing technology.

[1596] Step 2: Using video generation technology, the summarized text is converted into an animated video. This video generation technology uses a deep learning-based video generation model. This model is trained from a large amount of video data and its associated text data, and has the ability to generate corresponding videos from text.

[1597] Step 3: For example, convert the summary sentence "The project is progressing smoothly" into an animated video showing project members working. This video is intended to present the content of the meeting in a visually easy-to-understand way.

[1598] Example 1

[1599] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1600] There is a need to efficiently convert recordings of internal meetings into text, summarize the text, and automatically generate animated videos that are easy to understand visually. However, in conventional systems, the entire process from receiving and saving the recordings, calling up the speech recognition model, generating and saving the text data, and providing access to the user was not automated, and manual operations were required. This resulted in issues such as a lack of efficiency and accuracy.

[1601] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1602] In this invention, the server includes means for converting recorded data of an internal meeting into text using artificial intelligence, means for summarizing the textual data using generative artificial intelligence, means for automatically generating an animated video from the summarized text, means for receiving and saving the recorded data, means for calling a speech recognition model and inputting the recorded data to generate text data, and means for saving the generated text data and making it accessible to users. This automates the entire process from receiving the recorded data to generating, saving, and providing the text data to users, enabling efficient and accurate processing.

[1603] "Internal meeting recording data" refers to digital data that records the audio of meetings held within a company.

[1604] "Artificial intelligence" refers to the technology of data processing and decision-making that enables computer systems to mimic human intelligence.

[1605] "Means for converting text" refers to technology or devices for converting voice data into text data.

[1606] "Generative AI" is AI that has the ability to generate new data from existing data.

[1607] A "summarization tool" is a technique or device that extracts important information from a long piece of text and summarizes it in a short form.

[1608] "Means for automatically generating animated videos" refers to technology or devices that automatically create videos that are visually easy to understand based on text data.

[1609] "Means for receiving and storing recorded data" refers to technology or devices that receive audio data transmitted from an external source and store it in digital form.

[1610] A "speech recognition model" is an algorithm or program for analyzing speech data and generating corresponding text data.

[1611] The "means for generating text data" refers to a technique or device that creates text data from voice data using a voice recognition model.

[1612] "Means for storing the generated text data and making it accessible to users" refers to technologies and devices that store the generated text data in digital form and allow users to view or retrieve it.

[1613] This invention relates to a system that uses artificial intelligence to convert recorded data from an internal meeting into text, summarizes the text, and automatically generates an animated video that is visually easy to understand. A specific embodiment of this system is described below.

[1614] System configuration

[1615] Hardware and software used

[1616] Hardware: High-performance server (e.g., server equipped with GPU)

[1617] Software: Speech recognition models (e.g., Google Speech-to-Text API, DeepSpeech), generative artificial intelligence, animation generation software

[1618] Program processing

[1619] Subject: Server

[1620] 1. Receiving and storing recording data

[1621] The server receives the recording data of the internal meeting uploaded by the user. The received data is saved in the specified directory. For example, it is saved in the path " / uploads / meeting_recording_20231001.wav".

[1622] 2. Calling the speech recognition model

[1623] The server prepares an API to call the speech recognition model. For example, if you use the Google Speech-to-Text API, you set the API key and endpoint.

[1624] 3. Inputting the recorded data into the speech recognition model

[1625] The server inputs the saved recording data into the speech recognition model. Specifically, it sends an API request specifying the path of the recording data. For example, it sends " / uploads / MeetingRecordings_20231001.wav" to the Google Speech-to-Text API.

[1626] 4. Generating Text Data

[1627] The speech recognition model analyzes the recorded data and generates corresponding text data. For example, if the recorded data contains the sentence, "In today's meeting, we discussed the development schedule for a new product," the model generates text data with the same content.

[1628] 5. Saving the generated text data

[1629] The server stores the generated text data, for example, in a file named " / texts / meetingrecording_20231001.txt."

[1630] 6. Providing Access to Users

[1631] The server makes the generated text data accessible to the user. The user accesses " / texts / meetingrecording_20231001.txt" using a web browser to view the text data.

[1632] Specific examples

[1633] For example, if a recording of an internal meeting is uploaded to a server with the file name "Meeting_Recording_20231001.wav," the server inputs this file into a speech recognition model. The speech recognition model analyzes the recording and generates text data like the following:

[1634] Recording contents:

[1635] "At today's meeting, we discussed the development schedule for new products."

[1636] Generated text data:

[1637] "At today's meeting, we discussed the development schedule for new products."

[1638] Prompt Sentence Examples

[1639] Examples of prompts to be input to a generative AI model include:

[1640] "Please convert the following recording to text: Meeting Recording_20231001.wav"

[1641] Using this prompt, the server inputs the recorded data into a speech recognition model and begins the process of generating text data.

[1642] The above is a specific embodiment of the present invention. This system automates the entire process from receiving recorded data to generating text data, saving it, and providing it to the user, making it possible to process the data efficiently and accurately.

[1643] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1644] Step 1:

[1645] Users upload recordings

[1646] The user uploads the recording data of an internal meeting from their device to the server. Specifically, the user opens a web browser and accesses the system's upload page. They click the file selection button and select "Meeting Recording_20231001.wav" from their local disk. When they click the upload button, the recording data is sent to the server.

[1647] Input: Recording data file selected by the user (e.g. "Meeting Recording_20231001.wav")

[1648] Output: Recording data is sent to the server

[1649] Step 2:

[1650] The server receives and stores the recording data.

[1651] The server receives the recording data uploaded by the user. The received data is saved in the specified directory. For example, it is saved in the path " / uploads / meetingrecording_20231001.wav".

[1652] Input: Recording data sent by the user

[1653] Output: Recording data saved in the specified directory on the server

[1654] Step 3:

[1655] The server calls the speech recognition model

[1656] The server prepares an API to call the speech recognition model. For example, if you use the Google Speech-to-Text API, you set the API key and endpoint.

[1657] Input: Path to the recording data (e.g. " / uploads / MeetingRecording_20231001.wav")

[1658] Output: The speech recognition model is ready to be invoked

[1659] Step 4:

[1660] The server inputs the recording into a speech recognition model.

[1661] The server inputs the saved recording data into the speech recognition model. Specifically, it sends an API request specifying the path of the recording data. For example, it sends " / uploads / MeetingRecordings_20231001.wav" to the Google Speech-to-Text API.

[1662] Input: Path to the recording data (e.g. " / uploads / MeetingRecording_20231001.wav")

[1663] Output: The recording sent to the speech recognition model

[1664] Step 5:

[1665] The speech recognition model analyzes the recorded data and generates text data.

[1666] The speech recognition model analyzes the recorded data and generates corresponding text data. For example, if the recorded data contains the sentence, "In today's meeting, we discussed the development schedule for a new product," the model generates text data with the same content.

[1667] Input: The recording sent to the speech recognition model

[1668] Output: Generated text data

[1669] Step 6:

[1670] The server stores the generated text data.

[1671] The server stores the generated text data, for example, in a file named " / texts / meetingrecording_20231001.txt."

[1672] Input: Generated text data

[1673] Output: Text data saved in the specified directory on the server

[1674] Step 7:

[1675] The user checks the generated text data

[1676] The user accesses the server and checks the generated text data. Specifically, the user uses a web browser to access " / texts / MeetingRecording_20231001.txt" and view the text data.

[1677] Input: Path of the text data stored on the server

[1678] Output: Text data viewed by the user

[1679] (Application example 1)

[1680] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1681] Conventional systems for converting recorded data from internal meetings into text can summarize the contents of meetings and generate animated videos to aid visual understanding, but they lack the functionality to convert recorded data from work instructions and maintenance meetings within factories into text in real time and provide it to workers and managers. This poses the problem of not being able to fully improve work efficiency or reduce errors within factories.

[1682] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1683] In this invention, the server includes means for converting recorded data of in-house meetings into text using artificial intelligence, means for summarizing the textual data using generative artificial intelligence, means for automatically generating the summarized text into an animated video, means for converting recorded data of work instructions and maintenance meetings within a factory into text in real time, and means for providing the real-time textual data to workers and managers. This makes it possible to convert the contents of work instructions and maintenance meetings within a factory into text in real time and provide it to workers and managers.

[1684] An "internal meeting" is a meeting held within a company or organization, where business progress, problems, future policies, etc. are discussed.

[1685] "Recorded data" means data in which sound is recorded in digital or analog format and stored in a format that allows for later playback and analysis.

[1686] "Artificial intelligence" refers to a computer system that mimics human intelligence and has capabilities such as learning, reasoning, and recognition.

[1687] "Texting" is the process of converting non-text data, such as audio or video, into text form.

[1688] "Generative AI" is AI that has the ability to generate new information and content from given data.

[1689] "Summarization" is the process of extracting important information from long text or data and summarizing it in a concise manner.

[1690] "Animation video" is a video format that expresses movement by displaying a series of consecutive images in sequence.

[1691] "Inside a factory" refers to the facility where products are manufactured or processed in the manufacturing or production industries.

[1692] "Work instructions" are specific instructions and procedures for carrying out a specific task.

[1693] A "maintenance meeting" is a meeting where discussions and plans are made regarding the maintenance and inspection of equipment and machinery.

[1694] "Real-time" refers to data processing and information provision occurring immediately, without delay.

[1695] "Workers" are workers who actually perform work in factories or on-site.

[1696] An "administrator" is a person responsible for running and supervising an organization or project.

[1697] A system for implementing this invention is configured as follows: First, a server acquires recorded data of internal meetings, work instructions in a factory, and maintenance meetings. Next, the server converts this recorded data into text using speech recognition technology. This speech recognition technology uses Wav2Vec2, a speech recognition model based on deep learning.

[1698] The server summarizes the text data using a generative AI. This generative AI is implemented using the Transformers library. The summarized text is then automatically generated into an animated video. This animated video is intended to present the summarized text in a visually easy-to-understand format.

[1699] The server also converts work instructions and recordings of maintenance meetings into text in real time and provides it to workers and managers. This real-time text conversion contributes to improving work efficiency and reducing errors.

[1700] The specific hardware used is a microphone and a computer, and the software used is Python, a speech recognition library, the Transformers library, and a Wav2Vec2 model.

[1701] For example, if you use recorded data from a maintenance meeting held in a factory, when you input this data into the server, the following text will be output.

[1702] "At today's maintenance meeting, it was confirmed that Machine A requires a routine inspection. The next inspection is scheduled for next Monday."

[1703] An example of a prompt sentence to input to the generative AI model is as follows:

[1704] "Please convert the following audio data to text: [path to audio data]"

[1705] This prompt sentence can be used to efficiently convert voice data into text.

[1706] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1707] Step 1:

[1708] The server acquires recorded data from internal meetings, work instructions in factories, and maintenance meetings. The input is the recorded data, and the output is an audio data file. Specifically, audio is recorded using a microphone and the data is saved in digital format.

[1709] Step 2:

[1710] The server converts the acquired recorded data into text using speech recognition technology. The input is a voice data file, and the output is text data. Specifically, a speech recognition library is used to convert the voice data into text. In this process, speech recognition is performed using the Wav2Vec2 model.

[1711] Step 3:

[1712] The server summarizes the text data using generative artificial intelligence. The input is text data, and the output is summarized text data. Specifically, it uses the Transformers library to extract important information from the text data and generate a summary.

[1713] Step 4:

[1714] The server automatically generates an animated video from the summarized text. The input is the summarized text data, and the output is an animated video file. Specifically, animation generation software is used to express the summarized text as a video in a visually easy-to-understand format.

[1715] Step 5:

[1716] The server converts factory work instructions and recordings of maintenance meetings into text in real time. The input is real-time voice data, and the output is real-time text data. Specifically, it performs real-time voice recognition and instantly converts it into text.

[1717] Step 6:

[1718] The server provides documented data to workers and managers in real time. The input is real-time text data, and the output is text information displayed on the terminals of the workers and managers. Specifically, the text data is sent via the network and displayed on the terminals of the workers and managers.

[1719] Example 2

[1720] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1721] In conventional systems, the process of transcribing recordings of internal meetings and summarizing them was often done manually, which was time-consuming and labor-intensive. Furthermore, there was a lack of a way to present the summarized text in a visually easy-to-understand format. Furthermore, there was no system that could efficiently summarize text entered by users.

[1722] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1723] In this invention, the server includes means for converting recorded data of an internal meeting into text using artificial intelligence, means for summarizing the textual data using generative artificial intelligence, means for automatically generating an animated video from the summarized text, means for a terminal to send text entered by a user to the server, means for the server to preprocess the received text and input a prompt sentence to the generative artificial intelligence, means for the generative artificial intelligence to generate a summary and the server to send the summary result to the terminal, and means for the terminal to display the summary result to the user. This automates the process from converting recorded data into text to summarizing and visual representation, enabling efficient summarization of text entered by a user.

[1724] "Internal meeting recording data" refers to data in which the content of statements made during a meeting is recorded in audio format.

[1725] "Artificial intelligence" refers to computer systems that mimic human intelligence and can learn and reason.

[1726] "Texting" is the process of converting audio data or other forms of data into text form.

[1727] "Generative AI" is AI that has the ability to generate new text or information based on input data.

[1728] "Summarization" is the process of shortening long sentences or data and extracting only the important information.

[1729] "Animation video" is a video format that uses a series of images or videos to express movement.

[1730] A "terminal" is a device such as a computer or smartphone that is operated by a user.

[1731] A "server" is a computer system that processes and stores data on a network.

[1732] "Preprocessing" is the initial stage of processing to convert data into a format that is easier to analyze and process.

[1733] A "prompt sentence" is an input sentence that causes a generative artificial intelligence to perform a specific task.

[1734] The "summarization result" is the shortened text that is generated through the summarization process.

[1735] "User" refers to a person who uses the system.

[1736] This invention is a system that uses artificial intelligence to convert recorded data from internal meetings into text, then uses generative artificial intelligence to summarize the text, and then automatically generates an animated video from the summarized text. It also includes a function to efficiently summarize text entered by the user.

[1737] Hardware and software used

[1738] Server: A server equipped with a high-performance GPU

[1739] Software: OpenAI GPT-3, BERT, Python, Flask (web framework)

[1740] Device: A computer or smartphone that the user operates.

[1741] Data processing and calculation

[1742] Server Processing

[1743] The server receives text data provided by the user. This data is text stored in a text file or database. The server then uses a deep learning-based text generation model to summarize the received data using natural language processing techniques. Examples of such models include OpenAI's GPT-3 and BERT.

[1744] The server preprocesses the received text, tokenizing and normalizing it. For example, it uses NLTK or spaCy to tokenize the text and remove unnecessary whitespace and special characters. The preprocessed text is then converted into a format that can be input to the generative artificial intelligence.

[1745] Terminal handling

[1746] The terminal is responsible for transmitting the text data entered by the user to the server. When the user enters text through the terminal interface and presses the send button, the terminal transmits the data to the server. The terminal also displays the summary results returned from the server to the user.

[1747] User Action

[1748] The user inputs text data using the terminal. For example, the user inputs "Please summarize this text" and provides the specific text. The user can check the summary results displayed on the terminal and request another summary if necessary.

[1749] Specific examples

[1750] Prompt Sentence Examples

[1751] Examples of prompts that users can input into a generative AI model include:

[1752] "Please summarize the following sentence: 'This system is designed to summarize sentences entered by users. Specifically, it uses a deep learning-based text generation model to generate short summaries of the entered sentences.'"

[1753] In this way, the user can utilize a system that allows them to easily summarize text.

[1754] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1755] Step 1:

[1756] The user inputs a sentence.

[1757] A user inputs a sentence to be summarized through the terminal interface. For example, the user might input, "This system is designed to summarize the sentence entered by the user. Specifically, it uses a text generation model based on deep learning to summarize the entered sentence in a short form." The input data is in text format.

[1758] Step 2:

[1759] The terminal sends the input text to the server.

[1760] The terminal sends the text entered by the user to the server. Specifically, it uses an HTTP request to send the text data to the server. For example, if you are using Flask, you can send a POST request using the requests library. The input is the text data entered by the user, and the output is what is sent to the server.

[1761] Step 3:

[1762] The server receives the text and performs preprocessing.

[1763] The server receives text sent from the terminal. The received text is tokenized and normalized as preprocessing. For example, NLTK or spaCy is used to tokenize the text and remove unnecessary whitespace and special characters. The input is the text data received from the terminal, and the output is the preprocessed text data.

[1764] Step 4:

[1765] The server inputs a prompt sentence into the generative AI model.

[1766] The server converts the preprocessed text into a format that can be input to the generative AI model. Specifically, it creates a prompt sentence and inputs it to the generative AI model. For example, it creates a prompt sentence such as, "Please summarize the following sentence: 'This system is designed to summarize sentences entered by users. Specifically, it uses a text generation model based on deep learning to summarize the input sentence in a concise manner.'" The input is the preprocessed text data, and the output is the prompt sentence.

[1767] Step 5:

[1768] A generative AI model generates the summary.

[1769] The generative AI model analyzes the input prompt sentence and generates a summary. For example, if OpenAI's GPT-3 is used, an API is called to generate the summary. The generated summary will look something like, "This system summarizes the sentence entered by the user using a deep learning model." The input is the prompt sentence, and the output is the summarized text data.

[1770] Step 6:

[1771] The server sends the summary results to the terminal.

[1772] The server sends the generated summary to the terminal. Specifically, it returns the summary result using an HTTP response. For example, if you are using Flask, you can use the jsonify function to return the summary result in JSON format. The input is the summarized text data, and the output is what is sent to the terminal.

[1773] Step 7:

[1774] The terminal displays the summary results to the user.

[1775] The terminal displays the summary results received from the server to the user. Specifically, it uses HTML and JavaScript to display the summary results in a user interface. For example, it displays the summary results in a text area or a dialog box. The input is the summary results received from the server, and the output is what is displayed to the user.

[1776] (Application example 2)

[1777] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1778] In today's information society, a huge amount of information is generated every day. In particular, text data such as internal meetings and news articles contain a large amount of information, making it difficult to quickly grasp important information. In addition, there is a lack of means to present this information in a visually easy-to-understand format, so efficient information processing and understanding are required.

[1779] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for converting recorded data of an internal meeting into text using artificial intelligence, means for summarizing the textual data using generative artificial intelligence, means for visually displaying the summarized text, and means for providing the summarized text to the user. This makes it possible to quickly extract important information from huge amounts of text data and provide it in a visually easy-to-understand format.

[1780] "Internal meeting recording data" refers to data that records the audio of meetings held within a company or organization.

[1781] "Artificial intelligence" refers to computer systems that mimic human intelligence and can learn and reason.

[1782] "Means for converting text" refers to technology or devices for converting voice data into text data.

[1783] "Generative AI" is AI that has the ability to generate new text based on input data.

[1784] A "summarization method" is a technique or device for extracting important information from long text data and summarizing it in a concise manner.

[1785] A "visual display means" is any technique or device for displaying text data in a graphical format.

[1786] "Means for providing to the user" refers to the technology or device used to deliver the generated summary or visual display to the user.

[1787] The system for implementing this invention uses artificial intelligence to convert recorded data from internal meetings into text, then uses generative artificial intelligence to summarize the text, visually displaying the summarized text and providing it to the user.

[1788] System configuration

[1789] The system consists of the following main components:

[1790] 1. Means of converting recorded data into text: Speech recognition software to convert recorded data into text data. Specifically, we use Google Speech-to-Text API, etc.

[1791] 2. Generative AI: A generative AI model for summarizing textual data. Specifically, we use a model such as the BERT model using the Transformers library.

[1792] 3. Visual display means: A graphical user interface (GUI) for visually displaying the summarized text. Specifically, a web application using HTML / CSS and JavaScript.

[1793] 4. User delivery method: A notification system to provide users with summarized information, specifically via a smartphone app or web browser.

[1794] Program processing explanation

[1795] Transcription of recorded data

[1796] The server receives recordings of internal meetings and converts them into text using speech recognition software, using the Google Speech-to-Text API.

[1797] Documented data summary

[1798] Generative AI takes text data as input, extracts important information, and generates a summary. This process is done using the Transformers library.

[1799] Visual display of summarized data

[1800] The summarized text data is displayed in a visually understandable format, specifically through a web application using HTML / CSS and JavaScript.

[1801] Provision to users

[1802] Finally, the summarized information is provided to the user via a smartphone app or web browser.

[1803] Specific examples

[1804] For example, if a user selects "articles about Japan's economic growth," the system operates as follows:

[1805] 1. Transcription of audio recording: "Japan's economy continues to grow in 2023. Innovation in technology, in particular, is driving economic growth. The government has introduced new policies to support corporate innovation. This has led to the creation of many start-ups and the emergence of new business models. Furthermore, international cooperation is progressing, and Japanese technology is gaining recognition around the world."

[1806] 2. Summary of the written data: "Japan's economy will continue to grow in 2023, driven in particular by the technology sector. New government policies are encouraging the creation of new startups and promoting international collaboration."

[1807] 3. Visual display of summarized data: The summarized text is displayed on the web application.

[1808] 4. Provision to users: Users will be notified via a smartphone app.

[1809] Prompt Sentence Examples

[1810] "Summarize the following news article: Japan's economy continues to grow in 2023. Innovation in technology, in particular, is driving economic growth. The government has introduced new policies to support corporate innovation. This has led to the creation of many start-ups and the emergence of new business models. Furthermore, international cooperation is progressing, and Japanese technology is gaining recognition around the world."

[1811] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1812] Step 1:

[1813] The server receives the recording data of the internal meeting. The input is audio data, and the output is the raw audio data. This audio data is converted into text data in the next step.

[1814] Step 2:

[1815] The server uses speech recognition software (e.g., Google Speech-to-Text API) to convert the recorded data into text data. The input is voice data, and the output is text data. This conversion process turns the voice data into text.

[1816] Step 3:

[1817] The server uses generative artificial intelligence (e.g., the BERT model using the Transformers library) to summarize the text data. The input is text data, and the output is summarized text data. This summarization process extracts important information and summarizes it in a concise manner.

[1818] Step 4:

[1819] The server generates a graphical user interface (GUI) to visually display the summarized text data. The input is the summarized text data, and the output is visually displayable data (e.g., a web page using HTML / CSS and JavaScript). This display process makes it easier for users to visually understand the summarized information.

[1820] Step 5:

[1821] The server provides the user with visually displayed summary data. The input is visually displayable data, and the output is summary information displayed on the user's device. This provision process allows the user to receive the summary information through a smartphone app or web browser.

[1822] Step 6:

[1823] The user checks the provided summary information and obtains more detailed information as needed. The input is the user's operation, and the output is the display of detailed information. This checking process allows the user to efficiently grasp important information and obtain more detailed information as needed.

[1824] Example 3

[1825] Next, a third embodiment of the third embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1826] In the conventional method of managing meeting recording data, the recording data is converted into text and then summarized, which is time-consuming and laborious. Furthermore, there is a lack of a way to present the summarized text in a visually easy-to-understand format, which reduces the efficiency of information transmission. This makes it difficult to quickly and effectively share the contents of the meeting.

[1827] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[1828] In this invention, the server includes means for converting recorded data of an internal meeting into text using artificial intelligence, means for summarizing the textual data using generative artificial intelligence, means for automatically generating an animated video from the summarized text using a deep learning model, means for transmitting the generated animated video to a terminal, and means for displaying the received animated video to a user, thereby enabling the content of the meeting to be shared quickly and effectively.

[1829] "Internal meeting recording data" refers to data in which the content of statements made during a meeting is recorded in audio format.

[1830] "Artificial intelligence" is a technology that mimics human intelligence and automatically analyzes and processes data.

[1831] "Texting means" refers to techniques or methods for converting audio data into text format.

[1832] "Generative AI" is AI that has the ability to generate new information or summaries from input data.

[1833] "Methods of summarization" are techniques or methods for extracting important information from long pieces of text or data and summarizing it in a concise manner.

[1834] A "deep learning model" is a technology that uses multi-layered neural networks to analyze data and learn patterns.

[1835] "Animation video" is a video format that expresses movement by displaying a series of images.

[1836] "Means for automatic generation" refers to techniques or methods for automatically generating output data based on specific input data.

[1837] A "terminal" is a device operated by a user, and includes a computer, a smartphone, etc.

[1838] A "transmission medium" is a technique or method for transferring data from one place to another.

[1839] "Display means" refers to the techniques and methods used to visually present data to the user.

[1840] The present invention provides a system for efficiently managing recorded data of in-house meetings and providing summarized information in a visually easy-to-understand format. Specific embodiments of this system are described below.

[1841] Hardware and software used

[1842] This system is mainly composed of a server, terminals, and user operations. The specific hardware and software used are as follows:

[1843] Server: We use powerful computer servers to process and store data.

[1844] Terminal: A device operated by a user, including a PC or smartphone.

[1845] Artificial Intelligence: Artificial intelligence with speech recognition technology to convert recorded data into text.

[1846] Generative AI: AI that uses natural language processing techniques to summarize written data.

[1847] Deep learning model: A model for converting summarized text into animated videos.

[1848] System processing flow

[1849] 1. The user inputs the summarized text into the terminal.

[1850] The user inputs the summarized sentence into an input field on the terminal, for example, "The project is progressing smoothly."

[1851] 2. The device sends the input text to the server

[1852] The device sends the text entered by the user to the server as an HTTP request, converting the text into JSON format and sending it to the server's API endpoint.

[1853] 3. The server inputs the received sentences into the deep learning model

[1854] The server analyzes the JSON-formatted text received from the terminal and inputs it into the deep learning model.

[1855] 4. Deep learning models analyze text and generate animated videos

[1856] The deep learning model analyzes input sentences and generates animated videos based on their content. For example, based on the sentence "The project is progressing smoothly," it generates an animated video depicting project members working.

[1857] 5. The server sends the generated animation to the device.

[1858] The server receives the animation video generated by the deep learning model and sends it to the device using an HTTP response.

[1859] 6. The device displays the animation video to the user.

[1860] The terminal displays the animation received from the server to the user, who can then view the generated animation on the screen of the terminal.

[1861] Specific examples

[1862] The user inputs a summarized sentence, "The project is progressing smoothly," into the device. The device sends this sentence to the server. The server uses a deep learning-based video generation model to generate an animated video depicting the project members working. The generated video is then sent from the server to the device, where the user can view it.

[1863] Prompt Sentence Examples

[1864] Take the condensed sentence "The project is progressing smoothly" and turn it into an animated video.

[1865] In this way, the contents of the conference can be shared quickly and effectively. The flow of the identification process in the third embodiment will be described with reference to FIG.

[1866] Step 1:

[1867] The user inputs the summarized text into the terminal.

[1868] As a specific operation, the user enters "The project is progressing smoothly" into the input field of the terminal and clicks the send button.

[1869] Input: A summary of the sentence (e.g., "The project is progressing smoothly.")

[1870] Output: Text typed into the terminal

[1871] Step 2:

[1872] The terminal sends the input text to the server.

[1873] Specifically, the device converts the input text into JSON format and sends an HTTP POST request to the server's API endpoint.

[1874] Input: Text typed into the terminal

[1875] Output: JSON formatted document sent to the server

[1876] Step 3:

[1877] The server inputs the received sentences into the deep learning model.

[1878] Specifically, the server analyzes the received JSON data, extracts sentences, and inputs them into the deep learning model.

[1879] Input: JSON formatted text sent to the server

[1880] Output: The sentences fed into the deep learning model

[1881] Step 4:

[1882] A deep learning model analyzes text and generates animated videos.

[1883] Specifically, the deep learning model analyzes input sentences and generates animated scenes based on their content. For example, based on the sentence "The project is progressing smoothly," it generates a scene of project members working at their desks.

[1884] Input: The sentence fed into the deep learning model

[1885] Output: Generated animation video

[1886] Step 5:

[1887] The server transmits the generated animation to the terminal.

[1888] Specifically, the server encodes the generated animation video in MP4 format and sends it to the terminal as an HTTP response.

[1889] Input: Generated animation video

[1890] Output: Animated video in MP4 format sent to your device

[1891] Step 6:

[1892] The terminal displays the received animation video to the user.

[1893] Specifically, the device displays the received MP4 animation video in a player, and the user can play the video in a browser to view it.

[1894] Input: Animated video in MP4 format sent to the device

[1895] Output: Animated video displayed to the user

[1896] (Application example 3)

[1897] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1898] In conventional advertising production, creating visually appealing animated videos based on advertising copy requires a great deal of time and effort. Furthermore, because specialized knowledge and skills are required, it is difficult for advertisers to easily create videos themselves. This has led to issues such as reduced efficiency in advertising production and increased costs.

[1899] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[1900] In this invention, the server includes means for converting recorded data of an internal meeting into text using artificial intelligence, means for summarizing the text data using generative artificial intelligence, means for automatically generating an animated video from the summarized text, means for receiving the summarized advertising copy as input and generating an animated advertising video using an animation generation model based on deep learning, and means for displaying the generated animated advertising video. This makes it possible to automatically generate visually appealing animated advertising videos based on the summarized advertising copy provided by the advertiser.

[1901] "Internal meeting recording data" refers to audio recordings of meetings held within a company or organization.

[1902] "Artificial intelligence" is an information processing technology in which computer systems imitate human intelligence.

[1903] "Texting" is the process of converting audio data into text data.

[1904] "Generative AI" is AI that has the ability to generate new information and content from given data.

[1905] "Summarization" is the process of extracting important information from long text or data and summarizing it in a concise manner.

[1906] "Animation video" is a video format that uses a series of images to express movement.

[1907] "Deep learning" is a branch of machine learning that uses multi-layered neural networks to analyze data and learn patterns.

[1908] An "video generation model" is an algorithm or framework for generating video based on input data.

[1909] "Ad copy" is text created to promote a product or service.

[1910] An "animated advertising video" is a visually appealing animated advertisement generated based on advertising copy.

[1911] "Display" is the process of visually presenting generated content to a user.

[1912] A system for implementing the present invention has the following configuration.

[1913] System Configuration

[1914] 1. Hardware:

[1915] Server: A server equipped with a high-performance GPU

[1916] Devices: Smartphones, smart glasses

[1917] 2. Software:

[1918] Natural Language Processing (NLP) libraries (e.g., spaCy, NLTK)

[1919] Deep learning frameworks (e.g. TensorFlow, PyTorch)

[1920] Application Interface

[1921] Processing flow

[1922] 1. Receiving input data:

[1923] The user inputs the advertisement copy. For example, the advertisement copy is "The new smartphone is equipped with a high-performance camera."

[1924] 2. Data preprocessing:

[1925] The server uses a natural language processing library to tokenize the input ad copy and perform grammatical analysis.

[1926] 3. Application of video generation model:

[1927] The server passes the tokenized data to a video generation model pre-trained using a deep learning framework to generate animated advertising videos.

[1928] 4. Video output:

[1929] The generated animated advertising video is displayed to the user through the application interface of the terminal.

[1930] Specific examples

[1931] For example, if an advertiser enters the ad copy, "Our new smartphone is equipped with a high-performance camera," the system operates as follows:

[1932] 1. Receiving input data:

[1933] The user enters the ad copy through a smartphone application.

[1934] 2. Data preprocessing:

[1935] The server uses a natural language processing library to tokenize and parse the ad copy.

[1936] 3. Application of video generation model:

[1937] The server then inputs the tokenized data into a video generation model using a deep learning framework to generate animated advertising videos that highlight the smartphone's high-performance camera.

[1938] 4. Video output:

[1939] The generated animated advertising video is displayed to the user through the smartphone application interface.

[1940] In this way, it is possible to automatically generate visually appealing animated advertising videos based on the summarized advertising copy provided by the advertiser.

[1941] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[1942] Step 1:

[1943] The user inputs the advertisement copy through the smartphone application. For example, the advertisement copy might be, "Our new smartphone is equipped with a high-performance camera." This advertisement copy is then sent to the system.

[1944] Step 2:

[1945] The server receives the input ad copy and tokenizes it using a natural language processing library (e.g., spaCy, NLTK). The tokenized data is divided into each word and phrase of the sentence. Next, grammatical analysis is performed to analyze the structure of the sentence. This allows the meaning and important elements of the ad copy to be extracted.

[1946] Step 3:

[1947] The server then inputs the tokenized and parsed data into a pre-trained video generation model using a deep learning framework (e.g., TensorFlow, PyTorch). The video generation model generates animated ad videos based on the input data. The model creates appropriate visual effects and animations based on the content of the ad copy.

[1948] Step 4:

[1949] The generated animated advertising video is sent from the server to the terminal and displayed to the user through the terminal's application interface. The user can check the generated animated advertising video and modify or regenerate it as necessary.

[1950] In this way, it is possible to automatically generate visually appealing animated advertising videos based on the summarized advertising copy provided by the advertiser.

[1951] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1952] "Example 1"

[1953] One embodiment of the present invention is a system that combines an emotion engine. This system includes an emotion engine that recognizes emotions from a user's voice data. The emotion engine analyzes the acoustic characteristics and speaking patterns of the voice data and infers the user's emotional state from the results. Specifically, it analyzes characteristics of the voice data, such as pitch, volume, and speaking rate, and recognizes the emotional state indicated by these characteristics.

[1954] "Example 2"

[1955] The emotion engine also generates summaries based on the recognized emotional state. Specifically, if the user is recognized as being in a joyful or excited state, a positive summary reflecting that emotional state is generated. Conversely, if the user is recognized as being in a sad or angry state, a negative summary reflecting that emotional state is generated.

[1956] "Example 3"

[1957] Furthermore, the animation visually expresses the user's emotions. Specifically, if the user recognizes that they are in a happy emotional state, the animation is generated with bright colors and active movements that reflect that emotional state. Conversely, if the user recognizes that they are in a sad emotional state, the animation is generated with dark colors and quiet movements that reflect that emotional state.

[1958] The processing flow of each embodiment will be described below.

[1959] "Example 1"

[1960] Step 1: Collect voice data from the user.

[1961] Step 2: Analyze the acoustic characteristics and speaking patterns of the collected voice data.

[1962] Step 3: Estimate the user's emotional state from the analysis results.

[1963] "Example 2"

[1964] Step 1: Recognize the user's emotional state.

[1965] Step 2: Generate a summary based on the recognized emotional state.

[1966] Step 3: Present the generated summary to the user.

[1967] "Example 3"

[1968] Step 1: Recognize the user's emotional state.

[1969] Step 2: Generate an animated video that reflects the recognized emotional state.

[1970] Step 3: Present the generated animation video to the user.

[1971] Example 1

[1972] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1973] Conventional systems for transcribing recordings of internal meetings only provide a summary or visual representation of the meeting content, and are unable to grasp the emotional state of the speakers. This makes it difficult to understand the atmosphere of the meeting and the emotions of the speakers, making it difficult to accurately grasp the content of the meeting. Furthermore, because the generated report does not include the emotional state, there is a risk of missing important nuances in the meeting.

[1974] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1975] In this invention, the server includes means for converting recorded data of an internal meeting into text using artificial intelligence, means for summarizing the textual data using generative artificial intelligence, means for automatically generating animated videos from the summarized text, means for analyzing the acoustic characteristics of the audio data and speaking patterns to estimate emotional states, means for generating a report including the meeting content and emotional states, and means for providing the generated report to a user, thereby enabling an accurate understanding of the meeting content and the emotional states of speakers.

[1976] "Internal meeting recording data" refers to data that records the audio of an internal meeting.

[1977] "Artificial intelligence" is a technology that refers to the processing and judgment that computer systems perform by imitating human intelligence.

[1978] "Means of converting text" refers to techniques and methods for converting voice data into text data.

[1979] "Generative AI" is an AI technology that extracts important information from text data and generates summaries and new sentences.

[1980] "Methods of summarization" are techniques or methods for extracting important parts from long pieces of text or data and summarizing them in a concise manner.

[1981] "Means for automatically generating animated videos" refers to techniques and methods for automatically creating visually easy-to-understand animated videos based on text data.

[1982] "Acoustic characteristics of voice data" refers to acoustic features such as pitch, volume, and speaking rate contained in the voice data.

[1983] "Speaking patterns" refer to the characteristic rhythm, intonation, and word usage that can be seen in a speaker's speech.

[1984] "Means for estimating emotional state" refers to techniques or methods for estimating the speaker's emotions by analyzing the acoustic characteristics of speech data and speaking patterns.

[1985] The "means for generating a report" refers to a technique or method for creating a report including the meeting content and emotional state based on the results of speech recognition and emotion analysis.

[1986] The "means for providing to the user" refers to the technology or method for delivering the generated report to the user and making it viewable.

[1987] This invention is a system that uses artificial intelligence to convert recorded data from an internal meeting into text and analyze the emotional state of the participants. Specific embodiments of this system are described below.

[1988] First, the user uploads the recording of the internal meeting from their device to the server. The recording data is in audio file format such as WAV or MP3. The server receives the recording data uploaded by the user.

[1989] The server then converts the recording into text using speech recognition technology, using software such as Google Cloud Speech-to-Text API or IBM Watson Speech to Text, which uses deep learning-based speech recognition models to convert the audio data into text with high accuracy.

[1990] The text data is then passed to an emotion engine. The server uses an emotion engine, such as the Emotion API from Microsoft Azure Cognitive Services, to analyze the user's emotional state from the text data. The emotion engine analyzes the acoustic characteristics of the audio data (such as pitch, volume, and speaking rate) and speaking patterns to identify the emotional state indicated by these characteristics.

[1991] The server integrates the results of speech recognition and emotion analysis to generate a report containing the meeting content and the emotional state of each speaker. This report is generated in text or PDF format and provided to the user. The user can download the report via their device and check the meeting content and emotional state.

[1992] For example, the following prompts can be fed to a generative AI model:

[1993] "Upload recordings of internal meetings. The server uses speech recognition technology to convert the recordings into text, and then uses an emotion engine to analyze the emotional state of each speaker. Finally, it generates and provides a report showing the content and emotional state of the meeting."

[1994] In this way, the system processes while clarifying the roles of the server, terminal, and user. This system enables accurate understanding of the content of the meeting and the emotional state of the speaker, ensuring effective meeting management without overlooking important nuances.

[1995] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1996] Step 1:

[1997] A user uploads the recording data of an internal meeting from their device to the server.

[1998] Input: Recording data of internal meetings (audio file formats such as WAV and MP3)

[1999] Output: Audio file uploaded to the server

[2000] Specific operation: The user uses the file upload function of the device to send the recorded data to the server, and the server saves the received audio file in a specified directory.

[2001] Step 2:

[2002] The server receives the recorded data and begins the voice recognition process.

[2003] Input: Uploaded audio file

[2004] Output: Start signal for speech recognition processing

[2005] Specific operation: The server checks the existence of the received audio file and prepares to start the speech recognition process. Specifically, it checks the format of the audio file and converts it if necessary.

[2006] Step 3:

[2007] The server uses voice recognition technology to convert the recorded data into text.

[2008] Input: Audio file

[2009] Output: Text data

[2010] How it works: The server calls voice recognition software such as Google Cloud Speech-to-Text API or IBM Watson Speech to Text to convert the audio file into text, which is then temporarily stored on the server.

[2011] Step 4:

[2012] The server passes the converted data to the emotion engine.

[2013] Input: Text data

[2014] Output: Input data to the emotion engine

[2015] Specific operation: The server converts the text data into an appropriate format to be passed to the emotion engine. Specifically, the server converts the text data into JSON format or similar and sends it to the emotion engine's API.

[2016] Step 5:

[2017] The server uses an emotion engine to analyze the emotional state from the text data.

[2018] Input: Input data to the emotion engine

[2019] Output: Sentiment analysis results

[2020] How it works: The server uses an emotion engine, such as the Emotion API from Microsoft Azure Cognitive Services, to analyze the user's emotional state from the text data. The emotion engine analyzes the acoustic characteristics and speaking patterns of the audio data to estimate the user's emotional state. The analysis results are returned to the server.

[2021] Step 6:

[2022] The server generates a report containing the meeting content and emotional state.

[2023] Input: Speech recognition results, emotion analysis results

[2024] Output: Report (text or PDF format)

[2025] Specific operation: The server integrates the results of speech recognition and sentiment analysis to generate a report containing the meeting content and the emotional state of each speaker. The report is created in a specified format and saved on the server...

Claims

1. A means for converting meeting recording data into text data using artificial intelligence; A means for summarizing the converted text data using generative artificial intelligence; means for recognizing an emotional state of a user from said audio recording; means for generating an animated video based on the summarized text data and the recognized emotional state of the user; A system including:

2. the summarizing means extracts important information about the conference from the text data, and summarizes the text data based on the extracted important information. The system of claim 1 .

3. the generating means generates an animated video that visually represents information contained in the summarized text data and has colors and movements that reflect the emotional state of the user. The system of claim 1 .

Images