System

Abstract

A system is provided.SOLUTION: A system comprising: means for receiving an image requirement from a user; means for instructing an image generation AI model to generate an image based on the image requirement; means for returning the generated image to the user; means for receiving a correction requirement from the user and instructing the image generation AI model to generate a corrected image again; means for checking a copy right of the generated image; and means for recording a result of the copy right check.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] The process of preparing images for proposals and meeting materials requires a great deal of time and effort using conventional methods. In particular, the process of searching for materials both inside and outside the company and making appropriate edits reduces work efficiency and results in increased costs. There is also a risk of copyright infringement, and the associated verification work creates uncertainty and time costs. Furthermore, summarizing and recording comments made during meetings is also done manually, hindering the efficiency of meetings. To address these issues, a system utilizing image generation AI is needed. [Means for solving the problem]

[0005] To solve the above problems, the present invention provides the following means. A means is provided for receiving image requirements from a user, and based on these, an image generation AI model is instructed to generate an image. The generated image is returned to the user's device, and if there are any correction requirements, they are received again and a corrected version of the image is generated. A system is also built that performs a copyright check on the generated image and records the results. Furthermore, a means is included that automatically transcribes audio data recorded during meetings, generates summary data, and provides it to the user. This improves the efficiency of the image creation and correction process, reduces copyright risks, and quickly summarizes meeting records, improving the efficiency of the entire business.

[0006] A "user" is an entity that uses this system to input image requirements, check images, and give correction instructions.

[0007] A "terminal" is a device through which a user inputs image requirements and correction requirements, and is a device that performs data communication between the user and the server.

[0008] "Server" refers to a central processing unit that operates an image generation AI model, generates and corrects images based on user-entered requirements, checks copyright, and transcribes and summarizes meeting records.

[0009] "Image requirements" refer to specific conditions required by users, such as the style, theme, color tone, and resolution of images used in proposals and meeting materials.

[0010] An "image generation AI model" is an artificial intelligence algorithm that generates new images based on the user's image requirements.

[0011] "Generated images" are image data generated by an image generation AI model for use in proposals and meeting materials.

[0012] The "modification requirements" refer to additional changes or conditions that a user requests for the generated image.

[0013] A "copyright check" is an evaluation process to determine whether the generated image is similar to or relies on existing copyrighted work.

[0014] "Audio data" refers to audio information such as comments recorded during a meeting.

[0015] "Automatic transcription" refers to the process of converting recorded audio data into text data.

[0016] "Summary data" is information that summarizes text data generated by automatic transcription and extracts only the main points. [Brief explanation of the drawings]

[0017] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0018] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0019] First, the terms used in the following description will be explained.

[0020] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0021] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0022] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0023] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0024] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0025] [First embodiment]

[0026] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0027] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0028] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0029] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0030] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0031] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0032] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0033] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0034] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0035] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0036] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0037] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0038] This invention relates to a system for generating images for use in proposals and meeting materials, redacting images, checking copyrights, and automatically transcribing and summarizing meeting records. The program processing of the system is explained in natural language below.

[0039] System Overview

[0040] The system allows users to input image requirements through their devices and send them to a server. The server then uses an image generation AI model to generate images, returns them to the user, and makes any necessary corrections before providing the final image. It also includes a copyright check function to avoid the risk of copyright infringement, and the ability to automatically transcribe and summarize audio data during meetings.

[0041] Image Generation Process

[0042] 1. User Roles

[0043] Users use the device to input image requirements for proposals and meeting materials, such as "next-generation smartphone," "futuristic design," "blue color theme," and "high resolution."

[0044] After the image is returned, the user checks it, enters correction requirements as necessary, and resubmits it.

[0045] 2. Role of the terminal

[0046] The device sends the image requirements entered by the user to the server, which sends the request as JSON format data.

[0047] The terminal displays the generated and modified images returned from the server.

[0048] 3. Server Roles

[0049] The server analyzes the image requirements received from the device and instructs the image generation AI model to generate the image based on that information.

[0050] It receives image data generated by an image generation AI model, adds metadata (date and time of generation, conditions of use, etc.), and returns it to the device.

[0051] If correction instructions are given, the correction instructions are sent again to the image generation AI model, and a corrected version is generated.

[0052] It uses an internal copyright checking AI model to assess the copyright risk of the generated images and records the results.

[0053] Copyright Check Process

[0054] The server uses an internal copyright check AI model to evaluate the generated image for similarity and reliability with existing images. The evaluation results are recorded and provided to users only if there are no problems with using the image.

[0055] Meeting recording efficiency process

[0056] 1. Recording audio data

[0057] During a meeting, users use the terminal to record audio of comments and notes.

[0058] 2. Automatic transcription

[0059] The device sends the recorded audio data to a server, which passes it to an automatic transcription AI model to convert it into text.

[0060] 3. Summary Generation

[0061] The server passes the generated text data to a summarization AI model to generate a summary, which is returned to the user and used as a meeting record.

[0062] Specific examples

[0063] Image generation example

[0064] When a user requests an image with the theme of "next-generation smartphones," the device sends the request to the server.

[0065] The server passes the requirements to the image generation AI model and receives the generated high-resolution smartphone image of the futuristic design.

[0066] The server adds metadata to the generated image and returns it to the terminal.

[0067] The user checks the image and inputs correction requirements, such as "I want the gloss to be a little less." The terminal again sends the correction requirements to the server, and the server generates the corrected image.

[0068] Meeting recording example

[0069] During a conference, a user records a voice memo on a terminal, and the terminal transmits the voice data to a server.

[0070] The server automatically transcribes the audio data and summarizes the generated text.

[0071] The summary data is sent back to the terminal, and the user can use it efficiently as a follow-up after the meeting.

[0072] This system streamlines the image creation and editing process for proposals and meeting materials, reduces copyright risks, and enables quick summarization of meeting records.

[0073] The processing flow will be explained below.

[0074] Image Generation Process

[0075] Step 1:

[0076] The user logs in to a dedicated application on the device.

[0077] Users input image requirements for proposals and meeting materials, such as "next-generation smartphone," "futuristic design," "blue color theme," and "high resolution."

[0078] Step 2:

[0079] The device sends the entered image requirements to the server as JSON format data.

[0080] Step 3:

[0081] The server parses the JSON format image requirements received from the device.

[0082] Step 4:

[0083] The server instructs the image generation AI model to generate an image based on the analyzed image requirements.

[0084] Step 5:

[0085] The image generation AI model generates new images based on the specified requirements.

[0086] Step 6:

[0087] The server receives image data generated by the image generation AI model and adds metadata (such as the date and time of generation, conditions of use, etc.) to the generated image.

[0088] Step 7:

[0089] The server transmits the generated image to the terminal.

[0090] Step 8:

[0091] The user checks the image generated on the terminal. If the user determines that corrections are necessary, they input the correction requirements into the terminal. For example, they input correction instructions such as "make the design less glossy."

[0092] Step 9:

[0093] The terminal transmits the correction requirements entered by the user back to the server as JSON format data.

[0094] Step 10:

[0095] Based on the correction requirements received from the terminal, the server again instructs the image generation AI model to generate a corrected image.

[0096] Step 11:

[0097] The image generation AI model regenerates the image based on the correction requirements and sends it back to the server.

[0098] Step 12:

[0099] The server receives the corrected image data and sends it back to the terminal, where the user checks the corrected image and finally approves it.

[0100] Copyright Check Process

[0101] Step 1:

[0102] The server uses an internal copyright checking AI model to evaluate the generated image for similarity and authenticity with existing images.

[0103] Step 2:

[0104] The server records the evaluation results in an internal database, and if there are no problems, provides the image to the user.

[0105] Step 3:

[0106] The server saves the good images in the project folder and makes them accessible to users.

[0107] Meeting recording efficiency process

[0108] Step 1:

[0109] During a meeting, a user uses the terminal to record audio of comments and notes.

[0110] Step 2:

[0111] The device transmits the recorded audio data to the server.

[0112] Step 3:

[0113] The server passes the audio data to an automatic transcription AI model and converts it into text.

[0114] Step 4:

[0115] The server passes the generated text data to a summary AI model to generate summary data.

[0116] Step 5:

[0117] The server sends the generated summary data to the terminal, where the user can check it. The user can use it efficiently as a follow-up after the meeting.

[0118] Example 1

[0119] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0120] In today's world, the creation and editing of images for proposals and meeting materials, copyright verification, and automation of meeting recording are extremely important issues. However, these processes are extremely time-consuming and laborious, so systems that can handle them efficiently and quickly are needed. Furthermore, it is also necessary to check whether the generated images are similar to existing images and whether they infringe copyright. There is also a need for automatic transcription and summary generation of audio data recorded during meetings, but it is currently difficult to efficiently achieve these functions in a single system.

[0121] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0122] In this invention, the server includes a means for receiving image requirements from a user, a means for instructing an image generation AI model to generate an image, and a means for performing copyright checks on the generated images. This allows users to easily input specific image requirements, quickly generate and edit high-quality images, and evaluate copyright risks. Furthermore, by including a function for automatically transcribing audio data during meetings and generating summaries, it also automates meeting recording, significantly improving overall work efficiency.

[0123] The "means for receiving image requirements" is a function for receiving conditions and features related to an image specified by a user.

[0124] An "image generation AI model" is an algorithm or system that uses artificial intelligence to generate images based on specified conditions.

[0125] The "means for instructing image generation" is a function that conveys the received image requirements to the image generation AI model and causes it to generate the image.

[0126] "Means for returning generated images" refers to a function that returns images generated by an image generation AI model to the user's device.

[0127] The "means for receiving correction requirements" is a function for receiving a correction request for a generated image from a user.

[0128] The "means for performing copyright checks" is a function that evaluates the similarity and reliance of generated images against existing images, and checks for the risk of copyright infringement.

[0129] The "means for recording the results of copyright checks" is a function for saving the results of copyright checks in a database or the like.

[0130] "Means for automatically transcribing audio data" refers to a function that converts audio data recorded during a meeting into text data.

[0131] The "means for summarizing and providing transcribed data" is a function that summarizes automatically transcribed text data and provides it to the user.

[0132] The "means for adding and providing metadata" is a function for adding additional information such as the date and time of creation and the conditions of use to the created image and providing it to the user.

[0133] "Means for sending in JSON format" is a function that sends image requirements specified by the user to the server in a data format called JSON (JavaScript Object Notation).

[0134] The present invention relates to a system for generating images for use in proposals and meeting materials, redacting, copyright checking, and automatically transcribing and summarizing meeting records. The system is implemented as follows:

[0135] System Overview

[0136] In this system, users input image requirements through their devices and send them to the server. The server uses an image generation AI model to generate images, returns them to the user, makes any necessary corrections, and provides the final image. The system also has a copyright check function for generated images, and can automatically transcribe audio data from meetings and generate summaries.

[0137] Hardware and software used

[0138] The system uses the following hardware and software:

[0139] Image generation AI models: e.g., DALL-E and Stable Diffusion

[0140] Copyright checking AI model: For example, CopyChecker

[0141] Automatic transcription AI models: e.g., Google Cloud Speech-to-Text

[0142] Summary generation AI model: e.g., GPT-3

[0143] Device: The computer or smartphone used by the user

[0144] Server: A remote server for data processing and running AI models

[0145] Specific operation of the system

[0146] 1. Enter image requirements

[0147] Users can use their devices to input image requirements for proposals and meeting materials, such as "next-generation smartphone," "futuristic design," "blue color theme," and "high resolution." Users simply enter their specific requirements in the text box and click the send button.

[0148] 2. Submit your image requirements

[0149] The device sends the image requirements entered by the user to the server in JSON format. The data is sent via an HTTP POST request. The following JSON format data is used:

[0150] json

[0151] {

[0152] "theme": "Next-generation smartphone",

[0153] "style": "futuristic design",

[0154] "color": "blue",

[0155] "resolution": "High resolution"

[0156] }

[0157] 3. Execute image generation

[0158] The server analyzes the received image requirements and issues instructions to the image generation AI model based on them. For example, a prompt might look like this: "Create a high-resolution image of a futuristic next-generation smartphone with a blue color theme." The server receives the image data generated by the image generation AI model, adds metadata (such as the date and time of generation and conditions of use), and sends it back to the device.

[0159] 4. Check and correct the image

[0160] The user checks the displayed image and inputs correction requirements as necessary. For example, they can enter a command such as "reduce the gloss" in the text box and click the resend button. The device then sends the correction requirements back to the server in JSON format, and the server again sends the correction instructions to the image generation AI model.

[0161] 5. Copyright Check

[0162] The server uses an internal copyright check AI model to evaluate the generated image for similarity and reliability with existing images. The results of the copyright check are recorded and provided to the user only if there are no problems.

[0163] 6. Audio recording and transcription

[0164] During a meeting, users record audio notes on their device, which then sends the audio data to a server, which then converts the audio into text using an automatic transcription AI model. For example, "Transcribe the following meeting audio to text."

[0165] 7. Generating and Serving Abstracts

[0166] The server passes the generated text data to a summarization AI model to generate summary data. This summary is sent back to the user and used as a meeting record. A summary like the following is generated: "At the meeting, the progress and next steps of the new project were discussed."

[0167] This system streamlines the image creation and editing process for proposals and meeting materials, reduces copyright risks, and enables quick summarization of meeting records.

[0168] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0169] Image Generation Process

[0170] Step 1: User inputs image requirements

[0171] Input: Image requirements for user proposals and meeting materials (e.g., "next-generation smartphone," "futuristic design," "blue color theme," "high resolution")

[0172] Specific operation: The user enters the image requirements in the text box on the terminal and clicks the send button.

[0173] Output: Image requirements are entered into the terminal and prepared for sending to the server.

[0174] Step 2: Send image requirements from the device to the server

[0175] Input: User-entered image requirements

[0176] Specific operation: The device converts the image requirements into JSON format and sends it to the server as an HTTP POST request.

[0177] Output: The following JSON data is sent to the server:

[0178] json

[0179] {

[0180] "theme": "Next-generation smartphone",

[0181] "style": "futuristic design",

[0182] "color": "blue",

[0183] "resolution": "High resolution"

[0184] }

[0185] Step 3: Image generation instructions from the server

[0186] Input: Image requirements sent from the terminal (JSON data)

[0187] Specific operation: The server parses the received JSON data and generates an appropriate prompt for the image generation AI model. For example: "Create a high-resolution image of a futuristic next-generation smartphone with a blue color theme."

[0188] Output: A prompt sentence is input into the image generation AI model, and the generated image data is returned to the server.

[0189] Step 4: Returning the image from the server to the device

[0190] Input: Image data generated by an image generation AI model

[0191] Specific operation: The server adds metadata (date and time of creation, conditions of use, etc.) to the generated image and returns it to the terminal.

[0192] Output: Image data with metadata is sent to the device. Example:

[0193] json

[0194] {

[0195] "image": "base64_encoded_image_data",

[0196] "metadata": {

[0197] "generated_at": "2023-10-01T12:00:00Z",

[0198] "usage_conditions": "Commercial use allowed"

[0199] }

[0200] }

[0201] Step 5: User checks the image and gives correction instructions

[0202] Input: Image data returned from the server

[0203] Specific operation: The user checks the image on the device and inputs correction requirements as necessary (e.g., "I would like the glossiness to be reduced"). After inputting the correction requirements, the user clicks the resend button.

[0204] Output: The modified image requirements are entered into the terminal and are ready to be sent to the server again.

[0205] Step 6: Sending modification requirements from the terminal to the server

[0206] Input: User-entered correction requirements

[0207] Specific operation: The terminal converts the modification requirements into JSON format and resubmits them to the server as an HTTP POST request.

[0208] Output: The correction requirements are sent to the server and used again as prompts for the image generation AI model. Example of resubmitted JSON data:

[0209] json

[0210] {

[0211] "theme": "Next-generation smartphone",

[0212] "style": "futuristic design",

[0213] "color": "blue",

[0214] "resolution": "high resolution",

[0215] "modification": "Reduce glossiness"

[0216] }

[0217] Step 7: Copyright Check

[0218] Input: Generated image data

[0219] How it works: The server inputs the generated image into a copyright checking AI model to evaluate its similarity to existing images and its reliability. The evaluation results are recorded and provided to the user only if there are no problems.

[0220] Output: Copyright check results and usable image data based on the evaluation. Example:

[0221] json

[0222] {

[0223] "image": "base64_encoded_image_data",

[0224] "metadata": {

[0225] "generated_at": "2023-10-01T12:00:00Z",

[0226] "usage_conditions": "No copyright issues detected"

[0227] }

[0228] }

[0229] Meeting recording efficiency process

[0230] Step 1: Recording audio data

[0231] Input: Audio during the meeting

[0232] Specific operation: The user clicks the "Start Recording" button on the device to record conversations and notes in audio format.

[0233] Output: The recorded audio data is stored on the device and prepared for transmission to the server.

[0234] Step 2: Sending audio data from the device to the server

[0235] Input: Recorded audio data

[0236] Specific operation: The device converts the voice data into a digital audio file format (e.g., WAV, MP3) and sends it to the server as an HTTP POST request.

[0237] Output: The audio data is sent to the server. Example:

[0238] json

[0239] {

[0240] "audio_file": "base64_encoded_audio_data"

[0241] }

[0242] Step 3: Auto-transcription

[0243] Input: Audio data sent to the server

[0244] Specific operation: The server inputs the audio data into an automatic transcription AI model and converts the audio into text data. The prompt is as follows: "Transcribe the following meeting audio to text."

[0245] Output: A transcript of the audio is generated. Example:

[0246] text

[0247] "In the meeting, we discussed the progress of the new project and the next steps. Specifically, we had a detailed discussion about the design and features of the next-generation smartphone."

[0248] Step 4: Generate summary and return summary data

[0249] Input: Automatically transcribed text data

[0250] Specific operation: The server inputs the generated text data into the summary AI model and generates a summary. The prompt is as follows: "Summarize the following transcription of the meeting."

[0251] Output: Summarized text data is generated and sent back to the user. Example:

[0252] text

[0253] "During the meeting, the progress and next steps of the new project were discussed. In particular, there was detailed discussion on the design and functionality of the next generation smartphone."

[0254] Through these specific processing steps, the system streamlines the creation and editing of images required for proposals and meeting materials, reduces copyright risks, and enables quick summarization of meeting records.

[0255] (Application example 1)

[0256] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0257] In autonomous vehicles, there are problems with the efficient generation, editing, and copyright check of video surveillance and meeting records during operation, as well as the automatic transcription and summarization of audio data. In particular, the collection of surveillance data and the creation of driving reports within autonomous vehicles requires a great deal of time and effort using current manual processes, making effective operation difficult. In addition, copyright issues may arise with the generated data, so evaluation and management of these issues are required.

[0258] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0259] In this invention, the server includes means for receiving image requirements from a user, means for instructing an image generation AI model to generate an image based on the image requirements, means for returning the generated image to the user's terminal, means for receiving correction requirements from the user and again instructing the image generation AI model to generate a corrected image, means for performing a copyright check on the generated image, means for recording the results of the copyright check, means for automatically transcribing audio data recorded during a meeting, means for summarizing the transcribed data and providing it to the user, means for acquiring image surveillance data inside and outside the autonomous vehicle, means for transferring the recorded audio data, and means for automatically processing and summarizing audio memos. This enables efficient operation management of autonomous vehicles, rapid data generation and correction, reduced copyright risks, and rapid summarization of meeting records.

[0260] "User" refers to an entity that uses the system to generate images, specify correction requirements, and record audio data.

[0261] "Image generation AI model" refers to an algorithm or model that generates images based on specified image requirements.

[0262] "Image requirements" refer to the specific style, content, color tone, resolution, and other conditions of the image desired by the user.

[0263] "Modification requirements" refer to instructions for modifying or changing the generated image.

[0264] "Copyright check" refers to the process of assessing whether the generated image is similar to an existing image or infringes copyright.

[0265] "Audio data" refers to audio information recorded during a conference.

[0266] "Automatic transcription" refers to the process of automatically converting audio data into text data.

[0267] "Summarization" refers to the process of concisely summarizing transcribed text data.

[0268] An "autonomous vehicle" is a vehicle that is capable of operating independently without a human driver.

[0269] "Image surveillance data" refers to image information collected inside and outside an autonomous vehicle.

[0270] "Terminal" refers to a device used by a user, such as a computer or smartphone.

[0271] "Metadata" refers to data that includes information related to a generated image, such as the date and time of generation and the conditions of use.

[0272] This invention relates to a system for efficiently realizing image monitoring, copyright checking, and automatic transcription and summarization of meeting records in an autonomous vehicle. This system starts when a user inputs image requirements using a terminal and sends them to a server. Specific embodiments of this system are described below.

[0273] 1. System Overview

[0274] The system mainly uses the user's device, a server, and cameras and audio recording devices inside and outside the autonomous vehicle. The server utilizes an image generation AI model, a copyright check AI model, an automatic transcription AI model, and a summary AI model for efficient data processing.

[0275] 2. Hardware and Software

[0276] Hardware:

[0277] User's device (e.g. smartphone, tablet)

[0278] Cameras inside and outside the autonomous vehicle

[0279] Audio recording device

[0280] software:

[0281] Python

[0282] Flask (web server framework)

[0283] Image generation AI model (e.g., HuggingFace's DALL·E)

[0284] Copyright Check AI Model

[0285] Automatic transcription API (e.g., Google Speech-to-Text)

[0286] Summary AI models (e.g., HuggingFace's BART)

[0287] 3. Data processing and calculation

[0288] The server receives the request sent by the user from the terminal and processes the data through the following process.

[0289] Image generation process:

[0290] The user inputs image requirements from their device and sends them to the server, such as "futuristic design of an autonomous vehicle, blue color theme, high resolution."

[0291] The server uses an image generation AI model to generate images based on the specified requirements.

[0292] Metadata (date and time of creation, conditions of use, etc.) is added to the generated image, and the result is sent back to the user's device.

[0293] The user inputs the correction requirements for the image and sends it back to the server, which then generates the corrected image and sends it back to the user.

[0294] Copyright Check Process:

[0295] The server passes the generated images to a copyright checking AI model to evaluate their similarity to existing images.

[0296] The evaluation results are recorded and notified to the user. If there are no problems, the image becomes available for use.

[0297] Meeting recording efficiency process:

[0298] During the meeting, users record voice memos on their terminals and send them to the server.

[0299] The server transcribes the audio data using the Google Speech-to-Text API.

[0300] The transcribed text is passed to a summarization AI model to generate summary data.

[0301] The summary data is returned to the user's terminal and is effectively used as a record of the meeting.

[0302] 4. Specific Examples

[0303] Image generation example:

[0304] The user inputs image requirements such as "futuristic design, self-driving vehicle, blue color theme, high resolution."

[0305] The server calls an image generation AI model to generate a high-resolution image, adds metadata, and sends it back to the user's device.

[0306] The user inputs the correction requirement "reduce the glossiness a little" and sends it to the server again. The server generates a corrected image and provides it to the user.

[0307] Example of a recording:

[0308] A user records a voice memo during a conference, and the terminal transmits the voice data to a server.

[0309] The server automatically transcribes the audio data and passes the transcribed text to a summarization AI model to generate a summary.

[0310] The summary data is returned to the user's terminal, and the user can efficiently use the conference record.

[0311] This system not only streamlines the operation and management of autonomous vehicles, but also enables rapid data generation and correction, reduces copyright risks, and improves productivity by quickly summarizing meeting notes.

[0312] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0313] Step 1:

[0314] A user uses a terminal to input image requirements and create a request for image generation, such as "futuristic design of autonomous vehicle, blue color theme, high resolution."

[0315] Input: Image requirements (e.g. futuristic design, blue color theme, high resolution)

[0316] Output: Image generation request

[0317] Step 2:

[0318] The device converts the user's image requirements into JSON format and sends it to the server. The device properly formats the input requirements and prepares the request for sending to the server.

[0319] Input: Image generation request

[0320] Output: Request in JSON format

[0321] Step 3:

[0322] The server analyzes the received JSON-formatted request and instructs the image generation AI model to generate an image. The server analyzes the request, extracts the parameters necessary for image generation, and passes them to the AI ​​model.

[0323] Input: JSON formatted request

[0324] Output: Instructions to the AI ​​model

[0325] Step 4:

[0326] The image generation AI model generates images based on instructions. The model uses prompt text to generate high-resolution images according to the specified style and color tone.

[0327] Input: Instructions to the AI ​​model (e.g., futuristic design, self-driving vehicle, blue, resolution)

[0328] Output: The generated image

[0329] Step 5:

[0330] The server adds metadata (such as the date and time of creation and the conditions of use) to the created image and returns it to the user's device. The server records the date and time of creation and the conditions of use of the image and attaches them to the image data.

[0331] Input: Generated image

[0332] Output: Generated image with metadata

[0333] Step 6:

[0334] The user checks the generated image on the terminal and inputs the correction requirements. The user specifies the improvements and changes to the image and creates a new request.

[0335] Input: Generated image with metadata

[0336] Output: Fix requirements

[0337] Step 7:

[0338] The terminal converts the modification requirements into JSON format and sends it to the server. The terminal then sends a request based on the modification requirements to the server again.

[0339] Input: Correction Requirement

[0340] Output: The modification request in JSON format

[0341] Step 8:

[0342] The server analyzes the modification request and instructs the image generation AI model to generate the modified image again. The server instructs the AI ​​model to regenerate the image using the new modification parameters.

[0343] Input: JSON formatted modification request

[0344] Output: Corrective instructions for the AI ​​model

[0345] Step 9:

[0346] The image generation AI model generates a corrected image based on the correction instructions. It uses the prompt text again to generate an image that reflects the necessary corrections.

[0347] Input: Correction instructions for the AI ​​model

[0348] Output: Corrected image

[0349] Step 10:

[0350] The server returns the corrected image to the user's device. The corrected image data is sent to the user's device in JSON format.

[0351] Input: Modified image

[0352] Output: The corrected image sent back to the user

[0353] The above are the specific processing steps for image generation and correction. Next, we will show the processing steps for automatic transcription and summarization of meeting recordings.

[0354] Step 11:

[0355] During a meeting, the user records a voice memo, and the device sends the voice data to the server. The user then presses the record button to record the required voice.

[0356] Input: Voice memo

[0357] Output: Audio data

[0358] Step 12:

[0359] The server passes the audio data to an automated transcription method to convert it into text. The server uses the Google Speech-to-Text API to transcribe the audio data.

[0360] Input: Audio data

[0361] Output: Transcribed text

[0362] Step 13:

[0363] The server passes the generated text to a summarization model to generate summary data. The server then invokes a summarization AI model (e.g., BART) to generate a summary that extracts the key points.

[0364] Input: Transcribed text

[0365] Output: Summary data

[0366] Step 14:

[0367] The server returns the summary data to the user's terminal, and the user efficiently checks the meeting record. The points of the summarized meeting are provided to the user.

[0368] Input: Summary data

[0369] Output: Summary data returned to the user

[0370] These processing steps enable efficient management of image generation, editing, copyright checks, and meeting records in autonomous vehicles.

[0371] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0372] This invention combines a system for generating images for use in proposals and meeting materials, correcting them, checking copyrights, and automatically transcribing and summarizing meeting records with an emotion engine that recognizes the user's emotions. This system provides more human-centered and effective support by suggesting image requirements, making corrections, and adjusting the summaries of meeting records according to the user's emotional state.

[0373] System Overview

[0374] In this system, users input image requirements through their devices and send them to a server. The server then uses an image generation AI model to generate an image, which is returned to the user, where it is corrected as necessary and the final image is provided. The system also includes a copyright check function to avoid the risk of copyright infringement, and the ability to automatically transcribe and summarize audio data during meetings. A new emotion engine has also been added, enabling suggestions and corrections based on the user's emotions.

[0375] Image generation process and emotion engine integration

[0376] 1. User Roles

[0377] Users log in to a dedicated application on their device, and when entering image requirements for proposals or meeting materials, the system recognizes emotions from the user's facial expressions and voice.

[0378] The emotion engine analyzes the user's emotional state when they input and suggests appropriate image requirements, such as "if the user is excited, we suggest a bold design."

[0379] 2. Role of the terminal

[0380] The terminal transmits the input image requirements and additional requirements provided by the emotion engine to the server, thereby including requirements based on the user's emotions.

[0381] The terminal displays the generated or modified image returned from the server.

[0382] 3. Server Roles

[0383] The server analyzes the image requirements received from the device and instructs the image generation AI model to generate the image.

[0384] The image generation AI model generates new images based on the instruction requirements.

[0385] The server adds metadata to the generated image and returns it to the terminal.

[0386] When responding to correction instructions, the emotion engine evaluates the user's emotional response and automatically recommends correction requirements.

[0387] Copyright Check Process

[0388] The server uses its internal copyright checking AI model to evaluate the generated image for similarity and reliability with existing images. The evaluation results are recorded and provided to users only if there are no problems with using the image.

[0389] Meeting recording efficiency process

[0390] 1. Recording audio data

[0391] During a meeting, users use their devices to record audio of comments and notes, and their emotional state during the recording is also analyzed.

[0392] 2. Automatic transcription

[0393] The device sends the recorded audio data to a server, which then passes the data to an automatic transcription AI model and converts it into text.

[0394] 3. Summary Generation

[0395] The server passes the transcribed text data to a summary AI model to generate a summary. An emotion engine then applies the summary to adjust the content according to the user's emotional state.

[0396] Specific examples

[0397] Image generation example

[0398] When a user requests an image themed on "next-generation smartphones" and inputs their requirements into the device, the emotion engine analyzes the user's emotional state. For example, if a user is excited, it will recommend a "futuristic design."

[0399] The device sends the proposed requirements and the emotion engine's suggestions to the server. The server receives the image generated by the image generation AI model and returns it to the device. The user checks the image, and the emotion engine automatically suggests correction requirements.

[0400] Meeting recording example

[0401] During a meeting, when a user records a voice memo on the terminal, the emotion engine analyzes the user's emotional state.

[0402] The device sends the recorded voice data to a server, which then automatically transcribes it. At the same time, an emotion engine adjusts the summary content to match the user's emotions based on the generated text data.

[0403] The summary data is sent back to the terminal, and the user can use it efficiently as a follow-up after the meeting.

[0404] This system streamlines the image creation and editing process for proposals and meeting materials, improves user response based on user sentiment, reduces copyright risks, and enables quick summarization of meeting records.

[0405] The processing flow will be explained below.

[0406] Image generation process incorporating an emotion engine

[0407] User-defined image requirements and generation

[0408] Step 1:

[0409] The user logs in to a dedicated application on the device.

[0410] Step 2:

[0411] The user inputs the image requirements for proposals and meeting materials into the device, such as "next-generation smartphone," "futuristic design," "blue color theme," and "high resolution."

[0412] Step 3:

[0413] While the user is typing, the device uses a built-in camera and microphone to analyze the user's facial expressions and voice, and an emotion engine evaluates the user's emotional state.

[0414] Step 4:

[0415] The emotion engine makes additional suggestions for image requirements based on the user's emotional state, for example, suggesting a "daring design" if the user is excited.

[0416] Step 5:

[0417] The device sends image requirements and emotion engine suggestions to the server as JSON format data.

[0418] Step 6:

[0419] The server analyzes the JSON format image requirements received from the device and instructs the image generation AI model to generate the image.

[0420] Step 7:

[0421] The image generation AI model generates new images based on the specified requirements.

[0422] Step 8:

[0423] The server receives image data generated by the image generation AI model and adds metadata (such as the date and time of generation, conditions of use, etc.) to the generated image.

[0424] Step 9:

[0425] The server transmits the generated image to the terminal.

[0426] Step 10:

[0427] The user checks the image generated on the device. If the user determines that corrections are necessary, the user inputs the correction requirements into the device (for example, "make the design less glossy").

[0428] Step 11:

[0429] The emotion engine evaluates the user's emotional response when viewing an image and automatically recommends revision requirements.

[0430] Step 12:

[0431] The device sends the user's correction requirements and the emotion engine's suggestions back to the server as JSON format data.

[0432] Step 13:

[0433] Based on the correction requirements received from the terminal, the server again instructs the image generation AI model to generate a corrected image.

[0434] Step 14:

[0435] The image generation AI model regenerates the image based on the correction requirements and sends it back to the server.

[0436] Step 15:

[0437] The server receives the corrected image data and sends it back to the terminal, where the user checks the corrected image and finally approves it.

[0438] Copyright check process incorporating emotion engine

[0439] Step 1:

[0440] The server uses its internal copyright checking AI model to evaluate the generated image for similarity and reliability with existing images.

[0441] Step 2:

[0442] The server records the evaluation results in an internal database, and if there are no problems, the image is provided to the user.

[0443] Step 3:

[0444] The server saves the good images in the project folder and makes them accessible to users.

[0445] Emotion-driven meeting recording process

[0446] Audio data recording and processing

[0447] Step 1:

[0448] During a meeting, a user uses the terminal to record audio of comments and notes.

[0449] Step 2:

[0450] The device analyzes the tone and pace of the user's voice while recording, and an emotion engine evaluates the user's emotional state.

[0451] Step 3:

[0452] The device sends the recorded audio data to the server.

[0453] Step 4:

[0454] The server passes the audio data to an automatic transcription AI model and converts it into text.

[0455] Step 5:

[0456] The server passes the generated text data to a summary AI model to generate summary data.

[0457] Step 6:

[0458] The emotion engine uses the summary data to tailor the content to the user's emotional state, for example, emphasizing positive summary results if the user is feeling down.

[0459] Step 7:

[0460] The server sends the generated summary data to the terminal, where the user can check it. The user can use it efficiently as a follow-up after the meeting.

[0461] These steps provide a system incorporating an emotion engine that not only streamlines the process of creating proposals and meeting materials, but also allows for suggestions and revisions based on the user's emotions, reduces copyright risks, and enables quick summarization of meeting records.

[0462] Example 2

[0463] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0464] In today's busy business environment, creating proposals and meeting materials requires a lot of time and effort. The tasks of generating and editing high-quality images that reflect the user's intent and emotions, as well as efficiently transcribing and summarizing meeting recordings, are particularly burdensome. The risk of copyright infringement for generated images is also a significant issue. It is necessary to develop a system that streamlines these processes and improves the user experience.

[0465] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0466] In this invention, the server includes means for proposing appropriate image requirements in cooperation with an emotion engine that recognizes and analyzes emotions from a user's facial expressions and voice, means for instructing an image generation AI model to generate an image, means for performing copyright checks on the generated image, means for automatically transcribing audio data recorded during a meeting, and means for summarizing the transcribed data, adjusting the content based on the user's emotional state, and providing the summarization to the user. This streamlines the image creation and editing process for proposals and meeting materials, improves response based on user emotions, reduces copyright risks, and enables quick summarization of meeting records.

[0467] A "user" is a person who operates the system and inputs image and modification requirements.

[0468] "Terminal" refers to an electronic device such as a computer or smartphone used by a user.

[0469] "Image requirements" refer to specific conditions and specifications regarding the image that a user wants to create.

[0470] The "emotion engine" is an AI model that recognizes emotions from a user's facial expressions and voice and analyzes their emotional state.

[0471] An "image generation AI model" is an artificial intelligence model for automatically generating images based on input requirements.

[0472] "Copyright check" is the process of checking whether the generated image is similar to or relies on an existing image and assessing legal risk.

[0473] The "correction requirements" refer to the corrections that the user wants to make to the generated image.

[0474] "Automatic transcription" is the automated process of converting audio data into text.

[0475] A "summary" is a text that briefly summarizes the transcribed data.

[0476] "Metadata" is auxiliary information added to a generated image, and is data that indicates details and attributes of the image.

[0477] This invention combines a system for generating images for use in proposals and meeting materials, correcting them, checking copyrights, and automatically transcribing and summarizing meeting records with an emotion engine that recognizes the user's emotions. This system provides more human-centered and effective support by suggesting image requirements, making corrections, and adjusting the summaries of meeting records according to the user's emotional state.

[0478] System Overview

[0479] In this system, users input image requirements through their devices and send them to a server. The server then uses an image generation AI model to generate an image, which is returned to the user, where it is corrected as necessary and the final image is provided. The system also includes a copyright check function to avoid the risk of copyright infringement, and the ability to automatically transcribe and summarize audio data during meetings. A new emotion engine has also been added, enabling suggestions and corrections based on the user's emotions.

[0480] Specific examples

[0481] Image generation example

[0482] When a user requests an image themed around "next-generation smartphones" and inputs the requirements into the device, the emotion engine analyzes the user's emotional state. For example, if the user is in an excited state, it will recommend a "futuristic design." The device then sends the proposed requirements and the emotion engine's suggestions to the server. The server receives the image generated by the image generation AI model and returns it to the device. The user then reviews the image, and the emotion engine automatically suggests further revisions.

[0483] Example prompt for a generative AI model:

[0484] "Generate images with futuristic and exciting designs based on the theme of next-generation smartphones."

[0485] Meeting recording example

[0486] During a meeting, when a user records a voice memo on their device, the emotion engine analyzes the user's emotional state. The device then sends the recorded voice data to a server, which then automatically transcribes it. At the same time, the emotion engine adjusts the summary content to match the user's emotions based on the generated text data. The summary data is then sent back to the device, allowing the user to efficiently use it as a follow-up after the meeting.

[0487] The system streamlines the image creation and editing process for proposals and meeting materials, enables user-sensitive responses, mitigates copyright risks, and enables quick summarization of meeting records.

[0488] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0489] Step 1:

[0490] The user logs in to a dedicated application on their device. The user inputs the image requirements needed for proposals and meeting materials, and the emotion engine recognizes and analyzes the user's emotions from their facial expressions and voice. For example, when a user requests a "futuristic design for a next-generation smartphone," the emotion engine proposes requirements that reflect the user's excited state of mind. The input data includes image requirements, and the output is the emotion analysis results and proposed image requirements.

[0491] Step 2:

[0492] The device sends the image requirements entered by the user and the suggestions made by the emotion engine to the server. The input data here are the user's image requirements and the results of emotion analysis, and the output data is a request message containing these. Specifically, the device sends a request to "generate a futuristic design for the next generation smartphone" along with the user's emotion data to the server.

[0493] Step 3:

[0494] The server analyzes the image requirements received from the device and instructs the image generation AI model to generate the image. The input data includes image requirements and emotion data, and the output is the generated image. Specifically, the server sends a "futuristic design for next-generation smartphones" as a prompt to the image generation AI model, and after receiving the generated image, it adds metadata.

[0495] Step 4:

[0496] The server returns the generated image to the terminal. The input data here is the generated image and metadata, and the output data is the image returned to the user's terminal. Specifically, the server returns the generated image to the terminal together with the metadata.

[0497] Step 5:

[0498] The device displays the generated image returned from the server to the user. The user checks the image and inputs correction requirements, if necessary. The emotion engine again analyzes the user's emotional state and makes suggestions for correction. The input data includes the generated image and the user's correction requirements, and the output data is the correction suggestions. Specifically, the user requests "more colorful," and the emotion engine takes this into consideration and suggests an even bolder color palette.

[0499] Step 6:

[0500] The device sends the user's correction requirements to the server, and the server again instructs the image generation AI model to generate a corrected image. The input data includes the correction requirements and emotion data, and the output data is the corrected image. Specifically, the device sends the correction requirement "change to a colorful design" to the server, and the server sends it again to the image generation AI model to generate the corrected image.

[0501] Step 7:

[0502] The server performs a copyright check on the generated image. The input data is the generated image, and the output data is the result of the copyright check. Specifically, the server uses a copyright check AI model to evaluate the similarity between the generated image and existing images, and records the results.

[0503] Step 8:

[0504] During a meeting, the user uses a device to record voice comments and notes. The emotion engine simultaneously analyzes the user's emotional state during the recording. The input data is the recorded voice and emotional data, and the output data is the analysis results. Specifically, the user says, "Regarding the next action plan," which is recorded as a voice memo, and the emotion engine analyzes the user's state of tension.

[0505] Step 9:

[0506] The device sends the recorded voice data to a server, which then passes the data to an automatic transcription AI model and converts it into text. The input data is voice data, and the output data is the transcribed text. Specifically, the device sends voice data saying "What is the next action plan?" to the server, and the server converts the voice data into text.

[0507] Step 10:

[0508] The server passes the transcribed text data to a summarization AI model, which generates summary data. The emotion engine then adjusts the content based on the generated summary according to the user's emotional state. The input data is the transcribed text and emotion data, and the output data is the adjusted summary. Specifically, the summarization AI model generates a short summary such as "The next steps are A, B, and C," and the emotion engine adjusts it to use softer wording.

[0509] Step 11:

[0510] The terminal receives the summary data and displays it to the user. The input data is the adjusted summary data, and the output data is data for the user to confirm. Specifically, the adjusted summary data is displayed on the terminal, and the user can confirm the summary, "The next steps are A, B, and C."

[0511] (Application example 2)

[0512] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0513] Current image generation systems and conference support systems are unable to consider the emotional state of the user, and in autonomous vehicles, there is a lack of services that respond to the emotional state of the occupants. Furthermore, the summary generation during a conference does not adjust to reflect the emotions of participants, making improving the user experience a challenge. In particular, there is a need for a comfortable environment and efficient conference management in autonomous vehicles.

[0514] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving image requirements from a user, means for instructing an image generation AI model to generate an image based on the image requirements, means for returning the generated image to the user's terminal, means for receiving correction requirements from the user and again instructing the image generation AI model to generate a corrected image, means for performing a copyright check on the generated image, means for recording the results of the copyright check, means for automatically transcribing audio data recorded during a conference, means for summarizing the transcribed data and providing it to the user, means for analyzing the emotional states of occupants in an autonomous vehicle in real time, means for automatically adjusting the in-vehicle environment (music, air conditioning, lighting, etc.) based on the analysis, means for collecting driving data of the autonomous vehicle and generating a visual report, and means for automatically adjusting the summary content of the transcribed data based on the emotional states of participants during the conference. This enables a comfortable environment to be set up in an autonomous vehicle, enabling the provision of services that take occupants' emotions into consideration and efficient conference management.

[0515] "Image requirements" are requirements specified by the user regarding the image to be generated, and include theme, style, color tone, resolution, and the like.

[0516] An "image generation AI model" is an artificial intelligence model that generates new images based on input image requirements.

[0517] The "correction requirements" are the correction requirements that the user must meet for the generated image.

[0518] "Copyright check" refers to the process of inspecting whether the generated image is similar to an existing image and whether it infringes copyright.

[0519] "Audio data recorded during a conference" refers to audio recorded while the conference is in progress.

[0520] "Automatic transcription" is the process of converting recorded audio data into text data.

[0521] "Summarization" is the process of simplifying transcribed data and extracting and organizing the main points.

[0522] "Emotional state" refers to the psychological state of the occupant as perceived from facial expressions, voice, etc.

[0523] "In-car environment" is a general term for music, air conditioning, lighting, etc. inside an autonomous vehicle.

[0524] "Driving data" refers to various data such as location information and speed associated with the movement of an autonomous vehicle.

[0525] A "visual report" is a report that is generated based on collected data in a visually easy-to-understand format.

[0526] This invention is a system that integrates image generation, emotion recognition, and conferencing support to improve the user experience in an autonomous vehicle. This system is composed of multiple hardware and software components that operate in the autonomous vehicle.

[0527] 1. System Configuration

[0528] The system consists of the following main hardware and software:

[0529] 1.1 Hardware

[0530] Devices: tablets, smartphones, in-car displays

[0531] Emotion recognition camera: A camera for capturing passenger facial expressions

[0532] Microphone: A microphone for capturing passenger voices and conference audio

[0533] Vehicle sensors: Sensors for collecting driving data from autonomous vehicles, such as GPS and speed sensors

[0534] 1.2 Software

[0535] Emotion recognition engine: Software that analyzes the emotional state of passengers (e.g., Microsoft Azure Cognitive Services, Google Cloud Vision AI)

[0536] Image generation AI model: An artificial intelligence model for generating images (e.g., OpenAI's DALL-E)

[0537] Speech recognition AI model: Software for transcribing meeting audio (e.g., Google Cloud Speech-to-Text)

[0538] Summarization AI model: Software for summarizing transcribed text (e.g., GPT-4)

[0539] 2. Program Processing

[0540] 2.1 Emotion recognition and environmental adjustment

[0541] The device's camera and microphone capture the passenger's facial expressions and voice, and send them to an emotion recognition engine in the cloud. The emotion recognition engine analyzes the captured images to understand the passenger's emotional state. Based on the analysis results, the car's music, air conditioning, and lighting are automatically adjusted.

[0542] 2.2 Image Creation and Modification

[0543] The user inputs image requirements on their device and sends them to the server. The server generates an image using an image generation AI model based on the requirements and returns the generated image to the device. If the user requests any corrections, the details are input again into the image generation AI model to generate a corrected image. The generated image also undergoes copyright checks.

[0544] 2.3 Meeting Support

[0545] During the meeting, crew members record audio using a microphone and send the data to a server. The server's speech recognition AI model converts the audio into text, and the summarization AI model extracts key points and creates a summary. When creating the summary, an emotion recognition engine adjusts the content to reflect the emotional state of the participants.

[0546] 3. Examples and prompts

[0547] 3.1 Specific examples

[0548] For example, it can analyze emotions during a relaxed drive and optimize the background music and air conditioning in the car. It can also record conference audio, generate summaries, and adjust the settings to take into account the emotions of the participants.

[0549] 3.2 Prompt Sentence Examples

[0550] "Create a report featuring the beautiful scenery during the relaxing drive. The passengers were very happy."

[0551] "Generate a summary of today's meeting and shorten the sections that participants found boring."

[0552] This will enable a comfortable environment to be created inside an autonomous vehicle, efficient meeting management, and the provision of high-quality services that take into consideration the emotions of passengers.

[0553] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0554] Step 1:

[0555] The terminal receives image requirements from the user, including theme, style, color tone, resolution, etc. The input image requirements are sent to the server.

[0556] Step 2:

[0557] The server analyzes the received image requirements and instructs the image generation AI model to generate an image based on that information. The image generation AI model (e.g., OpenAI's DALL-E) generates a new image based on the prompt text.

[0558] Step 3:

[0559] The server returns the generated image to the terminal, which displays the returned image to the user. The user checks the image and inputs correction requirements as necessary.

[0560] Step 4:

[0561] The device sends the user's correction requirements to the server, which then instructs the image generation AI model to generate a corrected image again, generating a new image.

[0562] Step 5:

[0563] The server performs a copyright check on the new image generated, using an algorithm (e.g., image recognition AI) that evaluates the similarity and authenticity of the generated image with existing images. The results of the copyright check are recorded.

[0564] Step 6:

[0565] The device's camera and microphone capture the passenger's facial expressions and voice, which are then sent to an emotion recognition engine in the cloud (e.g., Microsoft Azure Cognitive Services), which analyzes the passenger's emotional state and sends the results back to the device.

[0566] Step 7:

[0567] The device receives the analysis results and automatically adjusts the in-car environment (music, air conditioning, lighting, etc.) For example, if the passenger is relaxing, the music will be soft and the temperature will be moderate.

[0568] Step 8:

[0569] When a user starts a conference in a moving autonomous vehicle, the microphone on the device records the conference audio, and the recorded audio data is sent to the server.

[0570] Step 9:

[0571] The server passes the recorded audio data to an automatic transcription AI model (e.g., Google Cloud Speech-to-Text) to convert it into text data, which is then passed to a summarization AI model (e.g., GPT-4) to generate a summary.

[0572] Step 10:

[0573] The emotion recognition engine analyzes the emotional state of participants during the meeting and reflects it in the summary. For example, if it detects that a participant is bored, it will extract only the important points and summarize them concisely.

[0574] Step 11:

[0575] The summarized text data is sent back to the terminal, where the user can review it and use it as a follow-up for the meeting if necessary. This series of processes enables a comfortable environment to be set up inside the autonomous vehicle and for efficient meeting management.

[0576] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0577] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0578] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0579] [Second embodiment]

[0580] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0581] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0582] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0583] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0584] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0585] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0586] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0587] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0588] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0589] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0590] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0591] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0592] This invention relates to a system for generating images for use in proposals and meeting materials, redacting images, checking copyrights, and automatically transcribing and summarizing meeting records. The program processing of the system is explained in natural language below.

[0593] System Overview

[0594] The system allows users to input image requirements through their devices and send them to a server. The server then uses an image generation AI model to generate images, returns them to the user, and makes any necessary corrections before providing the final image. It also includes a copyright check function to avoid the risk of copyright infringement, and the ability to automatically transcribe and summarize audio data during meetings.

[0595] Image Generation Process

[0596] 1. User Roles

[0597] Users use the device to input image requirements for proposals and meeting materials, such as "next-generation smartphone," "futuristic design," "blue color theme," and "high resolution."

[0598] After the image is returned, the user checks it, enters correction requirements as necessary, and resubmits it.

[0599] 2. Role of the terminal

[0600] The device sends the image requirements entered by the user to the server, which sends the request as JSON format data.

[0601] The terminal displays the generated and modified images returned from the server.

[0602] 3. Server Roles

[0603] The server analyzes the image requirements received from the device and instructs the image generation AI model to generate the image based on that information.

[0604] It receives image data generated by an image generation AI model, adds metadata (date and time of generation, conditions of use, etc.), and returns it to the device.

[0605] If correction instructions are given, the correction instructions are sent again to the image generation AI model, and a corrected version is generated.

[0606] It uses an internal copyright checking AI model to assess the copyright risk of the generated images and records the results.

[0607] Copyright Check Process

[0608] The server uses an internal copyright check AI model to evaluate the generated image for similarity and reliability with existing images. The evaluation results are recorded and provided to users only if there are no problems with using the image.

[0609] Meeting recording efficiency process

[0610] 1. Recording audio data

[0611] During a meeting, users use the terminal to record audio of comments and notes.

[0612] 2. Automatic transcription

[0613] The device sends the recorded audio data to a server, which passes it to an automatic transcription AI model to convert it into text.

[0614] 3. Summary Generation

[0615] The server passes the generated text data to a summarization AI model to generate a summary, which is returned to the user and used as a meeting record.

[0616] Specific examples

[0617] Image generation example

[0618] When a user requests an image with the theme of "next-generation smartphones," the device sends the request to the server.

[0619] The server passes the requirements to the image generation AI model and receives the generated high-resolution smartphone image of the futuristic design.

[0620] The server adds metadata to the generated image and returns it to the terminal.

[0621] The user checks the image and inputs correction requirements, such as "I want the gloss to be a little less." The terminal again sends the correction requirements to the server, and the server generates the corrected image.

[0622] Meeting recording example

[0623] During a conference, a user records a voice memo on a terminal, and the terminal transmits the voice data to a server.

[0624] The server automatically transcribes the audio data and summarizes the generated text.

[0625] The summary data is sent back to the terminal, and the user can use it efficiently as a follow-up after the meeting.

[0626] This system streamlines the image creation and editing process for proposals and meeting materials, reduces copyright risks, and enables quick summarization of meeting records.

[0627] The processing flow will be explained below.

[0628] Image Generation Process

[0629] Step 1:

[0630] The user logs in to a dedicated application on the device.

[0631] Users input image requirements for proposals and meeting materials, such as "next-generation smartphone," "futuristic design," "blue color theme," and "high resolution."

[0632] Step 2:

[0633] The device sends the entered image requirements to the server as JSON format data.

[0634] Step 3:

[0635] The server parses the JSON format image requirements received from the device.

[0636] Step 4:

[0637] The server instructs the image generation AI model to generate an image based on the analyzed image requirements.

[0638] Step 5:

[0639] The image generation AI model generates new images based on the specified requirements.

[0640] Step 6:

[0641] The server receives image data generated by the image generation AI model and adds metadata (such as the date and time of generation, conditions of use, etc.) to the generated image.

[0642] Step 7:

[0643] The server transmits the generated image to the terminal.

[0644] Step 8:

[0645] The user checks the image generated on the terminal. If the user determines that corrections are necessary, they input the correction requirements into the terminal. For example, they input correction instructions such as "make the design less glossy."

[0646] Step 9:

[0647] The terminal transmits the correction requirements entered by the user back to the server as JSON format data.

[0648] Step 10:

[0649] Based on the correction requirements received from the terminal, the server again instructs the image generation AI model to generate a corrected image.

[0650] Step 11:

[0651] The image generation AI model regenerates the image based on the correction requirements and sends it back to the server.

[0652] Step 12:

[0653] The server receives the corrected image data and sends it back to the terminal, where the user checks the corrected image and finally approves it.

[0654] Copyright Check Process

[0655] Step 1:

[0656] The server uses an internal copyright checking AI model to evaluate the generated image for similarity and authenticity with existing images.

[0657] Step 2:

[0658] The server records the evaluation results in an internal database, and if there are no problems, provides the image to the user.

[0659] Step 3:

[0660] The server saves the good images in the project folder and makes them accessible to users.

[0661] Meeting recording efficiency process

[0662] Step 1:

[0663] During a meeting, a user uses the terminal to record audio of comments and notes.

[0664] Step 2:

[0665] The device transmits the recorded audio data to the server.

[0666] Step 3:

[0667] The server passes the audio data to an automatic transcription AI model and converts it into text.

[0668] Step 4:

[0669] The server passes the generated text data to a summary AI model to generate summary data.

[0670] Step 5:

[0671] The server sends the generated summary data to the terminal, where the user can check it. The user can use it efficiently as a follow-up after the meeting.

[0672] Example 1

[0673] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0674] In today's world, the creation and editing of images for proposals and meeting materials, copyright verification, and automation of meeting recording are extremely important issues. However, these processes are extremely time-consuming and laborious, so systems that can handle them efficiently and quickly are needed. Furthermore, it is also necessary to check whether the generated images are similar to existing images and whether they infringe copyright. There is also a need for automatic transcription and summary generation of audio data recorded during meetings, but it is currently difficult to efficiently achieve these functions in a single system.

[0675] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0676] In this invention, the server includes a means for receiving image requirements from a user, a means for instructing an image generation AI model to generate an image, and a means for performing copyright checks on the generated images. This allows users to easily input specific image requirements, quickly generate and edit high-quality images, and evaluate copyright risks. Furthermore, by including a function for automatically transcribing audio data during meetings and generating summaries, it also automates meeting recording, significantly improving overall work efficiency.

[0677] The "means for receiving image requirements" is a function for receiving conditions and features related to an image specified by a user.

[0678] An "image generation AI model" is an algorithm or system that uses artificial intelligence to generate images based on specified conditions.

[0679] The "means for instructing image generation" is a function that conveys the received image requirements to the image generation AI model and causes it to generate the image.

[0680] "Means for returning generated images" refers to a function that returns images generated by an image generation AI model to the user's device.

[0681] The "means for receiving correction requirements" is a function for receiving a correction request for a generated image from a user.

[0682] The "means for performing copyright checks" is a function that evaluates the similarity and reliance of generated images against existing images, and checks for the risk of copyright infringement.

[0683] The "means for recording the results of copyright checks" is a function for saving the results of copyright checks in a database or the like.

[0684] "Means for automatically transcribing audio data" refers to a function that converts audio data recorded during a meeting into text data.

[0685] The "means for summarizing and providing transcribed data" is a function that summarizes automatically transcribed text data and provides it to the user.

[0686] The "means for adding and providing metadata" is a function for adding additional information such as the date and time of creation and the conditions of use to the created image and providing it to the user.

[0687] "Means for sending in JSON format" is a function that sends image requirements specified by the user to the server in a data format called JSON (JavaScript Object Notation).

[0688] The present invention relates to a system for generating images for use in proposals and meeting materials, redacting, copyright checking, and automatically transcribing and summarizing meeting records. The system is implemented as follows:

[0689] System Overview

[0690] In this system, users input image requirements through their devices and send them to the server. The server uses an image generation AI model to generate images, returns them to the user, makes any necessary corrections, and provides the final image. The system also has a copyright check function for generated images, and can automatically transcribe audio data from meetings and generate summaries.

[0691] Hardware and software used

[0692] The system uses the following hardware and software:

[0693] Image generation AI models: e.g., DALL-E and Stable Diffusion

[0694] Copyright checking AI model: For example, CopyChecker

[0695] Automatic transcription AI models: e.g., Google Cloud Speech-to-Text

[0696] Summary generation AI model: e.g., GPT-3

[0697] Device: The computer or smartphone used by the user

[0698] Server: A remote server for data processing and running AI models

[0699] Specific operation of the system

[0700] 1. Enter image requirements

[0701] Users can use their devices to input image requirements for proposals and meeting materials, such as "next-generation smartphone," "futuristic design," "blue color theme," and "high resolution." Users simply enter their specific requirements in the text box and click the send button.

[0702] 2. Submit your image requirements

[0703] The device sends the image requirements entered by the user to the server in JSON format. The data is sent via an HTTP POST request. The following JSON format data is used:

[0704] json

[0705] {

[0706] "theme": "Next-generation smartphone",

[0707] "style": "futuristic design",

[0708] "color": "blue",

[0709] "resolution": "High resolution"

[0710] }

[0711] 3. Execute image generation

[0712] The server analyzes the received image requirements and issues instructions to the image generation AI model based on them. For example, a prompt might look like this: "Create a high-resolution image of a futuristic next-generation smartphone with a blue color theme." The server receives the image data generated by the image generation AI model, adds metadata (such as the date and time of generation and conditions of use), and sends it back to the device.

[0713] 4. Check and correct the image

[0714] The user checks the displayed image and inputs correction requirements as necessary. For example, they can enter a command such as "reduce the gloss" in the text box and click the resend button. The device then sends the correction requirements back to the server in JSON format, and the server again sends the correction instructions to the image generation AI model.

[0715] 5. Copyright Check

[0716] The server uses an internal copyright check AI model to evaluate the generated image for similarity and reliability with existing images. The results of the copyright check are recorded and provided to the user only if there are no problems.

[0717] 6. Audio recording and transcription

[0718] During a meeting, users record audio notes on their device, which then sends the audio data to a server, which then converts the audio into text using an automatic transcription AI model. For example, "Transcribe the following meeting audio to text."

[0719] 7. Generating and Serving Abstracts

[0720] The server passes the generated text data to a summarization AI model to generate summary data. This summary is sent back to the user and used as a meeting record. A summary like the following is generated: "At the meeting, the progress and next steps of the new project were discussed."

[0721] This system streamlines the image creation and editing process for proposals and meeting materials, reduces copyright risks, and enables quick summarization of meeting records.

[0722] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0723] Image Generation Process

[0724] Step 1: User inputs image requirements

[0725] Input: Image requirements for user proposals and meeting materials (e.g., "next-generation smartphone," "futuristic design," "blue color theme," "high resolution")

[0726] Specific operation: The user enters the image requirements in the text box on the terminal and clicks the send button.

[0727] Output: Image requirements are entered into the terminal and prepared for sending to the server.

[0728] Step 2: Send image requirements from the device to the server

[0729] Input: User-entered image requirements

[0730] Specific operation: The device converts the image requirements into JSON format and sends it to the server as an HTTP POST request.

[0731] Output: The following JSON data is sent to the server:

[0732] json

[0733] {

[0734] "theme": "Next-generation smartphone",

[0735] "style": "futuristic design",

[0736] "color": "blue",

[0737] "resolution": "High resolution"

[0738] }

[0739] Step 3: Image generation instructions from the server

[0740] Input: Image requirements sent from the terminal (JSON data)

[0741] Specific operation: The server parses the received JSON data and generates an appropriate prompt for the image generation AI model. For example: "Create a high-resolution image of a futuristic next-generation smartphone with a blue color theme."

[0742] Output: A prompt sentence is input into the image generation AI model, and the generated image data is returned to the server.

[0743] Step 4: Returning the image from the server to the device

[0744] Input: Image data generated by an image generation AI model

[0745] Specific operation: The server adds metadata (date and time of creation, conditions of use, etc.) to the generated image and returns it to the terminal.

[0746] Output: Image data with metadata is sent to the device. Example:

[0747] json

[0748] {

[0749] "image": "base64_encoded_image_data",

[0750] "metadata": {

[0751] "generated_at": "2023-10-01T12:00:00Z",

[0752] "usage_conditions": "Commercial use allowed"

[0753] }

[0754] }

[0755] Step 5: User checks the image and gives correction instructions

[0756] Input: Image data returned from the server

[0757] Specific operation: The user checks the image on the device and inputs correction requirements as necessary (e.g., "I would like the glossiness to be reduced"). After inputting the correction requirements, the user clicks the resend button.

[0758] Output: The modified image requirements are entered into the terminal and are ready to be sent to the server again.

[0759] Step 6: Sending modification requirements from the terminal to the server

[0760] Input: User-entered correction requirements

[0761] Specific operation: The terminal converts the modification requirements into JSON format and resubmits them to the server as an HTTP POST request.

[0762] Output: The correction requirements are sent to the server and used again as prompts for the image generation AI model. Example of resubmitted JSON data:

[0763] json

[0764] {

[0765] "theme": "Next-generation smartphone",

[0766] "style": "futuristic design",

[0767] "color": "blue",

[0768] "resolution": "high resolution",

[0769] "modification": "Reduce glossiness"

[0770] }

[0771] Step 7: Copyright Check

[0772] Input: Generated image data

[0773] How it works: The server inputs the generated image into a copyright checking AI model to evaluate its similarity to existing images and its reliability. The evaluation results are recorded and provided to the user only if there are no problems.

[0774] Output: Copyright check results and usable image data based on the evaluation. Example:

[0775] json

[0776] {

[0777] "image": "base64_encoded_image_data",

[0778] "metadata": {

[0779] "generated_at": "2023-10-01T12:00:00Z",

[0780] "usage_conditions": "No copyright issues detected"

[0781] }

[0782] }

[0783] Meeting recording efficiency process

[0784] Step 1: Recording audio data

[0785] Input: Audio during the meeting

[0786] Specific operation: The user clicks the "Start Recording" button on the device to record conversations and notes in audio format.

[0787] Output: The recorded audio data is stored on the device and prepared for transmission to the server.

[0788] Step 2: Sending audio data from the device to the server

[0789] Input: Recorded audio data

[0790] Specific operation: The device converts the voice data into a digital audio file format (e.g., WAV, MP3) and sends it to the server as an HTTP POST request.

[0791] Output: The audio data is sent to the server. Example:

[0792] json

[0793] {

[0794] "audio_file": "base64_encoded_audio_data"

[0795] }

[0796] Step 3: Auto-transcription

[0797] Input: Audio data sent to the server

[0798] Specific operation: The server inputs the audio data into an automatic transcription AI model and converts the audio into text data. The prompt is as follows: "Transcribe the following meeting audio to text."

[0799] Output: A transcript of the audio is generated. Example:

[0800] text

[0801] "In the meeting, we discussed the progress of the new project and the next steps. Specifically, we had a detailed discussion about the design and features of the next-generation smartphone."

[0802] Step 4: Generate summary and return summary data

[0803] Input: Automatically transcribed text data

[0804] Specific operation: The server inputs the generated text data into the summary AI model and generates a summary. The prompt is as follows: "Summarize the following transcription of the meeting."

[0805] Output: Summarized text data is generated and sent back to the user. Example:

[0806] text

[0807] "During the meeting, the progress and next steps of the new project were discussed. In particular, there was detailed discussion on the design and functionality of the next generation smartphone."

[0808] Through these specific processing steps, the system streamlines the creation and editing of images required for proposals and meeting materials, reduces copyright risks, and enables quick summarization of meeting records.

[0809] (Application example 1)

[0810] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0811] In autonomous vehicles, there are problems with the efficient generation, editing, and copyright check of video surveillance and meeting records during operation, as well as the automatic transcription and summarization of audio data. In particular, the collection of surveillance data and the creation of driving reports within autonomous vehicles requires a great deal of time and effort using current manual processes, making effective operation difficult. In addition, copyright issues may arise with the generated data, so evaluation and management of these issues are required.

[0812] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0813] In this invention, the server includes means for receiving image requirements from a user, means for instructing an image generation AI model to generate an image based on the image requirements, means for returning the generated image to the user's terminal, means for receiving correction requirements from the user and again instructing the image generation AI model to generate a corrected image, means for performing a copyright check on the generated image, means for recording the results of the copyright check, means for automatically transcribing audio data recorded during a meeting, means for summarizing the transcribed data and providing it to the user, means for acquiring image surveillance data inside and outside the autonomous vehicle, means for transferring the recorded audio data, and means for automatically processing and summarizing audio memos. This enables efficient operation management of autonomous vehicles, rapid data generation and correction, reduced copyright risks, and rapid summarization of meeting records.

[0814] "User" refers to an entity that uses the system to generate images, specify correction requirements, and record audio data.

[0815] "Image generation AI model" refers to an algorithm or model that generates images based on specified image requirements.

[0816] "Image requirements" refer to the specific style, content, color tone, resolution, and other conditions of the image desired by the user.

[0817] "Modification requirements" refer to instructions for modifying or changing the generated image.

[0818] "Copyright check" refers to the process of assessing whether the generated image is similar to an existing image or infringes copyright.

[0819] "Audio data" refers to audio information recorded during a conference.

[0820] "Automatic transcription" refers to the process of automatically converting audio data into text data.

[0821] "Summarization" refers to the process of concisely summarizing transcribed text data.

[0822] An "autonomous vehicle" is a vehicle that is capable of operating independently without a human driver.

[0823] "Image surveillance data" refers to image information collected inside and outside an autonomous vehicle.

[0824] "Terminal" refers to a device used by a user, such as a computer or smartphone.

[0825] "Metadata" refers to data that includes information related to a generated image, such as the date and time of generation and the conditions of use.

[0826] This invention relates to a system for efficiently realizing image monitoring, copyright checking, and automatic transcription and summarization of meeting records in an autonomous vehicle. This system starts when a user inputs image requirements using a terminal and sends them to a server. Specific embodiments of this system are described below.

[0827] 1. System Overview

[0828] The system mainly uses the user's device, a server, and cameras and audio recording devices inside and outside the autonomous vehicle. The server utilizes an image generation AI model, a copyright check AI model, an automatic transcription AI model, and a summary AI model for efficient data processing.

[0829] 2. Hardware and Software

[0830] Hardware:

[0831] User's device (e.g. smartphone, tablet)

[0832] Cameras inside and outside the autonomous vehicle

[0833] Audio recording device

[0834] software:

[0835] Python

[0836] Flask (web server framework)

[0837] Image generation AI model (e.g., HuggingFace's DALL·E)

[0838] Copyright Check AI Model

[0839] Automatic transcription API (e.g., Google Speech-to-Text)

[0840] Summary AI models (e.g., HuggingFace's BART)

[0841] 3. Data processing and calculation

[0842] The server receives the request sent by the user from the terminal and processes the data through the following process.

[0843] Image generation process:

[0844] The user inputs image requirements from their device and sends them to the server, such as "futuristic design of an autonomous vehicle, blue color theme, high resolution."

[0845] The server uses an image generation AI model to generate images based on the specified requirements.

[0846] Metadata (date and time of creation, conditions of use, etc.) is added to the generated image, and the result is sent back to the user's device.

[0847] The user inputs the correction requirements for the image and sends it back to the server, which then generates the corrected image and sends it back to the user.

[0848] Copyright Check Process:

[0849] The server passes the generated images to a copyright checking AI model to evaluate their similarity to existing images.

[0850] The evaluation results are recorded and notified to the user. If there are no problems, the image becomes available for use.

[0851] Meeting recording efficiency process:

[0852] During the meeting, users record voice memos on their terminals and send them to the server.

[0853] The server transcribes the audio data using the Google Speech-to-Text API.

[0854] The transcribed text is passed to a summarization AI model to generate summary data.

[0855] The summary data is returned to the user's terminal and is effectively used as a record of the meeting.

[0856] 4. Specific Examples

[0857] Image generation example:

[0858] The user inputs image requirements such as "futuristic design, self-driving vehicle, blue color theme, high resolution."

[0859] The server calls an image generation AI model to generate a high-resolution image, adds metadata, and sends it back to the user's device.

[0860] The user inputs the correction requirement "reduce the glossiness a little" and sends it to the server again. The server generates a corrected image and provides it to the user.

[0861] Example of a recording:

[0862] A user records a voice memo during a conference, and the terminal transmits the voice data to a server.

[0863] The server automatically transcribes the audio data and passes the transcribed text to a summarization AI model to generate a summary.

[0864] The summary data is returned to the user's terminal, and the user can efficiently use the conference record.

[0865] This system not only streamlines the operation and management of autonomous vehicles, but also enables rapid data generation and correction, reduces copyright risks, and improves productivity by quickly summarizing meeting notes.

[0866] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0867] Step 1:

[0868] A user uses a terminal to input image requirements and create a request for image generation, such as "futuristic design of autonomous vehicle, blue color theme, high resolution."

[0869] Input: Image requirements (e.g. futuristic design, blue color theme, high resolution)

[0870] Output: Image generation request

[0871] Step 2:

[0872] The device converts the user's image requirements into JSON format and sends it to the server. The device properly formats the input requirements and prepares the request for sending to the server.

[0873] Input: Image generation request

[0874] Output: Request in JSON format

[0875] Step 3:

[0876] The server analyzes the received JSON-formatted request and instructs the image generation AI model to generate an image. The server analyzes the request, extracts the parameters necessary for image generation, and passes them to the AI ​​model.

[0877] Input: JSON formatted request

[0878] Output: Instructions to the AI ​​model

[0879] Step 4:

[0880] The image generation AI model generates images based on instructions. The model uses prompt text to generate high-resolution images according to the specified style and color tone.

[0881] Input: Instructions to the AI ​​model (e.g., futuristic design, self-driving vehicle, blue, resolution)

[0882] Output: The generated image

[0883] Step 5:

[0884] The server adds metadata (such as the date and time of creation and the conditions of use) to the created image and returns it to the user's device. The server records the date and time of creation and the conditions of use of the image and attaches them to the image data.

[0885] Input: Generated image

[0886] Output: Generated image with metadata

[0887] Step 6:

[0888] The user checks the generated image on the terminal and inputs the correction requirements. The user specifies the improvements and changes to the image and creates a new request.

[0889] Input: Generated image with metadata

[0890] Output: Fix requirements

[0891] Step 7:

[0892] The terminal converts the modification requirements into JSON format and sends it to the server. The terminal then sends a request based on the modification requirements to the server again.

[0893] Input: Correction Requirement

[0894] Output: The modification request in JSON format

[0895] Step 8:

[0896] The server analyzes the modification request and instructs the image generation AI model to generate the modified image again. The server instructs the AI ​​model to regenerate the image using the new modification parameters.

[0897] Input: JSON formatted modification request

[0898] Output: Corrective instructions for the AI ​​model

[0899] Step 9:

[0900] The image generation AI model generates a corrected image based on the correction instructions. It uses the prompt text again to generate an image that reflects the necessary corrections.

[0901] Input: Correction instructions for the AI ​​model

[0902] Output: Corrected image

[0903] Step 10:

[0904] The server returns the corrected image to the user's device. The corrected image data is sent to the user's device in JSON format.

[0905] Input: Modified image

[0906] Output: The corrected image sent back to the user

[0907] The above are the specific processing steps for image generation and correction. Next, we will show the processing steps for automatic transcription and summarization of meeting recordings.

[0908] Step 11:

[0909] During a meeting, the user records a voice memo, and the device sends the voice data to the server. The user then presses the record button to record the required voice.

[0910] Input: Voice memo

[0911] Output: Audio data

[0912] Step 12:

[0913] The server passes the audio data to an automated transcription method to convert it into text. The server uses the Google Speech-to-Text API to transcribe the audio data.

[0914] Input: Audio data

[0915] Output: Transcribed text

[0916] Step 13:

[0917] The server passes the generated text to a summarization model to generate summary data. The server then invokes a summarization AI model (e.g., BART) to generate a summary that extracts the key points.

[0918] Input: Transcribed text

[0919] Output: Summary data

[0920] Step 14:

[0921] The server returns the summary data to the user's terminal, and the user efficiently checks the meeting record. The points of the summarized meeting are provided to the user.

[0922] Input: Summary data

[0923] Output: Summary data returned to the user

[0924] These processing steps enable efficient management of image generation, editing, copyright checks, and meeting records in autonomous vehicles.

[0925] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0926] This invention combines a system for generating images for use in proposals and meeting materials, correcting them, checking copyrights, and automatically transcribing and summarizing meeting records with an emotion engine that recognizes the user's emotions. This system provides more human-centered and effective support by suggesting image requirements, making corrections, and adjusting the summaries of meeting records according to the user's emotional state.

[0927] System Overview

[0928] In this system, users input image requirements through their devices and send them to a server. The server then uses an image generation AI model to generate an image, which is returned to the user, where it is corrected as necessary and the final image is provided. The system also includes a copyright check function to avoid the risk of copyright infringement, and the ability to automatically transcribe and summarize audio data during meetings. A new emotion engine has also been added, enabling suggestions and corrections based on the user's emotions.

[0929] Image generation process and emotion engine integration

[0930] 1. User Roles

[0931] Users log in to a dedicated application on their device, and when entering image requirements for proposals or meeting materials, the system recognizes emotions from the user's facial expressions and voice.

[0932] The emotion engine analyzes the user's emotional state when they input and suggests appropriate image requirements, such as "if the user is excited, we suggest a bold design."

[0933] 2. Role of the terminal

[0934] The terminal transmits the input image requirements and additional requirements provided by the emotion engine to the server, thereby including requirements based on the user's emotions.

[0935] The terminal displays the generated or modified image returned from the server.

[0936] 3. Server Roles

[0937] The server analyzes the image requirements received from the device and instructs the image generation AI model to generate the image.

[0938] The image generation AI model generates new images based on the instruction requirements.

[0939] The server adds metadata to the generated image and returns it to the terminal.

[0940] When responding to correction instructions, the emotion engine evaluates the user's emotional response and automatically recommends correction requirements.

[0941] Copyright Check Process

[0942] The server uses its internal copyright checking AI model to evaluate the generated image for similarity and reliability with existing images. The evaluation results are recorded and provided to users only if there are no problems with using the image.

[0943] Meeting recording efficiency process

[0944] 1. Recording audio data

[0945] During a meeting, users use their devices to record audio of comments and notes, and their emotional state during the recording is also analyzed.

[0946] 2. Automatic transcription

[0947] The device sends the recorded audio data to a server, which then passes the data to an automatic transcription AI model and converts it into text.

[0948] 3. Summary Generation

[0949] The server passes the transcribed text data to a summary AI model to generate a summary. An emotion engine then applies the summary to adjust the content according to the user's emotional state.

[0950] Specific examples

[0951] Image generation example

[0952] When a user requests an image themed on "next-generation smartphones" and inputs their requirements into the device, the emotion engine analyzes the user's emotional state. For example, if a user is excited, it will recommend a "futuristic design."

[0953] The device sends the proposed requirements and the emotion engine's suggestions to the server. The server receives the image generated by the image generation AI model and returns it to the device. The user checks the image, and the emotion engine automatically suggests correction requirements.

[0954] Meeting recording example

[0955] During a meeting, when a user records a voice memo on the terminal, the emotion engine analyzes the user's emotional state.

[0956] The device sends the recorded voice data to a server, which then automatically transcribes it. At the same time, an emotion engine adjusts the summary content to match the user's emotions based on the generated text data.

[0957] The summary data is sent back to the terminal, and the user can use it efficiently as a follow-up after the meeting.

[0958] This system streamlines the image creation and editing process for proposals and meeting materials, improves user response based on user sentiment, reduces copyright risks, and enables quick summarization of meeting records.

[0959] The processing flow will be explained below.

[0960] Image generation process incorporating an emotion engine

[0961] User-defined image requirements and generation

[0962] Step 1:

[0963] The user logs in to a dedicated application on the device.

[0964] Step 2:

[0965] The user inputs the image requirements for proposals and meeting materials into the device, such as "next-generation smartphone," "futuristic design," "blue color theme," and "high resolution."

[0966] Step 3:

[0967] While the user is typing, the device uses a built-in camera and microphone to analyze the user's facial expressions and voice, and an emotion engine evaluates the user's emotional state.

[0968] Step 4:

[0969] The emotion engine makes additional suggestions for image requirements based on the user's emotional state, for example, suggesting a "daring design" if the user is excited.

[0970] Step 5:

[0971] The device sends image requirements and emotion engine suggestions to the server as JSON format data.

[0972] Step 6:

[0973] The server analyzes the JSON format image requirements received from the device and instructs the image generation AI model to generate the image.

[0974] Step 7:

[0975] The image generation AI model generates new images based on the specified requirements.

[0976] Step 8:

[0977] The server receives image data generated by the image generation AI model and adds metadata (such as the date and time of generation, conditions of use, etc.) to the generated image.

[0978] Step 9:

[0979] The server transmits the generated image to the terminal.

[0980] Step 10:

[0981] The user checks the image generated on the device. If the user determines that corrections are necessary, the user inputs the correction requirements into the device (for example, "make the design less glossy").

[0982] Step 11:

[0983] The emotion engine evaluates the user's emotional response when viewing an image and automatically recommends revision requirements.

[0984] Step 12:

[0985] The device sends the user's correction requirements and the emotion engine's suggestions back to the server as JSON format data.

[0986] Step 13:

[0987] Based on the correction requirements received from the terminal, the server again instructs the image generation AI model to generate a corrected image.

[0988] Step 14:

[0989] The image generation AI model regenerates the image based on the correction requirements and sends it back to the server.

[0990] Step 15:

[0991] The server receives the corrected image data and sends it back to the terminal, where the user checks the corrected image and finally approves it.

[0992] Copyright check process incorporating emotion engine

[0993] Step 1:

[0994] The server uses its internal copyright checking AI model to evaluate the generated image for similarity and reliability with existing images.

[0995] Step 2:

[0996] The server records the evaluation results in an internal database, and if there are no problems, the image is provided to the user.

[0997] Step 3:

[0998] The server saves the good images in the project folder and makes them accessible to users.

[0999] Emotion-driven meeting recording process

[1000] Audio data recording and processing

[1001] Step 1:

[1002] During a meeting, a user uses the terminal to record audio of comments and notes.

[1003] Step 2:

[1004] The device analyzes the tone and pace of the user's voice while recording, and an emotion engine evaluates the user's emotional state.

[1005] Step 3:

[1006] The device sends the recorded audio data to the server.

[1007] Step 4:

[1008] The server passes the audio data to an automatic transcription AI model and converts it into text.

[1009] Step 5:

[1010] The server passes the generated text data to a summary AI model to generate summary data.

[1011] Step 6:

[1012] The emotion engine uses the summary data to tailor the content to the user's emotional state, for example, emphasizing positive summary results if the user is feeling down.

[1013] Step 7:

[1014] The server sends the generated summary data to the terminal, where the user can check it. The user can use it efficiently as a follow-up after the meeting.

[1015] These steps provide a system incorporating an emotion engine that not only streamlines the process of creating proposals and meeting materials, but also allows for suggestions and revisions based on the user's emotions, reduces copyright risks, and enables quick summarization of meeting records.

[1016] Example 2

[1017] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1018] In today's busy business environment, creating proposals and meeting materials requires a lot of time and effort. The tasks of generating and editing high-quality images that reflect the user's intent and emotions, as well as efficiently transcribing and summarizing meeting recordings, are particularly burdensome. The risk of copyright infringement for generated images is also a significant issue. It is necessary to develop a system that streamlines these processes and improves the user experience.

[1019] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1020] In this invention, the server includes means for proposing appropriate image requirements in cooperation with an emotion engine that recognizes and analyzes emotions from a user's facial expressions and voice, means for instructing an image generation AI model to generate an image, means for performing copyright checks on the generated image, means for automatically transcribing audio data recorded during a meeting, and means for summarizing the transcribed data, adjusting the content based on the user's emotional state, and providing the summarization to the user. This streamlines the image creation and editing process for proposals and meeting materials, improves response based on user emotions, reduces copyright risks, and enables quick summarization of meeting records.

[1021] A "user" is a person who operates the system and inputs image and modification requirements.

[1022] "Terminal" refers to an electronic device such as a computer or smartphone used by a user.

[1023] "Image requirements" refer to specific conditions and specifications regarding the image that a user wants to create.

[1024] The "emotion engine" is an AI model that recognizes emotions from a user's facial expressions and voice and analyzes their emotional state.

[1025] An "image generation AI model" is an artificial intelligence model for automatically generating images based on input requirements.

[1026] "Copyright check" is the process of checking whether the generated image is similar to or relies on an existing image and assessing legal risk.

[1027] The "correction requirements" refer to the corrections that the user wants to make to the generated image.

[1028] "Automatic transcription" is the automated process of converting audio data into text.

[1029] A "summary" is a text that briefly summarizes the transcribed data.

[1030] "Metadata" is auxiliary information added to a generated image, and is data that indicates details and attributes of the image.

[1031] This invention combines a system for generating images for use in proposals and meeting materials, correcting them, checking copyrights, and automatically transcribing and summarizing meeting records with an emotion engine that recognizes the user's emotions. This system provides more human-centered and effective support by suggesting image requirements, making corrections, and adjusting the summaries of meeting records according to the user's emotional state.

[1032] System Overview

[1033] In this system, users input image requirements through their devices and send them to a server. The server then uses an image generation AI model to generate an image, which is returned to the user, where it is corrected as necessary and the final image is provided. The system also includes a copyright check function to avoid the risk of copyright infringement, and the ability to automatically transcribe and summarize audio data during meetings. A new emotion engine has also been added, enabling suggestions and corrections based on the user's emotions.

[1034] Specific examples

[1035] Image generation example

[1036] When a user requests an image themed around "next-generation smartphones" and inputs the requirements into the device, the emotion engine analyzes the user's emotional state. For example, if the user is in an excited state, it will recommend a "futuristic design." The device then sends the proposed requirements and the emotion engine's suggestions to the server. The server receives the image generated by the image generation AI model and returns it to the device. The user then reviews the image, and the emotion engine automatically suggests further revisions.

[1037] Example prompt for a generative AI model:

[1038] "Generate images with futuristic and exciting designs based on the theme of next-generation smartphones."

[1039] Meeting recording example

[1040] During a meeting, when a user records a voice memo on their device, the emotion engine analyzes the user's emotional state. The device then sends the recorded voice data to a server, which then automatically transcribes it. At the same time, the emotion engine adjusts the summary content to match the user's emotions based on the generated text data. The summary data is then sent back to the device, allowing the user to efficiently use it as a follow-up after the meeting.

[1041] The system streamlines the image creation and editing process for proposals and meeting materials, enables user-sensitive responses, mitigates copyright risks, and enables quick summarization of meeting records.

[1042] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1043] Step 1:

[1044] The user logs in to a dedicated application on their device. The user inputs the image requirements needed for proposals and meeting materials, and the emotion engine recognizes and analyzes the user's emotions from their facial expressions and voice. For example, when a user requests a "futuristic design for a next-generation smartphone," the emotion engine proposes requirements that reflect the user's excited state of mind. The input data includes image requirements, and the output is the emotion analysis results and proposed image requirements.

[1045] Step 2:

[1046] The device sends the image requirements entered by the user and the suggestions made by the emotion engine to the server. The input data here are the user's image requirements and the results of emotion analysis, and the output data is a request message containing these. Specifically, the device sends a request to "generate a futuristic design for the next generation smartphone" along with the user's emotion data to the server.

[1047] Step 3:

[1048] The server analyzes the image requirements received from the device and instructs the image generation AI model to generate the image. The input data includes image requirements and emotion data, and the output is the generated image. Specifically, the server sends a "futuristic design for next-generation smartphones" as a prompt to the image generation AI model, and after receiving the generated image, it adds metadata.

[1049] Step 4:

[1050] The server returns the generated image to the terminal. The input data here is the generated image and metadata, and the output data is the image returned to the user's terminal. Specifically, the server returns the generated image to the terminal together with the metadata.

[1051] Step 5:

[1052] The device displays the generated image returned from the server to the user. The user checks the image and inputs correction requirements, if necessary. The emotion engine again analyzes the user's emotional state and makes suggestions for correction. The input data includes the generated image and the user's correction requirements, and the output data is the correction suggestions. Specifically, the user requests "more colorful," and the emotion engine takes this into consideration and suggests an even bolder color palette.

[1053] Step 6:

[1054] The device sends the user's correction requirements to the server, and the server again instructs the image generation AI model to generate a corrected image. The input data includes the correction requirements and emotion data, and the output data is the corrected image. Specifically, the device sends the correction requirement "change to a colorful design" to the server, and the server sends it again to the image generation AI model to generate the corrected image.

[1055] Step 7:

[1056] The server performs a copyright check on the generated image. The input data is the generated image, and the output data is the result of the copyright check. Specifically, the server uses a copyright check AI model to evaluate the similarity between the generated image and existing images, and records the results.

[1057] Step 8:

[1058] During a meeting, the user uses a device to record voice comments and notes. The emotion engine simultaneously analyzes the user's emotional state during the recording. The input data is the recorded voice and emotional data, and the output data is the analysis results. Specifically, the user says, "Regarding the next action plan," which is recorded as a voice memo, and the emotion engine analyzes the user's state of tension.

[1059] Step 9:

[1060] The device sends the recorded voice data to a server, which then passes the data to an automatic transcription AI model and converts it into text. The input data is voice data, and the output data is the transcribed text. Specifically, the device sends voice data saying "What is the next action plan?" to the server, and the server converts the voice data into text.

[1061] Step 10:

[1062] The server passes the transcribed text data to a summarization AI model, which generates summary data. The emotion engine then adjusts the content based on the generated summary according to the user's emotional state. The input data is the transcribed text and emotion data, and the output data is the adjusted summary. Specifically, the summarization AI model generates a short summary such as "The next steps are A, B, and C," and the emotion engine adjusts it to use softer wording.

[1063] Step 11:

[1064] The terminal receives the summary data and displays it to the user. The input data is the adjusted summary data, and the output data is data for the user to confirm. Specifically, the adjusted summary data is displayed on the terminal, and the user can confirm the summary, "The next steps are A, B, and C."

[1065] (Application example 2)

[1066] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1067] Current image generation systems and conference support systems are unable to consider the emotional state of the user, and in autonomous vehicles, there is a lack of services that respond to the emotional state of the occupants. Furthermore, the summary generation during a conference does not adjust to reflect the emotions of participants, making improving the user experience a challenge. In particular, there is a need for a comfortable environment and efficient conference management in autonomous vehicles.

[1068] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving image requirements from a user, means for instructing an image generation AI model to generate an image based on the image requirements, means for returning the generated image to the user's terminal, means for receiving correction requirements from the user and again instructing the image generation AI model to generate a corrected image, means for performing a copyright check on the generated image, means for recording the results of the copyright check, means for automatically transcribing audio data recorded during a conference, means for summarizing the transcribed data and providing it to the user, means for analyzing the emotional states of occupants in an autonomous vehicle in real time, means for automatically adjusting the in-vehicle environment (music, air conditioning, lighting, etc.) based on the analysis, means for collecting driving data of the autonomous vehicle and generating a visual report, and means for automatically adjusting the summary content of the transcribed data based on the emotional states of participants during the conference. This enables a comfortable environment to be set up in an autonomous vehicle, enabling the provision of services that take occupants' emotions into consideration and efficient conference management.

[1069] "Image requirements" are requirements specified by the user regarding the image to be generated, and include theme, style, color tone, resolution, and the like.

[1070] An "image generation AI model" is an artificial intelligence model that generates new images based on input image requirements.

[1071] The "correction requirements" are the correction requirements that the user must meet for the generated image.

[1072] "Copyright check" refers to the process of inspecting whether the generated image is similar to an existing image and whether it infringes copyright.

[1073] "Audio data recorded during a conference" refers to audio recorded while the conference is in progress.

[1074] "Automatic transcription" is the process of converting recorded audio data into text data.

[1075] "Summarization" is the process of simplifying transcribed data and extracting and organizing the main points.

[1076] "Emotional state" refers to the psychological state of the occupant as perceived from facial expressions, voice, etc.

[1077] "In-car environment" is a general term for music, air conditioning, lighting, etc. inside an autonomous vehicle.

[1078] "Driving data" refers to various data such as location information and speed associated with the movement of an autonomous vehicle.

[1079] A "visual report" is a report that is generated based on collected data in a visually easy-to-understand format.

[1080] This invention is a system that integrates image generation, emotion recognition, and conferencing support to improve the user experience in an autonomous vehicle. This system is composed of multiple hardware and software components that operate in the autonomous vehicle.

[1081] 1. System Configuration

[1082] The system consists of the following main hardware and software:

[1083] 1.1 Hardware

[1084] Devices: tablets, smartphones, in-car displays

[1085] Emotion recognition camera: A camera for capturing passenger facial expressions

[1086] Microphone: A microphone for capturing passenger voices and conference audio

[1087] Vehicle sensors: Sensors for collecting driving data from autonomous vehicles, such as GPS and speed sensors

[1088] 1.2 Software

[1089] Emotion recognition engine: Software that analyzes the emotional state of passengers (e.g., Microsoft Azure Cognitive Services, Google Cloud Vision AI)

[1090] Image generation AI model: An artificial intelligence model for generating images (e.g., OpenAI's DALL-E)

[1091] Speech recognition AI model: Software for transcribing meeting audio (e.g., Google Cloud Speech-to-Text)

[1092] Summarization AI model: Software for summarizing transcribed text (e.g., GPT-4)

[1093] 2. Program Processing

[1094] 2.1 Emotion recognition and environmental adjustment

[1095] The device's camera and microphone capture the passenger's facial expressions and voice, and send them to an emotion recognition engine in the cloud. The emotion recognition engine analyzes the captured images to understand the passenger's emotional state. Based on the analysis results, the car's music, air conditioning, and lighting are automatically adjusted.

[1096] 2.2 Image Creation and Modification

[1097] The user inputs image requirements on their device and sends them to the server. The server generates an image using an image generation AI model based on the requirements and returns the generated image to the device. If the user requests any corrections, the details are input again into the image generation AI model to generate a corrected image. The generated image also undergoes copyright checks.

[1098] 2.3 Meeting Support

[1099] During the meeting, crew members record audio using a microphone and send the data to a server. The server's speech recognition AI model converts the audio into text, and the summarization AI model extracts key points and creates a summary. When creating the summary, an emotion recognition engine adjusts the content to reflect the emotional state of the participants.

[1100] 3. Examples and prompts

[1101] 3.1 Specific examples

[1102] For example, it can analyze emotions during a relaxed drive and optimize the background music and air conditioning in the car. It can also record conference audio, generate summaries, and adjust the settings to take into account the emotions of the participants.

[1103] 3.2 Prompt Sentence Examples

[1104] "Create a report featuring the beautiful scenery during the relaxing drive. The passengers were very happy."

[1105] "Generate a summary of today's meeting and shorten the sections that participants found boring."

[1106] This will enable a comfortable environment to be created inside an autonomous vehicle, efficient meeting management, and the provision of high-quality services that take into consideration the emotions of passengers.

[1107] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1108] Step 1:

[1109] The terminal receives image requirements from the user, including theme, style, color tone, resolution, etc. The input image requirements are sent to the server.

[1110] Step 2:

[1111] The server analyzes the received image requirements and instructs the image generation AI model to generate an image based on that information. The image generation AI model (e.g., OpenAI's DALL-E) generates a new image based on the prompt text.

[1112] Step 3:

[1113] The server returns the generated image to the terminal, which displays the returned image to the user. The user checks the image and inputs correction requirements as necessary.

[1114] Step 4:

[1115] The device sends the user's correction requirements to the server, which then instructs the image generation AI model to generate a corrected image again, generating a new image.

[1116] Step 5:

[1117] The server performs a copyright check on the new image generated, using an algorithm (e.g., image recognition AI) that evaluates the similarity and authenticity of the generated image with existing images. The results of the copyright check are recorded.

[1118] Step 6:

[1119] The device's camera and microphone capture the passenger's facial expressions and voice, which are then sent to an emotion recognition engine in the cloud (e.g., Microsoft Azure Cognitive Services), which analyzes the passenger's emotional state and sends the results back to the device.

[1120] Step 7:

[1121] The device receives the analysis results and automatically adjusts the in-car environment (music, air conditioning, lighting, etc.) For example, if the passenger is relaxing, the music will be soft and the temperature will be moderate.

[1122] Step 8:

[1123] When a user starts a conference in a moving autonomous vehicle, the microphone on the device records the conference audio, and the recorded audio data is sent to the server.

[1124] Step 9:

[1125] The server passes the recorded audio data to an automatic transcription AI model (e.g., Google Cloud Speech-to-Text) to convert it into text data, which is then passed to a summarization AI model (e.g., GPT-4) to generate a summary.

[1126] Step 10:

[1127] The emotion recognition engine analyzes the emotional state of participants during the meeting and reflects it in the summary. For example, if it detects that a participant is bored, it will extract only the important points and summarize them concisely.

[1128] Step 11:

[1129] The summarized text data is sent back to the terminal, where the user can review it and use it as a follow-up for the meeting if necessary. This series of processes enables a comfortable environment to be set up inside the autonomous vehicle and for efficient meeting management.

[1130] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1131] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1132] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1133] [Third embodiment]

[1134] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1135] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1136] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1137] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1138] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1139] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1140] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1141] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1142] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1143] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1144] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1145] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1146] This invention relates to a system for generating images for use in proposals and meeting materials, redacting images, checking copyrights, and automatically transcribing and summarizing meeting records. The program processing of the system is explained in natural language below.

[1147] System Overview

[1148] The system allows users to input image requirements through their devices and send them to a server. The server then uses an image generation AI model to generate images, returns them to the user, and makes any necessary corrections before providing the final image. It also includes a copyright check function to avoid the risk of copyright infringement, and the ability to automatically transcribe and summarize audio data during meetings.

[1149] Image Generation Process

[1150] 1. User Roles

[1151] Users use the device to input image requirements for proposals and meeting materials, such as "next-generation smartphone," "futuristic design," "blue color theme," and "high resolution."

[1152] After the image is returned, the user checks it, enters correction requirements as necessary, and resubmits it.

[1153] 2. Role of the terminal

[1154] The device sends the image requirements entered by the user to the server, which sends the request as JSON format data.

[1155] The terminal displays the generated and modified images returned from the server.

[1156] 3. Server Roles

[1157] The server analyzes the image requirements received from the device and instructs the image generation AI model to generate the image based on that information.

[1158] It receives image data generated by an image generation AI model, adds metadata (date and time of generation, conditions of use, etc.), and returns it to the device.

[1159] If correction instructions are given, the correction instructions are sent again to the image generation AI model, and a corrected version is generated.

[1160] It uses an internal copyright checking AI model to assess the copyright risk of the generated images and records the results.

[1161] Copyright Check Process

[1162] The server uses an internal copyright check AI model to evaluate the generated image for similarity and reliability with existing images. The evaluation results are recorded and provided to users only if there are no problems with using the image.

[1163] Meeting recording efficiency process

[1164] 1. Recording audio data

[1165] During a meeting, users use the terminal to record audio of comments and notes.

[1166] 2. Automatic transcription

[1167] The device sends the recorded audio data to a server, which passes it to an automatic transcription AI model to convert it into text.

[1168] 3. Summary Generation

[1169] The server passes the generated text data to a summarization AI model to generate a summary, which is returned to the user and used as a meeting record.

[1170] Specific examples

[1171] Image generation example

[1172] When a user requests an image with the theme of "next-generation smartphones," the device sends the request to the server.

[1173] The server passes the requirements to the image generation AI model and receives the generated high-resolution smartphone image of the futuristic design.

[1174] The server adds metadata to the generated image and returns it to the terminal.

[1175] The user checks the image and inputs correction requirements, such as "I want the gloss to be a little less." The terminal again sends the correction requirements to the server, and the server generates the corrected image.

[1176] Meeting recording example

[1177] During a conference, a user records a voice memo on a terminal, and the terminal transmits the voice data to a server.

[1178] The server automatically transcribes the audio data and summarizes the generated text.

[1179] The summary data is sent back to the terminal, and the user can use it efficiently as a follow-up after the meeting.

[1180] This system streamlines the image creation and editing process for proposals and meeting materials, reduces copyright risks, and enables quick summarization of meeting records.

[1181] The processing flow will be explained below.

[1182] Image Generation Process

[1183] Step 1:

[1184] The user logs in to a dedicated application on the device.

[1185] Users input image requirements for proposals and meeting materials, such as "next-generation smartphone," "futuristic design," "blue color theme," and "high resolution."

[1186] Step 2:

[1187] The device sends the entered image requirements to the server as JSON format data.

[1188] Step 3:

[1189] The server parses the JSON format image requirements received from the device.

[1190] Step 4:

[1191] The server instructs the image generation AI model to generate an image based on the analyzed image requirements.

[1192] Step 5:

[1193] The image generation AI model generates new images based on the specified requirements.

[1194] Step 6:

[1195] The server receives image data generated by the image generation AI model and adds metadata (such as the date and time of generation, conditions of use, etc.) to the generated image.

[1196] Step 7:

[1197] The server transmits the generated image to the terminal.

[1198] Step 8:

[1199] The user checks the image generated on the terminal. If the user determines that corrections are necessary, they input the correction requirements into the terminal. For example, they input correction instructions such as "make the design less glossy."

[1200] Step 9:

[1201] The terminal transmits the correction requirements entered by the user back to the server as JSON format data.

[1202] Step 10:

[1203] Based on the correction requirements received from the terminal, the server again instructs the image generation AI model to generate a corrected image.

[1204] Step 11:

[1205] The image generation AI model regenerates the image based on the correction requirements and sends it back to the server.

[1206] Step 12:

[1207] The server receives the corrected image data and sends it back to the terminal, where the user checks the corrected image and finally approves it.

[1208] Copyright Check Process

[1209] Step 1:

[1210] The server uses an internal copyright checking AI model to evaluate the generated image for similarity and authenticity with existing images.

[1211] Step 2:

[1212] The server records the evaluation results in an internal database, and if there are no problems, provides the image to the user.

[1213] Step 3:

[1214] The server saves the good images in the project folder and makes them accessible to users.

[1215] Meeting recording efficiency process

[1216] Step 1:

[1217] During a meeting, a user uses the terminal to record audio of comments and notes.

[1218] Step 2:

[1219] The device transmits the recorded audio data to the server.

[1220] Step 3:

[1221] The server passes the audio data to an automatic transcription AI model and converts it into text.

[1222] Step 4:

[1223] The server passes the generated text data to a summary AI model to generate summary data.

[1224] Step 5:

[1225] The server sends the generated summary data to the terminal, where the user can check it. The user can use it efficiently as a follow-up after the meeting.

[1226] Example 1

[1227] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1228] In today's world, the creation and editing of images for proposals and meeting materials, copyright verification, and automation of meeting recording are extremely important issues. However, these processes are extremely time-consuming and laborious, so systems that can handle them efficiently and quickly are needed. Furthermore, it is also necessary to check whether the generated images are similar to existing images and whether they infringe copyright. There is also a need for automatic transcription and summary generation of audio data recorded during meetings, but it is currently difficult to efficiently achieve these functions in a single system.

[1229] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1230] In this invention, the server includes a means for receiving image requirements from a user, a means for instructing an image generation AI model to generate an image, and a means for performing copyright checks on the generated images. This allows users to easily input specific image requirements, quickly generate and edit high-quality images, and evaluate copyright risks. Furthermore, by including a function for automatically transcribing audio data during meetings and generating summaries, it also automates meeting recording, significantly improving overall work efficiency.

[1231] The "means for receiving image requirements" is a function for receiving conditions and features related to an image specified by a user.

[1232] An "image generation AI model" is an algorithm or system that uses artificial intelligence to generate images based on specified conditions.

[1233] The "means for instructing image generation" is a function that conveys the received image requirements to the image generation AI model and causes it to generate the image.

[1234] "Means for returning generated images" refers to a function that returns images generated by an image generation AI model to the user's device.

[1235] The "means for receiving correction requirements" is a function for receiving a correction request for a generated image from a user.

[1236] The "means for performing copyright checks" is a function that evaluates the similarity and reliance of generated images against existing images, and checks for the risk of copyright infringement.

[1237] The "means for recording the results of copyright checks" is a function for saving the results of copyright checks in a database or the like.

[1238] "Means for automatically transcribing audio data" refers to a function that converts audio data recorded during a meeting into text data.

[1239] The "means for summarizing and providing transcribed data" is a function that summarizes automatically transcribed text data and provides it to the user.

[1240] The "means for adding and providing metadata" is a function for adding additional information such as the date and time of creation and the conditions of use to the created image and providing it to the user.

[1241] "Means for sending in JSON format" is a function that sends image requirements specified by the user to the server in a data format called JSON (JavaScript Object Notation).

[1242] The present invention relates to a system for generating images for use in proposals and meeting materials, redacting, copyright checking, and automatically transcribing and summarizing meeting records. The system is implemented as follows:

[1243] System Overview

[1244] In this system, users input image requirements through their devices and send them to the server. The server uses an image generation AI model to generate images, returns them to the user, makes any necessary corrections, and provides the final image. The system also has a copyright check function for generated images, and can automatically transcribe audio data from meetings and generate summaries.

[1245] Hardware and software used

[1246] The system uses the following hardware and software:

[1247] Image generation AI models: e.g., DALL-E and Stable Diffusion

[1248] Copyright checking AI model: For example, CopyChecker

[1249] Automatic transcription AI models: e.g., Google Cloud Speech-to-Text

[1250] Summary generation AI model: e.g., GPT-3

[1251] Device: The computer or smartphone used by the user

[1252] Server: A remote server for data processing and running AI models

[1253] Specific operation of the system

[1254] 1. Enter image requirements

[1255] Users can use their devices to input image requirements for proposals and meeting materials, such as "next-generation smartphone," "futuristic design," "blue color theme," and "high resolution." Users simply enter their specific requirements in the text box and click the send button.

[1256] 2. Submit your image requirements

[1257] The device sends the image requirements entered by the user to the server in JSON format. The data is sent via an HTTP POST request. The following JSON format data is used:

[1258] json

[1259] {

[1260] "theme": "Next-generation smartphone",

[1261] "style": "futuristic design",

[1262] "color": "blue",

[1263] "resolution": "High resolution"

[1264] }

[1265] 3. Execute image generation

[1266] The server analyzes the received image requirements and issues instructions to the image generation AI model based on them. For example, a prompt might look like this: "Create a high-resolution image of a futuristic next-generation smartphone with a blue color theme." The server receives the image data generated by the image generation AI model, adds metadata (such as the date and time of generation and conditions of use), and sends it back to the device.

[1267] 4. Check and correct the image

[1268] The user checks the displayed image and inputs correction requirements as necessary. For example, they can enter a command such as "reduce the gloss" in the text box and click the resend button. The device then sends the correction requirements back to the server in JSON format, and the server again sends the correction instructions to the image generation AI model.

[1269] 5. Copyright Check

[1270] The server uses an internal copyright check AI model to evaluate the generated image for similarity and reliability with existing images. The results of the copyright check are recorded and provided to the user only if there are no problems.

[1271] 6. Audio recording and transcription

[1272] During a meeting, users record audio notes on their device, which then sends the audio data to a server, which then converts the audio into text using an automatic transcription AI model. For example, "Transcribe the following meeting audio to text."

[1273] 7. Generating and Serving Abstracts

[1274] The server passes the generated text data to a summarization AI model to generate summary data. This summary is sent back to the user and used as a meeting record. A summary like the following is generated: "At the meeting, the progress and next steps of the new project were discussed."

[1275] This system streamlines the image creation and editing process for proposals and meeting materials, reduces copyright risks, and enables quick summarization of meeting records.

[1276] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1277] Image Generation Process

[1278] Step 1: User inputs image requirements

[1279] Input: Image requirements for user proposals and meeting materials (e.g., "next-generation smartphone," "futuristic design," "blue color theme," "high resolution")

[1280] Specific operation: The user enters the image requirements in the text box on the terminal and clicks the send button.

[1281] Output: Image requirements are entered into the terminal and prepared for sending to the server.

[1282] Step 2: Send image requirements from the device to the server

[1283] Input: User-entered image requirements

[1284] Specific operation: The device converts the image requirements into JSON format and sends it to the server as an HTTP POST request.

[1285] Output: The following JSON data is sent to the server:

[1286] json

[1287] {

[1288] "theme": "Next-generation smartphone",

[1289] "style": "futuristic design",

[1290] "color": "blue",

[1291] "resolution": "High resolution"

[1292] }

[1293] Step 3: Image generation instructions from the server

[1294] Input: Image requirements sent from the terminal (JSON data)

[1295] Specific operation: The server parses the received JSON data and generates an appropriate prompt for the image generation AI model. For example: "Create a high-resolution image of a futuristic next-generation smartphone with a blue color theme."

[1296] Output: A prompt sentence is input into the image generation AI model, and the generated image data is returned to the server.

[1297] Step 4: Returning the image from the server to the device

[1298] Input: Image data generated by an image generation AI model

[1299] Specific operation: The server adds metadata (date and time of creation, conditions of use, etc.) to the generated image and returns it to the terminal.

[1300] Output: Image data with metadata is sent to the device. Example:

[1301] json

[1302] {

[1303] "image": "base64_encoded_image_data",

[1304] "metadata": {

[1305] "generated_at": "2023-10-01T12:00:00Z",

[1306] "usage_conditions": "Commercial use allowed"

[1307] }

[1308] }

[1309] Step 5: User checks the image and gives correction instructions

[1310] Input: Image data returned from the server

[1311] Specific operation: The user checks the image on the device and inputs correction requirements as necessary (e.g., "I would like the glossiness to be reduced"). After inputting the correction requirements, the user clicks the resend button.

[1312] Output: The modified image requirements are entered into the terminal and are ready to be sent to the server again.

[1313] Step 6: Sending modification requirements from the terminal to the server

[1314] Input: User-entered correction requirements

[1315] Specific operation: The terminal converts the modification requirements into JSON format and resubmits them to the server as an HTTP POST request.

[1316] Output: The correction requirements are sent to the server and used again as prompts for the image generation AI model. Example of resubmitted JSON data:

[1317] json

[1318] {

[1319] "theme": "Next-generation smartphone",

[1320] "style": "futuristic design",

[1321] "color": "blue",

[1322] "resolution": "high resolution",

[1323] "modification": "Reduce glossiness"

[1324] }

[1325] Step 7: Copyright Check

[1326] Input: Generated image data

[1327] How it works: The server inputs the generated image into a copyright checking AI model to evaluate its similarity to existing images and its reliability. The evaluation results are recorded and provided to the user only if there are no problems.

[1328] Output: Copyright check results and usable image data based on the evaluation. Example:

[1329] json

[1330] {

[1331] "image": "base64_encoded_image_data",

[1332] "metadata": {

[1333] "generated_at": "2023-10-01T12:00:00Z",

[1334] "usage_conditions": "No copyright issues detected"

[1335] }

[1336] }

[1337] Meeting recording efficiency process

[1338] Step 1: Recording audio data

[1339] Input: Audio during the meeting

[1340] Specific operation: The user clicks the "Start Recording" button on the device to record conversations and notes in audio format.

[1341] Output: The recorded audio data is stored on the device and prepared for transmission to the server.

[1342] Step 2: Sending audio data from the device to the server

[1343] Input: Recorded audio data

[1344] Specific operation: The device converts the voice data into a digital audio file format (e.g., WAV, MP3) and sends it to the server as an HTTP POST request.

[1345] Output: The audio data is sent to the server. Example:

[1346] json

[1347] {

[1348] "audio_file": "base64_encoded_audio_data"

[1349] }

[1350] Step 3: Auto-transcription

[1351] Input: Audio data sent to the server

[1352] Specific operation: The server inputs the audio data into an automatic transcription AI model and converts the audio into text data. The prompt is as follows: "Transcribe the following meeting audio to text."

[1353] Output: A transcript of the audio is generated. Example:

[1354] text

[1355] "In the meeting, we discussed the progress of the new project and the next steps. Specifically, we had a detailed discussion about the design and features of the next-generation smartphone."

[1356] Step 4: Generate summary and return summary data

[1357] Input: Automatically transcribed text data

[1358] Specific operation: The server inputs the generated text data into the summary AI model and generates a summary. The prompt is as follows: "Summarize the following transcription of the meeting."

[1359] Output: Summarized text data is generated and sent back to the user. Example:

[1360] text

[1361] "During the meeting, the progress and next steps of the new project were discussed. In particular, there was detailed discussion on the design and functionality of the next generation smartphone."

[1362] Through these specific processing steps, the system streamlines the creation and editing of images required for proposals and meeting materials, reduces copyright risks, and enables quick summarization of meeting records.

[1363] (Application example 1)

[1364] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1365] In autonomous vehicles, there are problems with the efficient generation, editing, and copyright check of video surveillance and meeting records during operation, as well as the automatic transcription and summarization of audio data. In particular, the collection of surveillance data and the creation of driving reports within autonomous vehicles requires a great deal of time and effort using current manual processes, making effective operation difficult. In addition, copyright issues may arise with the generated data, so evaluation and management of these issues are required.

[1366] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1367] In this invention, the server includes means for receiving image requirements from a user, means for instructing an image generation AI model to generate an image based on the image requirements, means for returning the generated image to the user's terminal, means for receiving correction requirements from the user and again instructing the image generation AI model to generate a corrected image, means for performing a copyright check on the generated image, means for recording the results of the copyright check, means for automatically transcribing audio data recorded during a meeting, means for summarizing the transcribed data and providing it to the user, means for acquiring image surveillance data inside and outside the autonomous vehicle, means for transferring the recorded audio data, and means for automatically processing and summarizing audio memos. This enables efficient operation management of autonomous vehicles, rapid data generation and correction, reduced copyright risks, and rapid summarization of meeting records.

[1368] "User" refers to an entity that uses the system to generate images, specify correction requirements, and record audio data.

[1369] "Image generation AI model" refers to an algorithm or model that generates images based on specified image requirements.

[1370] "Image requirements" refer to the specific style, content, color tone, resolution, and other conditions of the image desired by the user.

[1371] "Modification requirements" refer to instructions for modifying or changing the generated image.

[1372] "Copyright check" refers to the process of assessing whether the generated image is similar to an existing image or infringes copyright.

[1373] "Audio data" refers to audio information recorded during a conference.

[1374] "Automatic transcription" refers to the process of automatically converting audio data into text data.

[1375] "Summarization" refers to the process of concisely summarizing transcribed text data.

[1376] An "autonomous vehicle" is a vehicle that is capable of operating independently without a human driver.

[1377] "Image surveillance data" refers to image information collected inside and outside an autonomous vehicle.

[1378] "Terminal" refers to a device used by a user, such as a computer or smartphone.

[1379] "Metadata" refers to data that includes information related to a generated image, such as the date and time of generation and the conditions of use.

[1380] This invention relates to a system for efficiently realizing image monitoring, copyright checking, and automatic transcription and summarization of meeting records in an autonomous vehicle. This system starts when a user inputs image requirements using a terminal and sends them to a server. Specific embodiments of this system are described below.

[1381] 1. System Overview

[1382] The system mainly uses the user's device, a server, and cameras and audio recording devices inside and outside the autonomous vehicle. The server utilizes an image generation AI model, a copyright check AI model, an automatic transcription AI model, and a summary AI model for efficient data processing.

[1383] 2. Hardware and Software

[1384] Hardware:

[1385] User's device (e.g. smartphone, tablet)

[1386] Cameras inside and outside the autonomous vehicle

[1387] Audio recording device

[1388] software:

[1389] Python

[1390] Flask (web server framework)

[1391] Image generation AI model (e.g., HuggingFace's DALL·E)

[1392] Copyright Check AI Model

[1393] Automatic transcription API (e.g., Google Speech-to-Text)

[1394] Summary AI models (e.g., HuggingFace's BART)

[1395] 3. Data processing and calculation

[1396] The server receives the request sent by the user from the terminal and processes the data through the following process.

[1397] Image generation process:

[1398] The user inputs image requirements from their device and sends them to the server, such as "futuristic design of an autonomous vehicle, blue color theme, high resolution."

[1399] The server uses an image generation AI model to generate images based on the specified requirements.

[1400] Metadata (date and time of creation, conditions of use, etc.) is added to the generated image, and the result is sent back to the user's device.

[1401] The user inputs the correction requirements for the image and sends it back to the server, which then generates the corrected image and sends it back to the user.

[1402] Copyright Check Process:

[1403] The server passes the generated images to a copyright checking AI model to evaluate their similarity to existing images.

[1404] The evaluation results are recorded and notified to the user. If there are no problems, the image becomes available for use.

[1405] Meeting recording efficiency process:

[1406] During the meeting, users record voice memos on their terminals and send them to the server.

[1407] The server transcribes the audio data using the Google Speech-to-Text API.

[1408] The transcribed text is passed to a summarization AI model to generate summary data.

[1409] The summary data is returned to the user's terminal and is effectively used as a record of the meeting.

[1410] 4. Specific Examples

[1411] Image generation example:

[1412] The user inputs image requirements such as "futuristic design, self-driving vehicle, blue color theme, high resolution."

[1413] The server calls an image generation AI model to generate a high-resolution image, adds metadata, and sends it back to the user's device.

[1414] The user inputs the correction requirement "reduce the glossiness a little" and sends it to the server again. The server generates a corrected image and provides it to the user.

[1415] Example of a recording:

[1416] A user records a voice memo during a conference, and the terminal transmits the voice data to a server.

[1417] The server automatically transcribes the audio data and passes the transcribed text to a summarization AI model to generate a summary.

[1418] The summary data is returned to the user's terminal, and the user can efficiently use the conference record.

[1419] This system not only streamlines the operation and management of autonomous vehicles, but also enables rapid data generation and correction, reduces copyright risks, and improves productivity by quickly summarizing meeting notes.

[1420] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1421] Step 1:

[1422] A user uses a terminal to input image requirements and create a request for image generation, such as "futuristic design of autonomous vehicle, blue color theme, high resolution."

[1423] Input: Image requirements (e.g. futuristic design, blue color theme, high resolution)

[1424] Output: Image generation request

[1425] Step 2:

[1426] The device converts the user's image requirements into JSON format and sends it to the server. The device properly formats the input requirements and prepares the request for sending to the server.

[1427] Input: Image generation request

[1428] Output: Request in JSON format

[1429] Step 3:

[1430] The server analyzes the received JSON-formatted request and instructs the image generation AI model to generate an image. The server analyzes the request, extracts the parameters necessary for image generation, and passes them to the AI ​​model.

[1431] Input: JSON formatted request

[1432] Output: Instructions to the AI ​​model

[1433] Step 4:

[1434] The image generation AI model generates images based on instructions. The model uses prompt text to generate high-resolution images according to the specified style and color tone.

[1435] Input: Instructions to the AI ​​model (e.g., futuristic design, self-driving vehicle, blue, resolution)

[1436] Output: The generated image

[1437] Step 5:

[1438] The server adds metadata (such as the date and time of creation and the conditions of use) to the created image and returns it to the user's device. The server records the date and time of creation and the conditions of use of the image and attaches them to the image data.

[1439] Input: Generated image

[1440] Output: Generated image with metadata

[1441] Step 6:

[1442] The user checks the generated image on the terminal and inputs the correction requirements. The user specifies the improvements and changes to the image and creates a new request.

[1443] Input: Generated image with metadata

[1444] Output: Fix requirements

[1445] Step 7:

[1446] The terminal converts the modification requirements into JSON format and sends it to the server. The terminal then sends a request based on the modification requirements to the server again.

[1447] Input: Correction Requirement

[1448] Output: The modification request in JSON format

[1449] Step 8:

[1450] The server analyzes the modification request and instructs the image generation AI model to generate the modified image again. The server instructs the AI ​​model to regenerate the image using the new modification parameters.

[1451] Input: JSON formatted modification request

[1452] Output: Corrective instructions for the AI ​​model

[1453] Step 9:

[1454] The image generation AI model generates a corrected image based on the correction instructions. It uses the prompt text again to generate an image that reflects the necessary corrections.

[1455] Input: Correction instructions for the AI ​​model

[1456] Output: Corrected image

[1457] Step 10:

[1458] The server returns the corrected image to the user's device. The corrected image data is sent to the user's device in JSON format.

[1459] Input: Modified image

[1460] Output: The corrected image sent back to the user

[1461] The above are the specific processing steps for image generation and correction. Next, we will show the processing steps for automatic transcription and summarization of meeting recordings.

[1462] Step 11:

[1463] During a meeting, the user records a voice memo, and the device sends the voice data to the server. The user then presses the record button to record the required voice.

[1464] Input: Voice memo

[1465] Output: Audio data

[1466] Step 12:

[1467] The server passes the audio data to an automated transcription method to convert it into text. The server uses the Google Speech-to-Text API to transcribe the audio data.

[1468] Input: Audio data

[1469] Output: Transcribed text

[1470] Step 13:

[1471] The server passes the generated text to a summarization model to generate summary data. The server then invokes a summarization AI model (e.g., BART) to generate a summary that extracts the key points.

[1472] Input: Transcribed text

[1473] Output: Summary data

[1474] Step 14:

[1475] The server returns the summary data to the user's terminal, and the user efficiently checks the meeting record. The points of the summarized meeting are provided to the user.

[1476] Input: Summary data

[1477] Output: Summary data returned to the user

[1478] These processing steps enable efficient management of image generation, editing, copyright checks, and meeting records in autonomous vehicles.

[1479] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1480] This invention combines a system for generating images for use in proposals and meeting materials, correcting them, checking copyrights, and automatically transcribing and summarizing meeting records with an emotion engine that recognizes the user's emotions. This system provides more human-centered and effective support by suggesting image requirements, making corrections, and adjusting the summaries of meeting records according to the user's emotional state.

[1481] System Overview

[1482] In this system, users input image requirements through their devices and send them to a server. The server then uses an image generation AI model to generate an image, which is returned to the user, where it is corrected as necessary and the final image is provided. The system also includes a copyright check function to avoid the risk of copyright infringement, and the ability to automatically transcribe and summarize audio data during meetings. A new emotion engine has also been added, enabling suggestions and corrections based on the user's emotions.

[1483] Image generation process and emotion engine integration

[1484] 1. User Roles

[1485] Users log in to a dedicated application on their device, and when entering image requirements for proposals or meeting materials, the system recognizes emotions from the user's facial expressions and voice.

[1486] The emotion engine analyzes the user's emotional state when they input and suggests appropriate image requirements, such as "if the user is excited, we suggest a bold design."

[1487] 2. Role of the terminal

[1488] The terminal transmits the input image requirements and additional requirements provided by the emotion engine to the server, thereby including requirements based on the user's emotions.

[1489] The terminal displays the generated or modified image returned from the server.

[1490] 3. Server Roles

[1491] The server analyzes the image requirements received from the device and instructs the image generation AI model to generate the image.

[1492] The image generation AI model generates new images based on the instruction requirements.

[1493] The server adds metadata to the generated image and returns it to the terminal.

[1494] When responding to correction instructions, the emotion engine evaluates the user's emotional response and automatically recommends correction requirements.

[1495] Copyright Check Process

[1496] The server uses its internal copyright checking AI model to evaluate the generated image for similarity and reliability with existing images. The evaluation results are recorded and provided to users only if there are no problems with using the image.

[1497] Meeting recording efficiency process

[1498] 1. Recording audio data

[1499] During a meeting, users use their devices to record audio of comments and notes, and their emotional state during the recording is also analyzed.

[1500] 2. Automatic transcription

[1501] The device sends the recorded audio data to a server, which then passes the data to an automatic transcription AI model and converts it into text.

[1502] 3. Summary Generation

[1503] The server passes the transcribed text data to a summary AI model to generate a summary. An emotion engine then applies the summary to adjust the content according to the user's emotional state.

[1504] Specific examples

[1505] Image generation example

[1506] When a user requests an image themed on "next-generation smartphones" and inputs their requirements into the device, the emotion engine analyzes the user's emotional state. For example, if a user is excited, it will recommend a "futuristic design."

[1507] The device sends the proposed requirements and the emotion engine's suggestions to the server. The server receives the image generated by the image generation AI model and returns it to the device. The user checks the image, and the emotion engine automatically suggests correction requirements.

[1508] Meeting recording example

[1509] During a meeting, when a user records a voice memo on the terminal, the emotion engine analyzes the user's emotional state.

[1510] The device sends the recorded voice data to a server, which then automatically transcribes it. At the same time, an emotion engine adjusts the summary content to match the user's emotions based on the generated text data.

[1511] The summary data is sent back to the terminal, and the user can use it efficiently as a follow-up after the meeting.

[1512] This system streamlines the image creation and editing process for proposals and meeting materials, improves user response based on user sentiment, reduces copyright risks, and enables quick summarization of meeting records.

[1513] The processing flow will be explained below.

[1514] Image generation process incorporating an emotion engine

[1515] User-defined image requirements and generation

[1516] Step 1:

[1517] The user logs in to a dedicated application on the device.

[1518] Step 2:

[1519] The user inputs the image requirements for proposals and meeting materials into the device, such as "next-generation smartphone," "futuristic design," "blue color theme," and "high resolution."

[1520] Step 3:

[1521] While the user is typing, the device uses a built-in camera and microphone to analyze the user's facial expressions and voice, and an emotion engine evaluates the user's emotional state.

[1522] Step 4:

[1523] The emotion engine makes additional suggestions for image requirements based on the user's emotional state, for example, suggesting a "daring design" if the user is excited.

[1524] Step 5:

[1525] The device sends image requirements and emotion engine suggestions to the server as JSON format data.

[1526] Step 6:

[1527] The server analyzes the JSON format image requirements received from the device and instructs the image generation AI model to generate the image.

[1528] Step 7:

[1529] The image generation AI model generates new images based on the specified requirements.

[1530] Step 8:

[1531] The server receives image data generated by the image generation AI model and adds metadata (such as the date and time of generation, conditions of use, etc.) to the generated image.

[1532] Step 9:

[1533] The server transmits the generated image to the terminal.

[1534] Step 10:

[1535] The user checks the image generated on the device. If the user determines that corrections are necessary, the user inputs the correction requirements into the device (for example, "make the design less glossy").

[1536] Step 11:

[1537] The emotion engine evaluates the user's emotional response when viewing an image and automatically recommends revision requirements.

[1538] Step 12:

[1539] The device sends the user's correction requirements and the emotion engine's suggestions back to the server as JSON format data.

[1540] Step 13:

[1541] Based on the correction requirements received from the terminal, the server again instructs the image generation AI model to generate a corrected image.

[1542] Step 14:

[1543] The image generation AI model regenerates the image based on the correction requirements and sends it back to the server.

[1544] Step 15:

[1545] The server receives the corrected image data and sends it back to the terminal, where the user checks the corrected image and finally approves it.

[1546] Copyright check process incorporating emotion engine

[1547] Step 1:

[1548] The server uses its internal copyright checking AI model to evaluate the generated image for similarity and reliability with existing images.

[1549] Step 2:

[1550] The server records the evaluation results in an internal database, and if there are no problems, the image is provided to the user.

[1551] Step 3:

[1552] The server saves the good images in the project folder and makes them accessible to users.

[1553] Emotion-driven meeting recording process

[1554] Audio data recording and processing

[1555] Step 1:

[1556] During a meeting, a user uses the terminal to record audio of comments and notes.

[1557] Step 2:

[1558] The device analyzes the tone and pace of the user's voice while recording, and an emotion engine evaluates the user's emotional state.

[1559] Step 3:

[1560] The device sends the recorded audio data to the server.

[1561] Step 4:

[1562] The server passes the audio data to an automatic transcription AI model and converts it into text.

[1563] Step 5:

[1564] The server passes the generated text data to a summary AI model to generate summary data.

[1565] Step 6:

[1566] The emotion engine uses the summary data to tailor the content to the user's emotional state, for example, emphasizing positive summary results if the user is feeling down.

[1567] Step 7:

[1568] The server sends the generated summary data to the terminal, where the user can check it. The user can use it efficiently as a follow-up after the meeting.

[1569] These steps provide a system incorporating an emotion engine that not only streamlines the process of creating proposals and meeting materials, but also allows for suggestions and revisions based on the user's emotions, reduces copyright risks, and enables quick summarization of meeting records.

[1570] Example 2

[1571] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1572] In today's busy business environment, creating proposals and meeting materials requires a lot of time and effort. The tasks of generating and editing high-quality images that reflect the user's intent and emotions, as well as efficiently transcribing and summarizing meeting recordings, are particularly burdensome. The risk of copyright infringement for generated images is also a significant issue. It is necessary to develop a system that streamlines these processes and improves the user experience.

[1573] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1574] In this invention, the server includes means for proposing appropriate image requirements in cooperation with an emotion engine that recognizes and analyzes emotions from a user's facial expressions and voice, means for instructing an image generation AI model to generate an image, means for performing copyright checks on the generated image, means for automatically transcribing audio data recorded during a meeting, and means for summarizing the transcribed data, adjusting the content based on the user's emotional state, and providing the summarization to the user. This streamlines the image creation and editing process for proposals and meeting materials, improves response based on user emotions, reduces copyright risks, and enables quick summarization of meeting records.

[1575] A "user" is a person who operates the system and inputs image and modification requirements.

[1576] "Terminal" refers to an electronic device such as a computer or smartphone used by a user.

[1577] "Image requirements" refer to specific conditions and specifications regarding the image that a user wants to create.

[1578] The "emotion engine" is an AI model that recognizes emotions from a user's facial expressions and voice and analyzes their emotional state.

[1579] An "image generation AI model" is an artificial intelligence model for automatically generating images based on input requirements.

[1580] "Copyright check" is the process of checking whether the generated image is similar to or relies on an existing image and assessing legal risk.

[1581] The "correction requirements" refer to the corrections that the user wants to make to the generated image.

[1582] "Automatic transcription" is the automated process of converting audio data into text.

[1583] A "summary" is a text that briefly summarizes the transcribed data.

[1584] "Metadata" is auxiliary information added to a generated image, and is data that indicates details and attributes of the image.

[1585] This invention combines a system for generating images for use in proposals and meeting materials, correcting them, checking copyrights, and automatically transcribing and summarizing meeting records with an emotion engine that recognizes the user's emotions. This system provides more human-centered and effective support by suggesting image requirements, making corrections, and adjusting the summaries of meeting records according to the user's emotional state.

[1586] System Overview

[1587] In this system, users input image requirements through their devices and send them to a server. The server then uses an image generation AI model to generate an image, which is returned to the user, where it is corrected as necessary and the final image is provided. The system also includes a copyright check function to avoid the risk of copyright infringement, and the ability to automatically transcribe and summarize audio data during meetings. A new emotion engine has also been added, enabling suggestions and corrections based on the user's emotions.

[1588] Specific examples

[1589] Image generation example

[1590] When a user requests an image themed around "next-generation smartphones" and inputs the requirements into the device, the emotion engine analyzes the user's emotional state. For example, if the user is in an excited state, it will recommend a "futuristic design." The device then sends the proposed requirements and the emotion engine's suggestions to the server. The server receives the image generated by the image generation AI model and returns it to the device. The user then reviews the image, and the emotion engine automatically suggests further revisions.

[1591] Example prompt for a generative AI model:

[1592] "Generate images with futuristic and exciting designs based on the theme of next-generation smartphones."

[1593] Meeting recording example

[1594] During a meeting, when a user records a voice memo on their device, the emotion engine analyzes the user's emotional state. The device then sends the recorded voice data to a server, which then automatically transcribes it. At the same time, the emotion engine adjusts the summary content to match the user's emotions based on the generated text data. The summary data is then sent back to the device, allowing the user to efficiently use it as a follow-up after the meeting.

[1595] The system streamlines the image creation and editing process for proposals and meeting materials, enables user-sensitive responses, mitigates copyright risks, and enables quick summarization of meeting records.

[1596] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1597] Step 1:

[1598] The user logs in to a dedicated application on their device. The user inputs the image requirements needed for proposals and meeting materials, and the emotion engine recognizes and analyzes the user's emotions from their facial expressions and voice. For example, when a user requests a "futuristic design for a next-generation smartphone," the emotion engine proposes requirements that reflect the user's excited state of mind. The input data includes image requirements, and the output is the emotion analysis results and proposed image requirements.

[1599] Step 2:

[1600] The device sends the image requirements entered by the user and the suggestions made by the emotion engine to the server. The input data here are the user's image requirements and the results of emotion analysis, and the output data is a request message containing these. Specifically, the device sends a request to "generate a futuristic design for the next generation smartphone" along with the user's emotion data to the server.

[1601] Step 3:

[1602] The server analyzes the image requirements received from the device and instructs the image generation AI model to generate the image. The input data includes image requirements and emotion data, and the output is the generated image. Specifically, the server sends a "futuristic design for next-generation smartphones" as a prompt to the image generation AI model, and after receiving the generated image, it adds metadata.

[1603] Step 4:

[1604] The server returns the generated image to the terminal. The input data here is the generated image and metadata, and the output data is the image returned to the user's terminal. Specifically, the server returns the generated image to the terminal together with the metadata.

[1605] Step 5:

[1606] The device displays the generated image returned from the server to the user. The user checks the image and inputs correction requirements, if necessary. The emotion engine again analyzes the user's emotional state and makes suggestions for correction. The input data includes the generated image and the user's correction requirements, and the output data is the correction suggestions. Specifically, the user requests "more colorful," and the emotion engine takes this into consideration and suggests an even bolder color palette.

[1607] Step 6:

[1608] The device sends the user's correction requirements to the server, and the server again instructs the image generation AI model to generate a corrected image. The input data includes the correction requirements and emotion data, and the output data is the corrected image. Specifically, the device sends the correction requirement "change to a colorful design" to the server, and the server sends it again to the image generation AI model to generate the corrected image.

[1609] Step 7:

[1610] The server performs a copyright check on the generated image. The input data is the generated image, and the output data is the result of the copyright check. Specifically, the server uses a copyright check AI model to evaluate the similarity between the generated image and existing images, and records the results.

[1611] Step 8:

[1612] During a meeting, the user uses a device to record voice comments and notes. The emotion engine simultaneously analyzes the user's emotional state during the recording. The input data is the recorded voice and emotional data, and the output data is the analysis results. Specifically, the user says, "Regarding the next action plan," which is recorded as a voice memo, and the emotion engine analyzes the user's state of tension.

[1613] Step 9:

[1614] The device sends the recorded voice data to a server, which then passes the data to an automatic transcription AI model and converts it into text. The input data is voice data, and the output data is the transcribed text. Specifically, the device sends voice data saying "What is the next action plan?" to the server, and the server converts the voice data into text.

[1615] Step 10:

[1616] The server passes the transcribed text data to a summarization AI model, which generates summary data. The emotion engine then adjusts the content based on the generated summary according to the user's emotional state. The input data is the transcribed text and emotion data, and the output data is the adjusted summary. Specifically, the summarization AI model generates a short summary such as "The next steps are A, B, and C," and the emotion engine adjusts it to use softer wording.

[1617] Step 11:

[1618] The terminal receives the summary data and displays it to the user. The input data is the adjusted summary data, and the output data is data for the user to confirm. Specifically, the adjusted summary data is displayed on the terminal, and the user can confirm the summary, "The next steps are A, B, and C."

[1619] (Application example 2)

[1620] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1621] Current image generation systems and conference support systems are unable to consider the emotional state of the user, and in autonomous vehicles, there is a lack of services that respond to the emotional state of the occupants. Furthermore, the summary generation during a conference does not adjust to reflect the emotions of participants, making improving the user experience a challenge. In particular, there is a need for a comfortable environment and efficient conference management in autonomous vehicles.

[1622] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving image requirements from a user, means for instructing an image generation AI model to generate an image based on the image requirements, means for returning the generated image to the user's terminal, means for receiving correction requirements from the user and again instructing the image generation AI model to generate a corrected image, means for performing a copyright check on the generated image, means for recording the results of the copyright check, means for automatically transcribing audio data recorded during a conference, means for summarizing the transcribed data and providing it to the user, means for analyzing the emotional states of occupants in an autonomous vehicle in real time, means for automatically adjusting the in-vehicle environment (music, air conditioning, lighting, etc.) based on the analysis, means for collecting driving data of the autonomous vehicle and generating a visual report, and means for automatically adjusting the summary content of the transcribed data based on the emotional states of participants during the conference. This enables a comfortable environment to be set up in an autonomous vehicle, enabling the provision of services that take occupants' emotions into consideration and efficient conference management.

[1623] "Image requirements" are requirements specified by the user regarding the image to be generated, and include theme, style, color tone, resolution, and the like.

[1624] An "image generation AI model" is an artificial intelligence model that generates new images based on input image requirements.

[1625] The "correction requirements" are the correction requirements that the user must meet for the generated image.

[1626] "Copyright check" refers to the process of inspecting whether the generated image is similar to an existing image and whether it infringes copyright.

[1627] "Audio data recorded during a conference" refers to audio recorded while the conference is in progress.

[1628] "Automatic transcription" is the process of converting recorded audio data into text data.

[1629] "Summarization" is the process of simplifying transcribed data and extracting and organizing the main points.

[1630] "Emotional state" refers to the psychological state of the occupant as perceived from facial expressions, voice, etc.

[1631] "In-car environment" is a general term for music, air conditioning, lighting, etc. inside an autonomous vehicle.

[1632] "Driving data" refers to various data such as location information and speed associated with the movement of an autonomous vehicle.

[1633] A "visual report" is a report that is generated based on collected data in a visually easy-to-understand format.

[1634] This invention is a system that integrates image generation, emotion recognition, and conferencing support to improve the user experience in an autonomous vehicle. This system is composed of multiple hardware and software components that operate in the autonomous vehicle.

[1635] 1. System Configuration

[1636] The system consists of the following main hardware and software:

[1637] 1.1 Hardware

[1638] Devices: tablets, smartphones, in-car displays

[1639] Emotion recognition camera: A camera for capturing passenger facial expressions

[1640] Microphone: A microphone for capturing passenger voices and conference audio

[1641] Vehicle sensors: Sensors for collecting driving data from autonomous vehicles, such as GPS and speed sensors

[1642] 1.2 Software

[1643] Emotion recognition engine: Software that analyzes the emotional state of passengers (e.g., Microsoft Azure Cognitive Services, Google Cloud Vision AI)

[1644] Image generation AI model: An artificial intelligence model for generating images (e.g., OpenAI's DALL-E)

[1645] Speech recognition AI model: Software for transcribing meeting audio (e.g., Google Cloud Speech-to-Text)

[1646] Summarization AI model: Software for summarizing transcribed text (e.g., GPT-4)

[1647] 2. Program Processing

[1648] 2.1 Emotion recognition and environmental adjustment

[1649] The device's camera and microphone capture the passenger's facial expressions and voice, and send them to an emotion recognition engine in the cloud. The emotion recognition engine analyzes the captured images to understand the passenger's emotional state. Based on the analysis results, the car's music, air conditioning, and lighting are automatically adjusted.

[1650] 2.2 Image Creation and Modification

[1651] The user inputs image requirements on their device and sends them to the server. The server generates an image using an image generation AI model based on the requirements and returns the generated image to the device. If the user requests any corrections, the details are input again into the image generation AI model to generate a corrected image. The generated image also undergoes copyright checks.

[1652] 2.3 Meeting Support

[1653] During the meeting, crew members record audio using a microphone and send the data to a server. The server's speech recognition AI model converts the audio into text, and the summarization AI model extracts key points and creates a summary. When creating the summary, an emotion recognition engine adjusts the content to reflect the emotional state of the participants.

[1654] 3. Examples and prompts

[1655] 3.1 Specific examples

[1656] For example, it can analyze emotions during a relaxed drive and optimize the background music and air conditioning in the car. It can also record conference audio, generate summaries, and adjust the settings to take into account the emotions of the participants.

[1657] 3.2 Prompt Sentence Examples

[1658] "Create a report featuring the beautiful scenery during the relaxing drive. The passengers were very happy."

[1659] "Generate a summary of today's meeting and shorten the sections that participants found boring."

[1660] This will enable a comfortable environment to be created inside an autonomous vehicle, efficient meeting management, and the provision of high-quality services that take into consideration the emotions of passengers.

[1661] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1662] Step 1:

[1663] The terminal receives image requirements from the user, including theme, style, color tone, resolution, etc. The input image requirements are sent to the server.

[1664] Step 2:

[1665] The server analyzes the received image requirements and instructs the image generation AI model to generate an image based on that information. The image generation AI model (e.g., OpenAI's DALL-E) generates a new image based on the prompt text.

[1666] Step 3:

[1667] The server returns the generated image to the terminal, which displays the returned image to the user. The user checks the image and inputs correction requirements as necessary.

[1668] Step 4:

[1669] The device sends the user's correction requirements to the server, which then instructs the image generation AI model to generate a corrected image again, generating a new image.

[1670] Step 5:

[1671] The server performs a copyright check on the new image generated, using an algorithm (e.g., image recognition AI) that evaluates the similarity and authenticity of the generated image with existing images. The results of the copyright check are recorded.

[1672] Step 6:

[1673] The device's camera and microphone capture the passenger's facial expressions and voice, which are then sent to an emotion recognition engine in the cloud (e.g., Microsoft Azure Cognitive Services), which analyzes the passenger's emotional state and sends the results back to the device.

[1674] Step 7:

[1675] The device receives the analysis results and automatically adjusts the in-car environment (music, air conditioning, lighting, etc.) For example, if the passenger is relaxing, the music will be soft and the temperature will be moderate.

[1676] Step 8:

[1677] When a user starts a conference in a moving autonomous vehicle, the microphone on the device records the conference audio, and the recorded audio data is sent to the server.

[1678] Step 9:

[1679] The server passes the recorded audio data to an automatic transcription AI model (e.g., Google Cloud Speech-to-Text) to convert it into text data, which is then passed to a summarization AI model (e.g., GPT-4) to generate a summary.

[1680] Step 10:

[1681] The emotion recognition engine analyzes the emotional state of participants during the meeting and reflects it in the summary. For example, if it detects that a participant is bored, it will extract only the important points and summarize them concisely.

[1682] Step 11:

[1683] The summarized text data is sent back to the terminal, where the user can review it and use it as a follow-up for the meeting if necessary. This series of processes enables a comfortable environment to be set up inside the autonomous vehicle and for efficient meeting management.

[1684] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1685] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1686] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1687] [Fourth embodiment]

[1688] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1689] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1690] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1691] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1692] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1693] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1694] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1695] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1696] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1697] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1698] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1699] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1700] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1701] This invention relates to a system for generating images for use in proposals and meeting materials, redacting images, checking copyrights, and automatically transcribing and summarizing meeting records. The program processing of the system is explained in natural language below.

[1702] System Overview

[1703] The system allows users to input image requirements through their devices and send them to a server. The server then uses an image generation AI model to generate images, returns them to the user, and makes any necessary corrections before providing the final image. It also includes a copyright check function to avoid the risk of copyright infringement, and the ability to automatically transcribe and summarize audio data during meetings.

[1704] Image Generation Process

[1705] 1. User Roles

[1706] Users use the device to input image requirements for proposals and meeting materials, such as "next-generation smartphone," "futuristic design," "blue color theme," and "high resolution."

[1707] After the image is returned, the user checks it, enters correction requirements as necessary, and resubmits it.

[1708] 2. Role of the terminal

[1709] The device sends the image requirements entered by the user to the server, which sends the request as JSON format data.

[1710] The terminal displays the generated and modified images returned from the server.

[1711] 3. Server Roles

[1712] The server analyzes the image requirements received from the device and instructs the image generation AI model to generate the image based on that information.

[1713] It receives image data generated by an image generation AI model, adds metadata (date and time of generation, conditions of use, etc.), and returns it to the device.

[1714] If correction instructions are given, the correction instructions are sent again to the image generation AI model, and a corrected version is generated.

[1715] It uses an internal copyright checking AI model to assess the copyright risk of the generated images and records the results.

[1716] Copyright Check Process

[1717] The server uses an internal copyright check AI model to evaluate the generated image for similarity and reliability with existing images. The evaluation results are recorded and provided to users only if there are no problems with using the image.

[1718] Meeting recording efficiency process

[1719] 1. Recording audio data

[1720] During a meeting, users use the terminal to record audio of comments and notes.

[1721] 2. Automatic transcription

[1722] The device sends the recorded audio data to a server, which passes it to an automatic transcription AI model to convert it into text.

[1723] 3. Summary Generation

[1724] The server passes the generated text data to a summarization AI model to generate a summary, which is returned to the user and used as a meeting record.

[1725] Specific examples

[1726] Image generation example

[1727] When a user requests an image with the theme of "next-generation smartphones," the device sends the request to the server.

[1728] The server passes the requirements to the image generation AI model and receives the generated high-resolution smartphone image of the futuristic design.

[1729] The server adds metadata to the generated image and returns it to the terminal.

[1730] The user checks the image and inputs correction requirements, such as "I want the gloss to be a little less." The terminal again sends the correction requirements to the server, and the server generates the corrected image.

[1731] Meeting recording example

[1732] During a conference, a user records a voice memo on a terminal, and the terminal transmits the voice data to a server.

[1733] The server automatically transcribes the audio data and summarizes the generated text.

[1734] The summary data is sent back to the terminal, and the user can use it efficiently as a follow-up after the meeting.

[1735] This system streamlines the image creation and editing process for proposals and meeting materials, reduces copyright risks, and enables quick summarization of meeting records.

[1736] The processing flow will be explained below.

[1737] Image Generation Process

[1738] Step 1:

[1739] The user logs in to a dedicated application on the device.

[1740] Users input image requirements for proposals and meeting materials, such as "next-generation smartphone," "futuristic design," "blue color theme," and "high resolution."

[1741] Step 2:

[1742] The device sends the entered image requirements to the server as JSON format data.

[1743] Step 3:

[1744] The server parses the JSON format image requirements received from the device.

[1745] Step 4:

[1746] The server instructs the image generation AI model to generate an image based on the analyzed image requirements.

[1747] Step 5:

[1748] The image generation AI model generates new images based on the specified requirements.

[1749] Step 6:

[1750] The server receives image data generated by the image generation AI model and adds metadata (such as the date and time of generation, conditions of use, etc.) to the generated image.

[1751] Step 7:

[1752] The server transmits the generated image to the terminal.

[1753] Step 8:

[1754] The user checks the image generated on the terminal. If the user determines that corrections are necessary, they input the correction requirements into the terminal. For example, they input correction instructions such as "make the design less glossy."

[1755] Step 9:

[1756] The terminal transmits the correction requirements entered by the user back to the server as JSON format data.

[1757] Step 10:

[1758] Based on the correction requirements received from the terminal, the server again instructs the image generation AI model to generate a corrected image.

[1759] Step 11:

[1760] The image generation AI model regenerates the image based on the correction requirements and sends it back to the server.

[1761] Step 12:

[1762] The server receives the corrected image data and sends it back to the terminal, where the user checks the corrected image and finally approves it.

[1763] Copyright Check Process

[1764] Step 1:

[1765] The server uses an internal copyright checking AI model to evaluate the generated image for similarity and authenticity with existing images.

[1766] Step 2:

[1767] The server records the evaluation results in an internal database, and if there are no problems, provides the image to the user.

[1768] Step 3:

[1769] The server saves the good images in the project folder and makes them accessible to users.

[1770] Meeting recording efficiency process

[1771] Step 1:

[1772] During a meeting, a user uses the terminal to record audio of comments and notes.

[1773] Step 2:

[1774] The device transmits the recorded audio data to the server.

[1775] Step 3:

[1776] The server passes the audio data to an automatic transcription AI model and converts it into text.

[1777] Step 4:

[1778] The server passes the generated text data to a summary AI model to generate summary data.

[1779] Step 5:

[1780] The server sends the generated summary data to the terminal, where the user can check it. The user can use it efficiently as a follow-up after the meeting.

[1781] Example 1

[1782] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1783] In today's world, the creation and editing of images for proposals and meeting materials, copyright verification, and automation of meeting recording are extremely important issues. However, these processes are extremely time-consuming and laborious, so systems that can handle them efficiently and quickly are needed. Furthermore, it is also necessary to check whether the generated images are similar to existing images and whether they infringe copyright. There is also a need for automatic transcription and summary generation of audio data recorded during meetings, but it is currently difficult to efficiently achieve these functions in a single system.

[1784] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1785] In this invention, the server includes a means for receiving image requirements from a user, a means for instructing an image generation AI model to generate an image, and a means for performing copyright checks on the generated images. This allows users to easily input specific image requirements, quickly generate and edit high-quality images, and evaluate copyright risks. Furthermore, by including a function for automatically transcribing audio data during meetings and generating summaries, it also automates meeting recording, significantly improving overall work efficiency.

[1786] The "means for receiving image requirements" is a function for receiving conditions and features related to an image specified by a user.

[1787] An "image generation AI model" is an algorithm or system that uses artificial intelligence to generate images based on specified conditions.

[1788] The "means for instructing image generation" is a function that conveys the received image requirements to the image generation AI model and causes it to generate the image.

[1789] "Means for returning generated images" refers to a function that returns images generated by an image generation AI model to the user's device.

[1790] The "means for receiving correction requirements" is a function for receiving a correction request for a generated image from a user.

[1791] The "means for performing copyright checks" is a function that evaluates the similarity and reliance of generated images against existing images, and checks for the risk of copyright infringement.

[1792] The "means for recording the results of copyright checks" is a function for saving the results of copyright checks in a database or the like.

[1793] "Means for automatically transcribing audio data" refers to a function that converts audio data recorded during a meeting into text data.

[1794] The "means for summarizing and providing transcribed data" is a function that summarizes automatically transcribed text data and provides it to the user.

[1795] The "means for adding and providing metadata" is a function for adding additional information such as the date and time of creation and the conditions of use to the created image and providing it to the user.

[1796] "Means for sending in JSON format" is a function that sends image requirements specified by the user to the server in a data format called JSON (JavaScript Object Notation).

[1797] The present invention relates to a system for generating images for use in proposals and meeting materials, redacting, copyright checking, and automatically transcribing and summarizing meeting records. The system is implemented as follows:

[1798] System Overview

[1799] In this system, users input image requirements through their devices and send them to the server. The server uses an image generation AI model to generate images, returns them to the user, makes any necessary corrections, and provides the final image. The system also has a copyright check function for generated images, and can automatically transcribe audio data from meetings and generate summaries.

[1800] Hardware and software used

[1801] The system uses the following hardware and software:

[1802] Image generation AI models: e.g., DALL-E and Stable Diffusion

[1803] Copyright checking AI model: For example, CopyChecker

[1804] Automatic transcription AI models: e.g., Google Cloud Speech-to-Text

[1805] Summary generation AI model: e.g., GPT-3

[1806] Device: The computer or smartphone used by the user

[1807] Server: A remote server for data processing and running AI models

[1808] Specific operation of the system

[1809] 1. Enter image requirements

[1810] Users can use their devices to input image requirements for proposals and meeting materials, such as "next-generation smartphone," "futuristic design," "blue color theme," and "high resolution." Users simply enter their specific requirements in the text box and click the send button.

[1811] 2. Submit your image requirements

[1812] The device sends the image requirements entered by the user to the server in JSON format. The data is sent via an HTTP POST request. The following JSON format data is used:

[1813] json

[1814] {

[1815] "theme": "Next-generation smartphone",

[1816] "style": "futuristic design",

[1817] "color": "blue",

[1818] "resolution": "High resolution"

[1819] }

[1820] 3. Execute image generation

[1821] The server analyzes the received image requirements and issues instructions to the image generation AI model based on them. For example, a prompt might look like this: "Create a high-resolution image of a futuristic next-generation smartphone with a blue color theme." The server receives the image data generated by the image generation AI model, adds metadata (such as the date and time of generation and conditions of use), and sends it back to the device.

[1822] 4. Check and correct the image

[1823] The user checks the displayed image and inputs correction requirements as necessary. For example, they can enter a command such as "reduce the gloss" in the text box and click the resend button. The device then sends the correction requirements back to the server in JSON format, and the server again sends the correction instructions to the image generation AI model.

[1824] 5. Copyright Check

[1825] The server uses an internal copyright check AI model to evaluate the generated image for similarity and reliability with existing images. The results of the copyright check are recorded and provided to the user only if there are no problems.

[1826] 6. Audio recording and transcription

[1827] During a meeting, users record audio notes on their device, which then sends the audio data to a server, which then converts the audio into text using an automatic transcription AI model. For example, "Transcribe the following meeting audio to text."

[1828] 7. Generating and Serving Abstracts

[1829] The server passes the generated text data to a summarization AI model to generate summary data. This summary is sent back to the user and used as a meeting record. A summary like the following is generated: "At the meeting, the progress and next steps of the new project were discussed."

[1830] This system streamlines the image creation and editing process for proposals and meeting materials, reduces copyright risks, and enables quick summarization of meeting records.

[1831] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1832] Image Generation Process

[1833] Step 1: User inputs image requirements

[1834] Input: Image requirements for user proposals and meeting materials (e.g., "next-generation smartphone," "futuristic design," "blue color theme," "high resolution")

[1835] Specific operation: The user enters the image requirements in the text box on the terminal and clicks the send button.

[1836] Output: Image requirements are entered into the terminal and prepared for sending to the server.

[1837] Step 2: Send image requirements from the device to the server

[1838] Input: User-entered image requirements

[1839] Specific operation: The device converts the image requirements into JSON format and sends it to the server as an HTTP POST request.

[1840] Output: The following JSON data is sent to the server:

[1841] json

[1842] {

[1843] "theme": "Next-generation smartphone",

[1844] "style": "futuristic design",

[1845] "color": "blue",

[1846] "resolution": "High resolution"

[1847] }

[1848] Step 3: Image generation instructions from the server

[1849] Input: Image requirements sent from the terminal (JSON data)

[1850] Specific operation: The server parses the received JSON data and generates an appropriate prompt for the image generation AI model. For example: "Create a high-resolution image of a futuristic next-generation smartphone with a blue color theme."

[1851] Output: A prompt sentence is input into the image generation AI model, and the generated image data is returned to the server.

[1852] Step 4: Returning the image from the server to the device

[1853] Input: Image data generated by an image generation AI model

[1854] Specific operation: The server adds metadata (date and time of creation, conditions of use, etc.) to the generated image and returns it to the terminal.

[1855] Output: Image data with metadata is sent to the device. Example:

[1856] json

[1857] {

[1858] "image": "base64_encoded_image_data",

[1859] "metadata": {

[1860] "generated_at": "2023-10-01T12:00:00Z",

[1861] "usage_conditions": "Commercial use allowed"

[1862] }

[1863] }

[1864] Step 5: User checks the image and gives correction instructions

[1865] Input: Image data returned from the server

[1866] Specific operation: The user checks the image on the device and inputs correction requirements as necessary (e.g., "I would like the glossiness to be reduced"). After inputting the correction requirements, the user clicks the resend button.

[1867] Output: The modified image requirements are entered into the terminal and are ready to be sent to the server again.

[1868] Step 6: Sending modification requirements from the terminal to the server

[1869] Input: User-entered correction requirements

[1870] Specific operation: The terminal converts the modification requirements into JSON format and resubmits them to the server as an HTTP POST request.

[1871] Output: The correction requirements are sent to the server and used again as prompts for the image generation AI model. Example of resubmitted JSON data:

[1872] json

[1873] {

[1874] "theme": "Next-generation smartphone",

[1875] "style": "futuristic design",

[1876] "color": "blue",

[1877] "resolution": "high resolution",

[1878] "modification": "Reduce glossiness"

[1879] }

[1880] Step 7: Copyright Check

[1881] Input: Generated image data

[1882] How it works: The server inputs the generated image into a copyright checking AI model to evaluate its similarity to existing images and its reliability. The evaluation results are recorded and provided to the user only if there are no problems.

[1883] Output: Copyright check results and usable image data based on the evaluation. Example:

[1884] json

[1885] {

[1886] "image": "base64_encoded_image_data",

[1887] "metadata": {

[1888] "generated_at": "2023-10-01T12:00:00Z",

[1889] "usage_conditions": "No copyright issues detected"

[1890] }

[1891] }

[1892] Meeting recording efficiency process

[1893] Step 1: Recording audio data

[1894] Input: Audio during the meeting

[1895] Specific operation: The user clicks the "Start Recording" button on the device to record conversations and notes in audio format.

[1896] Output: The recorded audio data is stored on the device and prepared for transmission to the server.

[1897] Step 2: Sending audio data from the device to the server

[1898] Input: Recorded audio data

[1899] Specific operation: The device converts the voice data into a digital audio file format (e.g., WAV, MP3) and sends it to the server as an HTTP POST request.

[1900] Output: The audio data is sent to the server. Example:

[1901] json

[1902] {

[1903] "audio_file": "base64_encoded_audio_data"

[1904] }

[1905] Step 3: Auto-transcription

[1906] Input: Audio data sent to the server

[1907] Specific operation: The server inputs the audio data into an automatic transcription AI model and converts the audio into text data. The prompt is as follows: "Transcribe the following meeting audio to text."

[1908] Output: A transcript of the audio is generated. Example:

[1909] text

[1910] "In the meeting, we discussed the progress of the new project and the next steps. Specifically, we had a detailed discussion about the design and features of the next-generation smartphone."

[1911] Step 4: Generate summary and return summary data

[1912] Input: Automatically transcribed text data

[1913] Specific operation: The server inputs the generated text data into the summary AI model and generates a summary. The prompt is as follows: "Summarize the following transcription of the meeting."

[1914] Output: Summarized text data is generated and sent back to the user. Example:

[1915] text

[1916] "During the meeting, the progress and next steps of the new project were discussed. In particular, there was detailed discussion on the design and functionality of the next generation smartphone."

[1917] Through these specific processing steps, the system streamlines the creation and editing of images required for proposals and meeting materials, reduces copyright risks, and enables quick summarization of meeting records.

[1918] (Application example 1)

[1919] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1920] In autonomous vehicles, there are problems with the efficient generation, editing, and copyright check of video surveillance and meeting records during operation, as well as the automatic transcription and summarization of audio data. In particular, the collection of surveillance data and the creation of driving reports within autonomous vehicles requires a great deal of time and effort using current manual processes, making effective operation difficult. In addition, copyright issues may arise with the generated data, so evaluation and management of these issues are required.

[1921] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1922] In this invention, the server includes means for receiving image requirements from a user, means for instructing an image generation AI model to generate an image based on the image requirements, means for returning the generated image to the user's terminal, means for receiving correction requirements from the user and again instructing the image generation AI model to generate a corrected image, means for performing a copyright check on the generated image, means for recording the results of the copyright check, means for automatically transcribing audio data recorded during a meeting, means for summarizing the transcribed data and providing it to the user, means for acquiring image surveillance data inside and outside the autonomous vehicle, means for transferring the recorded audio data, and means for automatically processing and summarizing audio memos. This enables efficient operation management of autonomous vehicles, rapid data generation and correction, reduced copyright risks, and rapid summarization of meeting records.

[1923] "User" refers to an entity that uses the system to generate images, specify correction requirements, and record audio data.

[1924] "Image generation AI model" refers to an algorithm or model that generates images based on specified image requirements.

[1925] "Image requirements" refer to the specific style, content, color tone, resolution, and other conditions of the image desired by the user.

[1926] "Modification requirements" refer to instructions for modifying or changing the generated image.

[1927] "Copyright check" refers to the process of assessing whether the generated image is similar to an existing image or infringes copyright.

[1928] "Audio data" refers to audio information recorded during a conference.

[1929] "Automatic transcription" refers to the process of automatically converting audio data into text data.

[1930] "Summarization" refers to the process of concisely summarizing transcribed text data.

[1931] An "autonomous vehicle" is a vehicle that is capable of operating independently without a human driver.

[1932] "Image surveillance data" refers to image information collected inside and outside an autonomous vehicle.

[1933] "Terminal" refers to a device used by a user, such as a computer or smartphone.

[1934] "Metadata" refers to data that includes information related to a generated image, such as the date and time of generation and the conditions of use.

[1935] This invention relates to a system for efficiently realizing image monitoring, copyright checking, and automatic transcription and summarization of meeting records in an autonomous vehicle. This system starts when a user inputs image requirements using a terminal and sends them to a server. Specific embodiments of this system are described below.

[1936] 1. System Overview

[1937] The system mainly uses the user's device, a server, and cameras and audio recording devices inside and outside the autonomous vehicle. The server utilizes an image generation AI model, a copyright check AI model, an automatic transcription AI model, and a summary AI model for efficient data processing.

[1938] 2. Hardware and Software

[1939] Hardware:

[1940] User's device (e.g. smartphone, tablet)

[1941] Cameras inside and outside the autonomous vehicle

[1942] Audio recording device

[1943] software:

[1944] Python

[1945] Flask (web server framework)

[1946] Image generation AI model (e.g., HuggingFace's DALL·E)

[1947] Copyright Check AI Model

[1948] Automatic transcription API (e.g., Google Speech-to-Text)

[1949] Summary AI models (e.g., HuggingFace's BART)

[1950] 3. Data processing and calculation

[1951] The server receives the request sent by the user from the terminal and processes the data through the following process.

[1952] Image generation process:

[1953] The user inputs image requirements from their device and sends them to the server, such as "futuristic design of an autonomous vehicle, blue color theme, high resolution."

[1954] The server uses an image generation AI model to generate images based on the specified requirements.

[1955] Metadata (date and time of creation, conditions of use, etc.) is added to the generated image, and the result is sent back to the user's device.

[1956] The user inputs the correction requirements for the image and sends it back to the server, which then generates the corrected image and sends it back to the user.

[1957] Copyright Check Process:

[1958] The server passes the generated images to a copyright checking AI model to evaluate their similarity to existing images.

[1959] The evaluation results are recorded and notified to the user. If there are no problems, the image becomes available for use.

[1960] Meeting recording efficiency process:

[1961] During the meeting, users record voice memos on their terminals and send them to the server.

[1962] The server transcribes the audio data using the Google Speech-to-Text API.

[1963] The transcribed text is passed to a summarization AI model to generate summary data.

[1964] The summary data is returned to the user's terminal and is effectively used as a record of the meeting.

[1965] 4. Specific Examples

[1966] Image generation example:

[1967] The user inputs image requirements such as "futuristic design, self-driving vehicle, blue color theme, high resolution."

[1968] The server calls an image generation AI model to generate a high-resolution image, adds metadata, and sends it back to the user's device.

[1969] The user inputs the correction requirement "reduce the glossiness a little" and sends it to the server again. The server generates a corrected image and provides it to the user.

[1970] Example of a recording:

[1971] A user records a voice memo during a conference, and the terminal transmits the voice data to a server.

[1972] The server automatically transcribes the audio data and passes the transcribed text to a summarization AI model to generate a summary.

[1973] The summary data is returned to the user's terminal, and the user can efficiently use the conference record.

[1974] This system not only streamlines the operation and management of autonomous vehicles, but also enables rapid data generation and correction, reduces copyright risks, and improves productivity by quickly summarizing meeting notes.

[1975] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1976] Step 1:

[1977] A user uses a terminal to input image requirements and create a request for image generation, such as "futuristic design of autonomous vehicle, blue color theme, high resolution."

[1978] Input: Image requirements (e.g. futuristic design, blue color theme, high resolution)

[1979] Output: Image generation request

[1980] Step 2:

[1981] The device converts the user's image requirements into JSON format and sends it to the server. The device properly formats the input requirements and prepares the request for sending to the server.

[1982] Input: Image generation request

[1983] Output: Request in JSON format

[1984] Step 3:

[1985] The server analyzes the received JSON-formatted request and instructs the image generation AI model to generate an image. The server analyzes the request, extracts the parameters necessary for image generation, and passes them to the AI ​​model.

[1986] Input: JSON formatted request

[1987] Output: Instructions to the AI ​​model

[1988] Step 4:

[1989] The image generation AI model generates images based on instructions. The model uses prompt text to generate high-resolution images according to the specified style and color tone.

[1990] Input: Instructions to the AI ​​model (e.g., futuristic design, self-driving vehicle, blue, resolution)

[1991] Output: The generated image

[1992] Step 5:

[1993] The server adds metadata (such as the date and time of creation and the conditions of use) to the created image and returns it to the user's device. The server records the date and time of creation and the conditions of use of the image and attaches them to the image data.

[1994] Input: Generated image

[1995] Output: Generated image with metadata

[1996] Step 6:

[1997] The user checks the generated image on the terminal and inputs the correction requirements. The user specifies the improvements and changes to the image and creates a new request.

[1998] Input: Generated image with metadata

[1999] Output: Fix requirements

[2000] Step 7:

[2001] The terminal converts the modification requirements into JSON format and sends it to the server. The terminal then sends a request based on the modification requirements to the server again.

[2002] Input: Correction Requirement

[2003] Output: The modification request in JSON format

[2004] Step 8:

[2005] The server analyzes the modification request and instructs the image generation AI model to generate the modified image again. The server instructs the AI ​​model to regenerate the image using the new modification parameters.

[2006] Input: JSON formatted modification request

[2007] Output: Corrective instructions for the AI ​​model

[2008] Step 9:

[2009] The image generation AI model generates a corrected image based on the correction instructions. It uses the prompt text again to generate an image that reflects the necessary corrections.

[2010] Input: Correction instructions for the AI ​​model

[2011] Output: Corrected image

[2012] Step 10:

[2013] The server returns the corrected image to the user's device. The corrected image data is sent to the user's device in JSON format.

[2014] Input: Modified image

[2015] Output: The corrected image sent back to the user

[2016] The above are the specific processing steps for image generation and correction. Next, we will show the processing steps for automatic transcription and summarization of meeting recordings.

[2017] Step 11:

[2018] During a meeting, the user records a voice memo, and the device sends the voice data to the server. The user then presses the record button to record the required voice.

[2019] Input: Voice memo

[2020] Output: Audio data

[2021] Step 12:

[2022] The server passes the audio data to an automated transcription method to convert it into text. The server uses the Google Speech-to-Text API to transcribe the audio data.

[2023] Input: Audio data

[2024] Output: Transcribed text

[2025] Step 13:

[2026] The server passes the generated text to a summarization model to generate summary data. The server then invokes a summarization AI model (e.g., BART) to generate a summary that extracts the key points.

[2027] Input: Transcribed text

[2028] Output: Summary data

[2029] Step 14:

[2030] The server returns the summary data to the user's terminal, and the user efficiently checks the meeting record. The points of the summarized meeting are provided to the user.

[2031] Input: Summary data

[2032] Output: Summary data returned to the user

[2033] These processing steps enable efficient management of image generation, editing, copyright checks, and meeting records in autonomous vehicles.

[2034] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[2035] This invention combines a system for generating images for use in proposals and meeting materials, correcting them, checking copyrights, and automatically transcribing and summarizing meeting records with an emotion engine that recognizes the user's emotions. This system provides more human-centered and effective support by suggesting image requirements, making corrections, and adjusting the summaries of meeting records according to the user's emotional state.

[2036] System Overview

[2037] In this system, users input image requirements through their devices and send them to a server. The server then uses an image generation AI model to generate an image, which is returned to the user, where it is corrected as necessary and the final image is provided. The system also includes a copyright check function to avoid the risk of copyright infringement, and the ability to automatically transcribe and summarize audio data during meetings. A new emotion engine has also been added, enabling suggestions and corrections based on the user's emotions.

[2038] Image generation process and emotion engine integration

[2039] 1. User Roles

[2040] Users log in to a dedicated application on their device, and when entering image requirements for proposals or meeting materials, the system recognizes emotions from the user's facial expressions and voice.

[2041] The emotion engine analyzes the user's emotional state when they input and suggests appropriate image requirements, such as "if the user is excited, we suggest a bold design."

[2042] 2. Role of the terminal

[2043] The terminal transmits the input image requirements and additional requirements provided by the emotion engine to the server, thereby including requirements based on the user's emotions.

[2044] The terminal displays the generated or modified image returned from the server.

[2045] 3. Server Roles

[2046] The server analyzes the image requirements received from the device and instructs the image generation AI model to generate the image.

[2047] The image generation AI model generates new images based on the instruction requirements.

[2048] The server adds metadata to the generated image and returns it to the terminal.

[2049] When responding to correction instructions, the emotion engine evaluates the user's emotional response and automatically recommends correction requirements.

[2050] Copyright Check Process

[2051] The server uses its internal copyright checking AI model to evaluate the generated image for similarity and reliability with existing images. The evaluation results are recorded and provided to users only if there are no problems with using the image.

[2052] Meeting recording efficiency process

[2053] 1. Recording audio data

[2054] During a meeting, users use their devices to record audio of comments and notes, and their emotional state during the recording is also analyzed.

[2055] 2. Automatic transcription

[2056] The device sends the recorded audio data to a server, which then passes the data to an automatic transcription AI model and converts it into text.

[2057] 3. Summary Generation

[2058] The server passes the transcribed text data to a summary AI model to generate a summary. An emotion engine then applies the summary to adjust the content according to the user's emotional state.

[2059] Specific examples

[2060] Image generation example

[2061] When a user requests an image themed on "next-generation smartphones" and inputs their requirements into the device, the emotion engine analyzes the user's emotional state. For example, if a user is excited, it will recommend a "futuristic design."

[2062] The device sends the proposed requirements and the emotion engine's suggestions to the server. The server receives the image generated by the image generation AI model and returns it to the device. The user checks the image, and the emotion engine automatically suggests correction requirements.

[2063] Meeting recording example

[2064] During a meeting, when a user records a voice memo on the terminal, the emotion engine analyzes the user's emotional state.

[2065] The device sends the recorded voice data to a server, which then automatically transcribes it. At the same time, an emotion engine adjusts the summary content to match the user's emotions based on the generated text data.

[2066] The summary data is sent back to the terminal, and the user can use it efficiently as a follow-up after the meeting.

[2067] This system streamlines the image creation and editing process for proposals and meeting materials, improves user response based on user sentiment, reduces copyright risks, and enables quick summarization of meeting records.

[2068] The processing flow will be explained below.

[2069] Image generation process incorporating an emotion engine

[2070] User-defined image requirements and generation

[2071] Step 1:

[2072] The user logs in to a dedicated application on the device.

[2073] Step 2:

[2074] The user inputs the image requirements for proposals and meeting materials into the device, such as "next-generation smartphone," "futuristic design," "blue color theme," and "high resolution."

[2075] Step 3:

[2076] While the user is typing, the device uses a built-in camera and microphone to analyze the user's facial expressions and voice, and an emotion engine evaluates the user's emotional state.

[2077] Step 4:

[2078] The emotion engine makes additional suggestions for image requirements based on the user's emotional state, for example, suggesting a "daring design" if the user is excited.

[2079] Step 5:

[2080] The device sends image requirements and emotion engine suggestions to the server as JSON format data.

[2081] Step 6:

[2082] The server analyzes the JSON format image requirements received from the device and instructs the image generation AI model to generate the image.

[2083] Step 7:

[2084] The image generation AI model generates new images based on the specified requirements.

[2085] Step 8:

[2086] The server receives image data generated by the image generation AI model and adds metadata (such as the date and time of generation, conditions of use, etc.) to the generated image.

[2087] Step 9:

[2088] The server transmits the generated image to the terminal.

[2089] Step 10:

[2090] The user checks the image generated on the device. If the user determines that corrections are necessary, the user inputs the correction requirements into the device (for example, "make the design less glossy").

[2091] Step 11:

[2092] The emotion engine evaluates the user's emotional response when viewing an image and automatically recommends revision requirements.

[2093] Step 12:

[2094] The device sends the user's correction requirements and the emotion engine's suggestions back to the server as JSON format data.

[2095] Step 13:

[2096] Based on the correction requirements received from the terminal, the server again instructs the image generation AI model to generate a corrected image.

[2097] Step 14:

[2098] The image generation AI model regenerates the image based on the correction requirements and sends it back to the server.

[2099] Step 15:

[2100] The server receives the corrected image data and sends it back to the terminal, where the user checks the corrected image and finally approves it.

[2101] Copyright check process incorporating emotion engine

[2102] Step 1:

[2103] The server uses its internal copyright checking AI model to evaluate the generated image for similarity and reliability with existing images.

[2104] Step 2:

[2105] The server records the evaluation results in an internal database, and if there are no problems, the image is provided to the user.

[2106] Step 3:

[2107] The server saves the good images in the project folder and makes them accessible to users.

[2108] Emotion-driven meeting recording process

[2109] Audio data recording and processing

[2110] Step 1:

[2111] During a meeting, a user uses the terminal to record audio of comments and notes.

[2112] Step 2:

[2113] The device analyzes the tone and pace of the user's voice while recording, and an emotion engine evaluates the user's emotional state.

[2114] Step 3:

[2115] The device sends the recorded audio data to the server.

[2116] Step 4:

[2117] The server passes the audio data to an automatic transcription AI model and converts it into text.

[2118] Step 5:

[2119] The server passes the generated text data to a summary AI model to generate summary data.

[2120] Step 6:

[2121] The emotion engine uses the summary data to tailor the content to the user's emotional state, for example, emphasizing positive summary results if the user is feeling down.

[2122] Step 7:

[2123] The server sends the generated summary data to the terminal, where the user can check it. The user can use it efficiently as a follow-up after the meeting.

[2124] These steps provide a system incorporating an emotion engine that not only streamlines the process of creating proposals and meeting materials, but also allows for suggestions and revisions based on the user's emotions, reduces copyright risks, and enables quick summarization of meeting records.

[2125] Example 2

[2126] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2127] In today's busy business environment, creating proposals and meeting materials requires a lot of time and effort. The tasks of generating and editing high-quality images that reflect the user's intent and emotions, as well as efficiently transcribing and summarizing meeting recordings, are particularly burdensome. The risk of copyright infringement for generated images is also a significant issue. It is necessary to develop a system that streamlines these processes and improves the user experience.

[2128] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[2129] In this invention, the server includes means for proposing appropriate image requirements in cooperation with an emotion engine that recognizes and analyzes emotions from a user's facial expressions and voice, means for instructing an image generation AI model to generate an image, means for performing copyright checks on the generated image, means for automatically transcribing audio data recorded during a meeting, and means for summarizing the transcribed data, adjusting the content based on the user's emotional state, and providing the summarization to the user. This streamlines the image creation and editing process for proposals and meeting materials, improves response based on user emotions, reduces copyright risks, and enables quick summarization of meeting records.

[2130] A "user" is a person who operates the system and inputs image and modification requirements.

[2131] "Terminal" refers to an electronic device such as a computer or smartphone used by a user.

[2132] "Image requirements" refer to specific conditions and specifications regarding the image that a user wants to create.

[2133] The "emotion engine" is an AI model that recognizes emotions from a user's facial expressions and voice and analyzes their emotional state.

[2134] An "image generation AI model" is an artificial intelligence model for automatically generating images based on input requirements.

[2135] "Copyright check" is the process of checking whether the generated image is similar to or relies on an existing image and assessing legal risk.

[2136] The "correction requirements" refer to the corrections that the user wants to make to the generated image.

[2137] "Automatic transcription" is the automated process of converting audio data into text.

[2138] A "summary" is a text that briefly summarizes the transcribed data.

[2139] "Metadata" is auxiliary information added to a generated image, and is data that indicates details and attributes of the image.

[2140] This invention combines a system for generating images for use in proposals and meeting materials, correcting them, checking copyrights, and automatically transcribing and summarizing meeting records with an emotion engine that recognizes the user's emotions. This system provides more human-centered and effective support by suggesting image requirements, making corrections, and adjusting the summaries of meeting records according to the user's emotional state.

[2141] System Overview

[2142] In this system, users input image requirements through their devices and send them to a server. The server then uses an image generation AI model to generate an image, which is returned to the user, where it is corrected as necessary and the final image is provided. The system also includes a copyright check function to avoid the risk of copyright infringement, and the ability to automatically transcribe and summarize audio data during meetings. A new emotion engine has also been added, enabling suggestions and corrections based on the user's emotions.

[2143] Specific examples

[2144] Image generation example

[2145] When a user requests an image themed around "next-generation smartphones" and inputs the requirements into the device, the emotion engine analyzes the user's emotional state. For example, if the user is in an excited state, it will recommend a "futuristic design." The device then sends the proposed requirements and the emotion engine's suggestions to the server. The server receives the image generated by the image generation AI model and returns it to the device. The user then reviews the image, and the emotion engine automatically suggests further revisions.

[2146] Example prompt for a generative AI model:

[2147] "Generate images with futuristic and exciting designs based on the theme of next-generation smartphones."

[2148] Meeting recording example

[2149] During a meeting, when a user records a voice memo on their device, the emotion engine analyzes the user's emotional state. The device then sends the recorded voice data to a server, which then automatically transcribes it. At the same time, the emotion engine adjusts the summary content to match the user's emotions based on the generated text data. The summary data is then sent back to the device, allowing the user to efficiently use it as a follow-up after the meeting.

[2150] The system streamlines the image creation and editing process for proposals and meeting materials, enables user-sensitive responses, mitigates copyright risks, and enables quick summarization of meeting records.

[2151] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2152] Step 1:

[2153] The user logs in to a dedicated application on their device. The user inputs the image requirements needed for proposals and meeting materials, and the emotion engine recognizes and analyzes the user's emotions from their facial expressions and voice. For example, when a user requests a "futuristic design for a next-generation smartphone," the emotion engine proposes requirements that reflect the user's excited state of mind. The input data includes image requirements, and the output is the emotion analysis results and proposed image requirements.

[2154] Step 2:

[2155] The device sends the image requirements entered by the user and the suggestions made by the emotion engine to the server. The input data here are the user's image requirements and the results of emotion analysis, and the output data is a request message containing these. Specifically, the device sends a request to "generate a futuristic design for the next generation smartphone" along with the user's emotion data to the server.

[2156] Step 3:

[2157] The server analyzes the image requirements received from the device and instructs the image generation AI model to generate the image. The input data includes image requirements and emotion data, and the output is the generated image. Specifically, the server sends a "futuristic design for next-generation smartphones" as a prompt to the image generation AI model, and after receiving the generated image, it adds metadata.

[2158] Step 4:

[2159] The server returns the generated image to the terminal. The input data here is the generated image and metadata, and the output data is the image returned to the user's terminal. Specifically, the server returns the generated image to the terminal together with the metadata.

[2160] Step 5:

[2161] The device displays the generated image returned from the server to the user. The user checks the image and inputs correction requirements, if necessary. The emotion engine again analyzes the user's emotional state and makes suggestions for correction. The input data includes the generated image and the user's correction requirements, and the output data is the correction suggestions. Specifically, the user requests "more colorful," and the emotion engine takes this into consideration and suggests an even bolder color palette.

[2162] Step 6:

[2163] The device sends the user's correction requirements to the server, and the server again instructs the image generation AI model to generate a corrected image. The input data includes the correction requirements and emotion data, and the output data is the corrected image. Specifically, the device sends the correction requirement "change to a colorful design" to the server, and the server sends it again to the image generation AI model to generate the corrected image.

[2164] Step 7:

[2165] The server performs a copyright check on the generated image. The input data is the generated image, and the output data is the result of the copyright check. Specifically, the server uses a copyright check AI model to evaluate the similarity between the generated image and existing images, and records the results.

[2166] Step 8:

[2167] During a meeting, the user uses a device to record voice comments and notes. The emotion engine simultaneously analyzes the user's emotional state during the recording. The input data is the recorded voice and emotional data, and the output data is the analysis results. Specifically, the user says, "Regarding the next action plan," which is recorded as a voice memo, and the emotion engine analyzes the user's state of tension.

[2168] Step 9:

[2169] The device sends the recorded voice data to a server, which then passes the data to an automatic transcription AI model and converts it into text. The input data is voice data, and the output data is the transcribed text. Specifically, the device sends voice data saying "What is the next action plan?" to the server, and the server converts the voice data into text.

[2170] Step 10:

[2171] The server passes the transcribed text data to a summarization AI model, which generates summary data. The emotion engine then adjusts the content based on the generated summary according to the user's emotional state. The input data is the transcribed text and emotion data, and the output data is the adjusted summary. Specifically, the summarization AI model generates a short summary such as "The next steps are A, B, and C," and the emotion engine adjusts it to use softer wording.

[2172] Step 11:

[2173] The terminal receives the summary data and displays it to the user. The input data is the adjusted summary data, and the output data is data for the user to confirm. Specifically, the adjusted summary data is displayed on the terminal, and the user can confirm the summary, "The next steps are A, B, and C."

[2174] (Application example 2)

[2175] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2176] Current image generation systems and conference support systems are unable to consider the emotional state of the user, and in autonomous vehicles, there is a lack of services that respond to the emotional state of the occupants. Furthermore, the summary generation during a conference does not adjust to reflect the emotions of participants, making improving the user experience a challenge. In particular, there is a need for a comfortable environment and efficient conference management in autonomous vehicles.

[2177] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving image requirements from a user, means for instructing an image generation AI model to generate an image based on the image requirements, means for returning the generated image to the user's terminal, means for receiving correction requirements from the user and again instructing the image generation AI model to generate a corrected image, means for performing a copyright check on the generated image, means for recording the results of the copyright check, means for automatically transcribing audio data recorded during a conference, means for summarizing the transcribed data and providing it to the user, means for analyzing the emotional states of occupants in an autonomous vehicle in real time, means for automatically adjusting the in-vehicle environment (music, air conditioning, lighting, etc.) based on the analysis, means for collecting driving data of the autonomous vehicle and generating a visual report, and means for automatically adjusting the summary content of the transcribed data based on the emotional states of participants during the conference. This enables a comfortable environment to be set up in an autonomous vehicle, enabling the provision of services that take occupants' emotions into consideration and efficient conference management.

[2178] "Image requirements" are requirements specified by the user regarding the image to be generated, and include theme, style, color tone, resolution, and the like.

[2179] An "image generation AI model" is an artificial intelligence model that generates new images based on input image requirements.

[2180] The "correction requirements" are the correction requirements that the user must meet for the generated image.

[2181] "Copyright check" refers to the process of inspecting whether the generated image is similar to an existing image and whether it infringes copyright.

[2182] "Audio data recorded during a conference" refers to audio recorded while the conference is in progress.

[2183] "Automatic transcription" is the process of converting recorded audio data into text data.

[2184] "Summarization" is the process of simplifying transcribed data and extracting and organizing the main points.

[2185] "Emotional state" refers to the psychological state of the occupant as perceived from facial expressions, voice, etc.

[2186] "In-car environment" is a general term for music, air conditioning, lighting, etc. inside an autonomous vehicle.

[2187] "Driving data" refers to various data such as location information and speed associated with the movement of an autonomous vehicle.

[2188] A "visual report" is a report that is generated based on collected data in a visually easy-to-understand format.

[2189] This invention is a system that integrates image generation, emotion recognition, and conferencing support to improve the user experience in an autonomous vehicle. This system is composed of multiple hardware and software components that operate in the autonomous vehicle.

[2190] 1. System Configuration

[2191] The system consists of the following main hardware and software:

[2192] 1.1 Hardware

[2193] Devices: tablets, smartphones, in-car displays

[2194] Emotion recognition camera: A camera for capturing passenger facial expressions

[2195] Microphone: A microphone for capturing passenger voices and conference audio

[2196] Vehicle sensors: Sensors for collecting driving data from autonomous vehicles, such as GPS and speed sensors

[2197] 1.2 Software

[2198] Emotion recognition engine: Software that analyzes the emotional state of passengers (e.g., Microsoft Azure Cognitive Services, Google Cloud Vision AI)

[2199] Image generation AI model: An artificial intelligence model for generating images (e.g., OpenAI's DALL-E)

[2200] Speech recognition AI model: Software for transcribing meeting audio (e.g., Google Cloud Speech-to-Text)

[2201] Summarization AI model: Software for summarizing transcribed text (e.g., GPT-4)

[2202] 2. Program Processing

[2203] 2.1 Emotion recognition and environmental adjustment

[2204] The device's camera and microphone capture the passenger's facial expressions and voice, and send them to an emotion recognition engine in the cloud. The emotion recognition engine analyzes the captured images to understand the passenger's emotional state. Based on the analysis results, the car's music, air conditioning, and lighting are automatically adjusted.

[2205] 2.2 Image Creation and Modification

[2206] The user inputs image requirements on their device and sends them to the server. The server generates an image using an image generation AI model based on the requirements and returns the generated image to the device. If the user requests any corrections, the details are input again into the image generation AI model to generate a corrected image. The generated image also undergoes copyright checks.

[2207] 2.3 Meeting Support

[2208] During the meeting, crew members record audio using a microphone and send the data to a server. The server's speech recognition AI model converts the audio into text, and the summarization AI model extracts key points and creates a summary. When creating the summary, an emotion recognition engine adjusts the content to reflect the emotional state of the participants.

[2209] 3. Examples and prompts

[2210] 3.1 Specific examples

[2211] For example, it can analyze emotions during a relaxed drive and optimize the background music and air conditioning in the car. It can also record conference audio, generate summaries, and adjust the settings to take into account the emotions of the participants.

[2212] 3.2 Prompt Sentence Examples

[2213] "Create a report featuring the beautiful scenery during the relaxing drive. The passengers were very happy."

[2214] "Generate a summary of today's meeting and shorten the sections that participants found boring."

[2215] This will enable a comfortable environment to be created inside an autonomous vehicle, efficient meeting management, and the provision of high-quality services that take into consideration the emotions of passengers.

[2216] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2217] Step 1:

[2218] The terminal receives image requirements from the user, including theme, style, color tone, resolution, etc. The input image requirements are sent to the server.

[2219] Step 2:

[2220] The server analyzes the received image requirements and instructs the image generation AI model to generate an image based on that information. The image generation AI model (e.g., OpenAI's DALL-E) generates a new image based on the prompt text.

[2221] Step 3:

[2222] The server returns the generated image to the terminal, which displays the returned image to the user. The user checks the image and inputs correction requirements as necessary.

[2223] Step 4:

[2224] The device sends the user's correction requirements to the server, which then instructs the image generation AI model to generate a corrected image again, generating a new image.

[2225] Step 5:

[2226] The server performs a copyright check on the new image generated, using an algorithm (e.g., image recognition AI) that evaluates the similarity and authenticity of the generated image with existing images. The results of the copyright check are recorded.

[2227] Step 6:

[2228] The device's camera and microphone capture the passenger's facial expressions and voice, which are then sent to an emotion recognition engine in the cloud (e.g., Microsoft Azure Cognitive Services), which analyzes the passenger's emotional state and sends the results back to the device.

[2229] Step 7:

[2230] The device receives the analysis results and automatically adjusts the in-car environment (music, air conditioning, lighting, etc.) For example, if the passenger is relaxing, the music will be soft and the temperature will be moderate.

[2231] Step 8:

[2232] When a user starts a conference in a moving autonomous vehicle, the microphone on the device records the conference audio, and the recorded audio data is sent to the server.

[2233] Step 9:

[2234] The server passes the recorded audio data to an automatic transcription AI model (e.g., Google Cloud Speech-to-Text) to convert it into text data, which is then passed to a summarization AI model (e.g., GPT-4) to generate a summary.

[2235] Step 10:

[2236] The emotion recognition engine analyzes the emotional state of participants during the meeting and reflects it in the summary. For example, if it detects that a participant is bored, it will extract only the important points and summarize them concisely.

[2237] Step 11:

[2238] The summarized text data is sent back to the terminal, where the user can review it and use it as a follow-up for the meeting if necessary. This series of processes enables a comfortable environment to be set up inside the autonomous vehicle and for efficient meeting management.

[2239] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2240] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2241] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[2242] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2243] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[2244] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a ...

Claims

1. means for receiving image requirements from a user; A means for instructing an image generation AI model to generate an image based on the image requirements; means for returning the generated image to the user's terminal; A means for receiving correction requirements from a user and instructing the image generation AI model to generate a corrected image again; means for performing a copyright check on the generated image; means for recording the results of said copyright check; A means of automatically transcribing audio data recorded during meetings, a means for summarizing the transcribed data and providing it to a user; A system including:

2. 2. The system according to claim 1, wherein the means for performing a copyright check on the generated image includes means for using an algorithm for evaluating the similarity and authenticity of the generated image with an existing image.

3. 10. The system of claim 1, further comprising means for setting a particular style, color tone, and resolution for the image generation AI model based on user-specified image requirements.

Images