System

Abstract

Provided is a system that collects camera image data of a drone and efficiently generates a learning model.SOLUTION: A system including means for using a generative AI to generate camera image data of a drone, means for storing the generated camera image data, means for providing the stored camera image data to a user, means for performing annotation work on the image data by the user, means for collecting annotated data and generating a learning model using a machine learning algorithm, and means for providing an online marketplace for selling the generated learning data set and the learning model.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] The modern drone industry, particularly in last-mile delivery, requires precise positioning control and improved safety. Conventional methods require the collection, labeling, and annotation of massive amounts of camera video data, placing a significant burden on labor and time. Furthermore, while high-quality datasets are necessary to improve drone autonomous driving technology, collecting and organizing this data can be extremely costly. This invention aims to address these challenges and provide a system that efficiently generates high-quality camera video data and learning models. [Means for solving the problem]

[0005] This invention provides a means for using a generative AI to generate drone camera video data. It also includes a means for saving the camera video data generated by the generative AI and a means for providing the saved data to a user. Furthermore, it provides a means for users to annotate the video data, collect the annotated data, and generate a learning model using a machine learning algorithm. Finally, it provides an online market for selling the generated learning datasets and learning models, making this technology available to other development companies. This enables the development and implementation of efficient and highly accurate autonomous driving technology, promoting technological innovation in the drone industry.

[0006] "Generative AI" is an artificial intelligence that automatically generates drone camera footage data based on parameters provided by the user.

[0007] "Camera video data" refers to digital data of video captured by a drone camera or video created by generative AI.

[0008] "Storage means" refers to technology for storing the generated camera image data in digital storage.

[0009] "Means of providing" refers to the technology that allows users to access stored camera video data.

[0010] "Annotation" is the process in which a user labels and annotates drone camera footage.

[0011] "Annotated data" is camera video data that has been labeled and annotated by a user.

[0012] The "means of collection" refers to the technology that imports and stores annotated data from users on a server.

[0013] A "machine learning algorithm" is a computational method for generating and training an autonomous driving model using collected annotation data.

[0014] "Learning Model" means a digital product trained by machine learning algorithms to provide autonomous driving technology.

[0015] An "online market" is an internet platform for selling and purchasing generated training datasets and learning models. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] This invention is a system for generating drone camera image data and using it for autonomous driving technology. This system uses generative AI to generate a variety of camera image data, store and sell that data, and accumulate annotation technology.

[0038] System Overview

[0039] 1. Data generation phase

[0040] The user logs in to the application on the server from their terminal, enters the necessary parameters (e.g., video resolution, duration, specific scenes, etc.) from the "Camera Video Data Generation" menu, and clicks the "Start Generation" button.

[0041] The server receives the user's request, passes parameters to the generation AI, and starts the video data generation process. The generation AI generates the necessary camera video data and stores it in the server's storage.

[0042] The server notifies the user that the generation of the video data is complete.

[0043] 2. Data annotation phase

[0044] The user logs in to the server from a terminal, selects the "annotation work" menu, and requests the video data that requires annotation.

[0045] The server extracts the requested video data from the storage and transmits it to the user's terminal.

[0046] The user performs annotation work by adding labels and annotations to the transmitted video data, and then transmits the completed data to the server.

[0047] The server stores the received annotation data in storage.

[0048] 3. Machine Learning Phase

[0049] The server collects the accumulated annotation data and generates a learning model using a machine learning algorithm, which is then periodically updated.

[0050] Along with the learning model, the generated dataset is also saved for the next phase.

[0051] 4. Data sales phase

[0052] The server then posts the generated training datasets and models on an online marketplace, making the data available to other development companies.

[0053] Users access the online market from their devices and purchase the necessary datasets and learning models.

[0054] After the purchase process is completed, the server sends the dataset and model to the user's device.

[0055] Specific examples

[0056] For example, consider the case where a person in a logistics company generates camera footage from a drone. The person in charge accesses the server from a secure terminal and requests the generation of camera footage. The server uses the generation AI to generate the footage according to the request and stores it. The person in charge is then notified and can download the generated video data and review it internally.

[0057] Furthermore, annotation work, which involves labeling and annotating this data, can be easily performed. This annotation data can be sent back to the server and used as training data for machine learning to enable autonomous driving for other drones. Ultimately, the generated learning models and datasets can be sold on online markets, allowing other companies to use this technology.

[0058] As such, this invention is a system that comprehensively supports everything from drone camera image data generation to annotation, machine learning, and data sales, which is expected to improve technology and efficiency in the drone industry.

[0059] The processing flow will be explained below.

[0060] Step 1:

[0061] The user logs in to the application on the server from their terminal. The user selects the "Camera Video Data Generation" menu and inputs the parameters required for video generation (e.g., resolution, duration, specific scenes, etc.). Then, the user clicks the "Start Generation" button to send the request to the server.

[0062] Step 2:

[0063] The server receives the user's request and passes parameters to the generation AI, which then generates camera image data based on these parameters and stores the generated data in the server's storage.

[0064] Step 3:

[0065] The server sends a notification to the user saying, "Camera video data has been generated." This notification includes an access link to the generated video data.

[0066] Step 4:

[0067] The user checks the notification on their device and downloads the generated camera video data from the provided link. The user can use this data for their own purposes.

[0068] Step 5:

[0069] When a user wants to perform annotation, he or she accesses the server from the terminal, selects the "annotation work" menu, and requests the video data that requires annotation.

[0070] Step 6:

[0071] The server extracts the requested video data from storage and sends it to the user's device, where the user can add labels and annotations to the data.

[0072] Step 7:

[0073] The user completes the annotation process on the device and sends the completed annotation data to the server, which then stores the received annotation data in storage and organizes it to improve the data quality.

[0074] Step 8:

[0075] The server collects the accumulated annotation data and generates a learning model using a machine learning algorithm, which is then used to improve the drone's autonomous driving technology.

[0076] Step 9:

[0077] The server stores the generated learning model in storage, periodically evaluates the model's performance, and updates the model by retraining it using the latest data.

[0078] Step 10:

[0079] The server then posts the generated training datasets and models on an online market, making them available to other development companies.

[0080] Step 11:

[0081] The user accesses the online market from their device, searches for and selects the required training dataset and training model, and then completes the purchase procedure, after which the server confirms the purchase details.

[0082] Step 12:

[0083] Once the server confirms that the purchase procedure has been completed, it sends the training dataset and model to the user's device, allowing the user to easily obtain the resources necessary for developing autonomous driving technology.

[0084] Example 1

[0085] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0086] Conventionally, systems that generate drone camera video data and use it to improve autonomous driving technology have had several problems. Specifically, the entire process from data generation to annotation and machine learning model generation is often done manually, which is time-consuming and labor-intensive. Additionally, the generated data and models are often sold individually, making it difficult to manage and use them all at once.

[0087] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0088] In this invention, the server includes a means for a user to log in to an application on the server from a terminal, a means for generating drone camera image data using a generative AI model based on a user request, and a means for saving the generated camera image data, thereby automating a series of processes and enabling efficient generation and management of high-quality data.

[0089] A "user" is an entity that uses this system to generate camera image data, perform annotation work, and purchase datasets and learning models.

[0090] A "terminal" is a computing device that a user uses to access applications on a server.

[0091] The "server" is a system that uses a generative AI model based on user requests to generate and store camera image data, and also collects annotation data and performs machine learning.

[0092] A "generative AI model" is an artificial intelligence algorithm that generates camera image data based on parameters specified by the user.

[0093] A "prompt sentence" is an instruction sentence that includes parameters required for generating camera image data input by the user.

[0094] "Camera video data" refers to digital video data generated by AI that imitates or generates video captured by a drone.

[0095] "Annotation" is the process of adding labels and annotations to video data.

[0096] A "machine learning algorithm" is a computational method for building and updating a learning model based on collected annotation data.

[0097] A "learning model" is a model generated by a machine learning algorithm to analyze data and perform a specific task.

[0098] A "dataset" is a collection of annotated video data used to train and test a learning model.

[0099] The "online market" is an e-commerce platform that provides generated training datasets and models to users for purchase and download.

[0100] This invention is a system for generating drone camera image data and using it for autonomous driving technology. This system uses a generative AI model to generate diverse camera image data, store and sell that data, and accumulate annotation technology.

[0101] Data generation phase

[0102] 1. The user accesses the application on the server from their terminal and logs in by entering their login information (user ID and password). An authentication server is used for logging in.

[0103] 2. The user selects the "Camera Video Data Generation" menu from the main menu and enters the required parameters (e.g., video resolution, duration, specific scene, etc.).

[0104] 3. The server receives the user's request and passes parameters to the generative AI model (e.g., GPT-4 (registered trademark)) to start the video data generation process. The prompt uses the following format:

[0105] "Resolution: 1920x1080" "Duration: 10 minutes" "Scene: Urban traffic"

[0106] 4. The generative AI model generates camera image data based on the input parameters. The generated data is stored in the server's storage (e.g., AWS (registered trademark) S3).

[0107] 5. The server notifies the user that the video data has been generated. This notification is sent via email or in-app notification.

[0108] Data annotation phase

[0109] 1. The user accesses the server from their terminal, selects the "Annotation" menu, and requests the video data that requires annotation.

[0110] 2. The server extracts the requested video data from storage (e.g., Google® Cloud Storage) and sends it to the user's device.

[0111] 3. The user adds labels and annotations to the transmitted video data using an annotation tool such as VGG Image Annotator.

[0112] 4. The user uploads the annotated data to the server, which then stores the received annotation data in its storage.

[0113] Machine Learning Phase

[0114] 1. The server collects the accumulated annotation data and generates or updates a learning model using a machine learning algorithm (e.g., TENSORFLOW (registered trademark) or PyTorch).

[0115] 2. The server stores the generated dataset along with the generated learning model in preparation for the next phase.

[0116] Data sales phase

[0117] 1. The server publishes the generated training dataset and model on an online marketplace, making the data available to other users.

[0118] 2. The user accesses the online market from their device and purchases the necessary datasets and learning models.

[0119] 3. After the purchase procedure is completed, the server sends the relevant dataset and model to the user's device.

[0120] Specific examples

[0121] When a logistics company employee wants to generate drone camera footage, they access the server from a secure device and request the generation of camera footage. This request includes a prompt like this:

[0122] "Resolution: 1920x1080" "Duration: 10 minutes" "Scene: Urban traffic"

[0123] The server uses the generative AI model to generate and store the video according to the request, after which the person in charge can download the video data and review it in-house.

[0124] Furthermore, annotation work, which involves labeling and annotating this data, can be easily performed. This annotation data can be sent back to the server and used as training data for machine learning to enable autonomous driving for other drones. Ultimately, the generated learning models and datasets can be sold on online markets, allowing other companies to use this technology.

[0125] As such, this invention is a system that comprehensively supports everything from drone camera image data generation to annotation, machine learning, and data sales, which is expected to improve technology and efficiency in the drone industry.

[0126] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0127] Step 1:

[0128] A user accesses an application on the server from a terminal and logs in by entering login information (user ID and password). The input is the user ID and password, which the authentication server receives and performs authentication processing. The output is the authentication result, and if the login is successful, the main menu screen is displayed.

[0129] Specific behavior:

[0130] 1. The user displays the login page on the terminal.

[0131] 2. Enter your user ID and password.

[0132] 3. The server sends a query to the authentication server.

[0133] 4. The authentication server returns the authentication result.

[0134] 5. If the server is successful in authenticating, it displays the main menu screen to the user.

[0135] Step 2:

[0136] The user selects the "Camera Video Data Generation" menu from the main menu and inputs parameters such as video resolution, duration, and specific scenes. The inputs are the video generation parameters. The server receives the input parameters and passes them to the generative AI model.

[0137] Specific behavior:

[0138] 1. The user clicks on the "Generate camera video data" menu.

[0139] 2. The user inputs parameters such as video resolution (e.g., 1920x1080), duration (e.g., 10 minutes), and scene (e.g., urban traffic).

[0140] 3. The user clicks the "Start Generation" button.

[0141] 4. The server retrieves the input parameters.

[0142] Step 3:

[0143] The server generates a prompt sentence from the acquired parameters and sends a request to the generative AI model. The input is the generated parameters, and the output is to generate a prompt sentence and pass it to the generative AI model. The generative AI model generates video data based on this prompt sentence.

[0144] Specific behavior:

[0145] 1. The server generates a prompt using the parameters it has obtained.

[0146] For example: "Resolution: 1920x1080", "Duration: 10 minutes", "Scene: City traffic".

[0147] 2. The server sends the generated prompt sentence to the generative AI model.

[0148] Step 4:

[0149] The generative AI model generates camera video data based on the prompt sentence and returns the generated data to the server. The input is the prompt sentence and the output is the generated camera video data. This data is sent to the server.

[0150] Specific behavior:

[0151] 1. The generative AI model parses the prompt.

[0152] 2. The AI ​​model uses its internal algorithm to generate video data.

[0153] 3. The generated video data is returned to the server.

[0154] Step 5:

[0155] The server saves the generated video data in storage (e.g., AWS S3) and notifies the user. The input is the generated camera video data, and the output is the URL or identification information of the saved data and a notification message.

[0156] Specific behavior:

[0157] 1. The server saves the generated video data in storage.

[0158] 2. Receive the URL and identification information of the stored data from the storage.

[0159] 3. The server notifies the user that video generation is complete (via email or in-app notification).

[0160] Step 6:

[0161] The user selects the "annotation work" menu and requests the required video data. The input is the user's request information, and the output is the video data provided by the server.

[0162] Specific behavior:

[0163] 1. The user clicks on the "Annotation Tasks" menu.

[0164] 2. The user selects the required video data and sends a request.

[0165] 3. The server receives the request and extracts the corresponding data.

[0166] Step 7:

[0167] The server sends the requested video data from storage (e.g., Google Cloud Storage) to the user's device. The input is the identification information of the requested video data, and the output is the video data to be sent to the user's device.

[0168] Specific behavior:

[0169] 1. The server queries the storage for the requested video data.

[0170] 2. The storage returns the video data to the server.

[0171] 3. The server sends the video data to the user's device.

[0172] Step 8:

[0173] The user adds labels and annotations to the transmitted video data using an annotation tool (e.g., VGG Image Annotator). The input is the received video data, and the output is the annotated video data.

[0174] Specific behavior:

[0175] 1. The user imports the received video data into the annotation tool.

[0176] 2. The user adds labels and annotations to the video data.

[0177] 3. Upload the annotated data to the server.

[0178] Step 9:

[0179] The server stores the annotation data received from the user. The input is the annotated video data, and the output is the stored annotation data.

[0180] Specific behavior:

[0181] 1. The server receives annotation data uploaded by the user.

[0182] 2. The server saves the annotation data in storage.

[0183] Step 10:

[0184] The server collects the accumulated annotation data and uses machine learning algorithms (e.g., TensorFlow or PyTorch) to generate new learning models or update existing models. The input is the accumulated annotation data, and the output is the generated or updated learning model.

[0185] Specific behavior:

[0186] 1. The server collects the accumulated annotation data.

[0187] 2. The server runs a machine learning algorithm and uses the data to train a model.

[0188] 3. Generate a new training model or update an existing model.

[0189] 4. Save the generated or updated model to storage.

[0190] Step 11:

[0191] The server posts the generated training model and dataset on an online market. The input is the training model and dataset, and the output is the posting information on the online market.

[0192] Specific behavior:

[0193] 1. The server uploads the training model and dataset to the online market.

[0194] 2. Sales information will be published on the online market.

[0195] Step 12:

[0196] Users access the online marketplace and purchase the datasets and learning models they need. The input is the user's purchase request, and the output is the purchased datasets and learning models.

[0197] Specific behavior:

[0198] 1. A user visits an online marketplace.

[0199] 2. The user selects the required dataset and learning model and completes the purchase process.

[0200] 3. The server sends the purchasing dataset and model to the user's device.

[0201] (Application example 1)

[0202] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0203] It is extremely important to generate drone camera video data under a variety of conditions and apply it to the development of autonomous driving and other technologies. However, existing systems have limited functionality for generating video data in real time under specific conditions desired by users, resulting in a lack of flexibility and diversity in the video data provided. Furthermore, the annotation process for the generated video data is cumbersome for users, and there is a need for more efficient annotation. Furthermore, there is a lack of efficient means for selling the generated datasets and learning models online, making it difficult to share and use the data.

[0204] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0205] In this invention, the server includes: means for using a generation AI to generate drone camera video data; means for saving the generated camera video data; means for providing the saved camera video data to a user; means for generating prompt statements based on user-specified conditions and generating camera video data in real time according to the prompt statements; means for the user to annotate the video data; means for collecting the annotated data and generating a learning model using a machine learning algorithm; and means for providing an online market for selling the generated learning datasets and learning models. This allows users to generate the video data they need under specific conditions in real time, enabling efficient annotation work. Furthermore, the generated datasets and learning models can be easily shared and used.

[0206] Key Word Definitions

[0207] "Generative AI" refers to artificial intelligence that generates camera image data based on user-specified conditions.

[0208] "Camera video data" refers to video information captured by a drone.

[0209] A "prompt statement" is an instruction statement that causes the generation AI to generate camera image data based on conditions specified by the user.

[0210] "Annotation" refers to the process of adding labels and annotations to camera video data.

[0211] "Learning model" refers to a trained model constructed by a machine learning algorithm using the generated camera image data and annotation data.

[0212] "Online market" refers to an internet platform for selling generated training datasets and learning models.

[0213] A "smartphone" refers to a mobile device that has Internet connectivity and can run a variety of applications.

[0214] "Real-time" refers to the property of responding immediately to user operations and inputs.

[0215] "User" refers to any individual or organization that uses this system to generate, store, annotate, and use drone camera footage data.

[0216] MODE FOR CARRYING OUT THE INVENTION

[0217] This invention is a system for generating drone camera image data and using it for autonomous driving technology and other applications. This system uses generative AI to generate camera image data under various conditions, stores and sells the data, and utilizes annotation technology. One embodiment of the present invention is described below.

[0218] 1. System Configuration

[0219] This system is connected via a terminal accessible by a user, a server that processes data, and the Internet.

[0220] 2. Data generation phase

[0221] Users can log in to the application on the server from their terminal, enter the necessary parameters (such as the video resolution, duration, and specific scenes) from the "Camera Video Data Generation" menu, and click the "Start Generation" button.

[0222] The server receives the user's request, passes parameters to the generation AI, and starts the video data generation process. The generation AI generates the necessary camera video data and stores it in the server's storage.

[0223] The server notifies the user that the generation of the video data is complete.

[0224] 3. Data annotation phase

[0225] The user logs in to the server from a terminal, selects the "annotation work" menu, and requests the video data that requires annotation.

[0226] The server extracts the requested video data from the storage and transmits it to the user's terminal.

[0227] The user performs annotation work by adding labels and annotations to the transmitted video data, and then transmits the completed data to the server.

[0228] The server stores the received annotation data in storage.

[0229] 4. Machine Learning Phase

[0230] The server collects the accumulated annotation data and generates a learning model using a machine learning algorithm, which is then periodically updated.

[0231] Along with the learning model, the generated dataset is also saved for the next phase.

[0232] 5. Data sales phase

[0233] The server posts the generated training datasets and learning models on an online marketplace, making the data available to other development companies.

[0234] Users access the online market from their devices and purchase the necessary datasets and learning models.

[0235] After the purchase process is completed, the server sends the dataset and model to the user's device.

[0236] 6. Hardware and Software Used

[0237] Servers are responsible for data processing and storage, and therefore use server machines equipped with high-performance processors and large-capacity storage.

[0238] The software uses TensorFlow to build generative AI models and web application frameworks such as Flask for annotation work.

[0239] The user terminals used are mobile terminals such as smartphones and tablets.

[0240] 7. Specific Examples

[0241] For example, if a user wants a video of a "sunny day in the mountains," they would input the following prompt sentence into the generation AI:

[0242] Prompt: "Generate a camera image of a mountainous area on a clear day."

[0243] The server runs a generation AI based on this prompt text and generates camera image data that matches the specified conditions.

[0244] The generated video data is displayed in real time on the user's device, and the user can then perform annotations on the video, which are then stored on the server.

[0245] This system allows users to generate the video data they need under specific conditions in real time, enabling them to perform annotation work efficiently. Furthermore, the generated datasets and learning models can be used by other companies and research institutions, improving the shareability and usefulness of data.

[0246] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0247] Program processing flow

[0248] Step 1:

[0249] A user logs in to the system from a terminal. As input, the user provides login information (user ID, password, etc.). The server receives this information, performs authentication, and determines whether the user can log in. If authentication is successful, the user gains access to the system. As output, it sends a message to the terminal indicating whether authentication was successful or not.

[0250] Step 2:

[0251] The user selects the "Generate Camera Video Data" menu on the terminal and inputs the required parameters (e.g., video resolution, duration, specific scene, etc.). As input, these parameters are sent from the terminal to the server. The server receives these parameters and generates a prompt sentence. As output, the generated prompt sentence is stored internally.

[0252] Step 3:

[0253] The server passes the prompt sentence to the generation AI and starts generating camera video data. As input, the server inputs the prompt sentence into the generation AI. The generation AI generates the drone camera video data based on this prompt sentence. As data processing, the generation AI performs the calculations necessary to generate the video data. As output, it obtains the generated video data.

[0254] Step 4:

[0255] The server saves the generated camera image data in storage and sends a notification to the user that generation is complete. As input, it receives the generated camera image data. As data calculation, it performs a procedure to save this data in storage. As output, it sends a message to the user notifying them that generation is complete.

[0256] Step 5:

[0257] The user selects the "Annotation" menu on the terminal and requests camera video data to be analyzed. As input, the user sends the request information. The server receives this information, extracts the requested video data from storage, and sends it to the user's terminal. As output, it sends the corresponding video data to the user.

[0258] Step 6:

[0259] The user performs annotation work by adding labels and annotations to the transmitted video data. As input, the video data is displayed on the device and the user adds annotations. As data processing, the annotation data is incorporated into the video data. As output, annotated data is generated.

[0260] Step 7:

[0261] The user sends the data after annotation work is completed to the server. The annotated data is sent to the server as input. The server receives this data and saves it in storage. A saving process is executed as data calculation. The annotation data is saved on the server as output.

[0262] Step 8:

[0263] The server collects the accumulated annotation data and generates a learning model using a machine learning algorithm. The annotation data is collected as input. As data processing, the machine learning algorithm uses the annotation data as training data to generate a learning model. As output, a new learning model is generated.

[0264] Step 9:

[0265] The generated training dataset and model are posted on the online market. As input, a new training dataset and model are provided. The server adds them to the online market platform. As data processing, a new item is registered in the market list. As output, it is displayed as an item available to users.

[0266] Step 10:

[0267] A user accesses the online market from their device and purchases the required dataset or learning model. The user's purchase request is sent as input. The server processes the purchase and, once the purchase is complete, sends the dataset or model to the user's device. The data calculation involves purchase processing and distribution. The output is a notification that the purchase has been completed, along with the dataset or model, sent to the user.

[0268] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0269] This invention relates to a system for generating drone camera video data and applying it to autonomous driving technology. Furthermore, by combining it with an emotion engine that recognizes user emotions, the user experience can be improved. This system generates camera video data using generative AI, stores and sells the data, and is equipped with annotation technology and emotion analysis functions.

[0270] System Overview

[0271] 1. Data generation phase

[0272] The user logs in to the application on the server from their terminal, enters the necessary parameters (e.g., video resolution, duration, specific scenes, etc.) from the "Camera Video Data Generation" menu, and clicks the "Start Generation" button.

[0273] The server receives the user's request, passes parameters to the generation AI, and starts the video data generation process. The generation AI generates the necessary camera video data and stores it in the server's storage.

[0274] The server notifies the user that the generation of the video data is complete.

[0275] 2. Use of Emotion Engine

[0276] The server monitors the user's emotional state using an emotion engine, for example, by analyzing the user's facial expressions and voice through a camera or microphone to determine the user's emotional state.

[0277] The server adaptively adjusts the parameters of the video data to be generated based on the analysis results of the emotion engine. For example, if the user is relaxed, the server can adjust the parameters to generate a video rich in scenery.

[0278] 3. Data annotation phase

[0279] The user accesses the server from a terminal, selects the "annotation work" menu, and requests the video data that requires annotation.

[0280] The server extracts the requested video data from storage and sends it to the user's device, where the user can add labels and annotations to the data.

[0281] The server uses an emotion engine to monitor the user's emotional state during annotation work. If the user feels stressed or fatigued, an alert is displayed to prompt the user to stop working.

[0282] 4. Machine Learning Phase

[0283] The server collects the accumulated annotation data and uses machine learning algorithms to generate learning models, which are then used to improve the drone's autonomous driving technology.

[0284] Along with the learning model, the generated dataset is also saved for the next phase.

[0285] 5. Data sales phase

[0286] The server then posts the generated training datasets and models on an online marketplace, making the data available to other development companies.

[0287] Users access the online market from their devices and purchase the necessary datasets and learning models.

[0288] After the purchase process is completed, the server sends the dataset and model to the user's device.

[0289] Specific examples

[0290] For example, consider the case where a person in a logistics company generates camera footage from a drone. The person in charge accesses the server from a secure terminal and requests the generation of camera footage. The server uses the generation AI to generate the footage according to the request and stores it. The person in charge is then notified and can download the generated video data and review it internally.

[0291] Next, as the annotation worker begins the task, the emotion engine monitors the worker's emotional state. If stress or fatigue is detected during the task, the system will display an alert urging the worker to stop, thereby supporting health management. The emotion engine can also adjust the way data is presented to improve the efficiency of the annotation task.

[0292] The resulting datasets and learning models will be sold to other development companies to help develop autonomous drone technology, which is expected to lead to the rapid adoption of the technology and improved safety.

[0293] The processing flow will be explained below.

[0294] Step 1:

[0295] The user logs in to the application on the server from their terminal. The user selects the "Camera Video Data Generation" menu and inputs the parameters required for video generation (e.g., resolution, duration, specific scenes, etc.). Then, the user clicks the "Start Generation" button to send the request to the server.

[0296] Step 2:

[0297] The server receives the user's request and activates the emotion engine, which analyzes the user's facial expressions and voice via a camera and microphone to recognize the user's emotional state (e.g., relaxed, focused, stressed, etc.).

[0298] Step 3:

[0299] Based on the analysis results of the emotion engine, the server issues instructions to the generation AI with adaptively adjusted parameters. For example, if the user is relaxed, the server instructs the AI ​​to generate images rich in scenery.

[0300] Step 4:

[0301] The generation AI generates camera image data based on parameters passed from the server, and the generated image data is stored in the server's storage.

[0302] Step 5:

[0303] The server sends a notification to the user saying, "Camera video data has been generated." This notification includes an access link to the generated video data.

[0304] Step 6:

[0305] The user checks the notification on their device and downloads the generated camera video data from the provided link. The user can then use this data for their own purposes.

[0306] Step 7:

[0307] When a user wants to perform annotation, he or she accesses the server from the terminal, selects the "annotation work" menu, and requests the video data that requires annotation.

[0308] Step 8:

[0309] The server extracts the requested video data from storage and sends it to the user's device, where the user can add labels and annotations to the data.

[0310] Step 9:

[0311] The server continuously monitors the user's emotional state during annotation work. If the emotion engine analyzes the user's emotional state and detects stress or fatigue, the server displays an alert urging the user to stop working.

[0312] Step 10:

[0313] The user completes the annotation process on the device and sends the completed annotation data to the server, which then stores the received annotation data in storage and organizes it to improve the data quality.

[0314] Step 11:

[0315] The server collects the accumulated annotation data and uses machine learning algorithms to generate learning models, which are then used to improve the drone's autonomous driving technology.

[0316] Step 12:

[0317] The server stores the generated learning model in storage, periodically evaluates the model's performance, and updates the model by retraining it using the latest data.

[0318] Step 13:

[0319] The server then posts the generated training datasets and models on an online market, making them available to other development companies.

[0320] Step 14:

[0321] The user accesses the online market from their device, searches for and selects the required training dataset and training model, and then completes the purchase procedure, after which the server confirms the purchase details.

[0322] Step 15:

[0323] Once the server confirms that the purchase procedure has been completed, it sends the training dataset and model to the user's device, allowing the user to easily obtain the resources necessary for developing autonomous driving technology.

[0324] Example 2

[0325] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0326] Conventional systems that generate drone camera video data and apply it to autonomous driving technology lack the ability to generate video that takes the user's emotional state into account, and provide insufficient support for efficient annotation. Furthermore, it has been difficult to properly manage and quickly provide the generated datasets and learning models for sale. This has hindered improvements to the user experience and efficient data generation.

[0327] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: a means for a user to log in to the server and input parameters for video data generation; a means for a generative AI model to generate camera video data based on a user request; a means for saving the generated camera video data in the server's storage; a means for notifying the user of the completion of generation; a means for monitoring the user's emotional state using an emotion engine and adjusting video generation parameters; a means for the user to perform annotation work to label and add annotations to the video data; a means for collecting the annotated data and generating a learning model using a machine learning algorithm; a means for listing the generated learning dataset and learning model on an online market; and a means for the user to purchase the dataset and model and transmit them from the server to a terminal. This enables video generation that takes the user's emotional state into consideration and efficient annotation work, thereby improving the quality and convenience of the datasets and learning models that are generated and provided.

[0328] A "user" is a person or organization that accesses the system through a terminal and generates and annotates camera image data.

[0329] A "server" is the central part of the system, a computer environment that generates, stores, notifies, analyzes, and sells data.

[0330] "Generative AI Model" means the artificial intelligence algorithms and programs used to generate camera footage data based on user requests.

[0331] "Camera image data" refers to still image or video data generated by a generative AI model.

[0332] "Storage" refers to a data storage device for temporarily or permanently storing the generated camera image data.

[0333] An "emotion engine" is a function or program that analyzes the user's emotional state and adjusts system operation and image generation accordingly.

[0334] "Annotation work" refers to the work in which a user labels or annotates camera video data.

[0335] A "machine learning algorithm" is an algorithm that learns from annotated data and acquires new knowledge.

[0336] A "learning model" is a computer model for performing a specific task, generated by a machine learning algorithm.

[0337] An "online market" is an e-commerce platform for selling generated training datasets and learning models.

[0338] A "dataset" is a set of data collected for a specific purpose.

[0339] This invention provides a system for generating drone camera video data and applying it to autonomous driving technology. Furthermore, by combining it with an emotion engine that recognizes user emotions, the user experience can be improved. This system generates camera video data using a generative AI model, stores and sells the data, and is equipped with annotation technology and emotion analysis functions.

[0340] Data generation phase

[0341] The user must log in to the application on the server from their terminal. This will display the home screen, allowing the user to begin operating the system. From the "Camera Video Data Generation" menu, enter the required parameters (e.g., video resolution 1920x1080, duration 10 minutes, specific scene "forest", etc.) and click the "Start Generation" button.

[0342] The server receives the parameters entered by the user and passes them to a generative AI model (e.g., OpenAI's (registered trademark) DALL-E or GAN) to begin generating video data. The generated camera video data is saved in the server's storage, and the server sends a notification to the user when generation is complete. Notifications are sent via email or in-app notification.

[0343] Use of emotion engine

[0344] The server monitors the user's emotional state using an emotion engine (e.g., Amazon Rekognition, Microsoft® Azure® Emotion API). It analyzes the user's facial expressions and voice through a camera and microphone to determine the user's emotional state. Based on the analysis results of the emotion engine, the server adaptively adjusts the parameters of the video data to be generated. For example, if the user is relaxed, the server may adjust the parameters to generate a video rich in scenery.

[0345] Data annotation phase

[0346] The user accesses the server from their device, selects the "Annotation" menu, and requests the video data that requires annotation. The server extracts the requested video data from storage and sends it to the user's device. The user then labels and annotates the data.

[0347] The server uses an emotion engine to monitor the user's emotional state while annotating. If the user feels stressed or fatigued, an alert is displayed to prompt the user to stop working. This also supports the user's health management.

[0348] Machine Learning Phase

[0349] The server collects the accumulated annotation data and generates a learning model using machine learning algorithms (e.g., TensorFlow, PyTorch). This learning model is used to improve the drone's autonomous driving technology. The generated dataset is also stored along with the learning model for the next phase.

[0350] Data sales phase

[0351] The server posts the generated training datasets and learning models on an online market, where other development companies can use this data. Users can access the online market from their devices and purchase the datasets and learning models they need. After the purchase process is complete, the server sends the datasets and models to the user's device.

[0352] Specific examples

[0353] For example, a logistics company employee generates camera footage from a drone. The employee accesses the server from a secure device and requests the generation of camera footage. The server uses AI to generate the footage requested and stores it. The employee is then notified and can download the generated video data for internal review.

[0354] Next, as the annotation worker begins the task, the emotion engine monitors the worker's emotional state. If stress or fatigue is detected during the task, the system will display an alert urging the worker to stop, supporting the worker's health management. The emotion engine can also adjust the way data is presented to improve the efficiency of the annotation task.

[0355] The resulting datasets and learning models will be sold to other development companies to help develop autonomous drone technology, which is expected to lead to the rapid adoption of the technology and improved safety.

[0356] Prompt Sentence Examples

[0357] "To generate drone camera footage, a logistics company employee requests the start of generation, specifying a resolution of 1920x1080, a duration of 10 minutes, and a scene of 'warehouse work.' Upon receiving this request, the server uses the generation AI to generate and save the footage."

[0358] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0359] Step 1:

[0360] A user logs in to the server.

[0361] Input: User ID and password.

[0362] The server sends the user ID and password to the authentication server and verifies the authentication information.

[0363] Output: A successful authentication message is displayed on the device, and the home screen is displayed.

[0364] Step 2:

[0365] The user inputs parameters for generating video data and requests generation.

[0366] Input: video resolution (e.g. 1920x1080), duration (e.g. 10 minutes), scene (e.g. forest).

[0367] The server receives the parameters entered by the user and passes them to the generative AI model.

[0368] Output: The parameters are passed to the generative AI model, which starts generating video data.

[0369] Step 3:

[0370] The server generates video data using the generative AI model.

[0371] Input: The parameters passed by the user.

[0372] The server inputs parameters into a generative AI model (e.g., OpenAI's DALL-E or GAN) and generates camera image data.

[0373] Output: The generated video data is saved in the server storage.

[0374] Step 4:

[0375] The server notifies the user that generation is complete.

[0376] Input: Generated video data.

[0377] The server sends a generation completion notification to the user's device (e.g., email or in-app notification).

[0378] Output: A generation completion notification is displayed on the user's terminal.

[0379] Step 5:

[0380] The server monitors the user's emotional state using an emotion engine.

[0381] Input: User video and audio data.

[0382] The server uses an emotion engine (e.g., Amazon Rekognition, Microsoft Azure Emotion API) to analyze the user's facial expressions and voice.

[0383] Output: The user's emotional state is determined and this information is fed back to the generative AI model.

[0384] Step 6:

[0385] The server adjusts the parameters of the video data it generates based on the emotional state.

[0386] Input: The user's emotional state.

[0387] The server adjusts the parameters of the generative AI model based on the analysis results of the emotion engine.

[0388] Output: Video data adapted to the user's emotions is generated.

[0389] Step 7:

[0390] A user requests an annotation task.

[0391] Input: A request for video data stored on the server.

[0392] The server extracts the requested video data from the storage and transmits it to the user terminal.

[0393] Output: The video data is sent to the user's device.

[0394] Step 8:

[0395] Users add labels and annotations to the video data.

[0396] Input: Transmitted video data.

[0397] The user plays back the video data and adds labels and annotations to each frame.

[0398] Output: Annotation data with labels and annotations added is generated.

[0399] Step 9:

[0400] The server uses an emotion engine to monitor the user's emotional state during the annotation process.

[0401] Input: User video and audio data.

[0402] The server uses an emotion engine to analyze the user's emotional state in real time.

[0403] Output: If stress or fatigue is detected, an alert will be displayed to prompt a break from work.

[0404] Step 10:

[0405] The server collects the annotated data and generates a learning model.

[0406] Input: annotation data.

[0407] The server generates a learning model using a machine learning algorithm (e.g., TensorFlow, PyTorch).

[0408] Output: The generated learning model is saved on the server.

[0409] Step 11:

[0410] The server posts the generated datasets and learning models on an online marketplace.

[0411] Input: A training dataset and a training model.

[0412] The server registers the dataset and the learning model in an online marketplace.

[0413] Output: The dataset and the training model are published on an online marketplace.

[0414] Step 12:

[0415] A user purchases a dataset or model.

[0416] Input: Datasets and training models published on online marketplaces.

[0417] Users select the required datasets and learning models from the online market and complete the purchase process.

[0418] Output: The purchase is completed.

[0419] Step 13:

[0420] The server sends the dataset and model to the user's device.

[0421] Input: Purchased dataset and training model.

[0422] After the purchase procedure is completed, the server sends the dataset and model to the user's terminal.

[0423] Output: The dataset and model are sent to the user's device.

[0424] (Application example 2)

[0425] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0426] While conventional drone technology can generate and store camera video data, it lacks a consistent process for annotating the video data and generating learning models using machine learning algorithms. Furthermore, the user experience is hindered by a lack of functionality for providing video data that takes into account the user's emotional state and for adaptively displaying real-time traffic information. This invention aims to solve these issues and provide more advanced drone technology.

[0427] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0428] a means for using generative AI to generate drone camera footage data;

[0429] a means for storing the generated camera image data;

[0430] means for providing the stored camera image data to a user;

[0431] A means for allowing a user to perform annotation work on video data;

[0432] a means for collecting the annotated data and generating a learning model using a machine learning algorithm;

[0433] a means for providing an online marketplace for selling the generated training datasets and training models;

[0434] emotion analysis means for recognizing a user's emotion and adaptively adjusting parameters of the generated video data;

[0435] means for providing real-time traffic information and displaying information according to the emotional state of the user;

[0436] This makes it possible to provide adaptive video data according to the user's emotional state and display traffic information in real time.

[0437] A "drone" is a type of unmanned aerial vehicle (UAV) that is equipped with remote control and autonomous flight capabilities and can capture camera footage from the air.

[0438] "Camera video data" is digital data of images or videos captured by a camera, and is data that includes visual information.

[0439] "Generative AI" refers to a system that uses artificial intelligence technology to automatically perform specific tasks, and in this context refers to the technology that automatically generates drone camera footage data.

[0440] "Storage means" refers to a function or device that temporarily or long-term stores the generated data.

[0441] "Providing means" refers to the functions and interfaces that allow users to access the stored data.

[0442] "Annotation work" refers to the work in which a user adds labels and annotations to video data.

[0443] "Machine learning algorithm" refers to a computational method or procedure for analyzing data and automatically building a learning model.

[0444] A "learning model" is a model generated using a machine learning algorithm, and is a data structure that contains knowledge and patterns for performing a specific task.

[0445] "Online market" refers to a platform for buying and selling goods and services over the Internet.

[0446] "Emotion analysis means" refers to functions and technologies that analyze a user's facial expressions, voice, etc. to determine their emotional state.

[0447] "Traffic information" refers to real-time or historical data on roads and traffic networks, including traffic volume, congestion, and accident information.

[0448] "Means for displaying information" refers to devices such as displays and speakers for providing information visually or audibly to a user.

[0449] This invention relates to a system for generating drone camera image data and applying it to autonomous driving technology and traffic information provision. The specific configuration and implementation method of the system are described below.

[0450] 1. System Configuration

[0451] The system consists of the following major components:

[0452] Drones equipped with generative AI

[0453] server

[0454] User devices (e.g., smartphones and PCs)

[0455] Sentiment Analysis Engine

[0456] Online Market Platform

[0457] 2. Program processing and hardware / software used

[0458] The server uses generative AI to generate drone camera footage data. The generated data is stored in the server's storage. Users can access and view this data from their smartphones or PCs. Users can then perform annotation work, and the data is collected again by the server. Using this annotation data, the server runs a machine learning algorithm to generate a learning model. The generated learning dataset and learning model are then sold on the online market.

[0459] Furthermore, by using an emotion analysis engine, the user's emotional state can be analyzed in real time and the video data parameters can be adjusted accordingly. Specifically, if the user is feeling emotionally stressed, adaptive adjustments can be made, such as providing videos with a relaxing effect.

[0460] The real-time traffic information sent from the drone is displayed on a smartphone or PC in a format that corresponds to the user's emotional state. For example, if the user is feeling irritated, information on relaxing routes and congestion avoidance will be displayed.

[0461] Specific examples of the hardware and software used are as follows:

[0462] Drones: General commercial drones (e.g., DJI Phantom)

[0463] Server: Cloud environment (e.g., Amazon Web Services, Google Cloud Platform)

[0464] Sentiment analysis engine: OpenCV, Keras

[0465] Machine learning algorithms: TensorFlow, PyTorch

[0466] 3. Examples of concrete examples and prompts

[0467] For example, consider the use of this system while a user is driving a car. The user uses a smartphone to receive real-time traffic information from a drone. The emotion analysis engine detects when the user is irritated and suggests a relaxing route or notifies the user of rest spots. This improves the user's driving experience and safety.

[0468] Example prompt sentence:

[0469] "Build an application that provides real-time traffic information from surveillance footage. It should also have the ability to determine the user's emotions and adaptively provide information as needed."

[0470] This will enable the realization of an advanced system that combines drone technology with emotion analysis, significantly improving the user experience.

[0471] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0472] Step 1:

[0473] The user logs in to the application on the server from their terminal. User authentication information (user ID and password) is required as input. The server receives the authentication information and performs user authentication. If authentication is successful, the dashboard screen is displayed to the user.

[0474] Step 2:

[0475] The user selects the "Generate Camera Video Data" menu from the dashboard screen. The parameters of the video data to be generated (resolution, duration, specific scene, etc.) are required as input. The server receives the user's request, passes these parameters to the generation AI, and begins the video data generation process.

[0476] Step 3:

[0477] The generation AI generates camera image data based on input parameters. It processes the data by performing the necessary modeling and simulations to create image data that meets the specified conditions. The generated image data is then stored in the server's storage.

[0478] Step 4:

[0479] The server notifies the user that the video data has been generated, and the user can access the server from their device to check that the video data is now available for download.

[0480] Step 5:

[0481] Users annotate the downloaded video data. The type and content of the annotation (labels, tags, annotations, etc.) are required as input. The annotation editing tool is used on the device to label and annotate the video data.

[0482] Step 6:

[0483] Once the annotation process is complete, the video data is uploaded back to the server. The server then collects the uploaded data and stores it in storage. It also uses an emotion analysis engine to monitor the user's emotional state and displays appropriate feedback as needed.

[0484] Step 7:

[0485] The server provides the collected annotation data to a machine learning algorithm to generate a learning model. The annotation data is required as input. As data processing, the algorithm learns patterns in the data and builds a prediction model or classification model.

[0486] Step 8:

[0487] The generated learning models and datasets are posted on an online marketplace. The server uploads these datasets to the marketplace and makes them available for users to view.

[0488] Step 9:

[0489] Users access the online marketplace and purchase the necessary learning models and datasets. Once the purchase process is complete, the server provides the purchased data to the user's device.

[0490] Step 10:

[0491] The server provides real-time traffic information and displays information according to the user's emotional state. Real-time video data from the drone and emotion analysis results are required as input. This allows the user to assess real-time traffic conditions and select the optimal travel route.

[0492] This completes the entire system processing, allowing users to use advanced drone technology to generate video data, annotate it, utilize learning models, and receive traffic information.

[0493] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0494] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0495] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0496] [Second embodiment]

[0497] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0498] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0499] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0500] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0501] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0502] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0503] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0504] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0505] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0506] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0507] In the smart glasses 214, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0508] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0509] This invention is a system for generating drone camera image data and using it for autonomous driving technology. This system uses generative AI to generate a variety of camera image data, store and sell that data, and accumulate annotation technology.

[0510] System Overview

[0511] 1. Data generation phase

[0512] The user logs in to the application on the server from their terminal, enters the necessary parameters (e.g., video resolution, duration, specific scenes, etc.) from the "Camera Video Data Generation" menu, and clicks the "Start Generation" button.

[0513] The server receives the user's request, passes parameters to the generation AI, and starts the video data generation process. The generation AI generates the necessary camera video data and stores it in the server's storage.

[0514] The server notifies the user that the generation of the video data is complete.

[0515] 2. Data annotation phase

[0516] The user logs in to the server from a terminal, selects the "annotation work" menu, and requests the video data that requires annotation.

[0517] The server extracts the requested video data from the storage and transmits it to the user's terminal.

[0518] The user performs annotation work by adding labels and annotations to the transmitted video data, and then transmits the completed data to the server.

[0519] The server stores the received annotation data in storage.

[0520] 3. Machine Learning Phase

[0521] The server collects the accumulated annotation data and generates a learning model using a machine learning algorithm, which is then periodically updated.

[0522] Along with the learning model, the generated dataset is also saved for the next phase.

[0523] 4. Data sales phase

[0524] The server then posts the generated training datasets and models on an online marketplace, making the data available to other development companies.

[0525] Users access the online market from their devices and purchase the necessary datasets and learning models.

[0526] After the purchase process is completed, the server sends the dataset and model to the user's device.

[0527] Specific examples

[0528] For example, consider the case where a person in a logistics company generates camera footage from a drone. The person in charge accesses the server from a secure terminal and requests the generation of camera footage. The server uses the generation AI to generate the footage according to the request and stores it. The person in charge is then notified and can download the generated video data and review it internally.

[0529] Furthermore, annotation work, which involves labeling and annotating this data, can be easily performed. This annotation data can be sent back to the server and used as training data for machine learning to enable autonomous driving for other drones. Ultimately, the generated learning models and datasets can be sold on online markets, allowing other companies to use this technology.

[0530] As such, this invention is a system that comprehensively supports everything from drone camera image data generation to annotation, machine learning, and data sales, which is expected to improve technology and efficiency in the drone industry.

[0531] The processing flow will be explained below.

[0532] Step 1:

[0533] The user logs in to the application on the server from their terminal. The user selects the "Camera Video Data Generation" menu and inputs the parameters required for video generation (e.g., resolution, duration, specific scenes, etc.). Then, the user clicks the "Start Generation" button to send the request to the server.

[0534] Step 2:

[0535] The server receives the user's request and passes parameters to the generation AI, which then generates camera image data based on these parameters and stores the generated data in the server's storage.

[0536] Step 3:

[0537] The server sends a notification to the user saying, "Camera video data has been generated." This notification includes an access link to the generated video data.

[0538] Step 4:

[0539] The user checks the notification on their device and downloads the generated camera video data from the provided link. The user can use this data for their own purposes.

[0540] Step 5:

[0541] When a user wants to perform annotation, he or she accesses the server from the terminal, selects the "annotation work" menu, and requests the video data that requires annotation.

[0542] Step 6:

[0543] The server extracts the requested video data from storage and sends it to the user's device, where the user can add labels and annotations to the data.

[0544] Step 7:

[0545] The user completes the annotation process on the device and sends the completed annotation data to the server, which then stores the received annotation data in storage and organizes it to improve the data quality.

[0546] Step 8:

[0547] The server collects the accumulated annotation data and generates a learning model using a machine learning algorithm, which is then used to improve the drone's autonomous driving technology.

[0548] Step 9:

[0549] The server stores the generated learning model in storage, periodically evaluates the model's performance, and updates the model by retraining it using the latest data.

[0550] Step 10:

[0551] The server then posts the generated training datasets and models on an online market, making them available to other development companies.

[0552] Step 11:

[0553] The user accesses the online market from their device, searches for and selects the required training dataset and training model, and then completes the purchase procedure, after which the server confirms the purchase details.

[0554] Step 12:

[0555] Once the server confirms that the purchase procedure has been completed, it sends the training dataset and model to the user's device, allowing the user to easily obtain the resources necessary for developing autonomous driving technology.

[0556] Example 1

[0557] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0558] Conventionally, systems that generate drone camera video data and use it to improve autonomous driving technology have had several problems. Specifically, the entire process from data generation to annotation and machine learning model generation is often done manually, which is time-consuming and labor-intensive. Additionally, the generated data and models are often sold individually, making it difficult to manage and use them all at once.

[0559] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0560] In this invention, the server includes a means for a user to log in to an application on the server from a terminal, a means for generating drone camera image data using a generative AI model based on a user request, and a means for saving the generated camera image data, thereby automating a series of processes and enabling efficient generation and management of high-quality data.

[0561] A "user" is an entity that uses this system to generate camera image data, perform annotation work, and purchase datasets and learning models.

[0562] A "terminal" is a computing device that a user uses to access applications on a server.

[0563] The "server" is a system that uses a generative AI model based on user requests to generate and store camera image data, and also collects annotation data and performs machine learning.

[0564] A "generative AI model" is an artificial intelligence algorithm that generates camera image data based on parameters specified by the user.

[0565] A "prompt sentence" is an instruction sentence that includes parameters required for generating camera image data input by the user.

[0566] "Camera video data" refers to digital video data generated by AI that imitates or generates video captured by a drone.

[0567] "Annotation" is the process of adding labels and annotations to video data.

[0568] A "machine learning algorithm" is a computational method for building and updating a learning model based on collected annotation data.

[0569] A "learning model" is a model generated by a machine learning algorithm to analyze data and perform a specific task.

[0570] A "dataset" is a collection of annotated video data used to train and test a learning model.

[0571] The "online market" is an e-commerce platform that provides generated training datasets and models to users for purchase and download.

[0572] This invention is a system for generating drone camera image data and using it for autonomous driving technology. This system uses a generative AI model to generate diverse camera image data, store and sell that data, and accumulate annotation technology.

[0573] Data generation phase

[0574] 1. The user accesses the application on the server from their terminal and logs in by entering their login information (user ID and password). An authentication server is used for logging in.

[0575] 2. The user selects the "Camera Video Data Generation" menu from the main menu and enters the required parameters (e.g., video resolution, duration, specific scene, etc.).

[0576] 3. The server receives the user's request and passes parameters to the generative AI model (e.g., GPT-4) to start the video data generation process. The prompt uses the following format:

[0577] "Resolution: 1920x1080" "Duration: 10 minutes" "Scene: Urban traffic"

[0578] 4. The generative AI model generates camera image data based on the input parameters. The generated data is stored in the server's storage (e.g., AWS S3).

[0579] 5. The server notifies the user that the video data has been generated. This notification is sent via email or in-app notification.

[0580] Data annotation phase

[0581] 1. The user accesses the server from their terminal, selects the "Annotation" menu, and requests the video data that requires annotation.

[0582] 2. The server extracts the requested video data from storage (e.g., Google Cloud Storage) and sends it to the user's device.

[0583] 3. The user adds labels and annotations to the transmitted video data using an annotation tool such as VGG Image Annotator.

[0584] 4. The user uploads the annotated data to the server, which then stores the received annotation data in its storage.

[0585] Machine Learning Phase

[0586] 1. The server collects the accumulated annotation data and generates or updates a learning model using a machine learning algorithm (e.g., TensorFlow or PyTorch).

[0587] 2. The server stores the generated dataset along with the generated learning model in preparation for the next phase.

[0588] Data sales phase

[0589] 1. The server publishes the generated training dataset and model on an online marketplace, making the data available to other users.

[0590] 2. The user accesses the online market from their device and purchases the necessary datasets and learning models.

[0591] 3. After the purchase procedure is completed, the server sends the relevant dataset and model to the user's device.

[0592] Specific examples

[0593] When a logistics company employee wants to generate drone camera footage, they access the server from a secure device and request the generation of camera footage. This request includes a prompt like this:

[0594] "Resolution: 1920x1080" "Duration: 10 minutes" "Scene: Urban traffic"

[0595] The server uses the generative AI model to generate and store the video according to the request, after which the person in charge can download the video data and review it in-house.

[0596] Furthermore, annotation work, which involves labeling and annotating this data, can be easily performed. This annotation data can be sent back to the server and used as training data for machine learning to enable autonomous driving for other drones. Ultimately, the generated learning models and datasets can be sold on online markets, allowing other companies to use this technology.

[0597] As such, this invention is a system that comprehensively supports everything from drone camera image data generation to annotation, machine learning, and data sales, which is expected to improve technology and efficiency in the drone industry.

[0598] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0599] Step 1:

[0600] A user accesses an application on the server from a terminal and logs in by entering login information (user ID and password). The input is the user ID and password, which the authentication server receives and performs authentication processing. The output is the authentication result, and if the login is successful, the main menu screen is displayed.

[0601] Specific behavior:

[0602] 1. The user displays the login page on the terminal.

[0603] 2. Enter your user ID and password.

[0604] 3. The server sends a query to the authentication server.

[0605] 4. The authentication server returns the authentication result.

[0606] 5. If the server is successful in authenticating, it displays the main menu screen to the user.

[0607] Step 2:

[0608] The user selects the "Camera Video Data Generation" menu from the main menu and inputs parameters such as video resolution, duration, and specific scenes. The inputs are the video generation parameters. The server receives the input parameters and passes them to the generative AI model.

[0609] Specific behavior:

[0610] 1. The user clicks on the "Generate camera video data" menu.

[0611] 2. The user inputs parameters such as video resolution (e.g., 1920x1080), duration (e.g., 10 minutes), and scene (e.g., urban traffic).

[0612] 3. The user clicks the "Start Generation" button.

[0613] 4. The server retrieves the input parameters.

[0614] Step 3:

[0615] The server generates a prompt sentence from the acquired parameters and sends a request to the generative AI model. The input is the generated parameters, and the output is to generate a prompt sentence and pass it to the generative AI model. The generative AI model generates video data based on this prompt sentence.

[0616] Specific behavior:

[0617] 1. The server generates a prompt using the parameters it has obtained.

[0618] For example: "Resolution: 1920x1080", "Duration: 10 minutes", "Scene: City traffic".

[0619] 2. The server sends the generated prompt sentence to the generative AI model.

[0620] Step 4:

[0621] The generative AI model generates camera video data based on the prompt sentence and returns the generated data to the server. The input is the prompt sentence and the output is the generated camera video data. This data is sent to the server.

[0622] Specific behavior:

[0623] 1. The generative AI model parses the prompt.

[0624] 2. The AI ​​model uses its internal algorithm to generate video data.

[0625] 3. The generated video data is returned to the server.

[0626] Step 5:

[0627] The server saves the generated video data in storage (e.g., AWS S3) and notifies the user. The input is the generated camera video data, and the output is the URL or identification information of the saved data and a notification message.

[0628] Specific behavior:

[0629] 1. The server saves the generated video data in storage.

[0630] 2. Receive the URL and identification information of the stored data from the storage.

[0631] 3. The server notifies the user that video generation is complete (via email or in-app notification).

[0632] Step 6:

[0633] The user selects the "annotation work" menu and requests the required video data. The input is the user's request information, and the output is the video data provided by the server.

[0634] Specific behavior:

[0635] 1. The user clicks on the "Annotation Tasks" menu.

[0636] 2. The user selects the required video data and sends a request.

[0637] 3. The server receives the request and extracts the corresponding data.

[0638] Step 7:

[0639] The server sends the requested video data from storage (e.g., Google Cloud Storage) to the user's device. The input is the identification information of the requested video data, and the output is the video data to be sent to the user's device.

[0640] Specific behavior:

[0641] 1. The server queries the storage for the requested video data.

[0642] 2. The storage returns the video data to the server.

[0643] 3. The server sends the video data to the user's device.

[0644] Step 8:

[0645] The user adds labels and annotations to the transmitted video data using an annotation tool (e.g., VGG Image Annotator). The input is the received video data, and the output is the annotated video data.

[0646] Specific behavior:

[0647] 1. The user imports the received video data into the annotation tool.

[0648] 2. The user adds labels and annotations to the video data.

[0649] 3. Upload the annotated data to the server.

[0650] Step 9:

[0651] The server stores the annotation data received from the user. The input is the annotated video data, and the output is the stored annotation data.

[0652] Specific behavior:

[0653] 1. The server receives annotation data uploaded by the user.

[0654] 2. The server saves the annotation data in storage.

[0655] Step 10:

[0656] The server collects the accumulated annotation data and uses machine learning algorithms (e.g., TensorFlow or PyTorch) to generate new learning models or update existing models. The input is the accumulated annotation data, and the output is the generated or updated learning model.

[0657] Specific behavior:

[0658] 1. The server collects the accumulated annotation data.

[0659] 2. The server runs a machine learning algorithm and uses the data to train a model.

[0660] 3. Generate a new training model or update an existing model.

[0661] 4. Save the generated or updated model to storage.

[0662] Step 11:

[0663] The server posts the generated training model and dataset on an online market. The input is the training model and dataset, and the output is the posting information on the online market.

[0664] Specific behavior:

[0665] 1. The server uploads the training model and dataset to the online market.

[0666] 2. Sales information will be published on the online market.

[0667] Step 12:

[0668] Users access the online marketplace and purchase the datasets and learning models they need. The input is the user's purchase request, and the output is the purchased datasets and learning models.

[0669] Specific behavior:

[0670] 1. A user visits an online marketplace.

[0671] 2. The user selects the required dataset and learning model and completes the purchase process.

[0672] 3. The server sends the purchasing dataset and model to the user's device.

[0673] (Application example 1)

[0674] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0675] It is extremely important to generate drone camera video data under a variety of conditions and apply it to the development of autonomous driving and other technologies. However, existing systems have limited functionality for generating video data in real time under specific conditions desired by users, resulting in a lack of flexibility and diversity in the video data provided. Furthermore, the annotation process for the generated video data is cumbersome for users, and there is a need for more efficient annotation. Furthermore, there is a lack of efficient means for selling the generated datasets and learning models online, making it difficult to share and use the data.

[0676] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0677] In this invention, the server includes: means for using a generation AI to generate drone camera video data; means for saving the generated camera video data; means for providing the saved camera video data to a user; means for generating prompt statements based on user-specified conditions and generating camera video data in real time according to the prompt statements; means for the user to annotate the video data; means for collecting the annotated data and generating a learning model using a machine learning algorithm; and means for providing an online market for selling the generated learning datasets and learning models. This allows users to generate the video data they need under specific conditions in real time, enabling efficient annotation work. Furthermore, the generated datasets and learning models can be easily shared and used.

[0678] Key Word Definitions

[0679] "Generative AI" refers to artificial intelligence that generates camera image data based on user-specified conditions.

[0680] "Camera video data" refers to video information captured by a drone.

[0681] A "prompt statement" is an instruction statement that causes the generation AI to generate camera image data based on conditions specified by the user.

[0682] "Annotation" refers to the process of adding labels and annotations to camera video data.

[0683] "Learning model" refers to a trained model constructed by a machine learning algorithm using the generated camera image data and annotation data.

[0684] "Online market" refers to an internet platform for selling generated training datasets and learning models.

[0685] A "smartphone" refers to a mobile device that has Internet connectivity and can run a variety of applications.

[0686] "Real-time" refers to the property of responding immediately to user operations and inputs.

[0687] "User" refers to any individual or organization that uses this system to generate, store, annotate, and use drone camera footage data.

[0688] MODE FOR CARRYING OUT THE INVENTION

[0689] This invention is a system for generating drone camera image data and using it for autonomous driving technology and other applications. This system uses generative AI to generate camera image data under various conditions, stores and sells the data, and utilizes annotation technology. One embodiment of the present invention is described below.

[0690] 1. System Configuration

[0691] This system is connected via a terminal accessible by a user, a server that processes data, and the Internet.

[0692] 2. Data generation phase

[0693] Users can log in to the application on the server from their terminal, enter the necessary parameters (such as the video resolution, duration, and specific scenes) from the "Camera Video Data Generation" menu, and click the "Start Generation" button.

[0694] The server receives the user's request, passes parameters to the generation AI, and starts the video data generation process. The generation AI generates the necessary camera video data and stores it in the server's storage.

[0695] The server notifies the user that the generation of the video data is complete.

[0696] 3. Data annotation phase

[0697] The user logs in to the server from a terminal, selects the "annotation work" menu, and requests the video data that requires annotation.

[0698] The server extracts the requested video data from the storage and transmits it to the user's terminal.

[0699] The user performs annotation work by adding labels and annotations to the transmitted video data, and then transmits the completed data to the server.

[0700] The server stores the received annotation data in storage.

[0701] 4. Machine Learning Phase

[0702] The server collects the accumulated annotation data and generates a learning model using a machine learning algorithm, which is then periodically updated.

[0703] Along with the learning model, the generated dataset is also saved for the next phase.

[0704] 5. Data sales phase

[0705] The server posts the generated training datasets and learning models on an online marketplace, making the data available to other development companies.

[0706] Users access the online market from their devices and purchase the necessary datasets and learning models.

[0707] After the purchase process is completed, the server sends the dataset and model to the user's device.

[0708] 6. Hardware and Software Used

[0709] Servers are responsible for processing and storing data, and therefore use server machines equipped with high-performance processors and large-capacity storage.

[0710] The software uses TensorFlow to build generative AI models and web application frameworks such as Flask for annotation work.

[0711] The user terminals used are mobile terminals such as smartphones and tablets.

[0712] 7. Specific Examples

[0713] For example, if a user wants a video of a "sunny day in the mountains," they would input the following prompt sentence into the generation AI:

[0714] Prompt: "Generate a camera image of a mountainous area on a clear day."

[0715] The server runs a generation AI based on this prompt text and generates camera image data that matches the specified conditions.

[0716] The generated video data is displayed in real time on the user's device, and the user can then perform annotations on the video, which are then stored on the server.

[0717] This system allows users to generate the video data they need under specific conditions in real time, enabling them to perform annotation work efficiently. Furthermore, the generated datasets and learning models can be used by other companies and research institutions, improving the shareability and usefulness of data.

[0718] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0719] Program processing flow

[0720] Step 1:

[0721] A user logs in to the system from a terminal. As input, the user provides login information (user ID, password, etc.). The server receives this information, performs authentication, and determines whether the user can log in. If authentication is successful, the user gains access to the system. As output, it sends a message to the terminal indicating whether authentication was successful or not.

[0722] Step 2:

[0723] The user selects the "Generate Camera Video Data" menu on the terminal and inputs the required parameters (e.g., video resolution, duration, specific scene, etc.). As input, these parameters are sent from the terminal to the server. The server receives these parameters and generates a prompt sentence. As output, the generated prompt sentence is stored internally.

[0724] Step 3:

[0725] The server passes the prompt sentence to the generation AI and starts generating camera video data. As input, the server inputs the prompt sentence into the generation AI. The generation AI generates the drone camera video data based on this prompt sentence. As data processing, the generation AI performs the calculations necessary to generate the video data. As output, it obtains the generated video data.

[0726] Step 4:

[0727] The server saves the generated camera image data in storage and sends a notification to the user that generation is complete. As input, it receives the generated camera image data. As data calculation, it performs a procedure to save this data in storage. As output, it sends a message to the user notifying them that generation is complete.

[0728] Step 5:

[0729] The user selects the "Annotation" menu on the terminal and requests camera video data to be analyzed. As input, the user sends the request information. The server receives this information, extracts the requested video data from storage, and sends it to the user's terminal. As output, it sends the corresponding video data to the user.

[0730] Step 6:

[0731] The user performs annotation work by adding labels and annotations to the transmitted video data. As input, the video data is displayed on the device and the user adds annotations. As data processing, the annotation data is incorporated into the video data. As output, annotated data is generated.

[0732] Step 7:

[0733] The user sends the data after annotation work is completed to the server. The annotated data is sent to the server as input. The server receives this data and saves it in storage. A saving process is executed as data calculation. The annotation data is saved on the server as output.

[0734] Step 8:

[0735] The server collects the accumulated annotation data and generates a learning model using a machine learning algorithm. The annotation data is collected as input. As data processing, the machine learning algorithm uses the annotation data as training data to generate a learning model. As output, a new learning model is generated.

[0736] Step 9:

[0737] The generated training dataset and model are posted on the online market. As input, a new training dataset and model are provided. The server adds them to the online market platform. As data processing, a new item is registered in the market list. As output, it is displayed as an item available to users.

[0738] Step 10:

[0739] A user accesses the online market from their device and purchases the required dataset or learning model. The user's purchase request is sent as input. The server processes the purchase and, once the purchase is complete, sends the dataset or model to the user's device. The data calculation involves purchase processing and distribution. The output is a notification that the purchase has been completed, along with the dataset or model, sent to the user.

[0740] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0741] This invention relates to a system for generating drone camera video data and applying it to autonomous driving technology. Furthermore, by combining it with an emotion engine that recognizes user emotions, the user experience can be improved. This system generates camera video data using generative AI, stores and sells the data, and is equipped with annotation technology and emotion analysis functions.

[0742] System Overview

[0743] 1. Data generation phase

[0744] The user logs in to the application on the server from their terminal, enters the necessary parameters (e.g., video resolution, duration, specific scenes, etc.) from the "Camera Video Data Generation" menu, and clicks the "Start Generation" button.

[0745] The server receives the user's request, passes parameters to the generation AI, and starts the video data generation process. The generation AI generates the necessary camera video data and stores it in the server's storage.

[0746] The server notifies the user that the generation of the video data is complete.

[0747] 2. Use of Emotion Engine

[0748] The server monitors the user's emotional state using an emotion engine, for example, by analyzing the user's facial expressions and voice through a camera or microphone to determine the user's emotional state.

[0749] The server adaptively adjusts the parameters of the video data to be generated based on the analysis results of the emotion engine. For example, if the user is relaxed, the server can adjust the parameters to generate a video rich in scenery.

[0750] 3. Data annotation phase

[0751] The user accesses the server from a terminal, selects the "annotation work" menu, and requests the video data that requires annotation.

[0752] The server extracts the requested video data from storage and sends it to the user's device, where the user can add labels and annotations to the data.

[0753] The server uses an emotion engine to monitor the user's emotional state during annotation work. If the user feels stressed or fatigued, an alert is displayed to prompt the user to stop working.

[0754] 4. Machine Learning Phase

[0755] The server collects the accumulated annotation data and uses machine learning algorithms to generate learning models, which are then used to improve the drone's autonomous driving technology.

[0756] Along with the learning model, the generated dataset is also saved for the next phase.

[0757] 5. Data sales phase

[0758] The server then posts the generated training datasets and models on an online marketplace, making the data available to other development companies.

[0759] Users access the online market from their devices and purchase the necessary datasets and learning models.

[0760] After the purchase process is completed, the server sends the dataset and model to the user's device.

[0761] Specific examples

[0762] For example, consider the case where a person in a logistics company generates camera footage from a drone. The person in charge accesses the server from a secure terminal and requests the generation of camera footage. The server uses the generation AI to generate the footage according to the request and stores it. The person in charge is then notified and can download the generated video data and review it internally.

[0763] Next, as the annotation worker begins the task, the emotion engine monitors the worker's emotional state. If stress or fatigue is detected during the task, the system will display an alert urging the worker to stop, thereby supporting health management. The emotion engine can also adjust the way data is presented to improve the efficiency of the annotation task.

[0764] The resulting datasets and learning models will be sold to other development companies to help develop autonomous drone technology, which is expected to lead to the rapid adoption of the technology and improved safety.

[0765] The processing flow will be explained below.

[0766] Step 1:

[0767] The user logs in to the application on the server from their terminal. The user selects the "Camera Video Data Generation" menu and inputs the parameters required for video generation (e.g., resolution, duration, specific scenes, etc.). Then, the user clicks the "Start Generation" button to send the request to the server.

[0768] Step 2:

[0769] The server receives the user's request and activates the emotion engine, which analyzes the user's facial expressions and voice via a camera and microphone to recognize the user's emotional state (e.g., relaxed, focused, stressed, etc.).

[0770] Step 3:

[0771] Based on the analysis results of the emotion engine, the server issues instructions to the generation AI with adaptively adjusted parameters. For example, if the user is relaxed, the server instructs the AI ​​to generate images rich in scenery.

[0772] Step 4:

[0773] The generation AI generates camera image data based on parameters passed from the server, and the generated image data is stored in the server's storage.

[0774] Step 5:

[0775] The server sends a notification to the user saying, "Camera video data has been generated." This notification includes an access link to the generated video data.

[0776] Step 6:

[0777] The user checks the notification on their device and downloads the generated camera video data from the provided link. The user can then use this data for their own purposes.

[0778] Step 7:

[0779] When a user wants to perform annotation, he or she accesses the server from the terminal, selects the "annotation work" menu, and requests the video data that requires annotation.

[0780] Step 8:

[0781] The server extracts the requested video data from storage and sends it to the user's device, where the user can add labels and annotations to the data.

[0782] Step 9:

[0783] The server continuously monitors the user's emotional state during annotation work. If the emotion engine analyzes the user's emotional state and detects stress or fatigue, the server displays an alert urging the user to stop working.

[0784] Step 10:

[0785] The user completes the annotation process on the device and sends the completed annotation data to the server, which then stores the received annotation data in storage and organizes it to improve the data quality.

[0786] Step 11:

[0787] The server collects the accumulated annotation data and uses machine learning algorithms to generate learning models, which are then used to improve the drone's autonomous driving technology.

[0788] Step 12:

[0789] The server stores the generated learning model in storage, periodically evaluates the model's performance, and updates the model by retraining it using the latest data.

[0790] Step 13:

[0791] The server then posts the generated training datasets and models on an online market, making them available to other development companies.

[0792] Step 14:

[0793] The user accesses the online market from their device, searches for and selects the required training dataset and training model, and then completes the purchase procedure, after which the server confirms the purchase details.

[0794] Step 15:

[0795] Once the server confirms that the purchase procedure has been completed, it sends the training dataset and model to the user's device, allowing the user to easily obtain the resources necessary for developing autonomous driving technology.

[0796] Example 2

[0797] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0798] Conventional systems that generate drone camera video data and apply it to autonomous driving technology lack the ability to generate video that takes the user's emotional state into account, and provide insufficient support for efficient annotation. Furthermore, it has been difficult to properly manage and quickly provide the generated datasets and learning models for sale. This has hindered improvements to the user experience and efficient data generation.

[0799] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: a means for a user to log in to the server and input parameters for video data generation; a means for a generative AI model to generate camera video data based on a user request; a means for saving the generated camera video data in the server's storage; a means for notifying the user of the completion of generation; a means for monitoring the user's emotional state using an emotion engine and adjusting video generation parameters; a means for the user to perform annotation work to label and add annotations to the video data; a means for collecting the annotated data and generating a learning model using a machine learning algorithm; a means for listing the generated learning dataset and learning model on an online market; and a means for the user to purchase the dataset and model and transmit them from the server to a terminal. This enables video generation that takes the user's emotional state into consideration and efficient annotation work, thereby improving the quality and convenience of the datasets and learning models that are generated and provided.

[0800] A "user" is a person or organization that accesses the system through a terminal and generates and annotates camera image data.

[0801] A "server" is the central part of the system, a computer environment that generates, stores, notifies, analyzes, and sells data.

[0802] "Generative AI Model" means the artificial intelligence algorithms and programs used to generate camera footage data based on user requests.

[0803] "Camera image data" refers to still image or video data generated by a generative AI model.

[0804] "Storage" refers to a data storage device for temporarily or permanently storing the generated camera image data.

[0805] An "emotion engine" is a function or program that analyzes the user's emotional state and adjusts system operation and image generation accordingly.

[0806] "Annotation work" refers to the work in which a user labels or annotates camera video data.

[0807] A "machine learning algorithm" is an algorithm that learns from annotated data and acquires new knowledge.

[0808] A "learning model" is a computer model for performing a specific task, generated by a machine learning algorithm.

[0809] An "online market" is an e-commerce platform for selling generated training datasets and learning models.

[0810] A "dataset" is a set of data collected for a specific purpose.

[0811] This invention provides a system for generating drone camera video data and applying it to autonomous driving technology. Furthermore, by combining it with an emotion engine that recognizes user emotions, the user experience can be improved. This system generates camera video data using a generative AI model, stores and sells the data, and is equipped with annotation technology and emotion analysis functions.

[0812] Data generation phase

[0813] The user must log in to the application on the server from their terminal. This will display the home screen, allowing the user to begin operating the system. From the "Camera Video Data Generation" menu, enter the required parameters (e.g., video resolution 1920x1080, duration 10 minutes, specific scene "forest", etc.) and click the "Start Generation" button.

[0814] The server receives the parameters entered by the user and passes them to a generative AI model (e.g., OpenAI's DALL-E or GAN) to begin generating video data. The generated camera video data is stored in the server's storage, and the server sends a notification to the user when generation is complete. Notifications are sent via email or in-app notification.

[0815] Use of emotion engine

[0816] The server monitors the user's emotional state using an emotion engine (e.g., Amazon Rekognition, Microsoft Azure Emotion API). It analyzes the user's facial expressions and voice through a camera and microphone to determine the user's emotional state. Based on the analysis results of the emotion engine, the server adaptively adjusts the parameters of the video data to be generated. For example, if the user is relaxed, the server will adjust the parameters to generate video with a richer landscape.

[0817] Data annotation phase

[0818] The user accesses the server from their device, selects the "Annotation" menu, and requests the video data that requires annotation. The server extracts the requested video data from storage and sends it to the user's device. The user then labels and annotates the data.

[0819] The server uses an emotion engine to monitor the user's emotional state while annotating. If the user feels stressed or fatigued, an alert is displayed to prompt the user to stop working. This also supports the user's health management.

[0820] Machine Learning Phase

[0821] The server collects the accumulated annotation data and generates a learning model using machine learning algorithms (e.g., TensorFlow, PyTorch). This learning model is used to improve the drone's autonomous driving technology. The generated dataset is also stored along with the learning model for the next phase.

[0822] Data sales phase

[0823] The server posts the generated training datasets and learning models on an online market, where other development companies can use this data. Users can access the online market from their devices and purchase the datasets and learning models they need. After the purchase process is complete, the server sends the datasets and models to the user's device.

[0824] Specific examples

[0825] For example, a logistics company employee generates camera footage from a drone. The employee accesses the server from a secure device and requests the generation of camera footage. The server uses AI to generate the footage requested and stores it. The employee is then notified and can download the generated video data for internal review.

[0826] Next, as the annotation worker begins the task, the emotion engine monitors the worker's emotional state. If stress or fatigue is detected during the task, the system will display an alert urging the worker to stop, supporting the worker's health management. The emotion engine can also adjust the way data is presented to improve the efficiency of the annotation task.

[0827] The resulting datasets and learning models will be sold to other development companies to help develop autonomous drone technology, which is expected to lead to the rapid adoption of the technology and improved safety.

[0828] Prompt Sentence Examples

[0829] "To generate drone camera footage, a logistics company employee requests the start of generation, specifying a resolution of 1920x1080, a duration of 10 minutes, and a scene of 'warehouse work.' Upon receiving this request, the server uses the generation AI to generate and save the footage."

[0830] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0831] Step 1:

[0832] A user logs in to the server.

[0833] Input: User ID and password.

[0834] The server sends the user ID and password to the authentication server and verifies the authentication information.

[0835] Output: A successful authentication message is displayed on the device, and the home screen is displayed.

[0836] Step 2:

[0837] The user inputs parameters for generating video data and requests generation.

[0838] Input: video resolution (e.g. 1920x1080), duration (e.g. 10 minutes), scene (e.g. forest).

[0839] The server receives the parameters entered by the user and passes them to the generative AI model.

[0840] Output: The parameters are passed to the generative AI model, which starts generating video data.

[0841] Step 3:

[0842] The server generates video data using the generative AI model.

[0843] Input: The parameters passed by the user.

[0844] The server inputs parameters into a generative AI model (e.g., OpenAI's DALL-E or GAN) and generates camera image data.

[0845] Output: The generated video data is saved in the server storage.

[0846] Step 4:

[0847] The server notifies the user that generation is complete.

[0848] Input: Generated video data.

[0849] The server sends a generation completion notification to the user's device (e.g., email or in-app notification).

[0850] Output: A generation completion notification is displayed on the user's terminal.

[0851] Step 5:

[0852] The server monitors the user's emotional state using an emotion engine.

[0853] Input: User video and audio data.

[0854] The server uses an emotion engine (e.g., Amazon Rekognition, Microsoft Azure Emotion API) to analyze the user's facial expressions and voice.

[0855] Output: The user's emotional state is determined and this information is fed back to the generative AI model.

[0856] Step 6:

[0857] The server adjusts the parameters of the video data it generates based on the emotional state.

[0858] Input: The user's emotional state.

[0859] The server adjusts the parameters of the generative AI model based on the analysis results of the emotion engine.

[0860] Output: Video data adapted to the user's emotions is generated.

[0861] Step 7:

[0862] A user requests an annotation task.

[0863] Input: A request for video data stored on the server.

[0864] The server extracts the requested video data from the storage and transmits it to the user terminal.

[0865] Output: The video data is sent to the user's device.

[0866] Step 8:

[0867] Users add labels and annotations to the video data.

[0868] Input: Transmitted video data.

[0869] The user plays back the video data and adds labels and annotations to each frame.

[0870] Output: Annotation data with labels and annotations added is generated.

[0871] Step 9:

[0872] The server uses an emotion engine to monitor the user's emotional state during the annotation process.

[0873] Input: User video and audio data.

[0874] The server uses an emotion engine to analyze the user's emotional state in real time.

[0875] Output: If stress or fatigue is detected, an alert will be displayed to prompt a break from work.

[0876] Step 10:

[0877] The server collects the annotated data and generates a learning model.

[0878] Input: annotation data.

[0879] The server generates a learning model using a machine learning algorithm (e.g., TensorFlow, PyTorch).

[0880] Output: The generated learning model is saved on the server.

[0881] Step 11:

[0882] The server posts the generated datasets and learning models on an online marketplace.

[0883] Input: A training dataset and a training model.

[0884] The server registers the dataset and the learning model in an online marketplace.

[0885] Output: The dataset and the training model are published on an online marketplace.

[0886] Step 12:

[0887] A user purchases a dataset or model.

[0888] Input: Datasets and training models published on online marketplaces.

[0889] Users select the required datasets and learning models from the online market and complete the purchase process.

[0890] Output: The purchase is completed.

[0891] Step 13:

[0892] The server sends the dataset and model to the user's device.

[0893] Input: Purchased dataset and training model.

[0894] After the purchase procedure is completed, the server sends the dataset and model to the user's terminal.

[0895] Output: The dataset and model are sent to the user's device.

[0896] (Application example 2)

[0897] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0898] While conventional drone technology can generate and store camera video data, it lacks a consistent process for annotating the video data and generating learning models using machine learning algorithms. Furthermore, the user experience is hindered by a lack of functionality for providing video data that takes into account the user's emotional state and for adaptively displaying real-time traffic information. This invention aims to solve these issues and provide more advanced drone technology.

[0899] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0900] a means for using generative AI to generate drone camera footage data;

[0901] a means for storing the generated camera image data;

[0902] means for providing the stored camera image data to a user;

[0903] A means for allowing a user to perform annotation work on video data;

[0904] a means for collecting the annotated data and generating a learning model using a machine learning algorithm;

[0905] a means for providing an online marketplace for selling the generated training datasets and training models;

[0906] emotion analysis means for recognizing a user's emotion and adaptively adjusting parameters of the generated video data;

[0907] means for providing real-time traffic information and displaying information according to the emotional state of the user;

[0908] This makes it possible to provide adaptive video data according to the user's emotional state and display traffic information in real time.

[0909] A "drone" is a type of unmanned aerial vehicle (UAV) that is equipped with remote control and autonomous flight capabilities and can capture camera footage from the air.

[0910] "Camera video data" is digital data of images or videos captured by a camera, and is data that includes visual information.

[0911] "Generative AI" refers to a system that uses artificial intelligence technology to automatically perform specific tasks, and in this context refers to the technology that automatically generates drone camera footage data.

[0912] "Storage means" refers to a function or device that temporarily or long-term stores the generated data.

[0913] "Providing means" refers to the functions and interfaces that allow users to access the stored data.

[0914] "Annotation work" refers to the work in which a user adds labels and annotations to video data.

[0915] "Machine learning algorithm" refers to a computational method or procedure for analyzing data and automatically building a learning model.

[0916] A "learning model" is a model generated using a machine learning algorithm, and is a data structure that contains knowledge and patterns for performing a specific task.

[0917] "Online market" refers to a platform for buying and selling goods and services over the Internet.

[0918] "Emotion analysis means" refers to functions and technologies that analyze a user's facial expressions, voice, etc. to determine their emotional state.

[0919] "Traffic information" refers to real-time or historical data on roads and traffic networks, including traffic volume, congestion, and accident information.

[0920] "Means for displaying information" refers to devices such as displays and speakers for providing information visually or audibly to a user.

[0921] This invention relates to a system for generating drone camera image data and applying it to autonomous driving technology and traffic information provision. The specific configuration and implementation method of the system are described below.

[0922] 1. System Configuration

[0923] The system consists of the following major components:

[0924] Drones equipped with generative AI

[0925] server

[0926] User devices (e.g., smartphones and PCs)

[0927] Sentiment Analysis Engine

[0928] Online Market Platform

[0929] 2. Program processing and hardware / software used

[0930] The server uses generative AI to generate drone camera footage data. The generated data is stored in the server's storage. Users can access and view this data from their smartphones or PCs. Users can then perform annotation work, and the data is collected again by the server. Using this annotation data, the server runs a machine learning algorithm to generate a learning model. The generated learning dataset and learning model are then sold on the online market.

[0931] Furthermore, by using an emotion analysis engine, the user's emotional state can be analyzed in real time and the video data parameters can be adjusted accordingly. Specifically, if the user is feeling emotionally stressed, adaptive adjustments can be made, such as providing videos with a relaxing effect.

[0932] The real-time traffic information sent from the drone is displayed on a smartphone or PC in a format that corresponds to the user's emotional state. For example, if the user is feeling irritated, information on relaxing routes and congestion avoidance will be displayed.

[0933] Specific examples of the hardware and software used are as follows:

[0934] Drones: General commercial drones (e.g., DJI Phantom)

[0935] Server: Cloud environment (e.g., Amazon Web Services, Google Cloud Platform)

[0936] Sentiment analysis engine: OpenCV, Keras

[0937] Machine learning algorithms: TensorFlow, PyTorch

[0938] 3. Examples of concrete examples and prompts

[0939] For example, consider the use of this system while a user is driving a car. The user uses a smartphone to receive real-time traffic information from a drone. The emotion analysis engine detects when the user is irritated and suggests a relaxing route or notifies the user of rest spots. This improves the user's driving experience and safety.

[0940] Example prompt sentence:

[0941] "Build an application that provides real-time traffic information from surveillance footage. It should also have the ability to determine the user's emotions and adaptively provide information as needed."

[0942] This will enable the realization of an advanced system that combines drone technology with emotion analysis, significantly improving the user experience.

[0943] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0944] Step 1:

[0945] The user logs in to the application on the server from their terminal. User authentication information (user ID and password) is required as input. The server receives the authentication information and performs user authentication. If authentication is successful, the dashboard screen is displayed to the user.

[0946] Step 2:

[0947] The user selects the "Generate Camera Video Data" menu from the dashboard screen. The parameters of the video data to be generated (resolution, duration, specific scene, etc.) are required as input. The server receives the user's request, passes these parameters to the generation AI, and begins the video data generation process.

[0948] Step 3:

[0949] The generation AI generates camera image data based on input parameters. It processes the data by performing the necessary modeling and simulations to create image data that meets the specified conditions. The generated image data is then stored in the server's storage.

[0950] Step 4:

[0951] The server notifies the user that the video data has been generated, and the user can access the server from their device to check that the video data is now available for download.

[0952] Step 5:

[0953] Users annotate the downloaded video data. The type and content of the annotation (labels, tags, annotations, etc.) are required as input. The annotation editing tool is used on the device to label and annotate the video data.

[0954] Step 6:

[0955] Once the annotation process is complete, the video data is uploaded back to the server. The server then collects the uploaded data and stores it in storage. It also uses an emotion analysis engine to monitor the user's emotional state and displays appropriate feedback as needed.

[0956] Step 7:

[0957] The server provides the collected annotation data to a machine learning algorithm to generate a learning model. The annotation data is required as input. As data processing, the algorithm learns patterns in the data and builds a prediction model or classification model.

[0958] Step 8:

[0959] The generated learning models and datasets are posted on an online marketplace. The server uploads these datasets to the marketplace and makes them available for users to view.

[0960] Step 9:

[0961] Users access the online marketplace and purchase the necessary learning models and datasets. Once the purchase process is complete, the server provides the purchased data to the user's device.

[0962] Step 10:

[0963] The server provides real-time traffic information and displays information according to the user's emotional state. Real-time video data from the drone and emotion analysis results are required as input. This allows the user to assess real-time traffic conditions and select the optimal travel route.

[0964] This completes the entire system processing, allowing users to use advanced drone technology to generate video data, annotate it, utilize learning models, and receive traffic information.

[0965] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0966] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0967] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0968] [Third embodiment]

[0969] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0970] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0971] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0972] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0973] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0974] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0975] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0976] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0977] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0978] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0979] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0980] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0981] This invention is a system for generating drone camera image data and using it for autonomous driving technology. This system uses generative AI to generate a variety of camera image data, store and sell that data, and accumulate annotation technology.

[0982] System Overview

[0983] 1. Data generation phase

[0984] The user logs in to the application on the server from their terminal, enters the necessary parameters (e.g., video resolution, duration, specific scenes, etc.) from the "Camera Video Data Generation" menu, and clicks the "Start Generation" button.

[0985] The server receives the user's request, passes parameters to the generation AI, and starts the video data generation process. The generation AI generates the necessary camera video data and stores it in the server's storage.

[0986] The server notifies the user that the generation of the video data is complete.

[0987] 2. Data annotation phase

[0988] The user logs in to the server from a terminal, selects the "annotation work" menu, and requests the video data that requires annotation.

[0989] The server extracts the requested video data from the storage and transmits it to the user's terminal.

[0990] The user performs annotation work by adding labels and annotations to the transmitted video data, and then transmits the completed data to the server.

[0991] The server stores the received annotation data in storage.

[0992] 3. Machine Learning Phase

[0993] The server collects the accumulated annotation data and generates a learning model using a machine learning algorithm, which is then periodically updated.

[0994] Along with the learning model, the generated dataset is also saved for the next phase.

[0995] 4. Data sales phase

[0996] The server then posts the generated training datasets and models on an online marketplace, making the data available to other development companies.

[0997] Users access the online market from their devices and purchase the necessary datasets and learning models.

[0998] After the purchase process is completed, the server sends the dataset and model to the user's device.

[0999] Specific examples

[1000] For example, consider the case where a person in a logistics company generates camera footage from a drone. The person in charge accesses the server from a secure terminal and requests the generation of camera footage. The server uses the generation AI to generate the footage according to the request and stores it. The person in charge is then notified and can download the generated video data and review it internally.

[1001] Furthermore, annotation work, which involves labeling and annotating this data, can be easily performed. This annotation data can be sent back to the server and used as training data for machine learning to enable autonomous driving for other drones. Ultimately, the generated learning models and datasets can be sold on online markets, allowing other companies to use this technology.

[1002] As such, this invention is a system that comprehensively supports everything from drone camera image data generation to annotation, machine learning, and data sales, which is expected to improve technology and efficiency in the drone industry.

[1003] The processing flow will be explained below.

[1004] Step 1:

[1005] The user logs in to the application on the server from their terminal. The user selects the "Camera Video Data Generation" menu and inputs the parameters required for video generation (e.g., resolution, duration, specific scenes, etc.). Then, the user clicks the "Start Generation" button to send the request to the server.

[1006] Step 2:

[1007] The server receives the user's request and passes parameters to the generation AI, which then generates camera image data based on these parameters and stores the generated data in the server's storage.

[1008] Step 3:

[1009] The server sends a notification to the user saying, "Camera video data has been generated." This notification includes an access link to the generated video data.

[1010] Step 4:

[1011] The user checks the notification on their device and downloads the generated camera video data from the provided link. The user can use this data for their own purposes.

[1012] Step 5:

[1013] When a user wants to perform annotation, he or she accesses the server from the terminal, selects the "annotation work" menu, and requests the video data that requires annotation.

[1014] Step 6:

[1015] The server extracts the requested video data from storage and sends it to the user's device, where the user can add labels and annotations to the data.

[1016] Step 7:

[1017] The user completes the annotation process on the device and sends the completed annotation data to the server, which then stores the received annotation data in storage and organizes it to improve the data quality.

[1018] Step 8:

[1019] The server collects the accumulated annotation data and generates a learning model using a machine learning algorithm, which is then used to improve the drone's autonomous driving technology.

[1020] Step 9:

[1021] The server stores the generated learning model in storage, periodically evaluates the model's performance, and updates the model by retraining it using the latest data.

[1022] Step 10:

[1023] The server then posts the generated training datasets and models on an online market, making them available to other development companies.

[1024] Step 11:

[1025] The user accesses the online market from their device, searches for and selects the required training dataset and training model, and then completes the purchase procedure, after which the server confirms the purchase details.

[1026] Step 12:

[1027] Once the server confirms that the purchase procedure has been completed, it sends the training dataset and model to the user's device, allowing the user to easily obtain the resources necessary for developing autonomous driving technology.

[1028] Example 1

[1029] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1030] Conventionally, systems that generate drone camera video data and use it to improve autonomous driving technology have had several problems. Specifically, the entire process from data generation to annotation and machine learning model generation is often done manually, which is time-consuming and labor-intensive. Additionally, the generated data and models are often sold individually, making it difficult to manage and use them all at once.

[1031] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1032] In this invention, the server includes a means for a user to log in to an application on the server from a terminal, a means for generating drone camera image data using a generative AI model based on a user request, and a means for saving the generated camera image data, thereby automating a series of processes and enabling efficient generation and management of high-quality data.

[1033] A "user" is an entity that uses this system to generate camera image data, perform annotation work, and purchase datasets and learning models.

[1034] A "terminal" is a computing device that a user uses to access applications on a server.

[1035] The "server" is a system that uses a generative AI model based on user requests to generate and store camera image data, and also collects annotation data and performs machine learning.

[1036] A "generative AI model" is an artificial intelligence algorithm that generates camera image data based on parameters specified by the user.

[1037] A "prompt sentence" is an instruction sentence that includes parameters required for generating camera image data input by the user.

[1038] "Camera video data" refers to digital video data generated by AI that imitates or generates video captured by a drone.

[1039] "Annotation" is the process of adding labels and annotations to video data.

[1040] A "machine learning algorithm" is a computational method for building and updating a learning model based on collected annotation data.

[1041] A "learning model" is a model generated by a machine learning algorithm to analyze data and perform a specific task.

[1042] A "dataset" is a collection of annotated video data used to train and test a learning model.

[1043] The "online market" is an e-commerce platform that provides generated training datasets and models to users for purchase and download.

[1044] This invention is a system for generating drone camera image data and using it for autonomous driving technology. This system uses a generative AI model to generate diverse camera image data, store and sell that data, and accumulate annotation technology.

[1045] Data generation phase

[1046] 1. The user accesses the application on the server from their terminal and logs in by entering their login information (user ID and password). An authentication server is used for logging in.

[1047] 2. The user selects the "Camera Video Data Generation" menu from the main menu and enters the required parameters (e.g., video resolution, duration, specific scene, etc.).

[1048] 3. The server receives the user's request and passes parameters to the generative AI model (e.g., GPT-4) to start the video data generation process. The prompt uses the following format:

[1049] "Resolution: 1920x1080" "Duration: 10 minutes" "Scene: Urban traffic"

[1050] 4. The generative AI model generates camera image data based on the input parameters. The generated data is stored in the server's storage (e.g., AWS S3).

[1051] 5. The server notifies the user that the video data has been generated. This notification is sent via email or in-app notification.

[1052] Data annotation phase

[1053] 1. The user accesses the server from their terminal, selects the "Annotation" menu, and requests the video data that requires annotation.

[1054] 2. The server extracts the requested video data from storage (e.g., Google Cloud Storage) and sends it to the user's device.

[1055] 3. The user adds labels and annotations to the transmitted video data using an annotation tool such as VGG Image Annotator.

[1056] 4. The user uploads the annotated data to the server, which then stores the received annotation data in its storage.

[1057] Machine Learning Phase

[1058] 1. The server collects the accumulated annotation data and generates or updates a learning model using a machine learning algorithm (e.g., TensorFlow or PyTorch).

[1059] 2. The server stores the generated dataset along with the generated learning model in preparation for the next phase.

[1060] Data sales phase

[1061] 1. The server publishes the generated training dataset and model on an online marketplace, making the data available to other users.

[1062] 2. The user accesses the online market from their device and purchases the necessary datasets and learning models.

[1063] 3. After the purchase procedure is completed, the server sends the relevant dataset and model to the user's device.

[1064] Specific examples

[1065] When a logistics company employee wants to generate drone camera footage, they access the server from a secure device and request the generation of camera footage. This request includes a prompt like this:

[1066] "Resolution: 1920x1080" "Duration: 10 minutes" "Scene: Urban traffic"

[1067] The server uses the generative AI model to generate and store the video according to the request, after which the person in charge can download the video data and review it in-house.

[1068] Furthermore, annotation work, which involves labeling and annotating this data, can be easily performed. This annotation data can be sent back to the server and used as training data for machine learning to enable autonomous driving for other drones. Ultimately, the generated learning models and datasets can be sold on online markets, allowing other companies to use this technology.

[1069] As such, this invention is a system that comprehensively supports everything from drone camera image data generation to annotation, machine learning, and data sales, which is expected to improve technology and efficiency in the drone industry.

[1070] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1071] Step 1:

[1072] A user accesses an application on the server from a terminal and logs in by entering login information (user ID and password). The input is the user ID and password, which the authentication server receives and performs authentication processing. The output is the authentication result, and if the login is successful, the main menu screen is displayed.

[1073] Specific behavior:

[1074] 1. The user displays the login page on the terminal.

[1075] 2. Enter your user ID and password.

[1076] 3. The server sends a query to the authentication server.

[1077] 4. The authentication server returns the authentication result.

[1078] 5. If the server is successful in authenticating, it displays the main menu screen to the user.

[1079] Step 2:

[1080] The user selects the "Camera Video Data Generation" menu from the main menu and inputs parameters such as video resolution, duration, and specific scenes. The inputs are the video generation parameters. The server receives the input parameters and passes them to the generative AI model.

[1081] Specific behavior:

[1082] 1. The user clicks on the "Generate camera video data" menu.

[1083] 2. The user inputs parameters such as video resolution (e.g., 1920x1080), duration (e.g., 10 minutes), and scene (e.g., urban traffic).

[1084] 3. The user clicks the "Start Generation" button.

[1085] 4. The server retrieves the input parameters.

[1086] Step 3:

[1087] The server generates a prompt sentence from the acquired parameters and sends a request to the generative AI model. The input is the generated parameters, and the output is to generate a prompt sentence and pass it to the generative AI model. The generative AI model generates video data based on this prompt sentence.

[1088] Specific behavior:

[1089] 1. The server generates a prompt using the parameters it has obtained.

[1090] For example: "Resolution: 1920x1080", "Duration: 10 minutes", "Scene: City traffic".

[1091] 2. The server sends the generated prompt sentence to the generative AI model.

[1092] Step 4:

[1093] The generative AI model generates camera video data based on the prompt sentence and returns the generated data to the server. The input is the prompt sentence and the output is the generated camera video data. This data is sent to the server.

[1094] Specific behavior:

[1095] 1. The generative AI model parses the prompt.

[1096] 2. The AI ​​model uses its internal algorithm to generate video data.

[1097] 3. The generated video data is returned to the server.

[1098] Step 5:

[1099] The server saves the generated video data in storage (e.g., AWS S3) and notifies the user. The input is the generated camera video data, and the output is the URL or identification information of the saved data and a notification message.

[1100] Specific behavior:

[1101] 1. The server saves the generated video data in storage.

[1102] 2. Receive the URL and identification information of the stored data from the storage.

[1103] 3. The server notifies the user that video generation is complete (via email or in-app notification).

[1104] Step 6:

[1105] The user selects the "annotation work" menu and requests the required video data. The input is the user's request information, and the output is the video data provided by the server.

[1106] Specific behavior:

[1107] 1. The user clicks on the "Annotation Tasks" menu.

[1108] 2. The user selects the required video data and sends a request.

[1109] 3. The server receives the request and extracts the corresponding data.

[1110] Step 7:

[1111] The server sends the requested video data from storage (e.g., Google Cloud Storage) to the user's device. The input is the identification information of the requested video data, and the output is the video data to be sent to the user's device.

[1112] Specific behavior:

[1113] 1. The server queries the storage for the requested video data.

[1114] 2. The storage returns the video data to the server.

[1115] 3. The server sends the video data to the user's device.

[1116] Step 8:

[1117] The user adds labels and annotations to the transmitted video data using an annotation tool (e.g., VGG Image Annotator). The input is the received video data, and the output is the annotated video data.

[1118] Specific behavior:

[1119] 1. The user imports the received video data into the annotation tool.

[1120] 2. The user adds labels and annotations to the video data.

[1121] 3. Upload the annotated data to the server.

[1122] Step 9:

[1123] The server stores the annotation data received from the user. The input is the annotated video data, and the output is the stored annotation data.

[1124] Specific behavior:

[1125] 1. The server receives annotation data uploaded by the user.

[1126] 2. The server saves the annotation data in storage.

[1127] Step 10:

[1128] The server collects the accumulated annotation data and uses machine learning algorithms (e.g., TensorFlow or PyTorch) to generate new learning models or update existing models. The input is the accumulated annotation data, and the output is the generated or updated learning model.

[1129] Specific behavior:

[1130] 1. The server collects the accumulated annotation data.

[1131] 2. The server runs a machine learning algorithm and uses the data to train a model.

[1132] 3. Generate a new training model or update an existing model.

[1133] 4. Save the generated or updated model to storage.

[1134] Step 11:

[1135] The server posts the generated training model and dataset on an online market. The input is the training model and dataset, and the output is the posting information on the online market.

[1136] Specific behavior:

[1137] 1. The server uploads the training model and dataset to the online market.

[1138] 2. Sales information will be published on the online market.

[1139] Step 12:

[1140] Users access the online marketplace and purchase the datasets and learning models they need. The input is the user's purchase request, and the output is the purchased datasets and learning models.

[1141] Specific behavior:

[1142] 1. A user visits an online marketplace.

[1143] 2. The user selects the required dataset and learning model and completes the purchase process.

[1144] 3. The server sends the purchasing dataset and model to the user's device.

[1145] (Application example 1)

[1146] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1147] It is extremely important to generate drone camera video data under a variety of conditions and apply it to the development of autonomous driving and other technologies. However, existing systems have limited functionality for generating video data in real time under specific conditions desired by users, resulting in a lack of flexibility and diversity in the video data provided. Furthermore, the annotation process for the generated video data is cumbersome for users, and there is a need for more efficient annotation. Furthermore, there is a lack of efficient means for selling the generated datasets and learning models online, making it difficult to share and use the data.

[1148] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1149] In this invention, the server includes: means for using a generation AI to generate drone camera video data; means for saving the generated camera video data; means for providing the saved camera video data to a user; means for generating prompt statements based on user-specified conditions and generating camera video data in real time according to the prompt statements; means for the user to annotate the video data; means for collecting the annotated data and generating a learning model using a machine learning algorithm; and means for providing an online market for selling the generated learning datasets and learning models. This allows users to generate the video data they need under specific conditions in real time, enabling efficient annotation work. Furthermore, the generated datasets and learning models can be easily shared and used.

[1150] Key Word Definitions

[1151] "Generative AI" refers to artificial intelligence that generates camera image data based on user-specified conditions.

[1152] "Camera video data" refers to video information captured by a drone.

[1153] A "prompt statement" is an instruction statement that causes the generation AI to generate camera image data based on conditions specified by the user.

[1154] "Annotation" refers to the process of adding labels and annotations to camera video data.

[1155] "Learning model" refers to a trained model constructed by a machine learning algorithm using the generated camera image data and annotation data.

[1156] "Online market" refers to an internet platform for selling generated training datasets and learning models.

[1157] A "smartphone" refers to a mobile device that has Internet connectivity and can run a variety of applications.

[1158] "Real-time" refers to the property of responding immediately to user operations and inputs.

[1159] "User" refers to any individual or organization that uses this system to generate, store, annotate, and use drone camera footage data.

[1160] MODE FOR CARRYING OUT THE INVENTION

[1161] This invention is a system for generating drone camera image data and using it for autonomous driving technology and other applications. This system uses generative AI to generate camera image data under various conditions, stores and sells the data, and utilizes annotation technology. One embodiment of the present invention is described below.

[1162] 1. System Configuration

[1163] This system is connected via a terminal accessible by a user, a server that processes data, and the Internet.

[1164] 2. Data generation phase

[1165] Users can log in to the application on the server from their terminal, enter the necessary parameters (such as the video resolution, duration, and specific scenes) from the "Camera Video Data Generation" menu, and click the "Start Generation" button.

[1166] The server receives the user's request, passes parameters to the generation AI, and starts the video data generation process. The generation AI generates the necessary camera video data and stores it in the server's storage.

[1167] The server notifies the user that the generation of the video data is complete.

[1168] 3. Data annotation phase

[1169] The user logs in to the server from a terminal, selects the "annotation work" menu, and requests the video data that requires annotation.

[1170] The server extracts the requested video data from the storage and transmits it to the user's terminal.

[1171] The user performs annotation work by adding labels and annotations to the transmitted video data, and then transmits the completed data to the server.

[1172] The server stores the received annotation data in storage.

[1173] 4. Machine Learning Phase

[1174] The server collects the accumulated annotation data and generates a learning model using a machine learning algorithm, which is then periodically updated.

[1175] Along with the learning model, the generated dataset is also saved for the next phase.

[1176] 5. Data sales phase

[1177] The server posts the generated training datasets and learning models on an online marketplace, making the data available to other development companies.

[1178] Users access the online market from their devices and purchase the necessary datasets and learning models.

[1179] After the purchase process is completed, the server sends the dataset and model to the user's device.

[1180] 6. Hardware and Software Used

[1181] Servers are responsible for processing and storing data, and therefore use server machines equipped with high-performance processors and large-capacity storage.

[1182] The software uses TensorFlow to build generative AI models and web application frameworks such as Flask for annotation work.

[1183] The user terminals used are mobile terminals such as smartphones and tablets.

[1184] 7. Specific Examples

[1185] For example, if a user wants a video of a "sunny day in the mountains," they would input the following prompt sentence into the generation AI:

[1186] Prompt: "Generate a camera image of a mountainous area on a clear day."

[1187] The server runs a generation AI based on this prompt text and generates camera image data that matches the specified conditions.

[1188] The generated video data is displayed in real time on the user's device, and the user can then perform annotations on the video, which are then stored on the server.

[1189] This system allows users to generate the video data they need under specific conditions in real time, enabling them to perform annotation work efficiently. Furthermore, the generated datasets and learning models can be used by other companies and research institutions, improving the shareability and usefulness of data.

[1190] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1191] Program processing flow

[1192] Step 1:

[1193] A user logs in to the system from a terminal. As input, the user provides login information (user ID, password, etc.). The server receives this information, performs authentication, and determines whether the user can log in. If authentication is successful, the user gains access to the system. As output, it sends a message to the terminal indicating whether authentication was successful or not.

[1194] Step 2:

[1195] The user selects the "Generate Camera Video Data" menu on the terminal and inputs the required parameters (e.g., video resolution, duration, specific scene, etc.). As input, these parameters are sent from the terminal to the server. The server receives these parameters and generates a prompt sentence. As output, the generated prompt sentence is stored internally.

[1196] Step 3:

[1197] The server passes the prompt sentence to the generation AI and starts generating camera video data. As input, the server inputs the prompt sentence into the generation AI. The generation AI generates the drone camera video data based on this prompt sentence. As data processing, the generation AI performs the calculations necessary to generate the video data. As output, it obtains the generated video data.

[1198] Step 4:

[1199] The server saves the generated camera image data in storage and sends a notification to the user that generation is complete. As input, it receives the generated camera image data. As data calculation, it performs a procedure to save this data in storage. As output, it sends a message to the user notifying them that generation is complete.

[1200] Step 5:

[1201] The user selects the "Annotation" menu on the terminal and requests camera video data to be analyzed. As input, the user sends the request information. The server receives this information, extracts the requested video data from storage, and sends it to the user's terminal. As output, it sends the corresponding video data to the user.

[1202] Step 6:

[1203] The user performs annotation work by adding labels and annotations to the transmitted video data. As input, the video data is displayed on the device and the user adds annotations. As data processing, the annotation data is incorporated into the video data. As output, annotated data is generated.

[1204] Step 7:

[1205] The user sends the data after annotation work is completed to the server. The annotated data is sent to the server as input. The server receives this data and saves it in storage. A saving process is executed as data calculation. The annotation data is saved on the server as output.

[1206] Step 8:

[1207] The server collects the accumulated annotation data and generates a learning model using a machine learning algorithm. The annotation data is collected as input. As data processing, the machine learning algorithm uses the annotation data as training data to generate a learning model. As output, a new learning model is generated.

[1208] Step 9:

[1209] The generated training dataset and model are posted on the online market. As input, a new training dataset and model are provided. The server adds them to the online market platform. As data processing, a new item is registered in the market list. As output, it is displayed as an item available to users.

[1210] Step 10:

[1211] A user accesses the online market from their device and purchases the required dataset or learning model. The user's purchase request is sent as input. The server processes the purchase and, once the purchase is complete, sends the dataset or model to the user's device. The data calculation involves purchase processing and distribution. The output is a notification that the purchase has been completed, along with the dataset or model, sent to the user.

[1212] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1213] This invention relates to a system for generating drone camera video data and applying it to autonomous driving technology. Furthermore, by combining it with an emotion engine that recognizes user emotions, the user experience can be improved. This system generates camera video data using generative AI, stores and sells the data, and is equipped with annotation technology and emotion analysis functions.

[1214] System Overview

[1215] 1. Data generation phase

[1216] The user logs in to the application on the server from their terminal, enters the necessary parameters (e.g., video resolution, duration, specific scenes, etc.) from the "Camera Video Data Generation" menu, and clicks the "Start Generation" button.

[1217] The server receives the user's request, passes parameters to the generation AI, and starts the video data generation process. The generation AI generates the necessary camera video data and stores it in the server's storage.

[1218] The server notifies the user that the generation of the video data is complete.

[1219] 2. Use of Emotion Engine

[1220] The server monitors the user's emotional state using an emotion engine, for example, by analyzing the user's facial expressions and voice through a camera or microphone to determine the user's emotional state.

[1221] The server adaptively adjusts the parameters of the video data to be generated based on the analysis results of the emotion engine. For example, if the user is relaxed, the server can adjust the parameters to generate a video rich in scenery.

[1222] 3. Data annotation phase

[1223] The user accesses the server from a terminal, selects the "annotation work" menu, and requests the video data that requires annotation.

[1224] The server extracts the requested video data from storage and sends it to the user's device, where the user can add labels and annotations to the data.

[1225] The server uses an emotion engine to monitor the user's emotional state during annotation work. If the user feels stressed or fatigued, an alert is displayed to prompt the user to stop working.

[1226] 4. Machine Learning Phase

[1227] The server collects the accumulated annotation data and uses machine learning algorithms to generate learning models, which are then used to improve the drone's autonomous driving technology.

[1228] Along with the learning model, the generated dataset is also saved for the next phase.

[1229] 5. Data sales phase

[1230] The server then posts the generated training datasets and models on an online marketplace, making the data available to other development companies.

[1231] Users access the online market from their devices and purchase the necessary datasets and learning models.

[1232] After the purchase process is completed, the server sends the dataset and model to the user's device.

[1233] Specific examples

[1234] For example, consider the case where a person in a logistics company generates camera footage from a drone. The person in charge accesses the server from a secure terminal and requests the generation of camera footage. The server uses the generation AI to generate the footage according to the request and stores it. The person in charge is then notified and can download the generated video data and review it internally.

[1235] Next, as the annotation worker begins the task, the emotion engine monitors the worker's emotional state. If stress or fatigue is detected during the task, the system will display an alert urging the worker to stop, thereby supporting health management. The emotion engine can also adjust the way data is presented to improve the efficiency of the annotation task.

[1236] The resulting datasets and learning models will be sold to other development companies to help develop autonomous drone technology, which is expected to lead to the rapid adoption of the technology and improved safety.

[1237] The processing flow will be explained below.

[1238] Step 1:

[1239] The user logs in to the application on the server from their terminal. The user selects the "Camera Video Data Generation" menu and inputs the parameters required for video generation (e.g., resolution, duration, specific scenes, etc.). Then, the user clicks the "Start Generation" button to send the request to the server.

[1240] Step 2:

[1241] The server receives the user's request and activates the emotion engine, which analyzes the user's facial expressions and voice via a camera and microphone to recognize the user's emotional state (e.g., relaxed, focused, stressed, etc.).

[1242] Step 3:

[1243] Based on the analysis results of the emotion engine, the server issues instructions to the generation AI with adaptively adjusted parameters. For example, if the user is relaxed, the server instructs the AI ​​to generate images rich in scenery.

[1244] Step 4:

[1245] The generation AI generates camera image data based on parameters passed from the server, and the generated image data is stored in the server's storage.

[1246] Step 5:

[1247] The server sends a notification to the user saying, "Camera video data has been generated." This notification includes an access link to the generated video data.

[1248] Step 6:

[1249] The user checks the notification on their device and downloads the generated camera video data from the provided link. The user can then use this data for their own purposes.

[1250] Step 7:

[1251] When a user wants to perform annotation, he or she accesses the server from the terminal, selects the "annotation work" menu, and requests the video data that requires annotation.

[1252] Step 8:

[1253] The server extracts the requested video data from storage and sends it to the user's device, where the user can add labels and annotations to the data.

[1254] Step 9:

[1255] The server continuously monitors the user's emotional state during annotation work. If the emotion engine analyzes the user's emotional state and detects stress or fatigue, the server displays an alert urging the user to stop working.

[1256] Step 10:

[1257] The user completes the annotation process on the device and sends the completed annotation data to the server, which then stores the received annotation data in storage and organizes it to improve the data quality.

[1258] Step 11:

[1259] The server collects the accumulated annotation data and uses machine learning algorithms to generate learning models, which are then used to improve the drone's autonomous driving technology.

[1260] Step 12:

[1261] The server stores the generated learning model in storage, periodically evaluates the model's performance, and updates the model by retraining it using the latest data.

[1262] Step 13:

[1263] The server then posts the generated training datasets and models on an online market, making them available to other development companies.

[1264] Step 14:

[1265] The user accesses the online market from their device, searches for and selects the required training dataset and training model, and then completes the purchase procedure, after which the server confirms the purchase details.

[1266] Step 15:

[1267] Once the server confirms that the purchase procedure has been completed, it sends the training dataset and model to the user's device, allowing the user to easily obtain the resources necessary for developing autonomous driving technology.

[1268] Example 2

[1269] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1270] Conventional systems that generate drone camera video data and apply it to autonomous driving technology lack the ability to generate video that takes the user's emotional state into account, and provide insufficient support for efficient annotation. Furthermore, it has been difficult to properly manage and quickly provide the generated datasets and learning models for sale. This has hindered improvements to the user experience and efficient data generation.

[1271] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: a means for a user to log in to the server and input parameters for video data generation; a means for a generative AI model to generate camera video data based on a user request; a means for saving the generated camera video data in the server's storage; a means for notifying the user of the completion of generation; a means for monitoring the user's emotional state using an emotion engine and adjusting video generation parameters; a means for the user to perform annotation work to label and add annotations to the video data; a means for collecting the annotated data and generating a learning model using a machine learning algorithm; a means for listing the generated learning dataset and learning model on an online market; and a means for the user to purchase the dataset and model and transmit them from the server to a terminal. This enables video generation that takes the user's emotional state into consideration and efficient annotation work, thereby improving the quality and convenience of the datasets and learning models that are generated and provided.

[1272] A "user" is a person or organization that accesses the system through a terminal and generates and annotates camera image data.

[1273] A "server" is the central part of the system, a computer environment that generates, stores, notifies, analyzes, and sells data.

[1274] "Generative AI Model" means the artificial intelligence algorithms and programs used to generate camera footage data based on user requests.

[1275] "Camera image data" refers to still image or video data generated by a generative AI model.

[1276] "Storage" refers to a data storage device for temporarily or permanently storing the generated camera image data.

[1277] An "emotion engine" is a function or program that analyzes the user's emotional state and adjusts system operation and image generation accordingly.

[1278] "Annotation work" refers to the work in which a user labels or annotates camera video data.

[1279] A "machine learning algorithm" is an algorithm that learns from annotated data and acquires new knowledge.

[1280] A "learning model" is a computer model for performing a specific task, generated by a machine learning algorithm.

[1281] An "online market" is an e-commerce platform for selling generated training datasets and learning models.

[1282] A "dataset" is a set of data collected for a specific purpose.

[1283] This invention provides a system for generating drone camera video data and applying it to autonomous driving technology. Furthermore, by combining it with an emotion engine that recognizes user emotions, the user experience can be improved. This system generates camera video data using a generative AI model, stores and sells the data, and is equipped with annotation technology and emotion analysis functions.

[1284] Data generation phase

[1285] The user must log in to the application on the server from their terminal. This will display the home screen, allowing the user to begin operating the system. From the "Camera Video Data Generation" menu, enter the required parameters (e.g., video resolution 1920x1080, duration 10 minutes, specific scene "forest", etc.) and click the "Start Generation" button.

[1286] The server receives the parameters entered by the user and passes them to a generative AI model (e.g., OpenAI's DALL-E or GAN) to begin generating video data. The generated camera video data is stored in the server's storage, and the server sends a notification to the user when generation is complete. Notifications are sent via email or in-app notification.

[1287] Use of emotion engine

[1288] The server monitors the user's emotional state using an emotion engine (e.g., Amazon Rekognition, Microsoft Azure Emotion API). It analyzes the user's facial expressions and voice through a camera and microphone to determine the user's emotional state. Based on the analysis results of the emotion engine, the server adaptively adjusts the parameters of the video data to be generated. For example, if the user is relaxed, the server will adjust the parameters to generate video with a richer landscape.

[1289] Data annotation phase

[1290] The user accesses the server from their device, selects the "Annotation" menu, and requests the video data that requires annotation. The server extracts the requested video data from storage and sends it to the user's device. The user then labels and annotates the data.

[1291] The server uses an emotion engine to monitor the user's emotional state while annotating. If the user feels stressed or fatigued, an alert is displayed to prompt the user to stop working. This also supports the user's health management.

[1292] Machine Learning Phase

[1293] The server collects the accumulated annotation data and generates a learning model using machine learning algorithms (e.g., TensorFlow, PyTorch). This learning model is used to improve the drone's autonomous driving technology. The generated dataset is also stored along with the learning model for the next phase.

[1294] Data sales phase

[1295] The server posts the generated training datasets and learning models on an online market, where other development companies can use this data. Users can access the online market from their devices and purchase the datasets and learning models they need. After the purchase process is complete, the server sends the datasets and models to the user's device.

[1296] Specific examples

[1297] For example, a logistics company employee generates camera footage from a drone. The employee accesses the server from a secure device and requests the generation of camera footage. The server uses AI to generate the footage requested and stores it. The employee is then notified and can download the generated video data for internal review.

[1298] Next, as the annotation worker begins the task, the emotion engine monitors the worker's emotional state. If stress or fatigue is detected during the task, the system will display an alert urging the worker to stop, supporting the worker's health management. The emotion engine can also adjust the way data is presented to improve the efficiency of the annotation task.

[1299] The resulting datasets and learning models will be sold to other development companies to help develop autonomous drone technology, which is expected to lead to the rapid adoption of the technology and improved safety.

[1300] Prompt Sentence Examples

[1301] "To generate drone camera footage, a logistics company employee requests the start of generation, specifying a resolution of 1920x1080, a duration of 10 minutes, and a scene of 'warehouse work.' Upon receiving this request, the server uses the generation AI to generate and save the footage."

[1302] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1303] Step 1:

[1304] A user logs in to the server.

[1305] Input: User ID and password.

[1306] The server sends the user ID and password to the authentication server and verifies the authentication information.

[1307] Output: A successful authentication message is displayed on the device, and the home screen is displayed.

[1308] Step 2:

[1309] The user inputs parameters for generating video data and requests generation.

[1310] Input: video resolution (e.g. 1920x1080), duration (e.g. 10 minutes), scene (e.g. forest).

[1311] The server receives the parameters entered by the user and passes them to the generative AI model.

[1312] Output: The parameters are passed to the generative AI model, which starts generating video data.

[1313] Step 3:

[1314] The server generates video data using the generative AI model.

[1315] Input: The parameters passed by the user.

[1316] The server inputs parameters into a generative AI model (e.g., OpenAI's DALL-E or GAN) and generates camera image data.

[1317] Output: The generated video data is saved in the server storage.

[1318] Step 4:

[1319] The server notifies the user that generation is complete.

[1320] Input: Generated video data.

[1321] The server sends a generation completion notification to the user's device (e.g., email or in-app notification).

[1322] Output: A generation completion notification is displayed on the user's terminal.

[1323] Step 5:

[1324] The server monitors the user's emotional state using an emotion engine.

[1325] Input: User video and audio data.

[1326] The server uses an emotion engine (e.g., Amazon Rekognition, Microsoft Azure Emotion API) to analyze the user's facial expressions and voice.

[1327] Output: The user's emotional state is determined and this information is fed back to the generative AI model.

[1328] Step 6:

[1329] The server adjusts the parameters of the video data it generates based on the emotional state.

[1330] Input: The user's emotional state.

[1331] The server adjusts the parameters of the generative AI model based on the analysis results of the emotion engine.

[1332] Output: Video data adapted to the user's emotions is generated.

[1333] Step 7:

[1334] A user requests an annotation task.

[1335] Input: A request for video data stored on the server.

[1336] The server extracts the requested video data from the storage and transmits it to the user terminal.

[1337] Output: The video data is sent to the user's device.

[1338] Step 8:

[1339] Users add labels and annotations to the video data.

[1340] Input: Transmitted video data.

[1341] The user plays back the video data and adds labels and annotations to each frame.

[1342] Output: Annotation data with labels and annotations added is generated.

[1343] Step 9:

[1344] The server uses an emotion engine to monitor the user's emotional state during the annotation process.

[1345] Input: User video and audio data.

[1346] The server uses an emotion engine to analyze the user's emotional state in real time.

[1347] Output: If stress or fatigue is detected, an alert will be displayed to prompt a break from work.

[1348] Step 10:

[1349] The server collects the annotated data and generates a learning model.

[1350] Input: annotation data.

[1351] The server generates a learning model using a machine learning algorithm (e.g., TensorFlow, PyTorch).

[1352] Output: The generated learning model is saved on the server.

[1353] Step 11:

[1354] The server posts the generated datasets and learning models on an online marketplace.

[1355] Input: A training dataset and a training model.

[1356] The server registers the dataset and the learning model in an online marketplace.

[1357] Output: The dataset and the training model are published on an online marketplace.

[1358] Step 12:

[1359] A user purchases a dataset or model.

[1360] Input: Datasets and training models published on online marketplaces.

[1361] Users select the required datasets and learning models from the online market and complete the purchase process.

[1362] Output: The purchase is completed.

[1363] Step 13:

[1364] The server sends the dataset and model to the user's device.

[1365] Input: Purchased dataset and training model.

[1366] After the purchase procedure is completed, the server sends the dataset and model to the user's terminal.

[1367] Output: The dataset and model are sent to the user's device.

[1368] (Application example 2)

[1369] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1370] While conventional drone technology can generate and store camera video data, it lacks a consistent process for annotating the video data and generating learning models using machine learning algorithms. Furthermore, the user experience is hindered by a lack of functionality for providing video data that takes into account the user's emotional state and for adaptively displaying real-time traffic information. This invention aims to solve these issues and provide more advanced drone technology.

[1371] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1372] a means for using generative AI to generate drone camera footage data;

[1373] a means for storing the generated camera image data;

[1374] means for providing the stored camera image data to a user;

[1375] A means for allowing a user to perform annotation work on video data;

[1376] a means for collecting the annotated data and generating a learning model using a machine learning algorithm;

[1377] a means for providing an online marketplace for selling the generated training datasets and training models;

[1378] emotion analysis means for recognizing a user's emotion and adaptively adjusting parameters of the generated video data;

[1379] means for providing real-time traffic information and displaying information according to the emotional state of the user;

[1380] This makes it possible to provide adaptive video data according to the user's emotional state and display traffic information in real time.

[1381] A "drone" is a type of unmanned aerial vehicle (UAV) that is equipped with remote control and autonomous flight capabilities and can capture camera footage from the air.

[1382] "Camera video data" is digital data of images or videos captured by a camera, and is data that includes visual information.

[1383] "Generative AI" refers to a system that uses artificial intelligence technology to automatically perform specific tasks, and in this context refers to the technology that automatically generates drone camera footage data.

[1384] "Storage means" refers to a function or device that temporarily or long-term stores the generated data.

[1385] "Providing means" refers to the functions and interfaces that allow users to access the stored data.

[1386] "Annotation work" refers to the work in which a user adds labels and annotations to video data.

[1387] "Machine learning algorithm" refers to a computational method or procedure for analyzing data and automatically building a learning model.

[1388] A "learning model" is a model generated using a machine learning algorithm, and is a data structure that contains knowledge and patterns for performing a specific task.

[1389] "Online market" refers to a platform for buying and selling goods and services over the Internet.

[1390] "Emotion analysis means" refers to functions and technologies that analyze a user's facial expressions, voice, etc. to determine their emotional state.

[1391] "Traffic information" refers to real-time or historical data on roads and traffic networks, including traffic volume, congestion, and accident information.

[1392] "Means for displaying information" refers to devices such as displays and speakers for providing information visually or audibly to a user.

[1393] This invention relates to a system for generating drone camera image data and applying it to autonomous driving technology and traffic information provision. The specific configuration and implementation method of the system are described below.

[1394] 1. System Configuration

[1395] The system consists of the following major components:

[1396] Drones equipped with generative AI

[1397] server

[1398] User devices (e.g., smartphones and PCs)

[1399] Sentiment Analysis Engine

[1400] Online Market Platform

[1401] 2. Program processing and hardware / software used

[1402] The server uses generative AI to generate drone camera footage data. The generated data is stored in the server's storage. Users can access and view this data from their smartphones or PCs. Users can then perform annotation work, and the data is collected again by the server. Using this annotation data, the server runs a machine learning algorithm to generate a learning model. The generated learning dataset and learning model are then sold on the online market.

[1403] Furthermore, by using an emotion analysis engine, the user's emotional state can be analyzed in real time and the video data parameters can be adjusted accordingly. Specifically, if the user is feeling emotionally stressed, adaptive adjustments can be made, such as providing videos with a relaxing effect.

[1404] The real-time traffic information sent from the drone is displayed on a smartphone or PC in a format that corresponds to the user's emotional state. For example, if the user is feeling irritated, information on relaxing routes and congestion avoidance will be displayed.

[1405] Specific examples of the hardware and software used are as follows:

[1406] Drones: General commercial drones (e.g., DJI Phantom)

[1407] Server: Cloud environment (e.g., Amazon Web Services, Google Cloud Platform)

[1408] Sentiment analysis engine: OpenCV, Keras

[1409] Machine learning algorithms: TensorFlow, PyTorch

[1410] 3. Examples of concrete examples and prompts

[1411] For example, consider the use of this system while a user is driving a car. The user uses a smartphone to receive real-time traffic information from a drone. The emotion analysis engine detects when the user is irritated and suggests a relaxing route or notifies the user of rest spots. This improves the user's driving experience and safety.

[1412] Example prompt sentence:

[1413] "Build an application that provides real-time traffic information from surveillance footage. It should also have the ability to determine the user's emotions and adaptively provide information as needed."

[1414] This will enable the realization of an advanced system that combines drone technology with emotion analysis, significantly improving the user experience.

[1415] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1416] Step 1:

[1417] The user logs in to the application on the server from their terminal. User authentication information (user ID and password) is required as input. The server receives the authentication information and performs user authentication. If authentication is successful, the dashboard screen is displayed to the user.

[1418] Step 2:

[1419] The user selects the "Generate Camera Video Data" menu from the dashboard screen. The parameters of the video data to be generated (resolution, duration, specific scene, etc.) are required as input. The server receives the user's request, passes these parameters to the generation AI, and begins the video data generation process.

[1420] Step 3:

[1421] The generation AI generates camera image data based on input parameters. It processes the data by performing the necessary modeling and simulations to create image data that meets the specified conditions. The generated image data is then stored in the server's storage.

[1422] Step 4:

[1423] The server notifies the user that the video data has been generated, and the user can access the server from their device to check that the video data is now available for download.

[1424] Step 5:

[1425] Users annotate the downloaded video data. The type and content of the annotation (labels, tags, annotations, etc.) are required as input. The annotation editing tool is used on the device to label and annotate the video data.

[1426] Step 6:

[1427] Once the annotation process is complete, the video data is uploaded back to the server. The server then collects the uploaded data and stores it in storage. It also uses an emotion analysis engine to monitor the user's emotional state and displays appropriate feedback as needed.

[1428] Step 7:

[1429] The server provides the collected annotation data to a machine learning algorithm to generate a learning model. The annotation data is required as input. As data processing, the algorithm learns patterns in the data and builds a prediction model or classification model.

[1430] Step 8:

[1431] The generated learning models and datasets are posted on an online marketplace. The server uploads these datasets to the marketplace and makes them available for users to view.

[1432] Step 9:

[1433] Users access the online marketplace and purchase the necessary learning models and datasets. Once the purchase process is complete, the server provides the purchased data to the user's device.

[1434] Step 10:

[1435] The server provides real-time traffic information and displays information according to the user's emotional state. Real-time video data from the drone and emotion analysis results are required as input. This allows the user to assess real-time traffic conditions and select the optimal travel route.

[1436] This completes the entire system processing, allowing users to use advanced drone technology to generate video data, annotate it, utilize learning models, and receive traffic information.

[1437] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1438] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1439] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1440] [Fourth embodiment]

[1441] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1442] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1443] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1444] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1445] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1446] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1447] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1448] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1449] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1450] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1451] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1452] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1453] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1454] This invention is a system for generating drone camera image data and using it for autonomous driving technology. This system uses generative AI to generate a variety of camera image data, store and sell that data, and accumulate annotation technology.

[1455] System Overview

[1456] 1. Data generation phase

[1457] The user logs in to the application on the server from their terminal, enters the necessary parameters (e.g., video resolution, duration, specific scenes, etc.) from the "Camera Video Data Generation" menu, and clicks the "Start Generation" button.

[1458] The server receives the user's request, passes parameters to the generation AI, and starts the video data generation process. The generation AI generates the necessary camera video data and stores it in the server's storage.

[1459] The server notifies the user that the generation of the video data is complete.

[1460] 2. Data annotation phase

[1461] The user logs in to the server from a terminal, selects the "annotation work" menu, and requests the video data that requires annotation.

[1462] The server extracts the requested video data from the storage and transmits it to the user's terminal.

[1463] The user performs annotation work by adding labels and annotations to the transmitted video data, and then transmits the completed data to the server.

[1464] The server stores the received annotation data in storage.

[1465] 3. Machine Learning Phase

[1466] The server collects the accumulated annotation data and generates a learning model using a machine learning algorithm, which is then periodically updated.

[1467] Along with the learning model, the generated dataset is also saved for the next phase.

[1468] 4. Data sales phase

[1469] The server then posts the generated training datasets and models on an online marketplace, making the data available to other development companies.

[1470] Users access the online market from their devices and purchase the necessary datasets and learning models.

[1471] After the purchase process is completed, the server sends the dataset and model to the user's device.

[1472] Specific examples

[1473] For example, consider the case where a person in a logistics company generates camera footage from a drone. The person in charge accesses the server from a secure terminal and requests the generation of camera footage. The server uses the generation AI to generate the footage according to the request and stores it. The person in charge is then notified and can download the generated video data and review it internally.

[1474] Furthermore, annotation work, which involves labeling and annotating this data, can be easily performed. This annotation data can be sent back to the server and used as training data for machine learning to enable autonomous driving for other drones. Ultimately, the generated learning models and datasets can be sold on online markets, allowing other companies to use this technology.

[1475] As such, this invention is a system that comprehensively supports everything from drone camera image data generation to annotation, machine learning, and data sales, which is expected to improve technology and efficiency in the drone industry.

[1476] The processing flow will be explained below.

[1477] Step 1:

[1478] The user logs in to the application on the server from their terminal. The user selects the "Camera Video Data Generation" menu and inputs the parameters required for video generation (e.g., resolution, duration, specific scenes, etc.). Then, the user clicks the "Start Generation" button to send the request to the server.

[1479] Step 2:

[1480] The server receives the user's request and passes parameters to the generation AI, which then generates camera image data based on these parameters and stores the generated data in the server's storage.

[1481] Step 3:

[1482] The server sends a notification to the user saying, "Camera video data has been generated." This notification includes an access link to the generated video data.

[1483] Step 4:

[1484] The user checks the notification on their device and downloads the generated camera video data from the provided link. The user can use this data for their own purposes.

[1485] Step 5:

[1486] When a user wants to perform annotation, he or she accesses the server from the terminal, selects the "annotation work" menu, and requests the video data that requires annotation.

[1487] Step 6:

[1488] The server extracts the requested video data from storage and sends it to the user's device, where the user can add labels and annotations to the data.

[1489] Step 7:

[1490] The user completes the annotation process on the device and sends the completed annotation data to the server, which then stores the received annotation data in storage and organizes it to improve the data quality.

[1491] Step 8:

[1492] The server collects the accumulated annotation data and generates a learning model using a machine learning algorithm, which is then used to improve the drone's autonomous driving technology.

[1493] Step 9:

[1494] The server stores the generated learning model in storage, periodically evaluates the model's performance, and updates the model by retraining it using the latest data.

[1495] Step 10:

[1496] The server then posts the generated training datasets and models on an online market, making them available to other development companies.

[1497] Step 11:

[1498] The user accesses the online market from their device, searches for and selects the required training dataset and training model, and then completes the purchase procedure, after which the server confirms the purchase details.

[1499] Step 12:

[1500] Once the server confirms that the purchase procedure has been completed, it sends the training dataset and model to the user's device, allowing the user to easily obtain the resources necessary for developing autonomous driving technology.

[1501] Example 1

[1502] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1503] Conventionally, systems that generate drone camera video data and use it to improve autonomous driving technology have had several problems. Specifically, the entire process from data generation to annotation and machine learning model generation is often done manually, which is time-consuming and labor-intensive. Additionally, the generated data and models are often sold individually, making it difficult to manage and use them all at once.

[1504] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1505] In this invention, the server includes a means for a user to log in to an application on the server from a terminal, a means for generating drone camera image data using a generative AI model based on a user request, and a means for saving the generated camera image data, thereby automating a series of processes and enabling efficient generation and management of high-quality data.

[1506] A "user" is an entity that uses this system to generate camera image data, perform annotation work, and purchase datasets and learning models.

[1507] A "terminal" is a computing device that a user uses to access applications on a server.

[1508] The "server" is a system that uses a generative AI model based on user requests to generate and store camera image data, and also collects annotation data and performs machine learning.

[1509] A "generative AI model" is an artificial intelligence algorithm that generates camera image data based on parameters specified by the user.

[1510] A "prompt sentence" is an instruction sentence that includes parameters required for generating camera image data input by the user.

[1511] "Camera video data" refers to digital video data generated by AI that imitates or generates video captured by a drone.

[1512] "Annotation" is the process of adding labels and annotations to video data.

[1513] A "machine learning algorithm" is a computational method for building and updating a learning model based on collected annotation data.

[1514] A "learning model" is a model generated by a machine learning algorithm to analyze data and perform a specific task.

[1515] A "dataset" is a collection of annotated video data used to train and test a learning model.

[1516] The "online market" is an e-commerce platform that provides generated training datasets and models to users for purchase and download.

[1517] This invention is a system for generating drone camera image data and using it for autonomous driving technology. This system uses a generative AI model to generate diverse camera image data, store and sell that data, and accumulate annotation technology.

[1518] Data generation phase

[1519] 1. The user accesses the application on the server from their terminal and logs in by entering their login information (user ID and password). An authentication server is used for logging in.

[1520] 2. The user selects the "Camera Video Data Generation" menu from the main menu and enters the required parameters (e.g., video resolution, duration, specific scene, etc.).

[1521] 3. The server receives the user's request and passes parameters to the generative AI model (e.g., GPT-4) to start the video data generation process. The prompt uses the following format:

[1522] "Resolution: 1920x1080" "Duration: 10 minutes" "Scene: Urban traffic"

[1523] 4. The generative AI model generates camera image data based on the input parameters. The generated data is stored in the server's storage (e.g., AWS S3).

[1524] 5. The server notifies the user that the video data has been generated. This notification is sent via email or in-app notification.

[1525] Data annotation phase

[1526] 1. The user accesses the server from their terminal, selects the "Annotation" menu, and requests the video data that requires annotation.

[1527] 2. The server extracts the requested video data from storage (e.g., Google Cloud Storage) and sends it to the user's device.

[1528] 3. The user adds labels and annotations to the transmitted video data using an annotation tool such as VGG Image Annotator.

[1529] 4. The user uploads the annotated data to the server, which then stores the received annotation data in its storage.

[1530] Machine Learning Phase

[1531] 1. The server collects the accumulated annotation data and generates or updates a learning model using a machine learning algorithm (e.g., TensorFlow or PyTorch).

[1532] 2. The server stores the generated dataset along with the generated learning model in preparation for the next phase.

[1533] Data sales phase

[1534] 1. The server publishes the generated training dataset and model on an online marketplace, making the data available to other users.

[1535] 2. The user accesses the online market from their device and purchases the necessary datasets and learning models.

[1536] 3. After the purchase procedure is completed, the server sends the relevant dataset and model to the user's device.

[1537] Specific examples

[1538] When a logistics company employee wants to generate drone camera footage, they access the server from a secure device and request the generation of camera footage. This request includes a prompt like this:

[1539] "Resolution: 1920x1080" "Duration: 10 minutes" "Scene: Urban traffic"

[1540] The server uses the generative AI model to generate and store the video according to the request, after which the person in charge can download the video data and review it in-house.

[1541] Furthermore, annotation work, which involves labeling and annotating this data, can be easily performed. This annotation data can be sent back to the server and used as training data for machine learning to enable autonomous driving for other drones. Ultimately, the generated learning models and datasets can be sold on online markets, allowing other companies to use this technology.

[1542] As such, this invention is a system that comprehensively supports everything from drone camera image data generation to annotation, machine learning, and data sales, which is expected to improve technology and efficiency in the drone industry.

[1543] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1544] Step 1:

[1545] A user accesses an application on the server from a terminal and logs in by entering login information (user ID and password). The input is the user ID and password, which the authentication server receives and performs authentication processing. The output is the authentication result, and if the login is successful, the main menu screen is displayed.

[1546] Specific behavior:

[1547] 1. The user displays the login page on the terminal.

[1548] 2. Enter your user ID and password.

[1549] 3. The server sends a query to the authentication server.

[1550] 4. The authentication server returns the authentication result.

[1551] 5. If the server is successful in authenticating, it displays the main menu screen to the user.

[1552] Step 2:

[1553] The user selects the "Camera Video Data Generation" menu from the main menu and inputs parameters such as video resolution, duration, and specific scenes. The inputs are the video generation parameters. The server receives the input parameters and passes them to the generative AI model.

[1554] Specific behavior:

[1555] 1. The user clicks on the "Generate camera video data" menu.

[1556] 2. The user inputs parameters such as video resolution (e.g., 1920x1080), duration (e.g., 10 minutes), and scene (e.g., urban traffic).

[1557] 3. The user clicks the "Start Generation" button.

[1558] 4. The server retrieves the input parameters.

[1559] Step 3:

[1560] The server generates a prompt sentence from the acquired parameters and sends a request to the generative AI model. The input is the generated parameters, and the output is to generate a prompt sentence and pass it to the generative AI model. The generative AI model generates video data based on this prompt sentence.

[1561] Specific behavior:

[1562] 1. The server generates a prompt using the parameters it has obtained.

[1563] For example: "Resolution: 1920x1080", "Duration: 10 minutes", "Scene: City traffic".

[1564] 2. The server sends the generated prompt sentence to the generative AI model.

[1565] Step 4:

[1566] The generative AI model generates camera video data based on the prompt sentence and returns the generated data to the server. The input is the prompt sentence and the output is the generated camera video data. This data is sent to the server.

[1567] Specific behavior:

[1568] 1. The generative AI model parses the prompt.

[1569] 2. The AI ​​model uses its internal algorithm to generate video data.

[1570] 3. The generated video data is returned to the server.

[1571] Step 5:

[1572] The server saves the generated video data in storage (e.g., AWS S3) and notifies the user. The input is the generated camera video data, and the output is the URL or identification information of the saved data and a notification message.

[1573] Specific behavior:

[1574] 1. The server saves the generated video data in storage.

[1575] 2. Receive the URL and identification information of the stored data from the storage.

[1576] 3. The server notifies the user that video generation is complete (via email or in-app notification).

[1577] Step 6:

[1578] The user selects the "annotation work" menu and requests the required video data. The input is the user's request information, and the output is the video data provided by the server.

[1579] Specific behavior:

[1580] 1. The user clicks on the "Annotation Tasks" menu.

[1581] 2. The user selects the required video data and sends a request.

[1582] 3. The server receives the request and extracts the corresponding data.

[1583] Step 7:

[1584] The server sends the requested video data from storage (e.g., Google Cloud Storage) to the user's device. The input is the identification information of the requested video data, and the output is the video data to be sent to the user's device.

[1585] Specific behavior:

[1586] 1. The server queries the storage for the requested video data.

[1587] 2. The storage returns the video data to the server.

[1588] 3. The server sends the video data to the user's device.

[1589] Step 8:

[1590] The user adds labels and annotations to the transmitted video data using an annotation tool (e.g., VGG Image Annotator). The input is the received video data, and the output is the annotated video data.

[1591] Specific behavior:

[1592] 1. The user imports the received video data into the annotation tool.

[1593] 2. The user adds labels and annotations to the video data.

[1594] 3. Upload the annotated data to the server.

[1595] Step 9:

[1596] The server stores the annotation data received from the user. The input is the annotated video data, and the output is the stored annotation data.

[1597] Specific behavior:

[1598] 1. The server receives annotation data uploaded by the user.

[1599] 2. The server saves the annotation data in storage.

[1600] Step 10:

[1601] The server collects the accumulated annotation data and uses machine learning algorithms (e.g., TensorFlow or PyTorch) to generate new learning models or update existing models. The input is the accumulated annotation data, and the output is the generated or updated learning model.

[1602] Specific behavior:

[1603] 1. The server collects the accumulated annotation data.

[1604] 2. The server runs a machine learning algorithm and uses the data to train a model.

[1605] 3. Generate a new training model or update an existing model.

[1606] 4. Save the generated or updated model to storage.

[1607] Step 11:

[1608] The server posts the generated training model and dataset on an online market. The input is the training model and dataset, and the output is the posting information on the online market.

[1609] Specific behavior:

[1610] 1. The server uploads the training model and dataset to the online market.

[1611] 2. Sales information will be published on the online market.

[1612] Step 12:

[1613] Users access the online marketplace and purchase the datasets and learning models they need. The input is the user's purchase request, and the output is the purchased datasets and learning models.

[1614] Specific behavior:

[1615] 1. A user visits an online marketplace.

[1616] 2. The user selects the required dataset and learning model and completes the purchase process.

[1617] 3. The server sends the purchasing dataset and model to the user's device.

[1618] (Application example 1)

[1619] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1620] It is extremely important to generate drone camera video data under a variety of conditions and apply it to the development of autonomous driving and other technologies. However, existing systems have limited functionality for generating video data in real time under specific conditions desired by users, resulting in a lack of flexibility and diversity in the video data provided. Furthermore, the annotation process for the generated video data is cumbersome for users, and there is a need for more efficient annotation. Furthermore, there is a lack of efficient means for selling the generated datasets and learning models online, making it difficult to share and use the data.

[1621] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1622] In this invention, the server includes: means for using a generation AI to generate drone camera video data; means for saving the generated camera video data; means for providing the saved camera video data to a user; means for generating prompt statements based on user-specified conditions and generating camera video data in real time according to the prompt statements; means for the user to annotate the video data; means for collecting the annotated data and generating a learning model using a machine learning algorithm; and means for providing an online market for selling the generated learning datasets and learning models. This allows users to generate the video data they need under specific conditions in real time, enabling efficient annotation work. Furthermore, the generated datasets and learning models can be easily shared and used.

[1623] Key Word Definitions

[1624] "Generative AI" refers to artificial intelligence that generates camera image data based on user-specified conditions.

[1625] "Camera video data" refers to video information captured by a drone.

[1626] A "prompt statement" is an instruction statement that causes the generation AI to generate camera image data based on conditions specified by the user.

[1627] "Annotation" refers to the process of adding labels and annotations to camera video data.

[1628] "Learning model" refers to a trained model constructed by a machine learning algorithm using the generated camera image data and annotation data.

[1629] "Online market" refers to an internet platform for selling generated training datasets and learning models.

[1630] A "smartphone" refers to a mobile device that has Internet connectivity and can run a variety of applications.

[1631] "Real-time" refers to the property of responding immediately to user operations and inputs.

[1632] "User" refers to any individual or organization that uses this system to generate, store, annotate, and use drone camera footage data.

[1633] MODE FOR CARRYING OUT THE INVENTION

[1634] This invention is a system for generating drone camera image data and using it for autonomous driving technology and other applications. This system uses generative AI to generate camera image data under various conditions, stores and sells the data, and utilizes annotation technology. One embodiment of the present invention is described below.

[1635] 1. System Configuration

[1636] This system is connected via a terminal accessible by a user, a server that processes data, and the Internet.

[1637] 2. Data generation phase

[1638] Users can log in to the application on the server from their terminal, enter the necessary parameters (such as the video resolution, duration, and specific scenes) from the "Camera Video Data Generation" menu, and click the "Start Generation" button.

[1639] The server receives the user's request, passes parameters to the generation AI, and starts the video data generation process. The generation AI generates the necessary camera video data and stores it in the server's storage.

[1640] The server notifies the user that the generation of the video data is complete.

[1641] 3. Data annotation phase

[1642] The user logs in to the server from a terminal, selects the "annotation work" menu, and requests the video data that requires annotation.

[1643] The server extracts the requested video data from the storage and transmits it to the user's terminal.

[1644] The user performs annotation work by adding labels and annotations to the transmitted video data, and then transmits the completed data to the server.

[1645] The server stores the received annotation data in storage.

[1646] 4. Machine Learning Phase

[1647] The server collects the accumulated annotation data and generates a learning model using a machine learning algorithm, which is then periodically updated.

[1648] Along with the learning model, the generated dataset is also saved for the next phase.

[1649] 5. Data sales phase

[1650] The server posts the generated training datasets and learning models on an online marketplace, making the data available to other development companies.

[1651] Users access the online market from their devices and purchase the necessary datasets and learning models.

[1652] After the purchase process is completed, the server sends the dataset and model to the user's device.

[1653] 6. Hardware and Software Used

[1654] Servers are responsible for processing and storing data, and therefore use server machines equipped with high-performance processors and large-capacity storage.

[1655] The software uses TensorFlow to build generative AI models and web application frameworks such as Flask for annotation work.

[1656] The user terminals used are mobile terminals such as smartphones and tablets.

[1657] 7. Specific Examples

[1658] For example, if a user wants a video of a "sunny day in the mountains," they would input the following prompt sentence into the generation AI:

[1659] Prompt: "Generate a camera image of a mountainous area on a clear day."

[1660] The server runs a generation AI based on this prompt text and generates camera image data that matches the specified conditions.

[1661] The generated video data is displayed in real time on the user's device, and the user can then perform annotations on the video, which are then stored on the server.

[1662] This system allows users to generate the video data they need under specific conditions in real time, enabling them to perform annotation work efficiently. Furthermore, the generated datasets and learning models can be used by other companies and research institutions, improving the shareability and usefulness of data.

[1663] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1664] Program processing flow

[1665] Step 1:

[1666] A user logs in to the system from a terminal. As input, the user provides login information (user ID, password, etc.). The server receives this information, performs authentication, and determines whether the user can log in. If authentication is successful, the user gains access to the system. As output, it sends a message to the terminal indicating whether authentication was successful or not.

[1667] Step 2:

[1668] The user selects the "Generate Camera Video Data" menu on the terminal and inputs the required parameters (e.g., video resolution, duration, specific scene, etc.). As input, these parameters are sent from the terminal to the server. The server receives these parameters and generates a prompt sentence. As output, the generated prompt sentence is stored internally.

[1669] Step 3:

[1670] The server passes the prompt sentence to the generation AI and starts generating camera video data. As input, the server inputs the prompt sentence into the generation AI. The generation AI generates the drone camera video data based on this prompt sentence. As data processing, the generation AI performs the calculations necessary to generate the video data. As output, it obtains the generated video data.

[1671] Step 4:

[1672] The server saves the generated camera image data in storage and sends a notification to the user that generation is complete. As input, it receives the generated camera image data. As data calculation, it performs a procedure to save this data in storage. As output, it sends a message to the user notifying them that generation is complete.

[1673] Step 5:

[1674] The user selects the "Annotation" menu on the terminal and requests camera video data to be analyzed. As input, the user sends the request information. The server receives this information, extracts the requested video data from storage, and sends it to the user's terminal. As output, it sends the corresponding video data to the user.

[1675] Step 6:

[1676] The user performs annotation work by adding labels and annotations to the transmitted video data. As input, the video data is displayed on the device and the user adds annotations. As data processing, the annotation data is incorporated into the video data. As output, annotated data is generated.

[1677] Step 7:

[1678] The user sends the data after annotation work is completed to the server. The annotated data is sent to the server as input. The server receives this data and saves it in storage. A saving process is executed as data calculation. The annotation data is saved on the server as output.

[1679] Step 8:

[1680] The server collects the accumulated annotation data and generates a learning model using a machine learning algorithm. The annotation data is collected as input. As data processing, the machine learning algorithm uses the annotation data as training data to generate a learning model. As output, a new learning model is generated.

[1681] Step 9:

[1682] The generated training dataset and model are posted on the online market. As input, a new training dataset and model are provided. The server adds them to the online market platform. As data processing, a new item is registered in the market list. As output, it is displayed as an item available to users.

[1683] Step 10:

[1684] A user accesses the online market from their device and purchases the required dataset or learning model. The user's purchase request is sent as input. The server processes the purchase and, once the purchase is complete, sends the dataset or model to the user's device. The data calculation involves purchase processing and distribution. The output is a notification that the purchase has been completed, along with the dataset or model, sent to the user.

[1685] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1686] This invention relates to a system for generating drone camera video data and applying it to autonomous driving technology. Furthermore, by combining it with an emotion engine that recognizes user emotions, the user experience can be improved. This system generates camera video data using generative AI, stores and sells the data, and is equipped with annotation technology and emotion analysis functions.

[1687] System Overview

[1688] 1. Data generation phase

[1689] The user logs in to the application on the server from their terminal, enters the necessary parameters (e.g., video resolution, duration, specific scenes, etc.) from the "Camera Video Data Generation" menu, and clicks the "Start Generation" button.

[1690] The server receives the user's request, passes parameters to the generation AI, and starts the video data generation process. The generation AI generates the necessary camera video data and stores it in the server's storage.

[1691] The server notifies the user that the generation of the video data is complete.

[1692] 2. Use of Emotion Engine

[1693] The server monitors the user's emotional state using an emotion engine, for example, by analyzing the user's facial expressions and voice through a camera or microphone to determine the user's emotional state.

[1694] The server adaptively adjusts the parameters of the video data to be generated based on the analysis results of the emotion engine. For example, if the user is relaxed, the server can adjust the parameters to generate a video rich in scenery.

[1695] 3. Data annotation phase

[1696] The user accesses the server from a terminal, selects the "annotation work" menu, and requests the video data that requires annotation.

[1697] The server extracts the requested video data from storage and sends it to the user's device, where the user can add labels and annotations to the data.

[1698] The server uses an emotion engine to monitor the user's emotional state during annotation work. If the user feels stressed or fatigued, an alert is displayed to prompt the user to stop working.

[1699] 4. Machine Learning Phase

[1700] The server collects the accumulated annotation data and uses machine learning algorithms to generate learning models, which are then used to improve the drone's autonomous driving technology.

[1701] Along with the learning model, the generated dataset is also saved for the next phase.

[1702] 5. Data sales phase

[1703] The server then posts the generated training datasets and models on an online marketplace, making the data available to other development companies.

[1704] Users access the online market from their devices and purchase the necessary datasets and learning models.

[1705] After the purchase process is completed, the server sends the dataset and model to the user's device.

[1706] Specific examples

[1707] For example, consider the case where a person in a logistics company generates camera footage from a drone. The person in charge accesses the server from a secure terminal and requests the generation of camera footage. The server uses the generation AI to generate the footage according to the request and stores it. The person in charge is then notified and can download the generated video data and review it internally.

[1708] Next, as the annotation worker begins the task, the emotion engine monitors the worker's emotional state. If stress or fatigue is detected during the task, the system will display an alert urging the worker to stop, thereby supporting health management. The emotion engine can also adjust the way data is presented to improve the efficiency of the annotation task.

[1709] The resulting datasets and learning models will be sold to other development companies to help develop autonomous drone technology, which is expected to lead to the rapid adoption of the technology and improved safety.

[1710] The processing flow will be explained below.

[1711] Step 1:

[1712] The user logs in to the application on the server from their terminal. The user selects the "Camera Video Data Generation" menu and inputs the parameters required for video generation (e.g., resolution, duration, specific scenes, etc.). Then, the user clicks the "Start Generation" button to send the request to the server.

[1713] Step 2:

[1714] The server receives the user's request and activates the emotion engine, which analyzes the user's facial expressions and voice via a camera and microphone to recognize the user's emotional state (e.g., relaxed, focused, stressed, etc.).

[1715] Step 3:

[1716] Based on the analysis results of the emotion engine, the server issues instructions to the generation AI with adaptively adjusted parameters. For example, if the user is relaxed, the server instructs the AI ​​to generate images rich in scenery.

[1717] Step 4:

[1718] The generation AI generates camera image data based on parameters passed from the server, and the generated image data is stored in the server's storage.

[1719] Step 5:

[1720] The server sends a notification to the user saying, "Camera video data has been generated." This notification includes an access link to the generated video data.

[1721] Step 6:

[1722] The user checks the notification on their device and downloads the generated camera video data from the provided link. The user can then use this data for their own purposes.

[1723] Step 7:

[1724] When a user wants to perform annotation, he or she accesses the server from the terminal, selects the "annotation work" menu, and requests the video data that requires annotation.

[1725] Step 8:

[1726] The server extracts the requested video data from storage and sends it to the user's device, where the user can add labels and annotations to the data.

[1727] Step 9:

[1728] The server continuously monitors the user's emotional state during annotation work. If the emotion engine analyzes the user's emotional state and detects stress or fatigue, the server displays an alert urging the user to stop working.

[1729] Step 10:

[1730] The user completes the annotation process on the device and sends the completed annotation data to the server, which then stores the received annotation data in storage and organizes it to improve the data quality.

[1731] Step 11:

[1732] The server collects the accumulated annotation data and uses machine learning algorithms to generate learning models, which are then used to improve the drone's autonomous driving technology.

[1733] Step 12:

[1734] The server stores the generated learning model in storage, periodically evaluates the model's performance, and updates the model by retraining it using the latest data.

[1735] Step 13:

[1736] The server then posts the generated training datasets and models on an online market, making them available to other development companies.

[1737] Step 14:

[1738] The user accesses the online market from their device, searches for and selects the required training dataset and training model, and then completes the purchase procedure, after which the server confirms the purchase details.

[1739] Step 15:

[1740] Once the server confirms that the purchase procedure has been completed, it sends the training dataset and model to the user's device, allowing the user to easily obtain the resources necessary for developing autonomous driving technology.

[1741] Example 2

[1742] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1743] Conventional systems that generate drone camera video data and apply it to autonomous driving technology lack the ability to generate video that takes the user's emotional state into account, and provide insufficient support for efficient annotation. Furthermore, it has been difficult to properly manage and quickly provide the generated datasets and learning models for sale. This has hindered improvements to the user experience and efficient data generation.

[1744] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: a means for a user to log in to the server and input parameters for video data generation; a means for a generative AI model to generate camera video data based on a user request; a means for saving the generated camera video data in the server's storage; a means for notifying the user of the completion of generation; a means for monitoring the user's emotional state using an emotion engine and adjusting video generation parameters; a means for the user to perform annotation work to label and add annotations to the video data; a means for collecting the annotated data and generating a learning model using a machine learning algorithm; a means for listing the generated learning dataset and learning model on an online market; and a means for the user to purchase the dataset and model and transmit them from the server to a terminal. This enables video generation that takes the user's emotional state into consideration and efficient annotation work, thereby improving the quality and convenience of the datasets and learning models that are generated and provided.

[1745] A "user" is a person or organization that accesses the system through a terminal and generates and annotates camera image data.

[1746] A "server" is the central part of the system, a computer environment that generates, stores, notifies, analyzes, and sells data.

[1747] "Generative AI Model" means the artificial intelligence algorithms and programs used to generate camera footage data based on user requests.

[1748] "Camera image data" refers to still image or video data generated by a generative AI model.

[1749] "Storage" refers to a data storage device for temporarily or permanently storing the generated camera image data.

[1750] An "emotion engine" is a function or program that analyzes the user's emotional state and adjusts system operation and image generation accordingly.

[1751] "Annotation work" refers to the work in which a user labels or annotates camera video data.

[1752] A "machine learning algorithm" is an algorithm that learns from annotated data and acquires new knowledge.

[1753] A "learning model" is a computer model for performing a specific task, generated by a machine learning algorithm.

[1754] An "online market" is an e-commerce platform for selling generated training datasets and learning models.

[1755] A "dataset" is a set of data collected for a specific purpose.

[1756] This invention provides a system for generating drone camera video data and applying it to autonomous driving technology. Furthermore, by combining it with an emotion engine that recognizes user emotions, the user experience can be improved. This system generates camera video data using a generative AI model, stores and sells the data, and is equipped with annotation technology and emotion analysis functions.

[1757] Data generation phase

[1758] The user must log in to the application on the server from their terminal. This will display the home screen, allowing the user to begin operating the system. From the "Camera Video Data Generation" menu, enter the required parameters (e.g., video resolution 1920x1080, duration 10 minutes, specific scene "forest", etc.) and click the "Start Generation" button.

[1759] The server receives the parameters entered by the user and passes them to a generative AI model (e.g., OpenAI's DALL-E or GAN) to begin generating video data. The generated camera video data is stored in the server's storage, and the server sends a notification to the user when generation is complete. Notifications are sent via email or in-app notification.

[1760] Use of emotion engine

[1761] The server monitors the user's emotional state using an emotion engine (e.g., Amazon Rekognition, Microsoft Azure Emotion API). It analyzes the user's facial expressions and voice through a camera and microphone to determine the user's emotional state. Based on the analysis results of the emotion engine, the server adaptively adjusts the parameters of the video data to be generated. For example, if the user is relaxed, the server will adjust the parameters to generate video with a richer landscape.

[1762] Data annotation phase

[1763] The user accesses the server from their device, selects the "Annotation" menu, and requests the video data that requires annotation. The server extracts the requested video data from storage and sends it to the user's device. The user then labels and annotates the data.

[1764] The server uses an emotion engine to monitor the user's emotional state while annotating. If the user feels stressed or fatigued, an alert is displayed to prompt the user to stop working. This also supports the user's health management.

[1765] Machine Learning Phase

[1766] The server collects the accumulated annotation data and generates a learning model using machine learning algorithms (e.g., TensorFlow, PyTorch). This learning model is used to improve the drone's autonomous driving technology. The generated dataset is also stored along with the learning model for the next phase.

[1767] Data sales phase

[1768] The server posts the generated training datasets and learning models on an online market, where other development companies can use this data. Users can access the online market from their devices and purchase the datasets and learning models they need. After the purchase process is complete, the server sends the datasets and models to the user's device.

[1769] Specific examples

[1770] For example, a logistics company employee generates camera footage from a drone. The employee accesses the server from a secure device and requests the generation of camera footage. The server uses AI to generate the footage requested and stores it. The employee is then notified and can download the generated video data for internal review.

[1771] Next, as the annotation worker begins the task, the emotion engine monitors the worker's emotional state. If stress or fatigue is detected during the task, the system will display an alert urging the worker to stop, supporting the worker's health management. The emotion engine can also adjust the way data is presented to improve the efficiency of the annotation task.

[1772] The resulting datasets and learning models will be sold to other development companies to help develop autonomous drone technology, which is expected to lead to the rapid adoption of the technology and improved safety.

[1773] Prompt Sentence Examples

[1774] "To generate drone camera footage, a logistics company employee requests the start of generation, specifying a resolution of 1920x1080, a duration of 10 minutes, and a scene of 'warehouse work.' Upon receiving this request, the server uses the generation AI to generate and save the footage."

[1775] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1776] Step 1:

[1777] A user logs in to the server.

[1778] Input: User ID and password.

[1779] The server sends the user ID and password to the authentication server and verifies the authentication information.

[1780] Output: A successful authentication message is displayed on the device, and the home screen is displayed.

[1781] Step 2:

[1782] The user inputs parameters for generating video data and requests generation.

[1783] Input: video resolution (e.g. 1920x1080), duration (e.g. 10 minutes), scene (e.g. forest).

[1784] The server receives the parameters entered by the user and passes them to the generative AI model.

[1785] Output: The parameters are passed to the generative AI model, which starts generating video data.

[1786] Step 3:

[1787] The server generates video data using the generative AI model.

[1788] Input: The parameters passed by the user.

[1789] The server inputs parameters into a generative AI model (e.g., OpenAI's DALL-E or GAN) and generates camera image data.

[1790] Output: The generated video data is saved in the server storage.

[1791] Step 4:

[1792] The server notifies the user that generation is complete.

[1793] Input: Generated video data.

[1794] The server sends a generation completion notification to the user's device (e.g., email or in-app notification).

[1795] Output: A generation completion notification is displayed on the user's terminal.

[1796] Step 5:

[1797] The server monitors the user's emotional state using an emotion engine.

[1798] Input: User video and audio data.

[1799] The server uses an emotion engine (e.g., Amazon Rekognition, Microsoft Azure Emotion API) to analyze the user's facial expressions and voice.

[1800] Output: The user's emotional state is determined and this information is fed back to the generative AI model.

[1801] Step 6:

[1802] The server adjusts the parameters of the video data it generates based on the emotional state.

[1803] Input: The user's emotional state.

[1804] The server adjusts the parameters of the generative AI model based on the analysis results of the emotion engine.

[1805] Output: Video data adapted to the user's emotions is generated.

[1806] Step 7:

[1807] A user requests an annotation task.

[1808] Input: A request for video data stored on the server.

[1809] The server extracts the requested video data from the storage and transmits it to the user terminal.

[1810] Output: The video data is sent to the user's device.

[1811] Step 8:

[1812] Users add labels and annotations to the video data.

[1813] Input: Transmitted video data.

[1814] The user plays back the video data and adds labels and annotations to each frame.

[1815] Output: Annotation data with labels and annotations added is generated.

[1816] Step 9:

[1817] The server uses an emotion engine to monitor the user's emotional state during the annotation process.

[1818] Input: User video and audio data.

[1819] The server uses an emotion engine to analyze the user's emotional state in real time.

[1820] Output: If stress or fatigue is detected, an alert will be displayed to prompt a break from work.

[1821] Step 10:

[1822] The server collects the annotated data and generates a learning model.

[1823] Input: annotation data.

[1824] The server generates a learning model using a machine learning algorithm (e.g., TensorFlow, PyTorch).

[1825] Output: The generated learning model is saved on the server.

[1826] Step 11:

[1827] The server posts the generated datasets and learning models on an online marketplace.

[1828] Input: A training dataset and a training model.

[1829] The server registers the dataset and the learning model in an online marketplace.

[1830] Output: The dataset and the training model are published on an online marketplace.

[1831] Step 12:

[1832] A user purchases a dataset or model.

[1833] Input: Datasets and training models published on online marketplaces.

[1834] Users select the required datasets and learning models from the online market and complete the purchase process.

[1835] Output: The purchase is completed.

[1836] Step 13:

[1837] The server sends the dataset and model to the user's device.

[1838] Input: Purchased dataset and training model.

[1839] After the purchase procedure is completed, the server sends the dataset and model to the user's terminal.

[1840] Output: The dataset and model are sent to the user's device.

[1841] (Application example 2)

[1842] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1843] While conventional drone technology can generate and store camera video data, it lacks a consistent process for annotating the video data and generating learning models using machine learning algorithms. Furthermore, the user experience is hindered by a lack of functionality for providing video data that takes into account the user's emotional state and for adaptively displaying real-time traffic information. This invention aims to solve these issues and provide more advanced drone technology.

[1844] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1845] a means for using generative AI to generate drone camera footage data;

[1846] a means for storing the generated camera image data;

[1847] means for providing the stored camera image data to a user;

[1848] A means for allowing a user to perform annotation work on video data;

[1849] a means for collecting the annotated data and generating a learning model using a machine learning algorithm;

[1850] a means for providing an online marketplace for selling the generated training datasets and training models;

[1851] emotion analysis means for recognizing a user's emotion and adaptively adjusting parameters of the generated video data;

[1852] means for providing real-time traffic information and displaying information according to the emotional state of the user;

[1853] This makes it possible to provide adaptive video data according to the user's emotional state and display traffic information in real time.

[1854] A "drone" is a type of unmanned aerial vehicle (UAV) that is equipped with remote control and autonomous flight capabilities and can capture camera footage from the air.

[1855] "Camera video data" is digital data of images or videos captured by a camera, and is data that includes visual information.

[1856] "Generative AI" refers to a system that uses artificial intelligence technology to automatically perform specific tasks, and in this context refers to the technology that automatically generates drone camera footage data.

[1857] "Storage means" refers to a function or device that temporarily or long-term stores the generated data.

[1858] "Providing means" refers to the functions and interfaces that allow users to access the stored data.

[1859] "Annotation work" refers to the work in which a user adds labels and annotations to video data.

[1860] "Machine learning algorithm" refers to a computational method or procedure for analyzing data and automatically building a learning model.

[1861] A "learning model" is a model generated using a machine learning algorithm, and is a data structure that contains knowledge and patterns for performing a specific task.

[1862] "Online market" refers to a platform for buying and selling goods and services over the Internet.

[1863] "Emotion analysis means" refers to functions and technologies that analyze a user's facial expressions, voice, etc. to determine their emotional state.

[1864] "Traffic information" refers to real-time or historical data on roads and traffic networks, including traffic volume, congestion, and accident information.

[1865] "Means for displaying information" refers to devices such as displays and speakers for providing information visually or audibly to a user.

[1866] This invention relates to a system for generating drone camera image data and applying it to autonomous driving technology and traffic information provision. The specific configuration and implementation method of the system are described below.

[1867] 1. System Configuration

[1868] The system consists of the following major components:

[1869] Drones equipped with generative AI

[1870] server

[1871] User devices (e.g., smartphones and PCs)

[1872] Sentiment Analysis Engine

[1873] Online Market Platform

[1874] 2. Program processing and hardware / software used

[1875] The server uses generative AI to generate drone camera footage data. The generated data is stored in the server's storage. Users can access and view this data from their smartphones or PCs. Users can then perform annotation work, and the data is collected again by the server. Using this annotation data, the server runs a machine learning algorithm to generate a learning model. The generated learning dataset and learning model are then sold on the online market.

[1876] Furthermore, by using an emotion analysis engine, the user's emotional state can be analyzed in real time and the video data parameters can be adjusted accordingly. Specifically, if the user is feeling emotionally stressed, adaptive adjustments can be made, such as providing videos with a relaxing effect.

[1877] The real-time traffic information sent from the drone is displayed on a smartphone or PC in a format that corresponds to the user's emotional state. For example, if the user is feeling irritated, information on relaxing routes and congestion avoidance will be displayed.

[1878] Specific examples of the hardware and software used are as follows:

[1879] Drones: General commercial drones (e.g., DJI Phantom)

[1880] Server: Cloud environment (e.g., Amazon Web Services, Google Cloud Platform)

[1881] Sentiment analysis engine: OpenCV, Keras

[1882] Machine learning algorithms: TensorFlow, PyTorch

[1883] 3. Examples of concrete examples and prompts

[1884] For example, consider the use of this system while a user is driving a car. The user uses a smartphone to receive real-time traffic information from a drone. The emotion analysis engine detects when the user is irritated and suggests a relaxing route or notifies the user of rest spots. This improves the user's driving experience and safety.

[1885] Example prompt sentence:

[1886] "Build an application that provides real-time traffic information from surveillance footage. It should also have the ability to determine the user's emotions and adaptively provide information as needed."

[1887] This will enable the realization of an advanced system that combines drone technology with emotion analysis, significantly improving the user experience.

[1888] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1889] Step 1:

[1890] The user logs in to the application on the server from their terminal. User authentication information (user ID and password) is required as input. The server receives the authentication information and performs user authentication. If authentication is successful, the dashboard screen is displayed to the user.

[1891] Step 2:

[1892] The user selects the "Generate Camera Video Data" menu from the dashboard screen. The parameters of the video data to be generated (resolution, duration, specific scene, etc.) are required as input. The server receives the user's request, passes these parameters to the generation AI, and begins the video data generation process.

[1893] Step 3:

[1894] The generation AI generates camera image data based on input parameters. It processes the data by performing the necessary modeling and simulations to create image data that meets the specified conditions. The generated image data is then stored in the server's storage.

[1895] Step 4:

[1896] The server notifies the user that the video data has been generated, and the user can access the server from their device to check that the video data is now available for download.

[1897] Step 5:

[1898] Users annotate the downloaded video data. The type and content of the annotation (labels, tags, annotations, etc.) are required as input. The annotation editing tool is used on the device to label and annotate the video data.

[1899] Step 6:

[1900] Once the annotation process is complete, the video data is uploaded back to the server. The server then collects the uploaded data and stores it in storage. It also uses an emotion analysis engine to monitor the user's emotional state and displays appropriate feedback as needed.

[1901] Step 7:

[1902] The server provides the collected annotation data to a machine learning algorithm to generate a learning model. The annotation data is required as input. As data processing, the algorithm learns patterns in the data and builds a prediction model or classification model.

[1903] Step 8:

[1904] The generated learning models and datasets are posted on an online marketplace. The server uploads these datasets to the marketplace and makes them available for users to view.

[1905] Step 9:

[1906] Users access the online marketplace and purchase the necessary learning models and datasets. Once the purchase process is complete, the server provides the purchased data to the user's device.

[1907] Step 10:

[1908] The server provides real-time traffic information and displays information according to the user's emotional state. Real-time video data from the drone and emotion analysis results are required as input. This allows the user to assess real-time traffic conditions and select the optimal travel route.

[1909] This completes the entire system processing, allowing users to use advanced drone technology to generate video data, annotate it, utilize learning models, and receive traffic information.

[1910] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1911] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1912] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1913] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1914] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1915] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1916] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1917] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1918] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1919] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1920] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1921] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1922] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1923] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1924] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1925] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1926] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1927] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1928] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1929] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1930] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1931] The following is further disclosed regarding the above embodiment.

[1932] (Claim 1)

[1933] a means for using generative AI to generate drone camera footage data;

[1934] a means for storing the generated camera image data;

[1935] means for providing the stored camera image data to a user;

[1936] A means for allowing a user to perform annotation work on video data;

[1937] a means for collecting the annotated data and generating a learning model using a machine learning algorithm;

[1938] a means for providing an online marketplace for selling the generated training datasets and training models;

[1939] A system including:

[1940] (Claim 2)

[1941] The system of claim 1, wherein the generation AI generates camera image data based on a user request.

[1942] (Claim 3)

[1943] 10. The system of claim 1, wherein the learning model is periodically updated using the collected annotation data.

[1944] "Example 1"

[1945] (Claim 1)

[1946] A means for a user to log in to an application on a server from a terminal;

[1947] A means for generating drone camera video data using a generative AI model based on a user request;

[1948] a means for storing the generated camera image data;

[1949] means for notifying and providing the stored camera image data to a user;

[1950] A means for a user to perform annotation work on video data;

[1951] A means for collecting data annotated by users and generating a learning model using a machine learning algorithm;

[1952] A means to sell the generated training datasets and learning models on an online market; and

[1953] A system including:

[1954] (Claim 2)

[1955] The system of claim 1, wherein the generative AI model generates camera image data based on a prompt sentence entered by a user.

[1956] (Claim 3)

[1957] The system of claim 1, wherein the collected annotation data is used to periodically update the learning model using a machine learning algorithm.

[1958] "Application Example 1"

[1959] (Claim 1)

[1960] a means for using generative AI to generate drone camera footage data;

[1961] a means for storing the generated camera image data;

[1962] means for providing the stored camera image data to a user;

[1963] means for generating a prompt sentence based on a condition specified by a user and generating camera image data in real time in accordance with the prompt sentence;

[1964] A means for allowing a user to perform annotation work on video data;

[1965] a means for collecting the annotated data and generating a learning model using a machine learning algorithm;

[1966] a means for providing an online marketplace for selling the generated training datasets and training models;

[1967] A system including:

[1968] (Claim 2)

[1969] The system of claim 1 provides an application that allows a user to experience drone camera footage in real time using a smartphone.

[1970] (Claim 3)

[1971] 10. The system of claim 1, wherein the learning model is periodically updated using the collected annotation data.

[1972] "Example 2: Combining Emotion Engines"

[1973] (Claim 1)

[1974] A means for a user to log in to the server and input parameters for generating video data;

[1975] A means for the generation AI model to generate camera image data based on a user request;

[1976] A means for storing the generated camera image data in a server storage;

[1977] means for notifying a user of the completion of generation;

[1978] means for monitoring a user's emotional state using an emotion engine and adjusting video generation parameters;

[1979] An annotation tool that allows a user to add labels and annotations to the video data;

[1980] a means for collecting the annotated data and generating a learning model using a machine learning algorithm;

[1981] A means to post the generated training datasets and learning models on an online marketplace; and

[1982] A means for users to purchase datasets and models and send them from the server to their devices;

[1983] A system including:

[1984] (Claim 2)

[1985] 10. The system of claim 1, wherein the emotion engine adjusts the generation parameters based on a user request.

[1986] (Claim 3)

[1987] 10. The system of claim 1, wherein the learning model is periodically updated using the collected annotation data.

[1988] "Application example 2 when combining emotion engines"

[1989] (Claim 1)

[1990] a means for using generative AI to generate drone camera footage data;

[1991] a means for storing the generated camera image data;

[1992] means for providing the stored camera image data to a user;

[1993] A means for allowing a user to perform annotation work on video data;

[1994] a means for collecting the annotated data and generating a learning model using a machine learning algorithm;

[1995] a means for providing an online marketplace for selling the generated training datasets and training models;

[1996] emotion analysis means for recognizing a user's emotion and adaptively adjusting parameters of the generated video data;

[1997] means for providing real-time traffic information and displaying information according to the emotional state of the user;

[1998] A system including:

[1999] (Claim 2)

[2000] The system of claim 1, wherein the generation AI generates camera image data based on a user request.

[2001] (Claim 3)

[2002] 10. The system of claim 1, wherein the learning model is periodically updated using the collected annotation data. [Explanation of symbols]

[2003] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. a means for using generative AI to generate drone camera footage data; a means for storing the generated camera image data; means for providing the stored camera image data to a user; A means for allowing a user to perform annotation work on video data; a means for collecting the annotated data and generating a learning model using a machine learning algorithm; a means for providing an online marketplace for selling the generated training datasets and training models; A system including:

2. The system according to claim 1, wherein the generation AI generates camera image data based on a user request.

3. The system of claim 1 , wherein the learning model is periodically updated using the collected annotation data.

Images