Data generation device, data generation method, and data generation program

Abstract

To provide a data generation device capable of improving the quality of a data set by surely adding data necessary for the data set in a short time.SOLUTION: A data generation device of the present invention includes an analysis unit configured to analyze a tendency of data in a data set related to a specific parameter with respect to the data set used for constructing a learned model, an acquisition unit configured to obtain information on insufficient data in the data set with respect to the parameter from an analysis result by the analysis unit, and a generation unit configured to generate additional data to be added to the data set based on the information on the insufficient data obtained by the acquisition unit.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to a data generation device, a data generation method, and a data generation program. [Background technology]

[0002] In machine learning, typified by deep learning, a trained model is generated by training a learning model using a large amount of data. To generate a high-quality trained model, high-quality training data (dataset) is required.

[0003] In this regard, the following Patent Documents 1 and 2 disclose techniques for analyzing a machine learning dataset and adding data necessary to improve the quality of the dataset. The techniques in Patent Documents 1 and 2 improve the quality of the dataset, enabling higher quality machine learning and generating a high-quality trained model. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Publication No. 2022-125222 [Patent Document 2] Japanese Patent Publication No. 2022-191682 Summary of the Invention [Problem to be solved by the invention]

[0005] However, the above technology adds necessary data from pre-prepared data to a dataset, which poses a problem that if necessary data is not available in the pre-prepared data, the data cannot be added to the dataset, and the quality of the dataset cannot be improved.

[0006] For example, when generating a trained model capable of detecting vehicles using image data of automobiles, bicycles, motorcycles, and other vehicles, even if an image with a bicycle in the bottom right corner is required, if there is no image data of such an image, the data cannot be added. Also, even if an attempt is made to take a photo of a bicycle with a camera and acquire new image data, it takes time to acquire the image data. In addition, if the AI ​​developer and the photographer are different people, the image data desired by the AI ​​developer may not be obtained.

[0007] The present invention has been made in view of the above-mentioned problems, and therefore an object of the present invention is to provide a data generation device, a data generation method, and a data generation program that can reliably add necessary data to a dataset in a short time to improve the quality of the dataset. [Means for solving the problem]

[0008] The above object of the present invention can be achieved by the following means.

[0009] (1) A data generation device having an analysis unit that analyzes trends in data in a dataset related to a specific parameter for a dataset used to construct a trained model; an acquisition unit that obtains information on missing data in the dataset for the parameter from the analysis results by the analysis unit; and a generation unit that generates additional data to add to the dataset based on the information on the missing data obtained by the acquisition unit.

[0010] (2) The data generation device described in (1) above, wherein the analysis unit determines the distribution of the data within the range of the parameter values ​​and calculates the frequency of occurrence of the data for each divided range into which the range is divided, and the acquisition unit obtains information on the divided ranges in which the frequency of occurrence is below a predetermined threshold as information on the missing data.

[0011] (3) The data generating device according to (2) above, further comprising a deletion unit that deletes part of the data in the divided ranges whose occurrence frequency is equal to or greater than a predetermined threshold.

[0012] (4) The data generation device described in (1) or (2) above, wherein the generation unit includes a trained model configured to generate the additional data based on information about the missing data, and the generation unit generates a prompt to be input to the trained model based on the information about the missing data.

[0013] (5) The data generation device according to (1) or (2) above, wherein the generation unit generates the additional data by modifying part of the data set based on information about the missing data.

[0014] (6) The data generation device according to (1) or (2) above, further comprising a reception unit that receives a user's selection of approval or disapproval of the additional data.

[0015] (7) The data generation device according to (6) above, wherein, when the selection of non-approval is accepted, the accepting unit further accepts conditions for generating additional data input by the user.

[0016] (8) The data generation device according to (7) above, further comprising a request unit that requests the user to input additional data if it is not possible to generate additional data according to the generation conditions.

[0017] (9) The data generating device according to (1) or (2) above, further comprising a deletion unit that deletes a portion of the additional data.

[0018] (10) The data generation device according to (1) or (2) above, wherein the dataset is a dataset of image data, and the parameters include the number, type, position, or size of objects contained in the image.

[0019] (11) The data generating device according to (1) or (2) above, wherein the dataset is a dataset of voice data, and the parameters include the gender or age of the person uttering the voice, the speed of the voice, or the pitch of the voice.

[0020] (12) A data generation method comprising the steps of: (a) analyzing trends in data in a dataset related to a specific parameter for a dataset used to build a trained model; (b) obtaining information on missing data in the dataset for the parameter from the analysis results of (a); and (c) generating additional data to add to the dataset based on the information on the missing data obtained in (b).

[0021] (13) A data generation program that causes a computer to execute the steps of: (a) analyzing trends in data in a dataset related to specific parameters for a dataset used to build a trained model; (b) obtaining information on missing data in the dataset for the parameters from the analysis results of (a); and (c) generating additional data to be added to the dataset based on the information on the missing data obtained in (b). [Effects of the Invention]

[0022] According to the present invention, necessary data can be reliably added to a dataset in a short time, thereby improving the quality of the dataset. [Brief explanation of the drawings]

[0023] Advantages and features provided by one or more embodiments of the present invention will be more fully understood from the following detailed description and the accompanying drawings, which are for purposes of illustration only and are not intended to define the limits of the invention. [Figure 1] FIG. 1 is a block diagram showing a schematic configuration of a data generating device. [Figure 2] FIG. 2 is a diagram illustrating the contents stored in a storage unit of the data generating device. [Figure 3] 5 is a flowchart showing the procedure of a data generation process according to the first embodiment. [Figure 4]FIG. 10 is a diagram illustrating an example of threshold condition information. [Figure 5] FIG. 10 is a diagram showing an example of a determination result of a threshold condition for a data distribution related to a parameter “position of an object.” [Figure 6] FIG. 10 is a diagram showing an example of a determination result of a threshold condition for a data distribution related to the parameter "size of object." [Figure 7] FIG. 10 is a diagram illustrating an example of prompt generation information. [Figure 8] 10 is a flowchart showing the procedure of a data generation process according to the second embodiment. [Figure 9] 9 is a flowchart following FIG. 8. [Figure 10] 10 is a flowchart showing the procedure of a data addition process. DETAILED DESCRIPTION OF THE INVENTION

[0024] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will be described below with reference to the accompanying drawings, but the scope of the present invention is not limited to the disclosed embodiments.

[0025] (First embodiment) 1 is a block diagram showing a schematic configuration of a data generating device 10 according to one embodiment of the present invention. The data generating device 10 is a computer such as a server or a PC (Personal Computer).

[0026] As shown in FIG. 1, the data generating device 10 includes a control unit 11, a storage unit 12, a communication unit 13, a display unit 14, and an input unit 15, which are interconnected by a bus.

[0027] The control unit 11 is configured with a CPU (Central Processing Unit) and memories such as RAM (Random Access Memory) and ROM (Read Only Memory), and controls the above-mentioned units and performs various arithmetic processing according to a program.

[0028] The storage unit 12 is configured by a hard disk drive (HDD) or a solid state drive (SSD), and stores various programs and various data.

[0029] The communication unit 13 is an interface for communicating with other devices, and uses a communication interface conforming to various wired or wireless standards.

[0030] The display unit 14 is, for example, a liquid crystal display, and displays various information.

[0031] The input unit 15 includes a keyboard, a numeric keypad, a mouse, etc., and receives input of various instructions and information.

[0032] FIG. 2 is a diagram showing the contents stored in the storage unit 12 of the data generating device 10. The storage unit 12 of the data generating device 10 stores a plurality of data sets 100 used to construct a trained model. The plurality of data sets 100 include a data set of image data and a data set of audio data. The image data and audio data have been annotated (e.g., labeled) by a user such as an AI developer.

[0033] Furthermore, the memory unit 12 of the data generating device 10 stores programs corresponding to the analysis unit 110, acquisition unit 120, generation unit 130, and deletion unit 140. The analysis unit 110 analyzes trends in data in the dataset 100 related to specific parameters. The acquisition unit 120 obtains information on data that is lacking in the dataset 100 from the analysis results by the analysis unit 110. The generation unit 130 generates additional data to be added to the dataset 100 based on the data information obtained by the acquisition unit 120. The generation unit 130 includes a generation AI (trained model) that automatically generates the additional data. The deletion unit 140 deletes excess data in the dataset 100. The functions of each of the above units are fulfilled by the control unit 11 executing the corresponding program.

[0034] The data generating device 10 may include components other than those described above, or may not include some of the components described above. For example, the data generating device 10 may include a GPU (Graphics Processing Unit).

[0035] In the data generating device 10 configured as described above, from the viewpoint of improving the quality of the dataset 100 used for machine learning, the trend of the data in the dataset 100 relating to a specific parameter is analyzed, and information on data that is missing in the dataset 100 is obtained. Then, additional data is generated based on the information on the missing data, and the generated additional data is added to the dataset 100. Hereinafter, the operation of the data generating device 10 will be described with reference to FIGS. 3 to 7.

[0036] In the following, an example will be described in which a trained model capable of detecting an object such as a car is generated by training the learning model using a large amount of image data. The image data of dataset 100 is image data obtained by photographing an object with a camera. To perform high-quality machine learning and generate a high-quality trained model, image data with various positions of objects within the photographed images is required. Also, image data with various sizes of objects within the photographed images is required.

[0037] 3 is a flowchart showing the procedure of the data generation process executed by the data generating device 10. The process of the flowchart shown in FIG. 3 is executed by the control unit 11 in accordance with a program stored in the storage unit 12 of the data generating device 10.

[0038] (Step S101) First, the control unit 11 determines a dataset to be analyzed. More specifically, the control unit 11 determines a dataset to be analyzed from among a plurality of datasets 100 stored in the storage unit 12 in accordance with a user instruction. In this embodiment, the control unit 11 determines a dataset 100 that includes a large amount of image data as the dataset to be analyzed.

[0039] (Step S102) Next, the control unit 11 identifies the format of the data included in the dataset to be analyzed. More specifically, the control unit 11 identifies the format of the data of the dataset 100 determined in the processing of step S101. In this embodiment, the control unit 11 identifies the format of the data of the dataset 100 as an image data format by referring to an extension such as "JPEG" or "TIFF," for example.

[0040] (Step S103) Next, the control unit 11 generates statistical data according to the format. More specifically, the control unit 11 obtains the distribution of data in the data set 100 within the parameter value range for the format parameters identified in the process of step S102. Then, the control unit 11 calculates the frequency of occurrence of data for each divided range obtained by dividing the parameter value range into a plurality of ranges.

[0041] In this embodiment, the control unit 11 obtains the distribution of image data of the dataset 100 within a rectangular range normalized to 1 x 1 for the image data parameter "position of the object." Then, the control unit 11 calculates the appearance frequency of image data for each of nine divided ranges, which are obtained by dividing the rectangular range normalized to 1 x 1 into 3 x 3 ranges.

[0042] Furthermore, for the image data parameter "object size," the control unit 11 obtains the distribution of image data of the data set 100 within a rectangular range normalized to 1 x 1. Then, the control unit 11 divides the rectangular range normalized to 1 x 1 into nine 3 x 3 ranges, and calculates the appearance frequency of image data for each of these divided ranges.

[0043] The control unit 11 refers to information on the labels attached to the image data (bounding box information, segmentation information) to determine the position and size of the object in the image.

[0044] (Step S104) Next, the control unit 11 identifies divided ranges in which the occurrence frequency of data is equal to or less than a predetermined threshold. More specifically, the control unit 11 identifies divided ranges in which the occurrence frequency is calculated in the process of step S103 and has an occurrence frequency equal to or less than a lower limit threshold set in advance for each divided range.

[0045] In this embodiment, the control unit 11 compares the appearance frequency of each of the nine divided ranges for which the appearance frequency of the parameter "object position" has been calculated with a predetermined lower threshold value by referring to the threshold condition information 200 (see FIG. 4) stored in the storage unit 12. Then, the control unit 11 identifies a divided range for which the appearance frequency is equal to or less than the lower threshold value.

[0046] Furthermore, the control unit 11 compares the appearance frequency of each of the nine divided ranges for which the appearance frequency of the parameter "object size" has been calculated with a predetermined lower threshold value by referring to the threshold value condition information 200. Then, the control unit 11 identifies a divided range whose appearance frequency is equal to or less than the lower threshold value.

[0047] In this embodiment, the control unit 11 refers to the threshold condition information 200 to identify not only divided ranges whose occurrence frequency is equal to or less than the lower threshold, but also divided ranges whose occurrence frequency is equal to or greater than the upper threshold.

[0048] 4 is a diagram showing an example of threshold condition information 200. As shown in FIG. 4, threshold condition information 200 includes identification information 210, category information 220, type information 230, quadrant information 240, upper / lower limit information 250, and threshold information 260.

[0049] Identification information 210 is information for identifying threshold conditions. Category information 220 is information indicating parameters (object position, object size) to which threshold conditions are applied. Type information 230 is information indicating the division type (3x3) of the division range for the parameter value to which threshold conditions are applied. Quadrant information 240 is information indicating the position of the division range to which threshold conditions are applied (see Figure 5). Upper / lower limit information 250 is information for identifying whether the threshold for the occurrence frequency is an upper limit threshold or a lower limit threshold, and threshold information 260 is information indicating the threshold for the occurrence frequency. The threshold may be a value set in advance as a default value or a value set by the user.

[0050] For example, if the upper / lower limit information 250 is "lower" and the threshold information 260 is "0.5," the threshold value of 0.5 becomes the lower threshold value. Therefore, for a divided range to which a threshold condition in which the upper / lower limit information 250 is "lower" and the threshold information 260 is "0.5" is applied, if the occurrence frequency calculated for the divided range is 0.5 or less, the divided range is identified as one in which the occurrence frequency is equal to or less than the lower threshold value.

[0051] Furthermore, if the upper / lower limit information 250 is "upper" and the threshold information 260 is "0.2," the threshold value of 0.2 becomes the upper threshold value. Therefore, for a divided range to which the threshold condition of the upper / lower limit information 250 being "upper" and the threshold information 260 being "0.2" is applied, if the occurrence frequency calculated for the divided range is 0.2 or more, the divided range is identified as one in which the occurrence frequency is equal to or greater than the upper threshold value.

[0052] FIG. 5 is a diagram showing an example of the determination result of the threshold condition for the data distribution related to the parameter "position of the object."

[0053] Figure 5 plots the positions of objects (labels) within an image, normalizing the image width and height range to 1x1. The horizontal axis of Figure 5 indicates the position of the object (label) in the width direction of the image, and the vertical axis indicates the position of the object (label) in the height direction of the image. The shading in Figure 5 indicates the number of image data (or labels), with a higher density indicating a larger number of image data (or labels).

[0054] 5, it can be seen that the density is high in the central portion and the number of image data (labels) is large, while the density is low in the peripheral portion and the number of image data (labels) is small. In other words, it can be seen from FIG. 5 that in data set 100, there are many image data items in which the object is located in the central portion of the image and few image data items in which the object is located in the peripheral portion of the image.

[0055] In this embodiment, the range of the parameter "object position" value (X: 0 to 1, Y: 0 to 1) is divided into 9 parts, 3 x 3, and the appearance frequency of image data (label) is calculated for each divided part. The appearance frequency of each divided part is then compared with an upper or lower threshold set for each divided part, and divided parts having an appearance frequency equal to or greater than the upper threshold or equal to or less than the lower threshold are identified.

[0056] The appearance frequency of image data is calculated based on, for example, the total number of image data in the data set 100 and the number of image data included in the divided range. Alternatively, the appearance frequency of image data is calculated based on, for example, the total number of labels of image data in the data set 100 and the number of labels included in the divided range.

[0057] 4 again, in the threshold condition information 200 of this embodiment, the threshold conditions are set so that for the six divided ranges located from the center down in Fig. 5, if the application frequency is equal to or less than the lower threshold of 0.5, the image data is deemed to be insufficient and the divided range is identified. Also, for the three divided ranges located at the top of Fig. 5, the threshold conditions are set so that for the three divided ranges located at the top, if the application frequency is equal to or greater than the upper threshold of 0.2, the image data is deemed to be excessive and the divided range is identified. This is because, in an image of a car, the sky is generally visible at the top of the image, so there is not a need for a large amount of image data in which the car is located at the top of the image.

[0058] 5, of the six divided ranges located below the center, five divided ranges excluding the central divided range have an appearance frequency equal to or less than the lower threshold, and are therefore judged as "NG" because there is insufficient image data. In other words, in this embodiment, these five divided ranges are identified as divided ranges in which the appearance frequency of image data is equal to or less than the lower threshold.

[0059] 5, the upper center divided range among the three divided ranges located at the top has an appearance frequency equal to or greater than the upper threshold, and is therefore judged as "NG" because the image data is excessive. In other words, in this embodiment, the upper center divided range is identified as the divided range in which the appearance frequency of image data is equal to or greater than the upper threshold.

[0060] FIG. 6 is a diagram showing an example of the determination result of the threshold condition for the data distribution related to the parameter "size of object."

[0061] Figure 6 plots the range of width and height of objects (labels) in an image, normalized to 1x1. The horizontal axis of Figure 6 indicates the width of the object (label), and the vertical axis indicates the height of the object (label). The shading in Figure 6 indicates the number of image data (or labels), with a higher density indicating a larger number of image data (or labels).

[0062] 6, it can be seen that, of the 3x3 divided ranges, the four divided ranges located on the upper right side have a large number of image data (labels), while the remaining five divided ranges have a small number of image data (labels). In other words, it can be seen from FIG. 6 that, in data set 100, there is a large amount of image data in which the object size is large, and there is a small amount of image data in which the object size is small.

[0063] 6, the five divided ranges with small object sizes have an appearance frequency equal to or less than the lower threshold, and are therefore judged as "NG" because there is insufficient image data. In other words, in this embodiment, these five divided ranges are identified as divided ranges in which the appearance frequency of image data is equal to or less than the predetermined threshold.

[0064] As described above, the process of step S104 identifies the divided ranges in which the appearance frequency of image data is equal to or less than the lower limit threshold. That is, information on image data that is missing in the data set 100 is obtained.

[0065] (Step S105) Next, the control unit 11 generates a data generation condition. More specifically, the control unit 11 generates a prompt for the generation AI to generate additional data (image data) to compensate for the lack of data for the divided range identified in the process of step S104.

[0066] 7 is a diagram showing an example of prompt generation information 300. As shown in FIG. 7, prompt generation information 300 includes identification information 310, upper / lower limit information 320, location information 330, and presence / absence information 340.

[0067] The identification information 310 is information for identifying the threshold condition. The upper / lower limit information 320 is information for identifying whether the threshold is an upper limit threshold or a lower limit threshold.

[0068] Location information 330 is information that indicates the position of the divided range to which the operation of the generation AI is applied. The top of each item in location information 330 contains information written in natural language, and the bottom contains information for the generation AI. For example, for the natural language "bottom right," it is written for the generation AI as "when the width of the image is x and the height is y, and x and y are normalized to 0-1, the range of (x, y) is the rectangular area formed by (0.66, 0.00) and (1.00, 0.33)."

[0069] The presence / absence information 340 is information indicating whether or not additional data needs to be generated. The top of each item in the presence / absence information 340 contains information written in natural language, while the bottom contains information for the generation AI. For example, for a segmented range where image data is insufficient, the natural language description is "in," while the generation AI description is "{number of missing images} uniformly distributed so that {label} has the maximum density {threshold} in the range." Furthermore, for a segmented range where image data is not insufficient, the natural language description is "avoid," while the generation AI description is "{label} does not exist in the range."

[0070] As described above, in the process of step S105, a prompt is generated to make the generation AI generate additional data for a divided range in which image data is insufficient.

[0071] (Step S106) Next, the control unit 11 generates additional data. More specifically, the control unit 11 inputs the prompt generated in the processing of step S105 to the generation AI to generate the additional data. Note that, from the viewpoint of improving the quality of the additional data, it is preferable to input image data of the dataset 100 to the generation AI and train the generation AI.

[0072] (Step S107) Next, the control unit 11 creates a new dataset. More specifically, the control unit 11 adds the additional data generated in the process of step S106 to the dataset 100 to be analyzed, thereby creating a new dataset 100.

[0073] Furthermore, for divided ranges in which the occurrence frequency is equal to or greater than a predetermined upper threshold, the control unit 11 deletes some of the image data from the new data set 100. The number of image data to be deleted is calculated based on the occurrence frequency and the upper threshold.

[0074] (Step S108) Then, the control unit 11 notifies the user that the additional data has been generated. More specifically, the control unit 11 notifies the user that the additional data has been generated by, for example, displaying information that the additional data has been generated on the display unit 14. The user performs annotations, such as labeling, on the additional data.

[0075] (Step S109) Next, the control unit 11 determines whether the annotation is complete. More specifically, the control unit 11 determines whether the annotation is complete by, for example, determining whether a button operation by the user indicating the completion of the annotation is recognized.

[0076] If it is determined that the annotation is not complete (step S109: NO), the control unit 11 waits until the annotation is complete.

[0077] On the other hand, if it is determined that the annotation is complete (step S109: YES), the control unit 11 proceeds to the process of step S110.

[0078] (Step S110) If it is determined that the annotation is complete (step S109: YES), the control unit 11 generates statistical data regarding the new data set. More specifically, for the new data set created in the process of step S107, the control unit 11 obtains the distribution of image data for each parameter, as in the process of step S103, and calculates the appearance frequency of image data for each divided range of the value of each parameter.

[0079] (Step S111) Next, the control unit 11 determines whether the shortage condition has been resolved. More specifically, the control unit 11 determines whether the shortage condition has been resolved by determining whether there is any divided range whose occurrence frequency is equal to or less than the lower threshold for the multiple divided ranges whose occurrence frequencies have been calculated in the processing of step S110. If there is no divided range whose occurrence frequency is equal to or less than the lower threshold, the control unit 11 determines that the shortage condition has been resolved.

[0080] When it is determined that the shortage condition has not been resolved (step S111: NO), the control unit 11 returns to the process of step S105. As a result, the processes of steps S105 to S111 are repeated until the shortage condition is resolved.

[0081] On the other hand, if it is determined that the shortage condition has been resolved (step S111: YES), the control unit 11 proceeds to the process of step S112.

[0082] (Step S112) If it is determined that the shortage condition has been resolved (step S111: YES), the control unit 11 officially registers the new dataset and ends the process. More specifically, the control unit 11 stores the dataset to which the additional data has been added in the process of step S107 as an official dataset in the storage unit 12, and ends the process.

[0083] As described above, according to the data generation process of this embodiment, the tendency of image data related to specific parameters is analyzed for the dataset 100 used for training the learning model. Then, information on image data that is missing in the dataset 100 is obtained from the analysis result of the tendency of the image data, and image data (additional data) to be added to the dataset 100 is generated based on the information on the missing image data.

[0084] According to this configuration, the missing image data is added to the dataset 100, thereby improving the quality of the dataset 100. Furthermore, because the missing image data is automatically generated by the generation AI, the quality of the dataset 100 can be reliably improved in a short time even if the missing image data is not prepared in advance.

[0085] Additionally, according to the data generation process of this embodiment, the process of analyzing the data trends is repeatedly executed for a new data set to which additional data has been added, so that the quality of the data set 100 can be improved more reliably.

[0086] In addition, according to the data generation process of this embodiment, excessive image data is deleted based on the analysis results of the image data trends, thereby preventing over-learning of the learning model.

[0087] Furthermore, machine learning requires a variety of image data under different conditions, and if the AI ​​developer and the photographer are different, it is difficult for the photographer, who is not the AI ​​developer, to determine whether the image data has been collected in sufficient quantity. Furthermore, if there is insufficient image data, the image must be taken again. However, if there is a lack of image data for environmental differences (rain, snow, etc.), the photographer's efforts alone cannot resolve the issue, and the only option is to wait for the conditions to occur. As a result, with typical machine learning, it takes a long time to obtain the necessary data, and situations may arise where the learning model must be trained using a low-quality dataset.

[0088] However, according to the data generating device 10 of this embodiment, the tendency of image data regarding specific parameters is analyzed and image data to be added to a dataset is generated, so that necessary image data can be reliably added in a short time, thereby avoiding a situation where a learning model has to be trained using a low-quality dataset.

[0089] (Variation) In the above-described embodiment, an example has been described in which additional data is generated using a generation AI (trained model). However, the additional data may also be generated using image processing such as augmentation. When generating additional data using augmentation, the shift amount and magnification rate of the object are calculated from information on missing data, and the additional data is generated by modifying part of the image data in the dataset 100.

[0090] In the above-described embodiment, the additional data is generated by a general generation AI that generates image data according to prompts. However, instead of a general generation AI, the additional data may be generated by an autonomous AI agent that repeats execution and evaluation until it achieves its goal. The autonomous AI agent repeatedly generates additional data until it determines that the missing condition has been resolved.

[0091] In the above-described embodiment, the position and size of an object are used as an example of parameters for image data. However, the parameters for image data are not limited to the position and size of an object, and may be the number and type of objects. Furthermore, the parameters for image data are not limited to annotation data such as the position, size, number, and type of objects, and may be metadata such as the location where the image in the image data was captured, the weather, and the subject of the capture. Both annotation data and metadata may be used as parameters for image data.

[0092] (Second embodiment) Next, a second embodiment of the present invention will be described with reference to Figures 8 to 10. This embodiment is an embodiment in which user approval for additional data is received. Note that the configuration of the data generation device according to this embodiment is the same as that of the first embodiment, and therefore a description of the configuration of the data generation device will be omitted.

[0093] 8 and 9 are flowcharts showing the procedure of the data generation process according to this embodiment. The process of the flowcharts shown in Fig. 8 and 9 is executed by the control unit 11 in accordance with a program stored in the storage unit 12 of the data generation device 10.

[0094] (Steps S201 to S208) Steps S201 to S208 are the same as steps S101 to S108 in Fig. 3, and therefore detailed description will be omitted. When the user is notified in the process of step S208, the user checks the generated additional data (image data). If the user determines that the additional data is appropriate, the user approves the additional data. On the other hand, if the user determines that the additional data is inappropriate, the user disapproves the additional data.

[0095] (Step S209) The control unit 11 determines whether or not a selection of approval or disapproval for the additional data has been accepted. More specifically, the control unit 11 determines whether or not a button operation by the user to select approval or disapproval for the additional data has been accepted.

[0096] When it is determined that the selection of approval or disapproval has not been received (step S209: NO), the control unit 11 waits until the selection of approval or disapproval is received.

[0097] On the other hand, if it is determined that the selection of approval or disapproval has been received (step S209: YES), the control unit 11 proceeds to the process of step S210.

[0098] (Step S210) When it is determined that the selection of approval or disapproval has been received (step S209: YES), the control unit 11 determines whether or not the selection of disapproval has been received. More specifically, the control unit 11 determines whether or not the selection received in the processing of step S209 is the selection of disapproval of the additional data, thereby determining whether or not the selection of disapproval has been received.

[0099] When it is determined that the selection of non-approval has not been accepted (step S210: NO), the control unit 11 proceeds to the process of step S213, assuming that the selection of approval has been accepted.

[0100] On the other hand, if it is determined that a non-approval selection has been received (step S210: YES), the control unit 11 proceeds to the process of step S211.

[0101] (Step S211) If it is determined that the selection of non-approval has been accepted (step S210: YES), the control unit 11 executes a data addition process for adding data in accordance with the user's instruction. The data addition process will be described in detail later.

[0102] (Step S212) Subsequently, the control unit 11 determines whether or not data has been added. More specifically, the control unit 11 determines whether or not new additional data has been added to the data set 100 in the data addition process of step S211.

[0103] If it is determined that no data has been added (step S212: NO), the control unit 11 ends the process.

[0104] On the other hand, if it is determined that data has been added (step S212: YES), the control unit 11 proceeds to the process of step S213.

[0105] (Steps S213 to S216) Steps S213 to S216 are similar to the processes in steps S109 to S112 in FIG. 3, and therefore detailed description thereof will be omitted.

[0106] As described above, according to the data generation process of this embodiment, the tendency of image data related to specific parameters is analyzed for the dataset 100 used for training the learning model. Then, information on image data that is missing in the dataset 100 is obtained from the analysis result of the tendency of the image data, and image data (additional data) to be added to the dataset 100 is generated based on the information on the missing image data.

[0107] According to the data generation process of this embodiment, the user checks the additional data, and if the user approves the additional data, the data set to which the additional data has been added is registered as an official data set. On the other hand, if the user does not approve the additional data, a data addition process is executed to add data in accordance with the user's instructions.

[0108] Fig. 10 is a flowchart showing the steps of the data addition process shown in step S211 of Fig. 8. The data addition process is executed, for example, when there is a problem with the additional data automatically generated by the generation AI and the user selects not to approve the additional data.

[0109] (Step S301) First, the control unit 11 requests the user to input data generation conditions. More specifically, the control unit 11 requests the user to input the generation conditions for additional data, for example, by displaying information prompting the user to input the generation conditions for additional data on the display unit 14. The user inputs the generation conditions for the desired additional data.

[0110] (Step S302) Next, the control unit 11 accepts the data generation conditions. More specifically, the control unit 11 accepts the generation conditions for the additional data input by the user as a result of the request made in the process of step S301.

[0111] (Step S303) Subsequently, the control unit 11 generates additional data. More specifically, the control unit 11 generates additional data using, for example, a generation AI based on the generation conditions for additional data received in the process of step S302.

[0112] (Step S304) Subsequently, the control unit 11 determines whether appropriate additional data has been generated. More specifically, the control unit 11 determines whether appropriate additional data has been generated by determining whether the additional data generated in the processing of step S303 matches the generation condition accepted in the processing of step S302. In this embodiment, the control unit 11 determines whether appropriate additional data has been generated, for example, by inputting the generated additional data into another trained model and referring to the discrimination result of the other trained model.

[0113] If it is determined that appropriate additional data has been generated (step S304: YES), the control unit 11 ends the process.

[0114] On the other hand, if it is determined that appropriate additional data has not been generated (step S304: NO), the control unit 11 proceeds to the process of step S305.

[0115] (Step S305) If it is determined that appropriate additional data has not been generated (step S304: NO), the control unit 11 requests the user to register the additional data. More specifically, the control unit 11 requests the user to register the additional data, for example, by displaying information prompting the user to register the additional data on the display unit 14. The user determines whether or not the additional data can be acquired and registered.

[0116] When requesting the registration of additional data, the control unit 11 can display the missing conditions and the number of sheets of additional data on the display unit 14. With this configuration, the user can understand the direction of the additional data from the missing conditions and the number of sheets, and can easily determine whether or not the additional data can be acquired.

[0117] (Step S306) Next, the control unit 11 determines whether or not it is impossible to register data. More specifically, the control unit 11 determines whether or not it is impossible to register additional data by, for example, determining whether or not a user button operation indicating that it is impossible to register additional data has been recognized.

[0118] If it is determined that the data cannot be registered (step S306: YES), the control unit 11 ends the process.

[0119] On the other hand, if it is determined that data registration is possible (step S306: NO), the control unit 11 proceeds to the process of step S307.

[0120] (Step S307) If it is determined that data registration is possible (step S306: NO), the control unit 11 accepts the additional data and ends the process. More specifically, the control unit 11 accepts the additional data input by the user as a result of the request in the process of step S305 and ends the process.

[0121] As described above, according to the data addition process of this embodiment, the user specifies the conditions for generating additional data, and the additional data is generated. Furthermore, if appropriate additional data is not generated, the user inputs the additional data. With this configuration, even if there is a problem with the additional data automatically generated by the generation AI, appropriate additional data can be added, thereby reliably improving the quality of the dataset 100.

[0122] The data addition process can also be used to add data for parameters other than those used in the data generation process. For example, if there are no problems with the position or size of the object but there is a lack of image data for a snowy day, the user can input data generation conditions such as "add a case where the weather is snowy based on the input image" and have the generation AI generate image data for a snowy day. If the generation AI cannot generate appropriate image data for a snowy day, the user can search for image data for a snowy day from the image data they have prepared in advance and register it as additional data.

[0123] Furthermore, in the data addition process of this embodiment, the control unit 11 may delete the additional data. For example, if the user selects not to approve the additional data, the control unit 11 causes the display unit 14 to display a list of the additional data. The user then checks the list of additional data and selects the additional data that the user wishes to delete. The control unit 11 then deletes the additional data selected by the user.

[0124] The present invention is not limited to the above-described embodiments, but can be modified in various ways within the scope of the claims.

[0125] For example, in the above-described embodiment, an example was described in which a trained model is generated by training an untrained training model using a dataset 100 of image data. However, a dataset from which additional data is generated by the data generation device of the present invention can be used not only for training an untrained training model, but also for new training of a trained model. For example, if a trained model capable of detecting vehicles such as cars, bicycles, and motorcycles is to be made capable of detecting scooters, the present invention can also be applied when collecting image data of scooters and training a new trained model. According to the present invention, the necessary image data related to scooters can be reliably added in a short time.

[0126] In the above-described embodiment, an example was described in which additional data is generated by analyzing the trend of image data relating to specific parameters in an image data set 100. However, the data set for which additional data is generated by the data generation device of the present invention is not limited to an image data set, and may be, for example, a voice data set. When generating additional data for a voice data set, the trend of the voice data in the data set 100 is analyzed using parameters such as the gender or age of the person uttering the voice, the voice speed, or the voice pitch, and information on missing voice data is obtained. Then, based on the information on the missing voice data, additional data is generated, for example, by a generation AI.

[0127] The means and methods for performing various processes in the data generation device according to the above-described embodiments can be realized by either a dedicated hardware circuit or a programmed computer. The program may be provided by a computer-readable recording medium such as a USB (Universal Serial Bus) memory or a DVD (Digital Versatile Disc)-ROM, or may be provided online via a network such as the Internet. In this case, the program recorded on the computer-readable recording medium is typically transferred to and stored in a storage unit such as a hard disk drive. The program may also be provided as standalone application software or may be incorporated into the software of the data generation device as a function of the device.

[0128] While embodiments of the present invention have been described and illustrated in detail, the disclosed embodiments are made for purposes of illustration and example only and are not intended to be limiting, and the scope of the present invention should be construed by the language of the appended claims. [Explanation of symbols]

[0129] 10. Data generating device; 11 control section, 12 storage section, 13 Communications Department, 14 Display section, 15 input section, 100 datasets.

Claims

1. An analysis unit that analyzes trends in data in a dataset regarding specific parameters for a dataset used to build a trained model; an acquisition unit that acquires information about data that is missing from the dataset for the parameter from the analysis result by the analysis unit; a generation unit that generates additional data to be added to the data set based on the information on the missing data obtained by the acquisition unit; A data generating device having:

2. the analysis unit obtains a distribution of the data within a range of the parameter values, and calculates an appearance frequency of the data for each divided range obtained by dividing the range; The data generating device according to claim 1 , wherein the acquiring unit acquires information on a divided range in which the occurrence frequency is equal to or less than a predetermined threshold as information on the missing data.

3. The data generating device according to claim 2 , further comprising a deletion unit that deletes part of the data in the divided ranges whose occurrence frequency is equal to or greater than a predetermined threshold.

4. The generation unit includes a trained model configured to generate the additional data based on information of the missing data, The data generation device according to claim 1 or 2, wherein the generation unit generates a prompt to be input to the trained model based on information about the missing data.

5. The data generating device according to claim 1 , wherein the generating unit generates the additional data by correcting a part of data in the data set based on information about the missing data.

6. 3. The data generating device according to claim 1, further comprising a receiving unit that receives a user's selection of approval or disapproval of the additional data.

7. The data generation device according to claim 6 , wherein, when the selection of non-approval is accepted, the accepting unit further accepts a generation condition for the additional data input by the user.

8. 8. The data generating device according to claim 7, further comprising a request unit that requests a user to input additional data when it is impossible to generate additional data according to said generation conditions.

9. The data generating device according to claim 1 , further comprising a deleting unit that deletes a portion of the additional data.

10. the dataset is an image data dataset, The data generating device according to claim 1 , wherein the parameters include the number, type, position, or size of objects included in the image.

11. the dataset is a dataset of audio data, 3. The data generating device according to claim 1, wherein the parameters include the gender or age of the person uttering the voice, the speed of the voice, or the pitch of the voice.

12. (a) analyzing trends in data in a dataset used to build a trained model with respect to specific parameters; (b) obtaining information on missing data in the data set for the parameter from the analysis result in the step (a); (c) generating additional data to be added to the dataset based on the information on the missing data obtained in step (b); A data generation method comprising:

13. (a) analyzing trends in data in a dataset used to build a trained model with respect to specific parameters; a step (b) of obtaining information on missing data in the data set for the parameter from the analysis result in the step (a); a step (c) of generating additional data to be added to the dataset based on the information on the missing data obtained in the step (b); A data generation program that causes a computer to execute the above.

Images