Training data generation device, training data generation method, and training data generation program

Abstract

To generate effective image data for improving detection accuracy without relying on human labor. [Solution] An image conversion unit 21 inputs image data, which is learning data 31 for training an object detection model that detects an object with a detection target attribute from the image data, to an image conversion model 41 that converts image data, and obtains a converted image in which the image data is converted by the image conversion model 41. A learning data addition unit 22 adds the converted image obtained by the image conversion unit 21 to the learning data 31.

Description

[Technical field]

[0001] The present disclosure relates to a technique for generating training data for an object detection model. [Background technology]

[0002] There is an object detection model that detects objects with target attributes from image data. In order to improve the detection accuracy of an object detection model, it is necessary to train the object detection model using various image data as training data. Training data is often created manually, which is a heavy workload. In addition, when training a model for dangerous situations, image data of dangerous situations is required, but it is difficult for humans to actually reproduce dangerous situations, and training data cannot be created easily.

[0003] Patent Document 1 describes a technique in which, when there are insufficient teacher images, processing such as enlargement, reduction, movement, and synthesis is performed on the teacher images to generate new teacher images. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2018-169672 A Summary of the Invention [Problem to be solved by the invention]

[0005] In the method described in Patent Document 1, basically only image data similar to the original teacher image can be generated as a new teacher image, which makes it difficult to generate effective image data to improve detection accuracy. An object of the present disclosure is to enable the generation of image data that is effective for improving detection accuracy without human intervention. [Means for solving the problem]

[0006] The training data generating device according to the present disclosure comprises: an image conversion unit that inputs image data, which is training data to be trained by an object detection model, to an image conversion model that converts image data, and acquires a converted image in which the image data is converted by the image conversion model; a learning data adding unit that adds the converted image acquired by the image conversion unit to the learning data; Equipped with. Effect of the Invention

[0007] In the present disclosure, image data, which is learning data, is input to an image transformation model, and a converted image obtained by converting the image data using the image transformation model is added to the learning data. This makes it possible to generate image data that has been modified in various ways, such as by changing the background, changing surrounding objects, and changing the way the light hits the image, in addition to simple processing such as enlarging, reducing, moving, and compositing. This makes it possible to generate image data that is effective in improving detection accuracy. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a configuration diagram of a training data generating device 10 according to a first embodiment. [Diagram 2] 3 is a flowchart of processing by the training data generation device 10 according to the first embodiment. [Diagram 3] FIG. 4 is an explanatory diagram of an input to an image transformation model 41 according to the first embodiment. [Figure 4] FIG. 11 is a configuration diagram of a training data generating device 10 according to a second embodiment. [Diagram 5] 11 is a flowchart of processing by the training data generation device 10 according to the second embodiment. [Figure 6] FIG. 11 is an explanatory diagram of an input to a sentence generation model 42 according to the second embodiment. [Figure 7] FIG. 11 is an explanatory diagram of an output from a sentence generation model 42 according to the second embodiment. [Figure 8] FIG. 11 is an explanatory diagram of an input to a document conversion model 43 according to the second embodiment. [Figure 9]FIG. 11 is a configuration diagram of a training data generating device 10 according to a third embodiment. [Figure 10] 13 is a flowchart of a learning data adding process according to the third embodiment. [Figure 11] FIG. 11 is an explanatory diagram of an input to an attribute determination model 46 according to the third embodiment. [Figure 12] FIG. 13 is a configuration diagram of a training data generating device 10 according to a fourth embodiment. [Figure 13] 11 is a flowchart of processing by the training data generation device 10 according to the fourth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Embodiment 1 ***Configuration Description*** The configuration of a training data generating device 10 according to the first embodiment will be described with reference to FIG. The training data generating device 10 is a computer. The training data generation device 10 includes the following hardware components: a processor 11, a memory 12, a storage 13, and a communication interface 14. The processor 11 is connected to other hardware components via signal lines and controls the other hardware components.

[0010] The processor 11 is an IC that performs processing. IC is an abbreviation for Integrated Circuit. Specific examples of the processor 11 include a CPU, a DSP, and a GPU. CPU is an abbreviation for Central Processing Unit. DSP is an abbreviation for Digital Signal Processor. GPU is an abbreviation for Graphics Processing Unit.

[0011] The memory 12 is a storage device that temporarily stores data. Specific examples of the memory 12 include SRAM and DRAM. SRAM is an abbreviation for Static Random Access Memory. DRAM is an abbreviation for Dynamic Random Access Memory.

[0012] The storage 13 is a storage device that stores data. A specific example of the storage 13 is an SSD. SSD is an abbreviation for Solid State Drive. The storage 13 may also be a portable recording medium such as an SD (registered trademark) memory card, CompactFlash (registered trademark), NAND flash, a flexible disk, an optical disk, a compact disk, a Blu-ray (registered trademark) disk, or a DVD. SD is an abbreviation for Secure Digital. DVD is an abbreviation for Digital Versatile Disk.

[0013] The communication interface 14 is an interface for communicating with an external device. Specific examples of the communication interface 14 include Ethernet (registered trademark), USB, and HDMI (registered trademark) ports. USB is an abbreviation for Universal Serial Bus. HDMI is an abbreviation for High-Definition Multimedia Interface.

[0014] The training data generation device 10 includes, as functional components, an image conversion unit 21 and a training data addition unit 22. The functions of the functional components of the training data generation device 10 are realized by software. The storage 13 stores a program that realizes the function of each functional component of the training data generation device 10. The program is loaded into the memory 12 by the processor 11 and executed by the processor 11. In this way, the function of each functional component of the training data generation device 10 is realized. The storage 13 stores learning data 31. The learning data 31 is data for learning an object detection model. The learning data 31 includes a plurality of image data. The object detection model is a learning model that detects an object with a detection target attribute from the image data. The object detection model is, for example, a model configured using deep learning.

[0015] The training data generation device 10 is connected to an image transformation model 41 via a communication interface 14 . The image transformation model 41 is a so-called generative AI that transforms an image. AI is an abbreviation for Artificial Intelligence. The image transformation model 41 may be configured using algorithms such as BERT and GPT, as specific examples. BERT is an abbreviation for Bidirectional Encoder Representations from Transformers. GPT is an abbreviation for Generative Pretrained Transformer. The learning model 112 may be configured by combining a plurality of algorithms including these algorithms.

[0016] Here, the image transformation model 41 is connected via the communication interface 14. In other words, the image transformation model 41 is located outside the training data generation device 10. However, the image transformation model 41 may be located inside the training data generation device 10.

[0017] 1 shows only one processor 11. However, there may be a plurality of processors 11, and the plurality of processors 11 may cooperate to execute programs that realize the respective functions.

[0018] ***Explanation of Operation*** The operation of the training data generating device 10 according to the first embodiment will be described with reference to FIGS. The operation procedure of the training data generation device 10 according to the embodiment 1 corresponds to the training data generation method according to the embodiment 1. Moreover, the program for realizing the operation of the training data generation device 10 according to the embodiment 1 corresponds to the training data generation program according to the embodiment 1.

[0019] The processing of the training data generating device 10 according to the first embodiment will be described with reference to FIG. (Step S11: Input process) The image conversion unit 21 extracts one image data from the learning data 31. The image conversion unit 21 inputs the extracted image data and attribute information indicating a designated attribute, which is an attribute of a detection target of the object detection model, to the image conversion model 41. When there are multiple attributes of a detection target of the object detection model, the image conversion unit 21 inputs attribute information designated to the extracted image data from among the multiple attributes. At this time, the image conversion unit 21 inputs a prompt indicating a conversion rule for the image data to the image conversion model 41. The piece of image data extracted in step S11 is preferably an image in which a specific attribute was not detected or a different attribute was detected in the pre-learning model evaluation stage of the object detection model.

[0020] For example, as shown in Fig. 3, the image conversion unit 21 inputs a prompt instructing a conversion rule for image data together with image data and attribute information to the image conversion model 41. The conversion rule specifies what should be changed randomly and what should never be changed. Items that are randomly changed include type of clothing, gender, background, etc. Other items such as how the light hits the person, the number of people, etc. may be specified depending on the situation. As things that are never changed, information about attribute information and information about the angle of view are specified. Information about attribute information includes not only the attribute information itself but also things that accompany the attribute information. For example, if the attribute information is a white cane, things that accompany the attribute information include holding the white cane in the right hand. In the prompt in FIG. 3, simply "information about ${attribute information}" is specified, but it is also possible to specify more specifically "${attribute information} and the hand holding ${attribute information}". Here, ${attribute information} represents the attribute information specified in the input information ("white cane" in the above example). In other words, "${attribute information} and the hand holding ${attribute information}" is read as "white cane and the hand holding the white cane". Other ${***} are similarly replaced with information specified as *** in the input information.

[0021] When the image data to be created is already determined, the image conversion unit 21 may specify the image data to be created in detail as a prompt.

[0022] (Step S12: Image conversion process) The image transformation model 41 generates a designated number of transformed images by transforming the image data input in step S21. Then, the image transformation model 41 outputs the designated number of transformed images. The designated number is, for example, about 10% of the number of image data in the learning data 31. At this time, image conversion model 41 follows the instructions in the prompt and randomly changes those designated as those to be changed randomly for each converted image. Image conversion model 41 does not change those designated as those to never be changed. In other words, a converted image is generated in which the designated attributes are not changed and are maintained. Those that are not designated as either those to be changed randomly or those to never be changed can be changed as desired. The image conversion unit 21 acquires the designated number of converted images output by the image conversion model 41.

[0023] (Step S13: Adding learning data) The learning data adding unit 22 adds to the learning data 31 the designated number of converted images acquired by the image conversion unit 21 in step S12. The learning data addition process shown in FIG. 2 is executed for each piece of attribute information that needs to be added to the learning data.

[0024] ***Advantages of the First Embodiment*** As described above, the training data generation device 10 according to the first embodiment inputs image data, which is training data, to the image transformation model 41, and adds the converted image obtained by converting the image data by the image transformation model 41 to the training data 31. This makes it possible to generate image data with various changes made to it, such as changes to the background, changes to objects present in the vicinity, and changes to the way the light hits the image. This makes it possible to generate image data that is effective in improving detection accuracy.

[0025] The training data generation device 10 according to the first embodiment is capable of creating training data that is difficult to collect, such as data for dangerous situations. Furthermore, the training data generation device 10 according to the first embodiment is capable of increasing the variety of training data that is difficult to collect.

[0026] Embodiment 2 The second embodiment differs from the first embodiment in that a converted image is generated by generating an explanatory text for image data, converting the explanatory text, and generating image data from the converted explanatory text. In the second embodiment, this difference will be described, and the same points will not be described.

[0027] ***Configuration Description*** The configuration of a training data generating device 10 according to the second embodiment will be described with reference to FIG. The image conversion model 41 includes a sentence generation model 42 , a document conversion model 43 , and an image generation model 44 . The text generation model 42 is a model that generates explanatory text for image data. The document conversion model 43 is a model that converts explanatory text. The image generation model 44 is a model that generates image data from explanatory text.

[0028] ***Explanation of Operation*** The process of the training data generating device 10 according to the second embodiment will be described with reference to FIG. (Step S21: Image input processing) The image conversion unit 21 extracts one image data from the learning data 31. The image conversion unit 21 inputs the extracted image data and attribute information indicating a designated attribute, which is an attribute of a detection target of the object detection model, to the sentence generation model 42. When there are multiple attributes of a detection target of the object detection model, the image conversion unit 21 inputs attribute information indicating one or more attributes of the multiple attributes. At this time, the image conversion unit 21 inputs a prompt indicating a generation rule of an explanatory sentence to the sentence generation model 42.

[0029] For example, as shown in Fig. 6, the image conversion unit 21 inputs a prompt instructing the generation rule of the explanatory sentence together with the image data and the attribute information to the sentence generation model 42. The generation rule of the explanatory sentence specifies that the information on the angle of view and the attribute information must be explained.

[0030] (Step S22: Explanation generation process) The sentence generation model 42 generates and outputs an explanatory text that explains the image data input in step S21. At this time, as shown in Fig. 7, the sentence generation model 42 generates an explanatory text for the entire image data including an explanation of the information on the angle of view and the attribute information in accordance with the instructions in the prompt. Then, the image conversion unit 21 obtains the explanatory text output by the sentence generation model 42.

[0031] (Step S23: Explanation input process) The image conversion unit 21 inputs the explanatory text acquired in step S22 and attribute information indicating the designated attribute, which is the attribute of the detection target of the object detection model, to the document conversion model 43. At this time, the image conversion unit 21 inputs a prompt indicating the conversion rule of the explanatory text to the image conversion model 41. For example, as shown in Fig. 8, the image conversion unit 21 inputs a prompt specifying the conversion rules for the explanatory text together with the explanatory text and attribute information to the document conversion model 43. The conversion rules specify what is to be changed randomly and what is to never be changed, as in Fig. 3. Also, in Fig. 8, the explanatory text is converted into English. This is because there is a tendency that the accuracy of generating an image from an explanatory text in English is higher.

[0032] (Step S24: Explanation conversion process) The document conversion model 43 generates and outputs a converted sentence by converting the explanatory sentence input in step S23. Then, the image conversion unit 21 acquires the converted sentence output by the document conversion model 43. At this time, the document conversion model 43 randomly changes each converted sentence that is specified to be changed randomly according to the instructions in the prompt. Furthermore, the document conversion model 43 does not change any sentence that is specified to never be changed. Any sentence that is not specified to be changed randomly or never be changed can be changed as desired. Furthermore, the document conversion model 43 converts the explanatory sentence into English. Note that here, the converted sentence is generated as a prompt for image generation.

[0033] (Step S25: Converted sentence input process) The image conversion unit 21 inputs the converted sentence (prompt for image generation) acquired in step S24 to the image generation model 44. At this time, the image conversion unit 21 inputs a prompt to the image generation model 44 to instruct it to generate a specified number of image data representing the converted sentence.

[0034] (Step S26: Image generation process) The image generation model 44 generates a specified number of image data representing the converted sentence input in step S25. The image generation model 44 generates image data by randomly modifying each image data while representing the converted sentence. The image generation model 44 then outputs the specified number of converted images. The image conversion unit 21 acquires the designated number of converted images output by the image generation model 44.

[0035] (Step S27: Learning data addition process) The learning data adding unit 22 adds to the learning data 31 the designated number of converted images acquired by the image conversion unit 21 in step S26.

[0036] ***Effects of the second embodiment*** As described above, the training data generation device 10 according to the second embodiment generates an explanatory text for image data, converts the explanatory text, and generates image data from the converted explanatory text to generate a converted image. By generating a converted text by converting the explanatory text, it is possible to clarify the converted content.

[0037] Furthermore, the training data generation device 10 according to the first embodiment can use separate generation AIs for the sentence generation model 42, the document conversion model 43, and the image generation model 44. This makes it possible to use the document conversion model 43 as a model specialized for language processing, and the sentence generation model 42 and the image generation model 44 as models specialized for image processing. As a result, it is possible to increase the processing speed and improve the processing accuracy.

[0038] ***Other configurations*** <Variation 1> In the second embodiment, one converted sentence is generated, and a specified number of converted images are generated from the one converted sentence. A specified number of converted sentences may be generated, and one converted image may be generated from the one converted sentence. Alternatively, M converted sentences may be generated, and N converted images may be generated from the one converted sentence. In this case, M and N are integers equal to or greater than 1, and M×N=specified number.

[0039] Embodiment 3 The third embodiment differs from the first and second embodiments in that it is determined whether or not to add a converted image to the learning data 31. In the third embodiment, this difference will be described, and a description of the same points will be omitted.

[0040] ***Configuration Description*** The configuration of a training data generating device 10 according to the third embodiment will be described with reference to FIG. The training data generation device 10 is connected to an object detection model 45 and an attribute determination model 46 via the communication interface 14. The object detection model 45 is a model that detects an object having a detection target attribute from image data, and is a model that learns the learning data 31. The attribute determination model 46 is a model that determines whether or not an object with a specified attribute is included in image data. The attribute determination model 46, like the image conversion model 41, is a so-called generative AI.

[0041] ***Explanation of Operation*** The learning data adding process (step S13 in FIG. 2 and step S27 in FIG. 5) according to the third embodiment will be described with reference to FIG. In the learning data addition process, the following process is executed for each transformed image.

[0042] (Step S31: First input process) The learning data adding unit 22 inputs the converted image of the target and the attribute information input when generating the converted image to the object detection model 45. The attribute information input when generating the converted image is the attribute information input in step S11 in Fig. 2 or step S21 in Fig. 5.

[0043] (Step S32: Object detection process) The object detection model 45 detects an object having attributes indicated by the attribute information input in step S31 from the converted image of the target input in step S31. The object detection model 45 outputs a detection result and a certainty of the detection. The learning data adding unit 22 acquires the detection result and the confidence level output by the object detection model 45.

[0044] (Step S33: Confidence Determination Process) The learning data adding unit 22 determines whether the confidence level acquired in step S32 is equal to or greater than a threshold value. If the confidence level is equal to or greater than the threshold, the learning data adding unit 22 does not add the target transformed image to the learning data 31 and ends the processing for the target transformed image. On the other hand, if the confidence level is less than the threshold, the learning data adding unit 22 advances the processing to step S34.

[0045] (Step S34: Second input process) The learning data adding unit 22 inputs the target converted image and the attribute information input when generating the converted image to the attribute determination model 46. At this time, as shown in Fig. 11, the learning data adding unit 22 inputs a prompt to the attribute determination model 46 to instruct it to determine whether or not an object having an attribute indicated by the input attribute information is present in the input converted image.

[0046] (Step S35: Attribute determination process) The attribute determination model 46 determines whether or not an object having the attribute indicated by the attribute information input in step S31 exists in the converted image of the target input in step S34. The attribute determination model 46 outputs the determination result. The learning data adding unit 22 acquires the judgment result output by the attribute judgment model 46 .

[0047] (Step S36: Result determination process) If the determination result obtained in step S35 indicates that an object exists in the target transformed image, the learning data adding unit 22 advances the process to step S37. On the other hand, if the determination result obtained in step S35 indicates that an object does not exist in the target transformed image, the learning data adding unit 22 does not add the target transformed image to the learning data 31, and ends the process for the target transformed image.

[0048] (Step S37: Additional processing) The learning data adding unit 22 adds the target transformed image to the learning data 31.

[0049] ***Effects of the Third Embodiment*** The training data generation device 10 according to the third embodiment uses the object detection model 45 and the attribute determination model 46 to determine whether or not to add a converted image to the training data 31. This makes it possible to prevent a converted image that may deteriorate the detection accuracy or a converted image that does not require training from being added to the training data 31. Specifically, in the judgment in step S33, overlearning of the object detection model is prevented by not adding to the learning data 31 anything that can be detected by the current object detection model. Also, in the judgment in step S36, a converted image that does not have attribute information to be learned is not added to the learning data 31, thereby preventing unrelated images from being included in the learning data 31.

[0050] ***Other configurations*** <Variation 2> In the third embodiment, only the converted images that became registrable using the object detection model 45 and the attribute determination model 46 were added to the learning data 31. Therefore, even if the specified number of converted images were generated, the specified number of converted images were not necessarily added to the learning data 31. Therefore, when at least some of the converted images have not been added to the learning data 31, the learning data adding unit 22 may return the process to the image conversion unit 21 to regenerate the deficient number of converted images. The deficient number is a value obtained by subtracting the number of converted images added to the learning data 31 from the designated number. In other words, the learning data adding unit 22 may set the deficient number as a new designated number and restart the process from step S11 in Fig. 2 or step S21 in Fig. 5.

[0051] Embodiment 4 The fourth embodiment differs from the third embodiment in that the fourth embodiment performs learning of the object detection model 45. In the fourth embodiment, this difference will be described, and a description of the same points will be omitted.

[0052] ***Configuration Description*** The configuration of a training data generating device 10 according to the fourth embodiment will be described with reference to FIG. 9 in that the training data generation device 10 includes a training unit 23 as a functional component. The training unit 23 is realized by software, similar to the image conversion unit 21 and the training data addition unit 22.

[0053] ***Explanation of Operation*** The process of the training data generation device 10 according to the fourth embodiment will be described with reference to FIG. (Step S41: Generation and addition process) The image conversion unit 21 and the learning data adding unit 22 add the converted image to the learning data 31 by the processing described in the third embodiment.

[0054] (Step S42: Learning process) The learning unit 23 trains the object detection model 45 using the learning data 31 to which the converted image has been added in step S41.

[0055] (Step S43: Model evaluation process) The learning unit 23 evaluates the object detection model 45 learned in step S42. The object detection model 45 can be evaluated using an existing technique. If the evaluation obtained in step S43 is lower than the previous evaluation, the learning unit 23 advances the process to step S44. If the evaluation obtained in step S43 is equal to or higher than the previous evaluation but does not reach the standard, the learning unit 23 returns the process to step S41. If the evaluation obtained in step S43 is equal to or higher than the previous evaluation but reaches the standard, the learning unit 23 ends the process.

[0056] (Step S44: Data deletion process) The learning unit 23 deletes the converted image added in step S41 from the learning data 31. The learning unit 23 also returns the object detection model 45 to the state before learning in step S42. Then, the learning unit 23 returns the process to step S41.

[0057] ***Effects of the fourth embodiment*** As described above, the training data generation device 10 according to the fourth embodiment repeats the addition of training data 31 and the training of the object detection model 45. This makes it possible to improve the detection accuracy of the object detection model 45 without manual intervention.

[0058] ***Other configurations*** <Variation 3> In the above embodiment, each functional component is realized by software. However, as Modification 3, each functional component may be realized by hardware. The following describes Modification 3, focusing on the differences from the above embodiment.

[0059] When each functional component is realized by hardware, the training data generation device 10 includes an electronic circuit instead of the processor 11, the memory 12, and the storage 13. The electronic circuit is a dedicated circuit for realizing the functions of each functional component, the memory 12, and the storage 13.

[0060] The electronic circuits include single circuits, composite circuits, programmed processors, parallel programmed processors, logic ICs, GAs, ASICs, and FPGAs. GA stands for Gate Array. ASIC stands for Application Specific Integrated Circuit. FPGA stands for Field-Programmable Gate Array. Each functional component may be realized by one electronic circuit, or each functional component may be realized by distributing it among a plurality of electronic circuits.

[0061] <Modification 4> As a fourth modification, some of the functional components may be realized by hardware, and other functional components may be realized by software.

[0062] The processor 11, the memory 12, the storage 13, and the electronic circuitry are collectively referred to as a processing circuit. In other words, the functions of the functional components are realized by the processing circuit.

[0063] Furthermore, the term "part" in the above description may be read as a "circuit," "step," "procedure," "processing," or "processing circuit."

[0064] Various aspects of the present disclosure are summarized below as appendices. (Appendix 1) an image conversion unit that inputs image data, which is training data to be trained by an object detection model, to an image conversion model that converts image data, and acquires a converted image in which the image data is converted by the image conversion model; a learning data adding unit that adds the converted image acquired by the image conversion unit to the learning data; A learning data generating device comprising: (Appendix 2) The image conversion unit instructs the image conversion model to maintain a state in which the image data includes a designated attribute, thereby converting the image data. 2. A training data generating device as described in claim 1. (Appendix 3) The image conversion unit converts the image data by instructing the image conversion model not to change a portion of the image data related to a specified attribute and to change other portions. 3. The training data generating device according to claim 1 or 2. (Appendix 4) The image conversion model includes a text generation model that generates a description of image data, a document conversion model that converts the description, and an image generation model that generates image data from the description, The image conversion unit inputs the image data to the sentence generation model, acquires an explanatory text of the image data generated by the sentence generation model, inputs the acquired explanatory text to the document conversion model, acquires a converted text converted by the document conversion model, and inputs the converted text to the image generation model, acquires the image data generated by the image generation model as the converted image. 3. The training data generating device according to claim 1 or 2. (Appendix 5) The learning data generation device according to claim 4, wherein the image conversion unit converts the explanatory text by instructing the document conversion model to leave the portion of the explanatory text relating to a specified attribute unchanged and to change other portions. (Appendix 6) The training data generation device further comprises: a data determination unit that inputs the converted image acquired by the image conversion unit to the object detection model and determines whether or not a degree of certainty that the object has been detected by the object detection model is equal to or greater than an evaluation threshold value; Equipped with The learning data adding unit adds the converted image to the learning data when the data determining unit determines that the confidence level is less than the evaluation threshold. 6. A training data generation device according to any one of claims 1 to 5. (Appendix 7) the data determination unit inputs the transformed image and the designated attribute to an attribute determination model that determines whether or not an object having a specific attribute exists in image data when the certainty factor is less than the evaluation threshold value, and obtains a determination result of whether or not an object having the designated attribute exists in the transformed image; The learning data adding unit adds the converted image to the learning data when the data determining unit obtains a determination result that an object having the specified attribute exists in the converted image. 7. A training data generating device according to claim 6. (Appendix 8) A computer inputs image data, which is training data to be trained by an object detection model, into an image transformation model that transforms image data, and obtains a transformed image in which the image data is transformed by the image transformation model; A training data generating method in which a computer adds the transformed image to the training data. (Appendix 9) an image conversion process for inputting image data, which is training data to be trained by an object detection model, into an image conversion model that converts image data, and acquiring a converted image in which the image data is converted by the image conversion model; a learning data addition process of adding the converted image acquired by the image conversion process to the learning data; A learning data generation program that causes a computer to function as a learning data generation device that performs the above.

[0065] The above describes the embodiments and modifications of the present disclosure. Some of these embodiments and modifications may be combined and implemented. Also, one or some of them may be partially implemented. Note that the present disclosure is not limited to the above embodiments and modifications, and various modifications are possible as necessary. [Explanation of symbols]

[0066] 10 training data generation device, 11 processor, 12 memory, 13 storage, 14 communication interface, 21 image conversion unit, 22 training data addition unit, 23 learning unit, 31 training data, 41 image conversion model, 42 sentence generation model, 43 document conversion model, 44 image generation model, 45 object detection model, 46 attribute determination model.

Claims

1. an image conversion unit that inputs image data, which is training data to be trained by an object detection model, to an image conversion model that converts image data, and acquires a converted image in which the image data is converted by the image conversion model; a learning data adding unit that adds the converted image acquired by the image conversion unit to the learning data; Equipped with the image conversion model includes a sentence generation model that generates a description of the image data, a document conversion model that converts the description, and an image generation model that generates image data from the description, The image conversion unit inputs the image data and attribute information indicating a specified attribute to the sentence generation model, and instructs the generation of an explanatory text including an explanation of the specified attribute indicated by the attribute information included in the image data and an explanation of the angle of view of the image data, thereby obtaining an explanatory text of the image data including an explanation of the specified attribute and an explanation of the angle of view generated by the sentence generation model, inputs the obtained explanatory text to the document conversion model, and instructs the designation to leave the specified attribute and the angle of view unchanged but to change other parts, thereby obtaining a converted text converted by the document conversion model, and inputs the converted text converted to the image generation model, thereby obtaining the image data generated by the image generation model as the converted image.

2. The training data generation device further comprises: The transformed image acquired by the image transform unit is input to the object detection model. Then, it is determined whether the confidence level of the object detection model is equal to or higher than an evaluation threshold. Data judgment section Equipped with The learning data adding unit is configured to add a learning data to a data If it is determined that the converted image exists, the converted image is added to the learning data. The training data generating device according to claim 1 .

3. the data determination unit inputs the transformed image and the designated attribute to an attribute determination model that determines whether or not an object having a specific attribute exists in image data when the certainty factor is less than the evaluation threshold value, and obtains a determination result of whether or not an object having the designated attribute exists in the transformed image; The learning data adding unit adds the converted image to the learning data when the data determining unit obtains a determination result that an object having the specified attribute exists in the converted image. The training data generating device according to claim 2 .

4. A computer inputs image data, which is training data to be trained by an object detection model, into an image transformation model that transforms image data, and obtains a transformed image in which the image data is transformed by the image transformation model; The computer adds the transformed image to the training data; the image conversion model includes a sentence generation model that generates a description of the image data, a document conversion model that converts the description, and an image generation model that generates image data from the description, A learning data generation method in which a computer inputs the image data and attribute information indicating a specified attribute to the sentence generation model, and instructs the generation of an explanatory text including an explanation of the specified attribute indicated by the attribute information contained in the image data and an explanation of the angle of view of the image data, thereby obtaining an explanatory text of the image data including an explanation of the specified attribute and an explanation of the angle of view generated by the sentence generation model, inputs the obtained explanatory text to the document conversion model, and instructs the computer to leave the specified attribute and the angle of view unchanged but to change other parts, thereby obtaining a converted text converted by the document conversion model, and inputs the converted text converted to the image generation model, thereby obtaining the image data generated by the image generation model as the converted image.

5. an image conversion process for inputting image data, which is training data to be trained by an object detection model, into an image conversion model that converts image data, and acquiring a converted image in which the image data is converted by the image conversion model; a learning data addition process of adding the converted image acquired by the image conversion process to the learning data; A computer is caused to function as a learning data generating device that performs the following: the image conversion model includes a sentence generation model that generates a description of the image data, a document conversion model that converts the description, and an image generation model that generates image data from the description, In the image conversion process, the image data and attribute information indicating a specified attribute are input to the sentence generation model, and an instruction is given to generate an explanatory text including an explanation of the specified attribute indicated by the attribute information contained in the image data and an explanation of the angle of view of the image data, thereby obtaining an explanatory text of the image data including an explanation of the specified attribute and an explanation of the angle of view generated by the sentence generation model, the explanatory text obtained is input to the document conversion model, and an instruction is given to change other parts while leaving the specified attribute and the angle of view unchanged, thereby obtaining a converted text converted by the document conversion model, and the converted text converted by the image generation model is input to the image generation model, thereby obtaining the image data generated by the image generation model as the converted image.

Images