A detection image generation method and system

Abstract

This invention discloses a detection image generation method and system, relating to the field of detection image generation technology, to solve the problem of inefficient detection image generation. The detection image generation method includes: first, obtaining ground truth values ​​and random noise; then, based on the ground truth values, generating a target foreground image using a foreground target generation model, and generating a background image using a background generation model based on the random noise; finally, superimposing the target foreground image and the background image to obtain the detection image. The detection image generation method and system provided by this invention are used to generate detection images for detection tasks, and to perform small-sample augmentation.

Description

Technical Field

[0001] This invention relates to the field of detection image generation technology, and in particular to a detection image generation method and system. Background Technology

[0002] Current image generation models are not tailored to specific detection tasks, leading to two main problems: First, they don't adjust for the application scenario of the generated object, but instead implicitly simulate and reconstruct the distribution of the target image. Second, they cannot generate corresponding ground truth values ​​simultaneously with the generated image to meet data augmentation needs, resulting in high manual annotation costs even after generating detection images. In fact, generating images targeting specific objects within an image has significant application value. For example, in small-sample target detection and recognition, image sample augmentation must revolve around the target of interest in the image to generate meaningful detection images.

[0003] Therefore, there is an urgent need for a method and system for generating detection images. Summary of the Invention

[0004] The purpose of this invention is to provide a method and system for generating detection images, which are used to generate detection images for detection and recognition tasks, while saving annotation costs.

[0005] To achieve the above objectives, the present invention provides the following technical solution:

[0006] A detection image generation method, the generation method comprising:

[0007] Obtain the detection ground truth and random noise; the detection ground truth includes the target category, target location, target size, and target quantity in text format;

[0008] Based on the detected ground truth values, a target foreground image is generated using a foreground target generation model;

[0009] Based on the random noise, a background image is generated using a background generation model;

[0010] The target foreground image and the background image are superimposed to obtain the detection image.

[0011] Compared with existing technologies, the detection image generation method provided by this invention generates a target foreground image based on the ground truth detection value using a foreground target generation model, and generates a background image based on random noise using a background generation model. The target foreground image and the background image are then superimposed to obtain the detection image. This allows for a focus on target generation, producing detection images tailored to the detection task. Furthermore, since the generation of the target foreground image uses the ground truth detection value as input, it enables the simultaneous generation of the detection image and its corresponding ground truth detection value, avoiding the need for bar charting on the detection image and saving annotation costs.

[0012] Optionally, generating a target foreground image using a foreground target generation model based on the detected ground truth specifically includes:

[0013] Based on the detection ground truth, a target image template is generated; the target image template includes multiple detection boxes; the position, number, and size of the detection boxes are determined by the target position, the target quantity, and the target size, respectively; the grayscale of the detection boxes is determined by the target category.

[0014] Using the target image template as input, a foreground target generation model is used to generate targets of the corresponding category within each detection box to obtain the target foreground image.

[0015] Optionally, the foreground target generation model is obtained by training a conditional GAN-based target generation network.

[0016] Optionally, before generating the target foreground image using the foreground target generation model based on the detected ground truth, the generation method further includes the step of training the conditional GAN-based target generation network to obtain the foreground target generation model, specifically including:

[0017] Obtain a first training sample set; the first training sample set includes multiple training target image templates and a training target foreground image corresponding to each training target image template;

[0018] Using the training target image template as input, the conditional GAN-based target generation network generates a predicted target foreground image corresponding to each training target image template.

[0019] Calculate the loss within the detection box range of the training target image template based on all the predicted target foreground images and all the training target foreground images;

[0020] Determine whether the first iteration termination condition has been met based on the loss.

[0021] If so, then the conditional GAN-based target generation network used in the current iteration shall be used as the foreground target generation model;

[0022] If not, the parameters of the conditional GAN-based target generation network are adjusted according to the loss, the predicted target foreground image is regenerated, and the loss within the detection box range of the training target image template is calculated until the first iteration termination condition is met.

[0023] Optionally, the random noise conforms to a standard normal distribution.

[0024] Optionally, the background generation model is obtained by training a background generation network based on VAE-GAN.

[0025] Optionally, the background generation network based on VAE-GAN includes an encoder, a decoder, and a discriminator connected in sequence.

[0026] Optionally, before generating a background image using the background generation model based on the random noise, the generation method further includes the step of training the VAE-GAN-based background generation network to obtain a background generation model, specifically including:

[0027] Obtain a second training sample set; the second training sample set includes multiple background images for training.

[0028] Using the training background image as input, the encoder is used to obtain a feature vector, and the KL divergence loss between the feature vector and the standard normal distribution vector is calculated.

[0029] Using the feature vector as input, the decoder generates a predicted background image, and the pixel-level loss between the predicted background image and the training background image is calculated.

[0030] The GAN loss is calculated using the discriminator based on the training background image and the predicted background image.

[0031] Based on the KL divergence loss, the pixel-level loss, and the GAN loss, determine whether the second iteration termination condition has been met;

[0032] If so, then the decoder used in the current iteration is used as the background generation model;

[0033] If not, the parameters of the encoder, the decoder and the discriminator are adjusted according to the KL divergence loss, the pixel-level loss and the GAN loss, the feature vector and the predicted background image are regenerated, and the KL divergence loss, pixel-level loss and GAN loss are calculated until the second iteration termination condition is met.

[0034] Optionally, the step of overlaying the target foreground image and the background image to obtain the detection image specifically includes:

[0035] The target foreground image and the background image are superimposed at the pixel level to obtain the detection image.

[0036] The present invention also provides a detection image generation system, the generation system comprising:

[0037] The acquisition module is used to acquire detection ground truth and random noise; the detection ground truth includes target category, target location, target size, and target quantity in text format;

[0038] The foreground generation module is used to generate a target foreground image based on the detected ground truth using a foreground target generation model;

[0039] The background generation module is used to generate a background image based on the random noise using a background generation model.

[0040] The overlay module is used to overlay the target foreground image and the background image to obtain a detection image.

[0041] Compared with the prior art, the beneficial effects of the detection image generation system provided by the present invention are the same as those of the detection image generation method described in the above technical solutions, and will not be repeated here. Attached Figure Description

[0042] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this invention, illustrate exemplary embodiments of the invention and are used to explain the invention, but do not constitute an undue limitation of the invention. In the drawings:

[0043] Figure 1 This is a flowchart of the generation method provided in Embodiment 1 of the present invention;

[0044] Figure 2 This is a schematic diagram of the generation process provided in Embodiment 1 of the present invention;

[0045] Figure 3 This is a schematic diagram of the background generation network based on VAE-GAN provided in Embodiment 1 of the present invention;

[0046] Figure 4 This is a system block diagram of the generation system provided in Embodiment 2 of the present invention. Detailed Implementation

[0047] To facilitate a clear description of the technical solutions in the embodiments of the present invention, the terms "first" and "second" are used to distinguish identical or similar items with essentially the same function and effect. For example, the first threshold and the second threshold are merely used to distinguish different thresholds and do not limit their order. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, and that the terms "first" and "second" are not necessarily different.

[0048] It should be noted that in this invention, the terms "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in this invention should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0049] Example 1:

[0050] Existing image generation models generate a single, general image without considering the target object within that image or distinguishing between the target and the background. For example, when detecting a ship on the sea, existing models might generate an image of the sea, possibly without a ship, or the ship might be generated but too blurry for detection. In other words, existing image generation models cannot generate detection images suitable for detection tasks. To address this issue, this embodiment provides a detection image generation method based on a hybrid conditional generative adversarial network, such as... Figure 1 As shown, the generation method includes:

[0051] S1: Obtain the detection ground truth and random noise; the detection ground truth includes the target category, target location, target size, and target quantity in text format;

[0052] The values ​​of the ground truth and random noise can be arbitrarily specified, and different detection images can be generated by changing the values ​​of the ground truth and random noise.

[0053] S2: Based on the detected ground truth, generate a target foreground image using a foreground target generation model;

[0054] S2 may include:

[0055] Based on the ground truth detection values, generate target image templates corresponding to the detection categories; such as Figure 2The image corresponding to the prior distribution of the detection conditions is shown. The target image template includes multiple detection boxes. The position, number, and size of the detection boxes are determined by the target position, target number, and target size in the ground truth. The grayscale of the detection boxes is determined by the target category, that is, the target category is represented by a pre-set grayscale value.

[0056] Using the target image template as input, a foreground target generation model generates targets of the corresponding category within each detection box, resulting in a target foreground image. The target foreground image is shown below. Figure 2 The image corresponding to the mid-foreground target sample is shown. The target category within the detection box is determined based on the gray value of the detection box, and then the corresponding target category is generated within the detection box at this known location using the foreground target generation model.

[0057] The foreground target generation model in this embodiment can be obtained by training a target generation network based on conditional GAN. The foreground target generation model can generate the corresponding target foreground image from the ground value of detection with arbitrary values.

[0058] Prior to S2, the generation method in this embodiment also includes the step of training a conditional GAN-based target generation network to obtain a foreground target generation model, specifically including:

[0059] Obtain a first training sample set; the first training sample set includes multiple training target image templates and a training target foreground image corresponding to each training target image template;

[0060] Using training target image templates as input, a conditional GAN-based target generation network is used to generate a predicted target foreground image corresponding to each training target image template.

[0061] Calculate the loss within the detection bounding box of the training target image template based on all predicted target foreground images and all training target foreground images;

[0062] Determine whether the first iteration termination condition has been met based on the loss;

[0063] If so, then the conditional GAN-based target generation network used in the current iteration will be used as the foreground target generation model;

[0064] If not, the parameters of the target generation network based on conditional GAN ​​are adjusted according to the loss, the predicted target foreground image is regenerated, and the loss within the detection box range of the target image template used for training is calculated until the first iteration termination condition is met.

[0065] In this embodiment, the target foreground image is generated using manually specified arbitrary ground truth values ​​as input. Therefore, it allows for the simultaneous generation of any number of detection images and their corresponding ground truth values, solving the problem of not being able to generate the corresponding ground truth values ​​while generating detection images. After generating the detection images, manual annotation is no longer required, avoiding a significant amount of work and saving annotation costs. Furthermore, since the target foreground image is generated based on the ground truth values, it's equivalent to pre-setting the target location in the target image template. The foreground target generation model can then generate targets of the specified category at the pre-set locations. Therefore, when training the conditional GAN-based target generation network, only the loss within the detection box needs to be considered, without considering the loss across the entire image, significantly improving training speed.

[0066] S3: Generate a background image using a background generation model based on the random noise;

[0067] The random noise in this embodiment conforms to a standard normal distribution, such as... Figure 2 The image corresponding to normally distributed random noise is shown. Background image as follows: Figure 2 The corresponding image generated from the background sample is shown below.

[0068] The background generation model in this embodiment is obtained by training a background generation network (variational coding generative adversarial network) based on VAE-GAN. For example... Figure 3 As shown, the background generation network based on VAE-GAN mainly consists of a variational encoder (VAE) and a generative adversarial network (GAN), including an encoder, a decoder (generator), and a discriminator connected in sequence.

[0069] Prior to S3, the generation method in this embodiment also includes the step of training a background generation network based on VAE-GAN to obtain a background generation model, specifically including:

[0070] Obtain a second training sample set; the second training sample set includes multiple background images for training.

[0071] Using the training background image as input, the encoder is used to obtain the feature vector, and the KL divergence loss between the feature vector and the standard normal distribution vector is calculated.

[0072] Using the feature vector as input, the decoder generates a predicted background image, and the pixel-level loss between the predicted background image and the training background image is calculated.

[0073] The GAN loss is calculated using a discriminator based on the training background image and the predicted background image.

[0074] Based on the KL divergence loss, pixel-level loss, and GAN loss, determine whether the termination condition for the second iteration has been met;

[0075] If so, the decoder used in the current iteration will be used as the background to generate the model;

[0076] If not, the parameters of the encoder, decoder, and discriminator are adjusted according to the KL divergence loss, pixel-level loss, and GAN loss. The feature vector and predicted background image are regenerated, and the KL divergence loss, pixel-level loss, and GAN loss are calculated until the termination condition of the second iteration is met.

[0077] This embodiment combines the advantages of VAE and GAN. It can reconstruct the intermediate feature vector from an irregular implicit distribution to a standard normal distribution during the image encoding and decoding process, while retaining the advantage of GAN in generating images with clearer edges and details. The background generation model obtained in this embodiment can generate background images that meet the requirements from random noise with any standard normal distribution because it shares the process of GAN generator with VAE decoder. The resulting background images have clearer edges and details and excellent high-frequency performance.

[0078] S4: Overlay the target foreground image and the background image to obtain a detection image.

[0079] S4 may include: pixel-level overlay of the target foreground image and background image, that is, combining the foreground target and background samples in a pixel-level overlay form to obtain a detection image for the detection task. The detection image is as follows: Figure 2 The image generated from the fusion detection sample is shown below.

[0080] To address the issue of data augmentation for small samples, this embodiment proposes a hybrid conditional generative adversarial network (GAN) image generation method for detection and recognition tasks. After all training is complete, only the ground truth detection values ​​and normally distributed random noise need to be manually specified to generate the foreground target and background respectively, which are then fused to obtain the desired detection image. This method is applicable to various industrial fields that require the generation of detection image data, including but not limited to applications requiring the generation of detection image samples such as visible light, infrared, and imaging radar. This embodiment proposes a hybrid conditional GAN ​​algorithm that generates the detection foreground target from a controllable conditional distribution and the detection background from random noise, combining the two to achieve simultaneous generation of image sample data and ground truth detection values ​​for detection and recognition, providing data support for small sample recognition research. The detection image constructed in this embodiment is generated from ground truth detection values ​​with customizable categories, positions, sizes, and quantities, which can greatly expand the original detection dataset, providing strong data support for improving the accuracy of corresponding detection algorithms, and can be widely used in big data-based artificial intelligence algorithm research.

[0081] The above implementation method uses image detection sample generation as an example. Once those skilled in the art understand the basic inventive concept, they can make corresponding changes and modifications to these examples. Therefore, the protection scope of this embodiment covers image detection, image segmentation, and other applications based on similar principles.

[0082] Example 2:

[0083] The embodiments of the present invention can divide functional modules according to the above method examples. For example, each function can be divided into its own functional module, or two or more functions can be integrated into one processing module. The integrated module can be implemented in hardware or as a software functional module. It should be noted that the module division in the embodiments of the present invention is illustrative and only represents one logical functional division; other division methods may be used in actual implementation.

[0084] When dividing each function into modules according to its corresponding function. Figure 4 A schematic diagram of the structure of a detection image generation system provided by an embodiment of the present invention is shown. Figure 4 As shown, the detection image generation system includes:

[0085] The acquisition module M1 is used to acquire detection ground truth and random noise; the detection ground truth includes the target category, target position, target size and target quantity in text format;

[0086] The foreground generation module M2 is used to generate a target foreground image based on the detected ground truth using a foreground target generation model;

[0087] Background generation module M3 is used to generate a background image based on the random noise using a background generation model;

[0088] The overlay module M4 is used to overlay the target foreground image and the background image to obtain a detection image.

[0089] All relevant content of each step involved in the above method embodiments can be referenced from the functional description of the corresponding functional module, and will not be repeated here.

[0090] Example 3:

[0091] This embodiment provides an electronic device for generating detection images, including:

[0092] Processor; and

[0093] Memory, in which computer-readable program instructions are stored.

[0094] The detection image generation method described in Example 1 is executed when the computer-readable program instructions are run by the processor.

[0095] Example 4:

[0096] This embodiment also provides a non-volatile computer-readable storage medium storing computer-readable instructions, which, when executed by a computer, can perform the detection image generation method as described in Embodiment 1.

[0097] The program portion of a technology can be considered a "product" or "artifact" existing in the form of executable code and / or related data, and is involved in or implemented through a computer-readable medium. Tangible, permanent storage media can include memory or storage used by any computer, processor, or similar device or related module. For example, various semiconductor memories, tape drives, disk drives, or any similar device capable of providing storage functionality for software.

[0098] Although the invention has been described herein in conjunction with various embodiments, those skilled in the art will understand and implement other variations of the disclosed embodiments by reviewing the accompanying drawings, the disclosure, and the appended claims in carrying out the claimed invention. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude a plurality. A single processor or other unit can implement several functions listed in the claims. While different dependent claims may recite certain measures, this does not mean that these measures cannot be combined to produce good results.

[0099] Although the invention has been described in conjunction with specific features and embodiments, it is obvious that various modifications and combinations can be made therein without departing from the spirit and scope of the invention. Accordingly, this specification and drawings are merely exemplary descriptions of the invention as defined by the appended claims, and are considered to cover any and all modifications, variations, combinations, or equivalents within the scope of the invention. Clearly, those skilled in the art can make various alterations and modifications to the invention without departing from its spirit and scope. Thus, if such modifications and modifications of the invention fall within the scope of the claims and their equivalents, the invention is also intended to include such modifications and modifications.

Claims

1. A method for generating a detection image, characterized in that, The generation method includes: Obtain the detection ground truth and random noise; the detection ground truth includes the target category, target location, target size, and target quantity in text format; Based on the detected ground truth values, a target foreground image is generated using a foreground target generation model; Based on the random noise, a background image is generated using a background generation model; The target foreground image and the background image are superimposed to obtain the detection image; The step of generating a target foreground image using a foreground target generation model based on the detected ground truth specifically includes: Based on the detection ground truth, a target image template is generated; the target image template includes multiple detection boxes; the position, number, and size of the detection boxes are determined by the target position, the target quantity, and the target size, respectively; the grayscale of the detection boxes is determined by the target category. Using the target image template as input, a foreground target generation model is used to generate targets of the corresponding category within each detection box to obtain the target foreground image; The foreground target generation model is obtained by training a target generation network based on conditional GAN. The background generation model is obtained by training a background generation network based on VAE-GAN; The background generation network based on VAE-GAN includes an encoder, a decoder, and a discriminator connected in sequence.

2. The generation method according to claim 1, characterized in that, Before generating a foreground image using the foreground object generation model based on the detected ground truth, the generation method further includes the step of training the conditional GAN-based object generation network to obtain the foreground object generation model, specifically including: Obtain a first training sample set; the first training sample set includes multiple training target image templates and a training target foreground image corresponding to each training target image template; Using the training target image template as input, the conditional GAN-based target generation network generates a predicted target foreground image corresponding to each training target image template. Calculate the loss within the detection box range of the training target image template based on all the predicted target foreground images and all the training target foreground images; Determine whether the first iteration termination condition has been met based on the loss. If so, then the conditional GAN-based target generation network used in the current iteration shall be used as the foreground target generation model; If not, the parameters of the conditional GAN-based target generation network are adjusted according to the loss, the predicted target foreground image is regenerated, and the loss within the detection box range of the training target image template is calculated until the first iteration termination condition is met.

3. The generation method according to claim 1, characterized in that, The random noise conforms to a standard normal distribution.

4. The generation method according to claim 1, characterized in that, Before generating a background image using the background generation model based on the random noise, the generation method further includes the step of training the VAE-GAN-based background generation network to obtain the background generation model, specifically including: Obtain a second training sample set; the second training sample set includes multiple background images for training. Using the training background image as input, the encoder is used to obtain a feature vector, and the KL divergence loss between the feature vector and the standard normal distribution vector is calculated. Using the feature vector as input, the decoder generates a predicted background image, and the pixel-level loss between the predicted background image and the training background image is calculated. The GAN loss is calculated using the discriminator based on the training background image and the predicted background image. Based on the KL divergence loss, the pixel-level loss, and the GAN loss, determine whether the second iteration termination condition has been met; If so, then the decoder used in the current iteration is used as the background generation model; If not, the parameters of the encoder, the decoder and the discriminator are adjusted according to the KL divergence loss, the pixel-level loss and the GAN loss, the feature vector and the predicted background image are regenerated, and the KL divergence loss, pixel-level loss and GAN loss are calculated until the second iteration termination condition is met.

5. The generation method according to claim 1, characterized in that, The process of overlaying the target foreground image and the background image to obtain the detection image specifically includes: The target foreground image and the background image are superimposed at the pixel level to obtain the detection image.

6. A detection image generation system, characterized in that, The generation system includes: The acquisition module is used to acquire detection ground truth and random noise; the detection ground truth includes target category, target location, target size, and target quantity in text format; The foreground generation module is used to generate a target foreground image based on the detected ground truth using a foreground target generation model; The background generation module is used to generate a background image based on the random noise using a background generation model. The overlay module is used to overlay the target foreground image and the background image to obtain a detection image; The step of generating a target foreground image using a foreground target generation model based on the detected ground truth specifically includes: Based on the detection ground truth, a target image template is generated; the target image template includes multiple detection boxes; the position, number, and size of the detection boxes are determined by the target position, the target quantity, and the target size, respectively; the grayscale of the detection boxes is determined by the target category. Using the target image template as input, a foreground target generation model is used to generate targets of the corresponding category within each detection box to obtain the target foreground image; The foreground target generation model is obtained by training a target generation network based on conditional GAN. The background generation model is obtained by training a background generation network based on VAE-GAN; The background generation network based on VAE-GAN includes an encoder, a decoder, and a discriminator connected in sequence.

Images