Low-contrast small target generation-detection combined training framework

Abstract

The invention relates to a low-contrast small target generation-detection combined training framework, belongs to the technical field of computer vision and traffic engineering data acquisition, and is suitable for all-weather real-time monitoring of low-contrast small targets in road videos acquired by a high-altitude fixed camera or a mooring unmanned aerial vehicle. In order to solve the problems that the improvement of the detection precision and the stability is limited and the application effect of a specific project is difficult to evaluate under the condition of improving the training round due to the separation of the existing generative model enhancement and detector training, the invention constructs a low-contrast small-target scene-oriented generation-detection combined training framework in a training stage; detection performance is used as a direct optimization target of a condition generator, joint optimization of a generation process and a detection process is realized, and small target detection precision and stability are improved under low contrast and complex all-weather conditions.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and traffic engineering data acquisition technology, and in particular to a joint training framework for low-contrast small target generation and detection. Background Technology

[0002] Existing data augmentation and YOLO detection network combinations based on generative models mostly employ a loosely coupled "augment first, then detect" approach: first, a GAN or style transfer network is used to convert clear periods / scenes into nighttime, foggy, or low-light images, and then the enhanced images are used to train the YOLO detection network separately. A typical example is the 1D0D-YOLOv7 framework proposed by Qiu et al., which concatenates the dehazing and adaptive augmentation modules with YOLOv7 to improve detection performance in low-light, foggy traffic scenes. However, the data augmentation part still focuses on image quality, resulting in a heavy network structure and high training costs. Bosquet et al. generated small target instances using GANs and pasted them onto the background, then used these synthetic images to train the detector. This completely separates generation and detection, focusing on "how realistic the synthesis is" and improving mAP on public datasets, making it difficult to evaluate the application effect in specific engineering projects.

[0003] In terms of domestic patents, existing published solutions also follow the same pattern. For example, Zheng Liang et al. (CN115984568A, A Target Detection Method Based on YOLOv3 Network in Haze Environment) first use a conditional generative adversarial network to dehaze and enhance haze images, and then use an improved YOLOv3 network with an embedded attention module for training and inference. The generative network mainly focuses on image restoration quality, while the detection network is optimized separately using supervised loss, and the detection network is trained separately using conventional supervised detection loss. The two are only indirectly related at the data level through the "synthesized image," lacking a unified joint loss function design. Similar ideas also appear in other small target or harsh environment scenarios. For example, Cheng Jianjie (CN115937794B) uses an independently trained super-resolution enhancement model to enlarge low-resolution surveillance images, and then uses a small target detection model for localization and recognition. The training objectives of the enhancement model and the detection model are also separate.

[0004] In summary, among the technical solutions for improving the detection accuracy of low-contrast small targets in YOLO detection networks based on generative models, the YOLO detection network often iterates repeatedly on a large number of synthetic images with limited contribution, which significantly increases training time and memory usage. Even if the number of training rounds is increased, the marginal benefits to detection accuracy and stability in complex all-weather scenes are very limited. There is still a lack of a generative-detection joint training method with the core objective of improving the detection performance of low-contrast small targets. Summary of the Invention

[0005] This invention provides a joint training framework for low-contrast small target generation and detection, which solves the technical problem that even with increased training rounds, the marginal benefits of existing training computations on the accuracy and stability of small target detection in complex all-weather scenarios are very limited.

[0006] The technical solution of this invention to solve the above-mentioned technical problems is as follows: a joint training framework for low-contrast small target generation and detection, comprising the following steps: Step 1: Input Image Preparation and Annotation. Input aerial images of the road taken during clear periods into the system, and perform detailed annotations on vehicle targets in the images to form an annotated image set (x, y) and detection annotations. Where x is the input image and y is the real target image paired with x. For the detection and labeling of vehicle targets; Step 2: Construct the condition generator G( ) and discriminator D( ), using Pix2Pix as the basis to construct a Conditional Generative Adversarial Network (cGAN), with a condition generator G( The system receives a sharp-period image x and its corresponding condition variable c (indicating the target domain / style category, such as "sharp-period / nighttime" or "high contrast / low contrast"), and outputs a generated image G(x|c). When the condition variable c is already determined, it can be abbreviated as G(x); the discriminator D( Given an input image x, the generator G(x) is used to distinguish between the real target image y and the generated image G(x); the objective function of the adversarial game between the generator and the discriminator is:

[0007] in, This represents the discriminant output of the discriminator for the real target image y, given the input image x. This represents the discriminant output of the discriminator on the generated image G(x) given the input image x; This represents the expectation operation on the true paired labeled image set (x, y). This represents the expected operation on the input sample x; Logarithmic operations; Let the objective function be the adversarial game objective function. The condition generator adversarial loss function is:

[0008] in, The adversarial loss of the conditional generator is used to prompt the generation of images. In discriminator D( The discrimination results continuously approach the real target image. Distribution; This represents the expected operation on the input sample x; Logarithmic operations; This indicates that, given an input image x, the discriminator will generate an image. Output that is judged as true; The discriminator adversarial loss function is:

[0009] in, This is the adversarial loss of the discriminator, used to improve the discriminator's ability to distinguish between the real target image y and the generated image G(x); This represents the probability that the discriminator will classify the generated image G(x) as non-real given the input image x; To enhance the structural consistency between the generated image and the real target image, pixel-level L1 consistency loss is introduced, with the loss function being:

[0010] in, is the pixel-level L1 consistency loss, used to constrain the absolute difference between the generated image G(x) and the real target image y at the pixel level; This represents the summation of the absolute errors over all pixel positions; Step 3: Construct a YOLO detection network and connect it to the condition generator. The YOLO detection network includes a backbone network, a neck network, and a detection head. The generated image G(x) is fed into the YOLO detection network as input. After feature extraction and fusion by the backbone network and the neck network, the detection head outputs the detection result and calculates the detection loss. Taking YOLOv11 as an example, its detection loss function is:

[0011] Among them, classification loss Bounding box regression loss is used to measure the difference between the predicted class probability and the true class. The target confidence loss is used to measure the difference between the predicted bounding box and the true bounding box. The detection loss is used to measure the difference between the predicted probability of the target's existence and the actual probability. By detection label Perform supervised calculations.

[0012] Step 4: Construct the joint loss function for the condition generator. And achieve reverse propagation, specifically by detecting the loss. As the joint loss function of the condition generator One of the components, and based on a differentiable computational graph, performs end-to-end backpropagation to update the trainable parameters of the conditional generator. The joint loss function for the condition generator is defined as follows:

[0013] in, The joint loss function of the condition generator is the adversarial loss generated by the condition generator. Pixel-level L1 consistency loss With detection loss Weighted composition; and These are weighting coefficients, used for balancing. and exist The contribution in the calculation defaults to 0.5. Since the generated image G(x) is used as input to the YOLO detection network, the detection loss... Trainable parameters of the condition generator The gradient can be calculated using the chain rule, and its expression is:

[0014] in, For condition generator G( Trainable parameters; Indicates detection loss The gradient with respect to the parameters of the condition generator; This represents the gradient of the detection loss with respect to the output image of the conditional generator; Indicates the generated image Trainable parameters of the condition generator The gradient; the detection loss Classification loss Bounding box regression loss With target confidence loss constitute; Therefore, the total gradient of the joint loss function of the conditional generator with respect to the trainable parameters of the conditional generator is:

[0015] in, Joint loss function The total gradient with respect to the condition generator parameters; , , These represent the gradient contributions of adversarial loss, pixel consistency loss, and detection loss to the condition generator parameters, respectively. and This is the weighting coefficient, used to balance the impact of different loss terms; the default value is 0.5.

[0016] The beneficial effects of this invention are: 1. By constructing a connection between the YOLO detection network and the conditional generator during the training phase, and incorporating the detection loss as a component of the joint loss function of the conditional generator, end-to-end backpropagation is performed based on a differentiable computation graph to update the trainable parameters of the conditional generator. This makes improving detection performance a direct optimization goal for the images generated by the conditional generator, thereby improving the detection accuracy and stability of small targets such as vehicles and pedestrians in complex all-weather low-contrast scenes.

[0017] 2. Since the condition generator is trained under the joint optimization of adversarial loss, pixel consistency loss and detection loss, it can suppress invalid synthetic samples that are difficult to improve detection performance by simply increasing the amount of training. This reduces the dependence on increasing the number of training rounds and effectively improves the accuracy and stability of small target detection in complex all-weather low-contrast scenes. As a result, it provides more stable basic detection results and data support for target recognition and trajectory extraction in high-altitude video monitoring.

[0018] 3. Because the present invention does not depend on the specific network structure or the specific form of the detection loss function of the YOLO detection network when constructing the differentiable connection structure between the condition generator and the YOLO detection network, it can be adapted to different versions such as YOLOv8 to YOLOv12, and can achieve seamless migration and rapid deployment across versions without the need for additional manual adjustments.

[0019] 4. Based on the above improvements in detection accuracy and stability, the continuity and availability of vehicle and pedestrian detection results and trajectory data can be enhanced, thereby providing high-quality data sources for road operation management, traffic research and third-party detection services, and supporting traffic simulation, driving behavior research, traffic capacity assessment and traffic safety evaluation. Attached Figure Description

[0020] Figure 1 This is a framework diagram of a joint training framework for low-contrast small target generation and detection according to the present invention. Detailed Implementation

[0021] The principles and features of the present invention are described below with reference to the accompanying drawings. The examples given are only for explaining the present invention and are not intended to limit the scope of the present invention.

[0022] A joint training framework for low-contrast small target generation and detection includes the following steps: Step 1: Input Image Preparation and Annotation The system inputs aerial images of the road taken during clear periods and performs detailed annotations on vehicle targets in the images, forming an annotated image set (x, y) and detection annotations. Where x is the input image and y is the real target image paired with x. For vehicle target detection labeling.

[0023] Step 2: Construct the condition generator G( ) and discriminator D( ) A Conditional Generative Adversarial Network (cGAN) is constructed based on Pix2Pix. The condition generator G( The system receives a clear time-period image x and its corresponding condition variable c (indicating the target domain / style category, such as "clear time / night"), and outputs a generated image G(x|c). When the condition variable c is already determined, it can be abbreviated as G(x). Discriminator D( Given an input image x, the generator G(x) is used to distinguish between the real target image y and the generated image G(x). The typical form of the adversarial objective function between the conditional generator and the discriminator is as follows: (1) The typical forms of the loss functions of the condition generator and discriminator are shown in formulas (2) and (3): (2) (3) To enhance image structure consistency, pixel-level L1 loss is introduced as shown in formula (4): (4) Step 3: Construct the YOLO detection network and connect it to the condition generator. A YOLO detection network is constructed and connected to a condition generator. The YOLO detection network includes a backbone network, a neck network, and a detection head. The generated image G(x) is fed into the YOLO detection network as input. After feature extraction and fusion by the backbone network and the neck network, the detection result is output at the detection head, and the detection loss is calculated. Taking YOLOv11 as an example, its detection loss function is: (5) Classification loss Bounding box regression loss is used to measure the difference between the predicted class probability and the true class. Target confidence loss is used to measure the difference between the predicted bounding box and the true bounding box. The detection loss is used to measure the difference between the predicted probability of the target's existence and the actual probability. By detection label Perform supervised calculations.

[0024] Step 4: Construct the joint loss function for the condition generator and implement backpropagation. Detect loss As the joint loss function of the condition generator One of the components, and based on a differentiable computational graph, performs end-to-end backpropagation to update the trainable parameters of the conditional generator. The joint loss function for the condition generator is defined as follows: (6) in, The joint loss function of the condition generator is the adversarial loss generated by the condition generator. Pixel-level L1 consistency loss With detection loss Weighted composition; and These are weighting coefficients, used for balancing. and exist The contribution is set to 0.5 by default.

[0025] Since the generated image G(x) is used as input to the YOLO detection network, the detection loss... Trainable parameters of the condition generator The gradient can be calculated using the chain rule, as shown in formula (7): (7) Therefore, the total gradient of the joint loss function of the conditional generator with respect to the trainable parameters of the conditional generator is given by formula (8): (8) Loss can be detected using the methods described above. During the training phase, the condition generator can be directly guided to generate images with appearance changes that are beneficial to improving the detection accuracy of small targets with low contrast, while suppressing invalid samples that only increase the amount of training but hardly improve the detection performance.

[0026] Because the generator-detection joint training method used in this invention does not depend on the specific network structure or detection loss form of the YOLO detection network when constructing the differentiable computational graph between the condition generator and the YOLO detection network, it can be adapted to different versions such as YOLOv8 to YOLOv12, and can achieve seamless migration and rapid deployment across versions without the need for additional manual adjustments.

[0027] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A joint training framework for low-contrast small target generation and detection, characterized in that, Includes the following steps: Step 1: Input Image Preparation and Annotation. Input aerial images of the road taken during clear periods into the system, and perform detailed annotations on vehicle targets in the images to form an annotated image set (x, y) and detection annotations. ,in For the input image, To and Paired real target images, For the detection and labeling of vehicle targets; Step 2: Build the condition generator With discriminator A conditional generative adversarial network is constructed based on Pix2Pix, and the condition generator receives images from clear time periods. and its corresponding condition variables Output the style-transformed image When condition variable When it has been determined, it can be abbreviated as The discriminator is used to distinguish between different input images. Under the condition of real target image With generated image To determine authenticity; Step 3: Construct the connection between the YOLO detection network and the condition generator. The YOLO detection network includes a backbone network, a neck network, and a detection head; generate the image. The data is fed into the YOLO detection network as input. After feature extraction and fusion by the backbone network and the neck region, the detection head outputs the detection result and calculates the detection loss. ; Step 4: Construct the joint loss function And implement the calculation method for backpropagation, specifically by calculating the detection loss. As the joint loss function of the condition generator One of the components, and based on a differentiable computational graph, performs end-to-end backpropagation to update the trainable parameters of the conditional generator. .

2. The low-contrast small target generation-detection joint training framework according to claim 1, characterized in that, In step 2, the objective function of the adversarial game in the conditional generative adversarial network is: Among them, the condition generator Input image In condition variables Convert to generated image under constraints ; Indicates that in a given input image Under these conditions, the discriminator can distinguish real target images. The discriminant output; Indicates that in a given input image Under these conditions, the discriminator analyzes the generated image. The discriminant output; This represents the expectation operation on the true paired labeled image set (x, y). Indicates the input sample Expectation calculation; For logarithmic operations; The objective function for adversarial games.

3. The low-contrast small target generation-detection joint training framework according to claim 1, characterized in that, In step 2, the adversarial loss function of the condition generator is: The loss function of the discriminator is: in, Used to prompt the generation of an image In discriminator D( The discrimination results continuously approach the real target image. Distribution; Indicates that in a given input image Under these conditions, the discriminator will generate an image. The probability of outputting the true value; Used to improve the discriminator's ability to distinguish real target images. With generated image Ability; 1 Indicates that in a given input image Under these conditions, the discriminator will generate an image. The corresponding probability is judged to be non-true.

4. The low-contrast small target generation-detection joint training framework according to claim 1, characterized in that, In step 2, to enhance the structural consistency between the generated image and the real target image, a pixel-level L1 consistency loss is introduced, the formula of which is as follows: in, Pixel-level L1 consistency loss is used to measure the generated image. Compared with real target image Absolute difference at the pixel level; This represents the summation of the absolute errors over all pixel positions.

5. The low-contrast small target generation-detection joint training framework according to claim 1, characterized in that, In step 3, the detection loss function is... The detection loss calculated by the YOLO detection network is given by the following formula: ， Among them, classification loss Bounding box regression loss is used to measure the difference between the predicted class probability and the true class. Used to measure the bounding box of the prediction With the true bounding box The difference between them, target confidence loss Used to measure the probability of the predicted target existing. Compared to the actual situation The differences between them.

6. The low-contrast small target generation-detection joint training framework according to claim 1, characterized in that, In step 4, the joint loss function of the condition generator is defined as: in, The joint loss function of the condition generator is the adversarial loss generated by the condition generator. Pixel-level L1 consistency loss With detection loss Weighted composition; and These are weighting coefficients, used for balancing. and exist The contribution is set to 0.5 by default.

7. The low-contrast small target generation-detection joint training framework according to claim 1, characterized in that, In step 4, the loss is detected. Trainable parameters of the condition generator The gradient is calculated using the chain rule, and its expression is: in, These are the trainable parameters of the condition generator; Indicates detection loss The gradient with respect to the parameters of the condition generator; This represents the gradient of the detection loss with respect to the output image of the conditional generator; Indicates the generated image Trainable parameters of the condition generator The gradient; Therefore, the joint loss function The total gradient of the trainable parameters of the conditional generator is: in, Joint loss function The total gradient with respect to the condition generator parameters; , , These represent the gradient contributions of adversarial loss, pixel consistency loss, and detection loss to the condition generator parameters, respectively. and This is the weighting coefficient, used to balance the impact of different loss terms; the default value is 0.5.

Images