A scale-aware target detection method for low-quality images

Abstract

This invention discloses a scale-aware target detection method for low-quality images, relating to the field of image information processing technology. Specifically, the images in the non-degraded image dataset are... I s The images in the low-quality image dataset are I t During training I s and I t Shared detection network weights; I s The features generated by the detector are fed into the detector to generate a score map for the detector, thus optimizing the detector. The features generated by the detector are then fed into the discriminator to produce the discriminator's predictions. I t The generated features are fed into the detector and then directly input into the discriminator, which distinguishes features originating from the image. I s still I t This invention reduces interference caused by distributional differences between training and test data, and ultimately enables the object detector to minimize interference from these differences. During testing, the detector is used only on low-quality images. This invention reduces the distributional differences between large-scale datasets of high-quality images and datasets of low-quality images, improving the accuracy and performance of object detection on low-quality images.

Description

Technical Field

[0001] This invention relates to the field of image information processing technology, specifically to a scale-aware target detection method for low-quality images, which can be used for target detection, target tracking, and other tasks in harsh environments. Background Technology

[0002] In recent years, object detection technology has made significant progress due to the rapid development of convolutional neural networks and the construction of large-scale datasets. Existing large-scale datasets often contain images of good quality, i.e., non-degraded images. However, in real-world applications, image quality is significantly degraded due to poor lighting, fog, rain, and other adverse environmental conditions, leading to an increasing demand for object detection in such scenarios. The significant data distribution difference between large-scale, high-quality training data and low-quality test data makes it difficult for models trained on such datasets to achieve the expected object detection performance on low-quality images. Cross-domain object detection technology can effectively alleviate this problem by reducing the data distribution difference between the training and test datasets through cross-domain adaptation techniques. In cross-domain object detection, the richly labeled dataset is used as the source domain, while the sparsely labeled dataset is used as the target domain. Commonly used methods include adversarial training, pseudo-label self-training, image-to-image transformation, domain randomization, and graph inference to reduce the distribution differences between different datasets.

[0003] Most current cross-domain object detection algorithms use anchored object detectors as the baseline detector, such as Faster R-CNN and SSD. However, anchored object detectors require setting a large number of hyperparameters, which is detrimental to the generalization ability of the object detector. In recent years, anchorless object detectors, which do not require setting hyperparameters, have received widespread attention from researchers. Most anchorless detectors, such as FCOS, FoveaBox, FSAF, and CenterNet, use center point regression methods for object localization. This design makes anchorless object detectors closer to segmentation tasks. In object detection technology, in order to better represent objects at different scales, the resolution of the final score map is different scales. This means that feature alignment at different scales also needs to be considered in cross-domain object detection. For domain adaptation techniques, semantic segmentation has two levels of domain adaptation: image-level and pixel-level, corresponding to global feature alignment and local feature alignment, respectively. Given the similarity between anchorless object detection technology and object segmentation tasks, image-level adaptation and pixel-level adaptation techniques are introduced into cross-domain object detection to extract multi-scale features of objects in low-quality images and improve object detection accuracy.

[0004] To overcome the above problems, it is necessary to improve existing low-quality image target detection methods. Summary of the Invention

[0005] To address the problem of low target detection accuracy in low-quality images mentioned in the background art, this invention provides a scale-aware target detection method for low-quality images. Due to the significant data distribution difference between large-scale training sets with good image quality and test sets with low-quality images, target detectors trained on high-quality image datasets often fail to achieve satisfactory detection results on low-quality images.

[0006] This invention is achieved through the following technical solution: a scale-aware low-quality image target detection method, comprising the following steps:

[0007] 1) The training dataset with good image quality is D. s The training dataset for low-quality images is D. t Training dataset D s The image in is I s The corresponding low-quality image dataset D t The image in is I t During training I s and I t Shared detection network weights;

[0008] 2) Training phase: Image I s The data is fed into the detector to generate a score map for the detector, which is then used to optimize the detector. Meanwhile, the multi-scale aligned features of the target generated by the detector are fed into the discriminator to produce the discriminator's predictions.

[0009] 3) Image I t The generated target multi-scale aligned features are input into the detector and then directly input into the discriminator to produce predicted values; the discriminator is used to distinguish features originating from image I. s Or I t ;

[0010] 4) Repeat steps 2) and 3) to train the detector and discriminator. Stop training when the performance verification metric starts to decline or the number of training rounds reaches 1000.

[0011] 5) After the network training is completed, during the testing phase, low-quality images are input into the target detector to obtain the target inspection category and location, and the experimental results are statistically analyzed and relevant indicators are calculated.

[0012] This invention utilizes domain adaptation techniques to reduce the data distribution differences between the training and testing datasets, specifically reducing the distribution discrepancy between large-scale datasets with good image quality and those with low-quality images. Furthermore, to better alleviate the multi-scale target problem, image-level adaptation techniques are introduced to align global and local features of the target, thereby perceiving multi-scale target information and improving the target detection performance on low-quality images. In summary, this invention proposes a scale-aware target detection method for low-quality images, utilizing anchorless domain adaptation techniques to improve target detection accuracy in low-quality images. The proposed domain adaptation techniques include image-level and pixel-level adaptation, and adversarial networks are used to achieve multi-scale feature alignment of the target.

[0013] Compared with the prior art, the present invention has the following beneficial effects: The scale-aware target detection method provided by the present invention aligns the features of the target at different scales through the fully convolutional domain adaptation technique, reduces the data distribution difference between low-quality images and non-degraded images (images with good quality), thereby improving the target detection accuracy and performance in low-quality images. It can perceive the multi-scale information of the target and achieve good technical results. Attached Figure Description

[0014] Figure 1 This is a network framework diagram of the present invention.

[0015] Figure 2 This is a diagram showing the target detection results of the present invention on low-quality images. Detailed Implementation

[0016] The present invention will be further described below with reference to specific embodiments.

[0017] The network framework of this embodiment is based on the anchorless target detector FCOS, and the generative adversarial network includes image-level adaptation (ILA) and pixel-level adaptation (PLA). Wherein, L... i These are the multi-scale features output by the FCOS network. Gray represents ILA, and black represents PLA. The image-level discriminator ILAD and the pixel-level discriminator PLAD are used for image-level adaptation and pixel-level adaptation, respectively.

[0018] A scale-aware target detection method for low-quality images, such as Figure 1 As shown: It includes the following steps:

[0019] 1) The training dataset with good image quality is D. s The training dataset for low-quality images is D. t Training dataset D s The image in is I s The corresponding low-quality image dataset Dt The image in is I t During training I s and I t Shared detection network weights; I s with I t This forms a pair of training images.

[0020] 2) Training phase: Image I s The data is input into the detector to generate a score map for the detector, thus optimizing the detector. Simultaneously, the multi-scale aligned features of the target generated by the detector are input into the discriminator to produce the discriminator's predictions. Specifically:

[0021] Will I s The input is fed into a detection network to optimize the object detection network, which includes a classification subnetwork and a location regression subnetwork. The loss function of the object detection network is:

[0022]

[0023] Where λ det Used to balance classification loss and regression loss, L cls To represent classification loss, use Focal Loss; L reg Represents the regression loss, using IOU loss; k represents the number of scale levels; CLS k and REG k These are the classification feature map and the regression feature map, respectively.

[0024] 3) Image I t The multi-scale aligned features of the generated target, fed into the detector, are directly input into the discriminator to produce predictions; the multi-scale features L generated by the backbone network FCOS are then used to generate predictions. i The features of the classification subnetwork (CN1 to CN4) are input into the pixel-level discriminator (PLAD); the features of the regression subnetwork (RN1 to RN4) are input into the pixel-level discriminator (PLAD) to distinguish features originating from image I. s Or I t .

[0025] 4) Repeat steps 2) and 3) to train the detector and discriminator. Stop training when the performance verification metric starts to decline or the number of training rounds reaches 1000. The specific process is as follows:

[0026] ① Treat the object detector as a generator, and the pixel-level discriminator and image-level discriminator as discriminators, construct a GAN paradigm adversarial network for adversarial training; the overall loss can be expressed as:

[0027] L adv =L det +λadv (L ILA +L PLA (2)

[0028] Where λ adv L is a trade-off parameter that balances the detection loss function and the adaptation loss function. ILA For image-level discriminator loss, L PLA For pixel-level image discriminator loss; where L ILA The loss after multi-scale global feature alignment of the target is calculated using the following formula:

[0029]

[0030] In this context, ID5 represents the fifth layer of ILDA, which is also the ILDA score map. The number of channels, width, and height of the fifth convolutional layer are represented by c, h, and w in ID5; k represents the number of scale layers.

[0031] L PLA To calculate the loss after aligning local features at multiple scales of the target, the formula is as follows:

[0032]

[0033] ② Transform the adversarial network loss function from step ① into the GAN paradigm.

[0034] L(I s ,I t ) = L det (I s )+λ adv L adv (I t (5)

[0035] Among them, I s The image representing the source domain, I t The image representing the target domain;

[0036] ③Place I s with I t Alternately feed the inputs into the adversarial network for training, and optimize the network loss:

[0037]

[0038] Where Dis is the discriminator network and Det is the detection network;

[0039] ④ Stop training when the performance verification metrics start to decline or the number of training rounds reaches 1000.

[0040] 5) After the network training is completed, during the testing phase, low-quality images are input into the target detector to obtain the target inspection category and location. The experimental results are statistically analyzed and the corresponding index scores are calculated, such as the evaluation accuracy AP.

[0041] The effects of the present invention in the embodiments are further illustrated by the following simulation experiments.

[0042] 1. Simulation conditions

[0043] This invention is a simulation performed using Python and other related toolkits on a CPU consisting of an Intel(R) Xeon(R) CPU E5-2650V4 @ 2.20GHz, 500GB of memory, and an Ubuntu 14 operating system.

[0044] 2. Simulation Content

[0045] A typical cross-domain object detection problem is detecting objects under adverse weather conditions. Typically, models are trained using images taken in clear weather. However, test images are taken under adverse weather conditions such as rain, fog, snow, haze, and dust, resulting in severely degraded image quality and a significantly different data distribution compared to the training data. To verify the effectiveness of this invention, the Foggy Cityscape dataset is used as the target domain, and the Cityscape dataset as the source domain. The target categories are people, riders, cars, trucks, buses, trains, motorcycles, and bicycles.

[0046] In this scenario, the invented SADA network was compared with Faster R-CNN, DAFR, DD+MRL, MAF, PDA, SWDA, RPC, CDTD, and EPM. Experimental results for image degradation are shown in Table 1. The invented method achieved highly competitive results, surpassing most previous domain adaptation algorithms for object detection.

[0047] Table 1. Experimental results of target detection under adverse weather conditions.

[0048]

[0049] The scope of protection claimed by this invention is not limited to the specific embodiments described above. Moreover, for those skilled in the art, this invention can have various modifications and alterations. Any modifications, improvements, and equivalent substitutions made within the concept and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for detecting low-quality images based on scale awareness, characterized in that: Includes the following steps: 1) The training dataset with good image quality is D. s The training dataset for low-quality images is D. t Training dataset D s The image in is I s The corresponding low-quality image dataset D t The image in is I t During training I s and I t Shared detection network weights; 2) Training phase: Image I s The data is fed into the detector to generate a score map for the detector, which is then used to optimize the detector. Meanwhile, the multi-scale aligned features of the target generated by the detector are fed into the discriminator to produce the discriminator's predictions. 3) Image I t The generated target multi-scale aligned features are input into the detector and then directly input into the discriminator to produce predicted values; the discriminator is used to distinguish features originating from image I. s Or I t ; Specifically, this involves: utilizing the multi-scale features L generated by the backbone network FCOS... i The features of the classification subnetwork (CN1 to CN4) are input into the pixel-level discriminator (PLAD); the features of the regression subnetwork (RN1 to RN4) are input into the pixel-level discriminator (PLAD) to distinguish features originating from image I. s Or I t ; 4) Repeat steps 2) and 3) to train the detector and discriminator. Stop training when the performance verification metric starts to decline or the number of training rounds reaches 1000. 5) After the network training is completed, during the testing phase, low-quality images are input into the target detector to obtain the target inspection category and location, and the experimental results are statistically analyzed and relevant indicators are calculated. The target detection method uses fully convolutional domain adaptation to align features of the target at different scales, and uses generative adversarial networks to reduce the data distribution difference between low-quality images and non-degraded images, thereby improving the target detection accuracy in low-quality images. Non-degraded images are high-quality images. The generative adversarial network includes image-level adaptation (ILA) and pixel-level adaptation (PLA).

2. The method for detecting low-quality images based on scale awareness according to claim 1, characterized in that: Step 2) is as follows: Place I s The input is fed into a detection network to optimize the object detection network, which includes a classification subnetwork and a location regression subnetwork. The loss function of the object detection network is: Where λ det Used to balance classification loss and regression loss, L cls To represent classification loss, use Focal Loss; L reg Represents the regression loss, using IOU loss; k represents the number of scale levels; CLS k and REG k These are the classification feature map and the regression feature map, respectively.

3. The method for detecting low-quality images based on scale awareness according to claim 2, characterized in that: Step 4) specifically involves: ① Treat the object detector as a generator, and the pixel-level discriminator and image-level discriminator as discriminators, construct a GAN paradigm adversarial network for adversarial training; the overall loss can be expressed as: L adv =L det +λ adv (L ILA +L PLA ) (2) Where λ adv L is a trade-off parameter that balances the detection loss function and the adaptation loss function. ILA For image-level discriminator loss, L PLA For pixel-level image discriminator loss; where L ILA The loss after multi-scale global feature alignment of the target is calculated using the following formula: In this context, ID5 represents the fifth layer of ILDA, which is also the ILDA score map. The number of channels, width, and height of the fifth convolutional layer are represented by c, h, and w in ID5; k represents the number of scale layers. L PLA To calculate the loss after aligning local features at multiple scales of the target, the formula is as follows: ② Transform the adversarial network loss function from step ① into the GAN paradigm. L(I s ,AND t )=L det (AND s )+λ adv L adv (AND t ) (5) Among them, I s The image representing the source domain, I t The image representing the target domain; ③Place I s with I t Alternately feed the inputs into the adversarial network for training, and optimize the network loss: Where Dis is the discriminator network and Det is the detection network; ④ Stop training when the performance verification metrics start to decline or the number of training rounds reaches 1000.

4. The method for detecting low-quality images based on scale awareness according to claim 2, characterized in that: Step 5) Testing phase: Input test image, obtain target detection category and location, and statistically analyze experimental results to calculate the target recognition accuracy (AP) and other index scores.

Images