Image pixel-level hybrid enhanced unmanned aerial vehicle domain adaptive detection method

Abstract

The invention discloses an image pixel-level hybrid enhanced unmanned aerial vehicle domain adaptive detection method, and belongs to the field of infrared target detection application. The method is oriented to infrared unmanned aerial vehicle data application, infrared unmanned aerial vehicle images are collected through an infrared camera, and the method comprises the steps of data preparation, pseudo label generation, region division and confidence calculation, confidence-based mixed image generation, bidirectional dynamic region pseudo label filtering, mixed label generation and loss calculation, model training optimization and the like. And domain adaptive target detection of the infrared unmanned aerial vehicle image is realized. The method can be used for infrared unmanned aerial vehicle target detection tasks in scenes such as night monitoring, disaster rescue and military reconnaissance, the problem of performance reduction in infrared image cross-domain application (such as different equipment and environment conditions) can be effectively solved, and the precision and robustness of target detection are improved.

Description

Technical Field

[0001] The present invention belongs to the field of drone applications, and more specifically, relates to an infrared image drone domain adaptive detection method based on pixel-level hybrid enhancement of regional pseudo-label confidence images. Background Art

[0002] With the development of drone technology, infrared imaging has been widely used in drone target detection tasks, especially in scenarios such as nighttime surveillance, disaster relief, and military reconnaissance. Infrared images, by capturing the target's thermal radiation, can identify targets in low light or complex weather conditions. However, infrared drone imagery is relatively difficult to acquire, and a lack of samples can severely impact the performance of deep learning-based infrared target detection models. Directly applying models trained on visible light images to infrared images often results in poor performance due to differences in domain distribution.

[0003] To address the challenges posed by domain distribution discrepancies, domain adaptation technology has become a research hotspot in recent years. Domain adaptive object detection methods aim to adapt models trained in a source domain (a domain rich in labeled data) to a target domain (a domain with scarce or no labeled data) through transfer learning or feature alignment. Currently, common domain adaptation methods include adversarial training, Mean Teacher-based methods, and pseudo-label self-training. For example, adversarial training aligns the feature distributions of the source and target domains through a generator and a discriminator; pseudo-label self-training utilizes the source domain model to generate preliminary annotations for the target domain data, followed by iterative optimization. However, these methods still face numerous challenges in infrared UAV scenarios: First, feature alignment based on adversarial training struggles to capture regional-level thermal radiation semantics, impacting detection accuracy; second, the small scale and unclear thermal features of targets in infrared images result in low pseudo-label quality; and third, existing methods struggle to meet the practical requirements of UAV platforms in terms of computational complexity and real-time performance.

[0004] To address these issues, some research has attempted to introduce data augmentation or confidence filtering strategies, for example, by enhancing data diversity through image region-level blending methods. However, these methods are primarily designed for visible light images and fail to fully consider the thermal imaging characteristics of infrared images and the dynamic nature of drone scenes. Therefore, combining regional information with pixel-level features in infrared drone target detection to improve pseudo-labeling quality and optimize detection performance remains a pressing technical challenge. Summary of the Invention

[0005] To address the shortcomings of existing technologies in domain-adaptive detection of drones in infrared imagery, this paper proposes a domain-adaptive drone detection method based on pixel-level hybrid enhancement of regional pseudo-label confidence images. This method aims to address the performance degradation caused by domain distribution differences in infrared image drone detection tasks. By implementing a regional pixel-level hybrid strategy and a bidirectional dynamic regional pseudo-label filtering mechanism, it achieves effective knowledge transfer from the source domain (visible light images) to the target domain (infrared images), improving the accuracy and robustness of the model for drone detection in infrared images.

[0006] To achieve the above objectives, the present invention provides a drone domain adaptation detection method based on pixel-level hybrid enhancement of images, comprising the following steps:

[0007] S1. Read data: read visible light image x S and its label y S (source domain data), read infrared image x T (target domain data);

[0008] S2, pseudo label generation: the infrared image x T Input to the target detector F (the first iteration is obtained by training with source domain data) to generate a set of pseudo labels for the target domain where N T is the number of detected targets, is the pseudo label of the i-th target (including bounding box coordinates and confidence);

[0009] S3, region division and confidence calculation: the visible light image x S and infrared image x T Divide into four equal areas, denoted as and According to the pseudo-label The area where the center point is located, assign each pseudo label to the corresponding area, and calculate the average pseudo label confidence of each area, which is recorded as represents the average pseudo-label confidence of the jth region;

[0010] S4, Bidirectional dynamic area pseudo label filtering: adopt bidirectional dynamic area pseudo label filtering mechanism to filter the infrared image x T Generated pseudo labels Perform filtering and record the filtered pseudo label set as

[0011] S5. Confidence-based hybrid image generation: for infrared image x T and the visible light image x S Perform regional confidence weighted pixel-by-pixel fusion to generate a mixed image x M ;

[0012] S6, hybrid label generation: the source domain image label y S and the target domain pseudo labels after filtering by bidirectional dynamic region pseudo labels Merge to generate mixed image pseudo labels

[0013] S7. Model training and optimization: Using source domain image x S And the generated mixed image x M Perform domain adaptation update training on the target detector F to obtain the updated target detector F, and use the updated target detector to replace the target detector F used in the previous round. Repeat steps S2 to S7, and obtain the final target detector F through multiple rounds of iterations.

[0014] Furthermore, in the bidirectional dynamic region pseudo-label filtering step, the present invention designs a bidirectional dynamic region pseudo-label filtering mechanism. By combining global filtering from loose to strict and bridging filtering from strict to loose, more reliable pseudo-labels can be obtained, thereby improving the effectiveness of domain adaptation learning.

[0015] Furthermore, in the global filtering process from loose to strict, a dynamic adjustment strategy of pseudo-label confidence is adopted, through a pre-set fixed threshold C th , filter out pseudo labels with confidence greater than the threshold, and implement pseudo label filtering from loose to strict. That is, in the early stage of training, a relatively loose pseudo label confidence scoring strategy is sampled to allow more pseudo labels to participate in training. As the number of training rounds increases, the strictness of the confidence score is gradually increased to filter out unreliable pseudo labels. The pseudo label confidence C(t) is obtained using the following calculation formula:

[0016]

[0017] δ(t) is a factor that is dynamically adjusted with the iteration round t, and its value range is (0,1); M is the number of samples used in each iteration; α is a constant (for example, 5); C det (t) and C bbx (t) are the category confidence and position confidence of the pseudo label at each iteration, which are predicted by the target detector F.

[0018] Furthermore, in the bridge filtering process from strict to loose, a threshold adaptive dynamic adjustment strategy is adopted. The average confidence of the highest confidence area is used as the benchmark, and the threshold C is dynamically adjusted. p (t), it is realized that pseudo labels are introduced from non-highest confidence areas from strict to loose, and the dynamic threshold is calculated as:

[0019]

[0020] Among them Cb and C e are the initial and final thresholds (e.g., 1 and 0.9, respectively), n(t) and N are the current training iteration and the total training iteration, is the average confidence of the pseudo-labels in the tth round in the highest confidence region.

[0021] Furthermore, in the confidence-based hybrid image generation step, a pixel-level image fusion method with regional confidence weighting is designed. First, the average pseudo-label confidence of each region is calculated. The four regions are sorted by size, and the four regions are divided into high, medium, medium and low confidence regions according to the ranking order from high to low. Then, for the three types of high, medium and low confidence, three different parameters of Beta (α, β) function are used to randomly generate a weight ratio λ for each region when generating the mixed image. i For high confidence areas, a Beta function with a value distribution biased towards 1 is used to make the visible image x S has a higher weight ratio; for low confidence areas, a Beta function with a value distribution biased towards 0 is used to make the visible light image x S The weight ratio is too small; for the medium confidence area, the Beta function with a value distribution biased towards 0.5 is used to make the visible light image x S The weight ratio value is equal to that of the infrared image. After obtaining the weight, the four regions are mixed according to the following formula to obtain the mixed image of the four regions. The final mixed image x can be obtained by merging M .

[0022]

[0023] Where ⊙ represents the pixel-level blending operation.

[0024] Furthermore, in model training and optimization, supervision loss and consistency loss are used for constraints, and the overall loss function can be expressed as:

[0025] L total =L det +γL cons

[0026] in To supervise the loss, we optimize the object detector using labeled visible light images, y S is the detection result of the detector, is the corresponding reference label; is the consistency loss, is the detection result of the target detector on the mixed image, is the reference label corresponding to the mixed image; γ is the dynamic weight, which is adjusted according to the reliability of the pseudo label; L() is the universal cross entropy loss, which is used to measure the classification loss.

[0027] In general, the above technical solutions conceived by the present invention have the following beneficial effects compared with the prior art:

[0028] (1) Global information utilization: Through the regional pixel-level mixing strategy, the mixing ratio is dynamically adjusted in combination with the pseudo-label confidence, which overcomes the limitation of existing methods that only focus on high-confidence areas. It enhances the model's ability to learn the global thermal features of the target domain in infrared images, and is particularly suitable for scenes with small target scales and blurred thermal features.

[0029] (2) Improved pseudo-label reliability: A bidirectional dynamic region pseudo-label filtering mechanism is adopted to introduce more infrared target information in the early stage of training and filter reliable pseudo-labels in the later stage, which improves the quality of pseudo-labels and effectively improves the accuracy of infrared UAV detection.

[0030] (3) High efficiency and real-time performance: Based on a single-stage detector (such as YOLOv8), combined with pixel-level mixing and consistency loss optimization, it meets the real-time and lightweight requirements of the infrared UAV platform. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 : Flowchart of adaptive target detection in the UAV domain using infrared images provided by an embodiment of the present invention;

[0032] Figure 2 : A structural diagram of a domain adaptation network based on pixel-level mixing of regional pseudo-label confidences provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0033] In order to make the objectives, technical solutions, and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to specific embodiments. It should be understood that the specific embodiments described herein are intended only to illustrate the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.

[0034] To address the performance degradation of adaptive target detection in drone infrared target domains due to domain distribution differences, this paper provides a method for adaptive target detection in drone image domains based on pixel-level blending of regional pseudo-label confidences. By utilizing a regional pixel-level blending strategy and a bidirectional dynamic regional pseudo-label filtering mechanism, this method achieves effective knowledge transfer from the source domain to the target domain, improving the model's detection accuracy and generalization capabilities in drone scenarios. The following describes a specific implementation of this invention using a single-stage detector based on YOLOv8 as an example.

[0035] refer to Figure 1-Figure 2 This embodiment takes the domain adaptive object detection task of UAV images from sunny scenes (source domain) to foggy scenes (target domain) as an example. The specific implementation steps are as follows:

[0036] Data preparation: Prepare source domain datasets, such as visible light images collected by drones using visible light imaging equipment (denoted as x S ), and mark the drone in the image to form the corresponding annotation data y S (Including the bounding box and category of the drone target). Prepare the target domain dataset, such as the drone infrared image collected by infrared thermal imaging equipment (denoted as x T ).

[0037] Source domain detector training: YOLOv8 is used as the target detection network, and the source domain dataset is used to train the network using supervised learning training to obtain the source domain target detector F VL and F VL As the initial model of the target detector in the target domain, that is, the target detector F = F IR =F VL .

[0038] Pseudo label generation: using target detector F IR Perform drone target detection on the target domain image and use the detection output as a pseudo label (including the bounding box and confidence of the detected target) to form a pseudo label set where N T is the number of detected targets, is the pseudo label of the i-th target (including bounding box coordinates and confidence).

[0039] Region division and confidence calculation: The source domain image x S and target domain infrared image x T Divide it into 4 regions of equal size in the form of a 2×2 grid, which are respectively and According to the center coordinates of each pseudo label, it is assigned to the corresponding area and the average confidence of each area is calculated. The result is recorded as represents the average pseudo-label confidence of the jth region; if there is no pseudo-label in the region, the confidence is set to 0.

[0040] Confidence-based regional pixel-level mixing: based on the average confidence of each region Ranking, assigning Beta function parameters: the highest confidence region uses Beta(32,18) with a bias toward 1, the lowest confidence region uses Beta(18,32) with a bias toward 0, and the medium confidence region uses Beta(32,32) with a bias toward 0.5. The weight ratio generated by the Beta function with the corresponding parameters for the i-th region is λ i After obtaining the weights, the four regions are mixed according to the following formula to obtain the mixed image of the four regions The final mixed image x can be obtained by merging M :

[0041]

[0042] Bidirectional dynamic region pseudo-label filtering: Bidirectional dynamic region pseudo-label filtering mechanism is used to filter the infrared image x T Generated pseudo labels Perform filtering and record the filtered pseudo label set as In the global filtering process from loose to strict, the filtering threshold C th Set to 0.3. In the bridge filtering process from strict to loose, the dynamic threshold C p (t)From C b =0.9 to C e =0.8 transition.

[0043] Hybrid label generation: the source domain image label y S and the target domain pseudo labels after filtering by bidirectional dynamic region pseudo labels Merge to generate mixed image pseudo labels

[0044] Model training and optimization: Using source domain image x S And the generated mixed image x M The target detector F is trained for domain adaptation and updated to obtain an updated target detector, which is then used to replace the target detector F used in the previous round.

[0045] Repeat the above six steps from pseudo-label generation to model training and optimization for 100 rounds to obtain the final target detector F. The final target detector F is then applied to the infrared UAV target detection application.

[0046] Those skilled in the art will readily understand that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention.

Claims

1. A drone domain adaptation detection method based on pixel-level hybrid enhancement of images, characterized by: The following steps are included: S1. Read data: read source domain data and target domain data, and take the source domain data as the visible light image x S and its label y S , the target domain data is the infrared image x T ; S2, pseudo label generation: the infrared image x T Input to the target detector F to generate a set of pseudo labels for the target domain where N T is the number of detected targets, is the pseudo label of the i-th target; wherein the target detector F is initially trained using source domain data, and the pseudo label includes bounding box coordinates and confidence; S3, region division and confidence calculation: the visible light image x S and infrared image x T Divide into four equal areas, denoted as and Represents the visible light image x S and infrared image x T The jth equally divided area; according to the pseudo label In the area where the center point is located, each pseudo label is assigned to the corresponding equally divided area, and the average pseudo label confidence of each area is calculated, which is recorded as represents the average pseudo-label confidence of the jth region; S4, Bidirectional dynamic area pseudo label filtering: adopt bidirectional dynamic area pseudo label filtering mechanism to filter the infrared image x T Generated pseudo labels Perform filtering and record the filtered target domain pseudo label as S5. Confidence-based hybrid image generation: for infrared image x T and the visible light image x S Perform regional confidence weighted pixel-by-pixel fusion to generate a mixed image x M ; S6, hybrid label generation: the label y of the source domain data S and the target domain pseudo labels after filtering by bidirectional dynamic region pseudo labels Merge to generate mixed image pseudo labels S7. Model training and optimization: Using visible light images x S And the generated mixed image x M The target detector F is trained for domain adaptation and updated to obtain the updated target detector F. The updated target detector F is used to replace the target detector F used in the previous round. Steps S2 to S7 are repeated. The final target detector F is obtained through iteration, thereby performing domain adaptive detection of infrared image drones based on image pixel-level hybrid enhancement.

2. The drone domain adaptation detection method based on pixel-level hybrid enhancement of images according to claim 1 is characterized in that: In the bidirectional dynamic area pseudo-label filtering step, the bidirectional dynamic area pseudo-label filtering mechanism adopts a combination of global filtering processing from loose to strict and bridging filtering processing from strict to loose; In the global filtering process from loose to strict, a dynamic adjustment strategy of pseudo-label confidence is adopted, through a pre-set fixed threshold C th , filter out pseudo labels with confidence greater than the threshold, and implement pseudo label filtering from loose to strict; In the bridge filtering process from strict to loose, the threshold adaptive dynamic adjustment strategy is adopted. The average confidence of the highest confidence area is used as the benchmark. By dynamically adjusting the threshold C p (t), achieving the goal of introducing pseudo labels from non-highest confidence regions from strict to loose.

3. The drone domain adaptation detection method based on pixel-level hybrid enhancement of images according to claim 2 is characterized in that: The global filtering process from loose to strict specifically includes: In the early stage of training, a relatively loose pseudo-label confidence scoring strategy is adopted to allow more pseudo-labels to participate in training. As the number of training rounds increases, the strictness of the confidence score is gradually increased to filter out unreliable pseudo-labels. The pseudo-label confidence C(t) is obtained using the following calculation formula: δ(t) is a factor that is dynamically adjusted with the iteration round t, and its value range is (0,1); M is the number of samples used in each iteration; α is a preset constant; C det (t) and C bbx (t) are the category confidence and position confidence of the pseudo label at each iteration, predicted by the target detector F.

4. The drone domain adaptation detection method based on pixel-level hybrid enhancement of images according to claim 2 is characterized in that: In the bridge filtering process from strict to loose, the threshold adaptive dynamic adjustment strategy is adopted. The average confidence of the highest confidence area is used as the benchmark. By dynamically adjusting the threshold C p (t), realize the introduction of pseudo labels from non-highest confidence areas from strict to loose, threshold C p (t) is calculated as: Among them C b and C e are the initial and final thresholds respectively, n(t) and N are the current training iteration t and the total training iterations, is the average confidence of the pseudo-labels in the tth round in the highest confidence region.

5. The drone domain adaptation detection method based on pixel-level hybrid enhancement of images according to claim 1 is characterized in that: The confidence-based mixed image generation step specifically includes the following steps: First, the pseudo-label confidence is averaged for each region Sort the four regions by size, and then assign them to high, medium, medium and low confidence levels in descending order. Then, for the three types of high, medium and low confidence, three Beta (α, β) functions with different parameters are used to randomly generate a weight ratio λ for each region when generating a mixed image. i ; For high confidence areas, a Beta function with a value distribution biased towards 1 is used to make the visible image x S has a higher weight ratio; for low confidence areas, a Beta function with a value distribution biased towards 0 is used to make the visible light image x S The weight ratio is too small; for the medium confidence area, the Beta function with a value distribution biased towards 0.5 is used to make the visible light image x S It has the same weight ratio value as the infrared image; In generating the weight ratio λ i Then, the four regions are mixed according to the following formula to obtain the mixed image of the four regions. The final mixed image x can be obtained by merging M : Where ⊙ represents the pixel-level blending operation.

6. The drone domain adaptation detection method based on pixel-level hybrid enhancement of images according to claim 1 is characterized in that: In the model training and optimization steps, when the target detector F is trained for domain adaptation, supervision loss and consistency loss are used for constraints, and the overall loss function L total Expressed as: THE total =L det +γL cons in To supervise the loss, we optimize the object detector using labeled visible light images, y S is the detection result of the detector, is the corresponding reference label; is the consistency loss, is the detection result of the target detector on the mixed image, is the reference label corresponding to the mixed image; γ is the dynamic weight, which is adjusted according to the reliability of the pseudo label; L() is the universal cross entropy loss, which is used to measure the classification loss.

Images