An unmanned aerial vehicle RGB-IR traffic small target detection method and device based on a main anchor frame constraint, an electronic device, and a readable storage medium

Abstract

This invention provides a method, apparatus, electronic device, and readable storage medium for UAV RGB-IR traffic target detection based on master anchor box constraints. The method includes the following steps: S1: forming RGB-IR paired inputs; S2: obtaining a set of master anchor boxes, which serves as the source constraint for the final detection candidate boxes; S3: obtaining a set of RGB expert boxes and a set of IR expert boxes; S4: finding expert matching boxes with consistent categories and spatial overlap satisfying a matching threshold; S5: using the corresponding expert boxes as auxiliary evidence of expert reliability; S6: performing low-confidence no-support filtering and confidence updates on the master anchor boxes; S7: performing weighted bounding box correction dominated by the master anchor boxes; S8: performing intra-category non-maximum suppression on the corrected candidate boxes. This invention improves the recall capability, localization stability, and deployment convenience of UAV RGB-IR traffic target detection by restricting unconstrained merging of multi-path detection boxes and utilizing expert matching boxes as auxiliary evidence.

Description

Technical Field

[0001] This invention relates to the field of target detection technology, specifically to a method, apparatus, electronic device, and readable storage medium for detecting small traffic targets on UAVs based on a master anchor frame constraint using RGB-IR. Background Technology

[0002] With the development of low-altitude intelligent sensing and drone inspection technologies, drones have gradually become an important mobile visual platform for traffic monitoring in urban roads, parking lots, park roads, and residential areas. Drones can acquire images from top-down or oblique top-down perspectives, covering a large area of ​​continuous road. However, traffic targets such as vehicles usually have small pixel ratios, dense target distribution, frequent occlusion, and complex background textures, making it easy for detectors to miss detections, make false detections, and experience positioning errors.

[0003] RGB visible light images offer rich texture, color, and edge information, but they are prone to texture degradation in low-light, shadow, and high-reflection scenes. Infrared images are more robust to changes in illumination, but target boundaries are often blurred, and their ability to distinguish categories may be insufficient. Therefore, RGB and infrared images are complementary in small target detection for UAVs in traffic.

[0004] Existing RGB-IR fusion detection methods typically include image-level fusion, input-level early fusion, feature-level fusion, and decision-level fusion. Image-level fusion can easily alter the original modality distribution, feature-level fusion often requires redesigning the network structure or training objectives, and ordinary decision-level box fusion tends to merge output boxes from different detectors as equivalent candidates. When low-confidence single-modal false detection boxes are introduced into the final result without constraints, it can lead to the accumulation of false detections.

[0005] Therefore, how to provide a UAV RGB-IR traffic target detection method based on main anchor frame constraints that can provide auxiliary evidence using RGB and infrared single-mode detection results without redesigning the backbone detection network, while avoiding unconstrained merging of multiple detection boxes, in order to improve the recall capability, positioning stability and deployment convenience of UAV RGB-IR traffic target detection, is the problem that this invention urgently needs to solve. Summary of the Invention

[0006] The purpose of this invention is to propose a method, device, electronic device, and readable storage medium for detecting small traffic targets in UAVs based on main anchor frame constraints. This solves the problems in existing multimodal detection result fusion, such as the easy introduction of single-modal false detections due to lack of constraints, the difficulty of adapting fixed-weight fusion to the reliability differences of target-level modalities, the large changes to the network structure in feature-level fusion, and the insufficient recall capability of small targets.

[0007] To achieve the above objectives, the technical solution adopted by the present invention is as follows: On the one hand, the present invention provides a method for detecting small traffic targets by UAVs based on master anchor frame constraints, comprising the following steps:

[0008] S1: Acquire RGB and infrared images of the same target area simultaneously from the visible light imaging device and infrared imaging device mounted on the UAV, and form an RGB-IR paired input; S2: Send the RGB-IR paired input to the fusion detector to obtain a set of master anchor boxes. The set of master anchor boxes serves as the source constraint for the final detection candidate boxes. The set of master anchor boxes includes multiple master anchor boxes, each of which includes box coordinates, category identifier, and confidence score; S3: Input the RGB image into the RGB single-modal expert detector to obtain an RGB expert box set, and input the infrared image into the IR single-modal expert detector to obtain an IR expert box set; S4: For each master anchor box in the set of master anchor boxes, search for expert matching boxes in the RGB expert box set and the IR expert box set that have the same category and spatial overlap and meet the matching threshold. Among them, expert candidate boxes that are found in the RGB expert box set and meet the matching conditions are denoted as RGB expert matching boxes, and expert candidate boxes that are found in the IR expert box set and meet the matching conditions are denoted as IR expert matching boxes; if there are no expert candidate boxes that meet the matching conditions in the corresponding modality, the expert matching boxes for that modality are empty; S5: Determine the expert support status of the main anchor box based on the found expert matching boxes and their sources, and use the corresponding expert boxes as auxiliary evidence of expert reliability; S6: Perform low-confidence no-support filtering and confidence update on the main anchor box based on the expert support status and the auxiliary evidence of expert reliability; S7: Perform weighted bounding box correction with the main anchor box as the main body on the retained main anchor box and its expert matching boxes; S8: Perform intra-class nonmaximum suppression on the corrected candidate boxes and output the traffic small object detection results containing category, confidence score and coordinates; wherein, the expert boxes in the RGB expert box set and the IR expert box set are not directly incorporated into the final detection result as unconstrained candidate boxes, but only participate in filtering, confidence score update and bounding box correction as auxiliary evidence of the main anchor box when the category consistency and spatial overlap conditions are met.

[0009] As a further improvement, the visible light imaging device and infrared imaging device in step S1 are installed on the UAV platform, or communicate with the edge sensing device that works in conjunction with the UAV platform.

[0010] The RGB-IR pairing input in step S2 is formed through one or a combination of channel splicing, time synchronization, and spatial registration; the fusion detector is an input-level early fusion detector, a four-channel RGB-IR detector, a feature fusion detector, or a trained multimodal target detector.

[0011] In step S3, the RGB single-modal expert detector and the IR single-modal expert detector are target detectors trained based on RGB images and infrared images, respectively.

[0012] As a further improvement, before performing expert matching on expert matching boxes that meet the threshold in step S4, low-confidence filtering and intra-source non-maximum suppression are performed on the main anchor box set, the RGB expert box set, and the IR expert box set, respectively, to remove duplicate candidate boxes within the same source; the expert matching process includes constructing the expert matching set according to the following conditions:

[0013] ;

[0014] in, This indicates a modal identifier used to distinguish between the RGB modal expert box set and the IR modal expert box set; Representing modes The corresponding set of single-modal expert boxes; Indicates the expert candidate box; Indicates the coordinates of the expert candidate boxes; Indicates the category of expert candidate boxes; Indicates the first [number] before the correction. The coordinates of the main anchor frame; Indicates the first One main anchor box category; Indicates expert candidate boxes With the The crossover ratio between the main anchor frames is used to measure the degree of spatial overlap between them. This represents the matching threshold. When multiple expert candidate boxes satisfy the matching conditions exist in the same modality, the expert candidate box with the largest intersection-union ratio (IoU) with the main anchor box is selected as the expert matching box for that modality.

[0015] As a further improvement, the expert support status in step S5 is determined by whether the RGB expert matching box and the IR expert matching box exist; when neither exists, there is no expert support; when only the RGB expert matching box exists, it is RGB single expert support; when only the IR expert matching box exists, it is IR single expert support; and when both exist, it is RGB-IR dual expert support.

[0016] As a further improvement, the low-confidence no-support filtering in step S6 includes: deleting the main anchor frame when it does not receive RGB expert matching frame support and IR expert matching frame support, and the main anchor frame confidence is lower than a preset filtering threshold; retaining the main anchor frame and maintaining or penalizing its confidence when it does not receive expert matching frame support but its confidence is not lower than the preset filtering threshold; the confidence update includes: enhancing the confidence of main anchor frames that receive RGB single expert support, IR single expert support, or RGB-IR dual expert support based on the main anchor frame confidence, expert matching frame confidence, number of supporting sources, and modal reliability parameters.

[0017] As a further improvement, the weighted bounding box correction in step S7 satisfies:

[0018] ;

[0019] in, Indicates participation in the The set of boxes corrected by the weighted bounding box of the first main anchor box, the set of boxes including the first... One main anchor box and its expert matching box; Represents the set of boxes Any of the participating correction boxes in the text; Indicates participation in the correction box The coordinates of the bounding box; Indicates participation in the correction box The corresponding coordinate weights; The coordinates represent the corrected candidate box coordinates, and the coordinate weight of the main anchor box is greater than that of the RGB expert matching box and the IR expert matching box. The coordinate weights of the main anchor box, the RGB expert matching box and the IR expert matching box are determined by one or a combination of candidate box confidence, category source prior and modal reliability parameters.

[0020] As a further improvement, the traffic target includes a vehicle target, which includes at least one of cars, trucks, and buses.

[0021] On the other hand, the present invention also provides a UAV RGB-IR traffic target detection device based on master anchor frame constraints. The UAV RGB-IR traffic target detection device based on master anchor frame constraints includes: an image acquisition unit, a synchronous registration unit, a fusion detection unit, an RGB expert unit, an IR expert unit, an expert matching unit, a state determination unit, a filtering update unit, a frame correction unit, a post-processing unit, and an output interface.

[0022] On the other hand, the present invention also provides an electronic device, characterized in that it includes a processor, a memory, and a program stored in the memory and executable on the processor, wherein when the program is executed by the processor, it implements a UAV RGB-IR traffic small target detection method based on a master anchor frame constraint.

[0023] On the other hand, the present invention also provides a computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program, which, when executed by a processor, implements a method for detecting small traffic targets in UAVs based on a master anchor frame constraint.

[0024] Compared with the prior art, the beneficial effects of the present invention are:

[0025] This invention uses the output of the fused detector as the main anchor box source, maintaining a stable source of the final candidate boxes and reducing redundancy and false detections caused by unconstrained merging of boxes from multiple detectors. It utilizes RGB and IR single-modal experts to provide target-level auxiliary evidence, enabling complementary local information from different modalities to participate in the final decision. Through low-confidence filtering without expert support and confidence updates based on support states, this invention suppresses false detections while retaining low-confidence small targets supported by expert evidence. Furthermore, this invention employs a weighted bounding box correction method dominated by the main anchor box, requiring only lightweight post-processing and not altering the backbone detection network structure, making it easy to deploy at the edge of a UAV or on a server. Attached Figure Description

[0026] Figure 1 This is a flowchart illustrating a UAV RGB-IR traffic small target detection method based on main anchor frame constraints according to the present invention.

[0027] Figure 2 This is a schematic diagram showing the composition and connection relationship of a UAV RGB-IR traffic small target detection device based on main anchor frame constraints according to the present invention;

[0028] Figure 3 This is a schematic diagram illustrating the expert consensus matching and support status determination of the present invention;

[0029] Figure 4 This is a schematic diagram of the filtering, confidence update, bounding box correction and post-processing process of the present invention;

[0030] Figure 5 This is a schematic diagram illustrating the weighted correction of the main anchor frame and the expert matching frame in this invention;

[0031] Figure 6 This is a schematic diagram of the application deployment and device implementation structure of the present invention.

[0032] Among them, 100 is the image acquisition unit; 110 is the synchronous registration unit; 120 is the fusion detection unit; 130 is the RGB expert unit; 140 is the IR expert unit; 150 is the expert matching unit; 160 is the state determination unit; 170 is the filter update unit; 180 is the box correction unit; 190 is the post-processing unit; and 200 is the output interface. Detailed Implementation

[0033] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments. In the drawings, arrows (unless otherwise specified) indicate data flow or processing direction, and dashed boxes indicate functional groups or deployment boundaries.

[0034] Example 1:

[0035] like Figure 1 As shown, this embodiment provides a UAV RGB-IR traffic small target detection method based on master anchor frame constraints. First, RGB and infrared images of the same target area are acquired simultaneously by a visible light imaging device and an infrared imaging device mounted on the UAV, forming an RGB-IR paired input. The RGB image retains texture, color, and edge information, while the infrared image retains thermal radiation and brightness thermal information. The RGB-IR paired input can be formed by time synchronization, spatial registration, and channel stitching. Specifically, the visible light camera acquires the RGB image; the infrared camera acquires the IR image; and the synchronization registration module forms the RGB-IR paired input.

[0036] The RGB-IR paired input is fed into the fusion detector to obtain the set of main anchor frames. Input the RGB image into the RGB single-modal expert detector to obtain the RGB expert box set. The infrared image is input into an IR single-mode expert detector to obtain an IR expert box set. Main anchor frame set Each main anchor box in the table includes coordinates, category, and confidence level.

[0037] For each main anchor box, in the RGB expert box set and IR expert box set The system searches for expert matching boxes that are of the same category and whose spatial overlap with the main anchor box meets a threshold. Then, it determines the expert support state based on the existence of RGB expert matching boxes and IR expert matching boxes, and performs filtering, confidence update, bounding box correction, and intra-category NMS accordingly, outputting the final traffic small object detection result.

[0038] Device composition and detector construction:

[0039] like Figure 2As shown, the device includes an image acquisition unit 100, a synchronous registration unit 110, a fusion detection unit 120, an RGB expert unit 130, an IR expert unit 140, an expert matching unit 150, a state determination unit 160, a filter update unit 170, a bounding box correction unit 180, a post-processing unit 190, and an output interface 200. The image acquisition unit 100 acquires RGB and infrared images; the synchronous registration unit 110 forms RGB-IR paired inputs; the fusion detection unit 120 generates master anchor boxes; the RGB expert unit 130 and the IR expert unit 140 generate RGB and IR expert boxes, respectively; the expert matching unit 150, the state determination unit 160, the filter update unit 170, the bounding box correction unit 180, and the post-processing unit 190 sequentially perform expert matching, state determination, filtering and confidence update, bounding box correction, and in-class NMS; the output interface 200 outputs the detection results.

[0040] exist Figure 2 In the illustrated device, the expert detection results output by the RGB expert unit 130 and the IR expert unit 140 serve only as auxiliary evidence for the main anchor boxes and are not directly incorporated into the final detection results as unconstrained candidate boxes. Each unit can be implemented through software modules, hardware modules, or a combination of software and hardware.

[0041] In a preferred embodiment, the fusion detector may be a four-channel RGB-IR early fusion detector, whose input includes three RGB channels and one infrared channel; the RGB single-modal expert detector and the IR single-modal expert detector may be trained on the RGB image and the infrared image respectively using the same basic detection framework. The above detectors may be YOLO series detectors, single-stage detectors, two-stage detectors, or other target detection models capable of outputting detection boxes, categories, and confidence scores.

[0042] To reduce the impact of duplicate candidate boxes from the same source, the set of main anchor boxes can be evaluated before expert matching. RGB expert box set and IR expert box set Low-confidence filtering and in-source non-maximum suppression are performed separately. The pre-confidence threshold and the in-source NMS threshold can be adjusted based on the validation set results, the computing power of the deployed devices, and the application scenario.

[0043] Expert consensus matching and support status determination:

[0044] like Figure 3As shown, for ease of representation, the expert support states are denoted as U0, U1, U2, and U3, where U0 represents no expert support, U1 represents RGB expert support, U2 represents IR expert support, and U3 represents RGB-IR dual expert support. The expert consistency matching and support state determination process includes three parts: input candidate set, matching rule execution, and support state determination. For each main anchor box, matching is performed in both the RGB and IR expert box sets according to the condition of consistent category and cross-union ratio (CUI) not less than the matching threshold. When multiple expert candidate boxes satisfy the condition for the same modality, the candidate box with the largest CUI is selected as the expert matching box for that modality. Subsequently, the states of no expert support, RGB single expert support, IR single expert support, or RGB-IR dual expert support are determined based on the RGB and IR modality matching results. Let the set of main anchor boxes output by the fusion detector be... Single-mode expert detectors in modality The set of expert boxes output below is ,in Choose either RGB or IR. Each set can be represented as:

[0045] (1);

[0046] In equation (1), , and These represent the first and second digits before the correction. The coordinates, category, and confidence level of each main anchor box; , and They represent the first The coordinates, category, and confidence level of each expert candidate box; and These represent the number of main anchor boxes and the number of expert candidate boxes for the corresponding modality, respectively. For any expert candidate box... A bounding box is considered a valid expert matching box that matches the main anchor box only if it satisfies the constraints of category consistency and spatial overlap. The matching conditions are as follows:

[0047] (2);

[0048] Based on equation (2), the first The main anchor frame in modality The expert matching set below can be represented as:

[0049] (3);

[0050] In equation (3), The threshold value represents the matching threshold, IoU represents the intersection-over-union function, and RGB and IR are modal identifiers represented in upright form. When multiple expert candidate boxes satisfy the conditions for the same modality, the candidate box with the largest IoU with the main anchor box is preferably selected as the expert matching box for that modality.

[0051] set up Indicates the first The set of expert matches found in the RGB expert box set for each main anchor box. Indicates the first The expert matching set found in the IR expert box set for each main anchor box, based on... and Whether it is empty or not, four support states can be obtained: when both are empty, there is no expert support; only... When not empty, RGB single expert support is available; only When the data is not empty, it uses IR single-expert support; when both are not empty, it uses RGB-IR dual-expert support. To facilitate subsequent confidence level updates, the expert support strength can also be defined. for:

[0052] (4);

[0053] Filtering, confidence update, and bounding box correction:

[0054] like Figure 4 As shown, when the main anchor box is not supported by RGB expert matching boxes and is not supported by IR expert matching boxes, and the confidence level of the main anchor box is lower than the preset filtering threshold, it is judged as a low-reliability candidate box and deleted. For main anchor boxes that are not supported by expert matching boxes but whose confidence level is not lower than the preset filtering threshold, they can be retained and a slight confidence level penalty can be applied, or their original confidence level can be maintained according to the needs of the scenario.

[0055] For anchor frames that receive RGB single-expert support, IR single-expert support, or RGB-IR dual-expert support, the confidence level is updated based on the anchor frame confidence level, the expert matching frame confidence level, and the number of supporting sources. The more supporting sources, the higher the enhancement factor the updated confidence level can achieve; if only a single expert support exists, a smaller enhancement is applied based on the corresponding expert's confidence level and modal reliability parameters.

[0056] like Figure 5 As shown, a weighted bounding box correction, primarily driven by the main anchor box, is performed on the retained main anchor box and its expert matching boxes. Specifically, Figure 5In the model, IR expert boxes are represented by dashed lines, RGB expert boxes by thin solid lines, and main anchor boxes by thick solid lines. Corrected boxes are obtained by a weighted average of the main anchor boxes and their matching expert boxes. The correction process does not include RGB and IR expert boxes in the final candidate set without constraints. RGB expert boxes are auxiliary evidence boxes output by the RGB single-modal expert detector that satisfy the matching conditions; IR expert boxes are auxiliary evidence boxes output by the IR single-modal expert detector that satisfy the matching conditions. Let... and These represent the optimal expert matching boxes selected from the corresponding modal expert matching sets, participating in the... The set of boxes for correcting the main anchor box. It can be represented as:

[0057] (5);

[0058] In Equation (5), Ai is the i-th main anchor box. Based on Equation (5), the corrected candidate box coordinates can be calculated using the following weighted method:

[0059] (6);

[0060] In equation (6), Indicates the corrected candidate box coordinates; Represents a set Any of the participating correction boxes in the text; Indicates participation in the correction box The coordinates; Indicates participation in the correction box The corresponding coordinate weights. To maintain stable output from the main detector, the coordinate weight of the main anchor box is preferably greater than that of the RGB expert matching box and the IR expert matching box.

[0061] After completing the confidence update and bounding box correction, intra-category nonmaximum suppression is performed on the candidate boxes to suppress duplicate detection boxes and output the final traffic small target detection results. The processor, memory, input interface, master anchor box constraint decision fusion program, post-processing and output interface together constitute the implementation structure of the electronic device; vehicle category, confidence and target coordinates are the output content of the detection results.

[0062] Reliability statistics parameters and threshold settings:

[0063] In one alternative implementation, the reliability differences between RGB experts and IR experts under different categories, scales, or scenarios can be statistically analyzed based on the training or validation set, and category source priors or modal reliability parameters can be set accordingly. For example, when the average reliability of the infrared expert is high in a certain type of target, the auxiliary weight of the IR expert box in that category can be appropriately increased; when the RGB expert performs well in small-scale targets, the weight of the RGB expert box participating in the correction can be appropriately increased.

[0064] Matching threshold, filtering threshold, confidence enhancement factor, coordinate weight, and NMS threshold can all be configurable parameters. In one set of implementation parameters, the matching IoU threshold can be set to 0.30 to 0.60, the source-level NMS threshold can be set to 0.60 to 0.75, the final category-level NMS threshold can be set to 0.50 to 0.70, and the low-confidence no-support filtering threshold can be adaptively determined based on the main anchor box confidence distribution. The above parameters are only preferred implementations and do not constitute a limitation on the scope of protection of this invention.

[0065] Application Example 1:

[0066] like Figure 6 As shown, this invention can be deployed on a drone platform, an edge computing box, an in-vehicle computing device, or a cloud server. The drone platform acquires RGB and infrared images through a visible light camera and an infrared camera, and transmits them to the edge computing device or server after synchronization and pairing. The edge computing device runs a fusion detector, a single-modal expert detector, and a master anchor box constraint decision fusion program, outputting vehicle category, confidence level, and coordinates.

[0067] This invention is applicable to scenarios such as road traffic monitoring, parking lot inspection, park vehicle management, low-altitude traffic perception, and multimodal target detection. Traffic targets can include categories such as car, truck, bus, and van, and can also be expanded to motorcycles, pedestrians, non-motorized vehicles, or other smaller target categories based on training data.

[0068] In one embodiment, the device may be implemented by an electronic device, including a processor, a memory, and a program stored in the memory and executable on the processor. When the program is executed by the processor, it implements the method described in any of the above embodiments. The electronic device may be an onboard computing unit of a drone, an edge computing box, a workstation, a server, or an embedded device with neural network inference capabilities.

[0069] In one embodiment, the program may be stored in a computer-readable storage medium. The computer-readable storage medium may be a hard disk, solid-state drive, memory card, read-only memory, random access memory, or other media capable of storing programs.

[0070] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any other way. Any modifications or equivalent changes made based on the technical essence of the present invention shall still fall within the scope of protection claimed by the present invention.

Claims

1. A method for detecting small traffic targets by UAVs based on master anchor frame constraints, characterized in that, Includes the following steps: S1: Acquire RGB and infrared images of the same target area simultaneously captured by the visible light imaging device and infrared imaging device on the UAV, and form RGB-IR paired input; S2: The RGB-IR paired input is fed into the fusion detector to obtain a set of master anchor boxes. The set of master anchor boxes serves as the source constraint for the final detection candidate boxes. The set of master anchor boxes includes multiple master anchor boxes, each of which includes box coordinates, class identifier, and confidence score. S3: The RGB image is input into the RGB single-modal expert detector to obtain an RGB expert box set. The infrared image is input into the IR single-modal expert detector to obtain an IR expert box set. S4: For each main anchor box in the main anchor box set, search for expert matching boxes in the RGB expert box set and the IR expert box set respectively that have the same category and spatial overlap and meet the matching threshold. Expert candidate boxes found in the RGB expert box set and meeting the matching conditions are denoted as RGB expert matching boxes, and expert candidate boxes found in the IR expert box set and meeting the matching conditions are denoted as IR expert matching boxes. If there are no expert candidate boxes meeting the matching conditions in the corresponding modality, the expert matching boxes for that modality are empty. S5: Determine the expert support state of the main anchor box based on the found expert matching boxes and their sources, and use the corresponding expert boxes as auxiliary evidence of expert reliability. S6: Based on the expert support state and the auxiliary evidence of expert reliability, perform low-confidence no-support filtering and confidence update on the main anchor boxes. S7: Perform weighted bounding box correction with the main anchor box as the dominant factor on the retained main anchor boxes and their expert matching boxes. S8: Perform intra-category non-maximum suppression on the corrected candidate boxes, and output the traffic small target detection results including category, confidence, and coordinates. In this context, the expert boxes in the RGB expert box set and the IR expert box set are not directly incorporated into the final detection result as unconstrained candidate boxes, but only participate in filtering, confidence update and bounding box correction as auxiliary evidence of the main anchor box when the conditions of class consistency and spatial overlap are met.

2. The method for detecting small traffic targets by UAVs based on master anchor frame constraints according to claim 1, characterized in that, The visible light imaging device and infrared imaging device in step S1 are installed on the UAV platform or communicate with the edge sensing device that works in conjunction with the UAV platform. The RGB-IR pairing input in step S2 is formed through one or a combination of channel splicing, time synchronization, and spatial registration; the fusion detector is an input-level early fusion detector, a four-channel RGB-IR detector, a feature fusion detector, or a trained multimodal target detector. In step S3, the RGB single-modal expert detector and the IR single-modal expert detector are target detectors trained based on RGB images and infrared images, respectively.

3. The method for detecting small traffic targets by UAVs based on master anchor frame constraints according to claim 1, characterized in that, Before performing expert matching on expert matching boxes that meet the threshold in step S4, low-confidence filtering and intra-source non-maximum suppression are performed on the main anchor box set, RGB expert box set, and IR expert box set, respectively, to remove duplicate candidate boxes from the same source; the expert matching process includes constructing the expert matching set according to the following conditions: ； in, This indicates a modal identifier used to distinguish between the RGB modal expert box set and the IR modal expert box set; Representing modes The corresponding set of single-modal expert boxes; Indicates the expert candidate box; Indicates the coordinates of the expert candidate boxes; Indicates the category of expert candidate boxes; Indicates the first number before the correction. The coordinates of the main anchor frame; Indicates the first One main anchor box category; Indicates expert candidate boxes With the The crossover ratio between the main anchor frames is used to measure the degree of spatial overlap between them. This represents the matching threshold. When there are multiple expert candidate boxes that meet the matching conditions in the same modality, the expert candidate box with the largest intersection-union ratio with the main anchor box is selected as the expert matching box for that modality.

4. The method for detecting small traffic targets by UAVs based on master anchor frame constraints according to claim 3, characterized in that, The expert support status in step S5 is determined by whether the RGB expert matching box and the IR expert matching box are empty; when both the RGB expert matching box and the IR expert matching box are empty, there is no expert support; when only the RGB expert matching box is not empty, there is RGB single expert support; when only the IR expert matching box is not empty, there is IR single expert support; when both the RGB expert matching box and the IR expert matching box are not empty, there is RGB-IR dual expert support.

5. The method for detecting small traffic targets by UAVs based on master anchor frame constraints according to claim 1, characterized in that, The low-confidence no-support filtering in step S6 includes: setting a preset filtering threshold, which is used to determine whether a main anchor box that has not obtained expert matching box support should be deleted as a low-reliability candidate box; when the main anchor box has not obtained RGB expert matching box support and has not obtained IR expert matching box support, and the confidence of the main anchor box is lower than the preset filtering threshold, the main anchor box is deleted; when the main anchor box has not obtained expert matching box support but the confidence is not lower than the preset filtering threshold, the main anchor box is retained and its confidence is maintained or penalized; the confidence update includes: for main anchor boxes that have obtained RGB single expert support, IR single expert support, or RGB-IR dual expert support, confidence is enhanced based on the main anchor box confidence, expert matching box confidence, number of supporting sources, and modal reliability parameters.

6. The method for detecting small traffic targets by UAVs based on master anchor frame constraints according to claim 4, characterized in that, The weighted bounding box correction in step S7 satisfies: ； in, Indicates participation in the The set of boxes corrected by the weighted bounding box of the first main anchor box, the set of boxes including the first... One main anchor box and its expert matching box; Represents the set of boxes Any of the participating correction boxes in the table; Indicates participation in the correction box The coordinates of the bounding box; Indicates participation in the correction box The corresponding coordinate weights; The coordinates represent the corrected candidate box coordinates, and the coordinate weight of the main anchor box is greater than that of the RGB expert matching box and the IR expert matching box. The coordinate weights of the main anchor box, the RGB expert matching box and the IR expert matching box are determined by one or a combination of candidate box confidence, category source prior and modal reliability parameters.

7. The method for detecting small traffic targets by UAVs based on master anchor frame constraints according to claim 1, characterized in that, The traffic targets include vehicle targets, which include at least one of cars, trucks, and buses.

8. A UAV RGB-IR traffic small target detection device based on master anchor frame constraints, characterized in that, The UAV RGB-IR traffic target detection device based on the main anchor frame constraint includes: an image acquisition unit (100), a synchronous registration unit (110), a fusion detection unit (120), an RGB expert unit (130), an IR expert unit (140), an expert matching unit (150), a state determination unit (160), a filter update unit (170), a box correction unit (180), a post-processing unit (190), and an output interface (200).

9. An electronic device, characterized in that, It includes a processor, a memory, and a program stored in the memory and executable on the processor, wherein the program, when executed by the processor, implements the method of any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method described in any one of claims 1-7.

Images