A reliability-gated unmanned aerial vehicle dense small target detection training method and system
By constructing a reliability gating mechanism in the inspection of civil aviation airports using drones, and by utilizing airport functional area masks and target detection networks, the problems of missed detection and confusion in target detection were solved, resulting in more efficient target detection.
Patent Information
- Application Number
- CN202611587122.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-10-14
- Publication Date
- 2026-11-10
AI Technical Summary
Existing technologies for target detection in UAV inspections at civil aviation airports suffer from problems such as missed detections, duplicate detections, and confusion between adjacent targets. This is especially true in scenarios with dense targets and frequent occlusion, where the candidate quality of the auxiliary prediction branch is inconsistent and erroneous auxiliary supervision interferes with the main detection branch.
By constructing a reliability gating mechanism, utilizing airport functional area masks and target detection networks, the reliability status within candidate groups is calculated, adaptive threshold generation and weight redistribution are performed, forming a continuous control closed loop with data dependencies, and auxiliary candidates are aggregated to uniformly control the supervision of the main detection branch.
It effectively reduces the interference of duplicate candidates and location offset candidates on the main detection branch, improves the accuracy and consistency of target detection, adapts to the training needs of different airport functional areas, and reduces the computational overhead during the training period.
Smart Images

Figure CN123071801A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of computer vision technology, and in particular to a training method and system for detecting dense small targets in unmanned aerial vehicles based on reliability gating. Background Technology
[0002] In existing technologies, Transformer-based object detection methods typically achieve end-to-end detection through object queries, bipartite graph matching, multi-scale features, and decoder prediction heads. To increase positive sample coverage during the training phase, some methods set up one-to-many auxiliary prediction branches, enabling a real target to match multiple auxiliary queries, and utilize auxiliary candidates to enhance the main detection branch. However, images from UAV inspections at civil aviation airports are characterized by small target size, dense clusters of similar targets, frequent occlusion, regular pavement textures, and complex background reflections. Small FODs on runways or taxiways have few effective pixels, and personnel and vehicles on the apron easily generate multiple overlapping candidate responses, leading to missed detections, duplicate detections, and confusion between adjacent targets.
[0003] Currently, while one-to-many auxiliary prediction branches can generate multiple candidates for the same real target, these candidates exhibit significant differences in classification confidence, localization quality, and consistency with the main detection branch. If multiple auxiliary candidates are used indiscriminately for training the main detection branch, low-quality candidates, localization-offset candidates, and duplicate response candidates will transmit conflicting supervisory information to the main detection branch, causing auxiliary knowledge pollution. Furthermore, existing candidate selection methods typically rely on a single confidence level, a fixed threshold, or independent candidate deduplication rules. Whether a candidate participates in training, how multiple candidates for the same target compete, and the intensity with which candidates affect the main detection branch are often subject to independent control criteria. Simultaneously, general selection methods struggle to leverage the differences in target scale, target density, and missed detection risk between runways, taxiways, aprons, and ordinary areas. Therefore, one-to-many auxiliary prediction suffers from inconsistent candidate quality, duplicate candidates, and erroneous auxiliary supervision interfering with the main detection branch.
[0004] Therefore, there is still room for improvement in existing technologies for target identification in airport environments. Summary of the Invention
[0005] The purpose of this invention is to overcome the shortcomings of the prior art and provide a training method and system for UAV dense small target detection based on reliability gating. The method constructs a reliability state and uses this state to sequentially drive the candidate participation degree, the weight allocation within a one-to-many candidate group corresponding to the same real target, and the auxiliary supervision strength of the main detection branch, forming a continuous control closed loop with data dependence before and after, thereby forming specific training adaptations for small and dense targets in different functional areas of civil aviation airports.
[0006] The objective of this invention is achieved through the following technical solution:
[0007] In a first aspect, this application discloses a training method for detecting dense small targets on unmanned aerial vehicles (UAVs) based on reliability gating, comprising the following steps:
[0008] Acquire training images of civil aviation airport inspections, target annotations, and multi-category airport functional area masks corresponding to the image space, and perform synchronous transformations on the images, target boxes, and area masks during spatial data augmentation. Input the training images into a target detection network. The main detection branch of the target detection network determines a unique main query for each real target through one-to-one matching, while the auxiliary prediction branch determines auxiliary candidates for the same real target through one-to-many matching. For each auxiliary candidate in the candidate group, calculate the reliability state within the same candidate group based on its classification confidence, localization quality, prediction consistency with the corresponding main query, real target scale, and importance of the airport functional area where the real target is located. Generate an adaptive threshold within the group based on the mean and dispersion of the reliability state within the same candidate group, calculate the candidate soft gating value, and redistribute the weights of multiple auxiliary candidates within the candidate group based on the candidate soft gating value and the reliability state within the candidate group. Aggregate the auxiliary candidates using the redistributed weights to obtain classification auxiliary targets and localization auxiliary targets oriented towards the main detection branch, and uniformly control the classification auxiliary targets and localization auxiliary targets with the effective weights of the candidate group to perform classification supervision and localization supervision for the corresponding main query.
[0009] Secondly, this application discloses a reliability-gated UAV dense small target detection training system, including a processor, a memory, and program instructions stored in the memory and executable by the processor; the program instructions are used to implement the reliability-gated UAV dense small target detection training method.
[0010] The beneficial effects of this invention are:
[0011] First, candidate gating, intra-group redistribution, and auxiliary supervision share the same reliability state, avoiding control fragmentation caused by multiple independent modules using different criteria. Second, one-to-many candidates for the same real target form an aggregated auxiliary target after intra-group competition, reducing the interference of duplicate candidates and location offset candidates on the main detection branch. Third, airport functional area masks directly participate in reliability aggregation during training, enabling small FODs on the runway, taxiway intrusion targets, and densely populated personnel and vehicles on the apron to obtain training control adapted to the scenario. Fourth, only the main detection branch is retained during the inference phase, without increasing the inference computation overhead brought by the auxiliary structure during training. Attached Figure Description
[0012] Figure 1This is a schematic diagram of the overall process of a reliability-gated UAV dense small target detection training method according to some embodiments of this application.
[0013] Figure 2 This is a schematic diagram of the architecture of an object detection network according to some embodiments of this application. Detailed Implementation
[0014] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0015] See Figures 1-2 This invention provides a training method for detecting dense small targets by UAVs based on reliability gating. The method in this application is applicable to low-altitude UAV inspection images of civil aviation airports and flight training bases.
[0016] First embodiment.
[0017] The training method for UAV dense small target detection based on reliability gating in the first embodiment refers to... Figure 1 Understanding includes the following steps:
[0018] S1. Acquire training images of civil aviation airport inspections, target annotations, and multi-category airport functional area masks corresponding to the image space, and perform synchronous transformations on the image, target bounding boxes, and area masks when performing spatial data augmentation.
[0019] In some embodiments, visible light inspection images collected by the UAV in the runway, taxiway, apron, parking positions, and surrounding areas are acquired, and targets such as personnel, vehicles, FOD, cones, lights, and facility components in the images are labeled with categories and bounding boxes. Simultaneously, an airport functional area mask M(x,y) corresponding to the spatial location of each training image is set. The mask distinguishes between general areas, apron operation areas, taxiways and their safety boundaries, and runways and their protected areas. The area mask is represented as follows:
[0020]
[0021] Different regions correspond to preset importance coefficients , , and The coefficients can be set according to the sensitivity of different areas to missed inspections in the inspection task, and preferably, the importance of ordinary areas should not be higher than that of the apron, taxiway, and runway areas. The area categories and importance coefficients are not limited to the above numbers and can be adjusted according to airport operating rules and inspection tasks. In this example, the importance coefficients are set as follows: .
[0022] During the training phase, color perturbation, random cropping, scaling, and horizontal flipping can be performed on the images. For operations that alter the spatial structure of the image, the same cropping, scaling, flipping, or stitching transformations are performed on the bounding boxes and airport functional region masks; when using Mosaic, the corresponding region masks of each image are stitched together synchronously; Mixup enhancements that cannot maintain the spatial correspondence between region masks and image space are not used in region-aware training batches.
[0023] S2. Input the training images into the object detection network.
[0024] refer to Figure 2 In this embodiment, a target detection network is constructed that includes a main detection branch and a training-phase one-to-many auxiliary prediction branch. The main detection branch uses one-to-one matching to determine the main query for each real target, and the auxiliary prediction branch uses one-to-many matching to determine auxiliary candidates for the same real target. The auxiliary candidates are then matched and their validity is determined. The auxiliary candidates that pass the validity determination form a candidate group. Only non-empty candidate groups are included in the subsequent candidate reliability aggregation, intra-group weight redistribution, and main detection branch auxiliary supervision.
[0025] Specifically, the target detection network includes a backbone network, a multi-scale feature extraction structure, a Transformer encoder, a main detection branch, and an auxiliary prediction branch.
[0026] The main detection branch generates the target category, bounding box, and query features for the final deployment model. It calculates the main detection loss based on the main query output, which includes classification loss and localization loss. The auxiliary prediction branch only operates during the training phase. For models containing... The number of training images of real targets is set to be no less than [number] auxiliary queries. ,in It is an integer not less than 1.
[0027] For the g-th true target, the main detection branch obtains the unique corresponding main query m(g) through bipartite graph matching. The auxiliary prediction branch calculates the matching cost between the i-th auxiliary candidate and the g-th true target based on the classification cost, bounding box distance cost, and intersection-union ratio cost, as follows:
[0028]
[0029] In the formula, Indicates the first The class probability distribution of the auxiliary candidate outputs. This indicates its predicted bounding box. and They represent the first The category labels and ground truth bounding boxes of each real target; Represents the classification cost function. , and These are the non-negative matching weights. Copy each real target... This forms an auxiliary matching target set, and bipartite graph matching is performed. For the matched i... Auxiliary candidates for replicating the target of the real target, ranked according to classification confidence level. and positioning quality Validity determination after matching; the classification confidence level The following embodiments will be described in detail.
[0030] when The candidate is retained at that time, where, This is the classification confidence threshold used for determining the validity of the match. These are the location quality thresholds used for determining the validity of a match, where both validity thresholds are between 0 and 1. This determination is only used to exclude invalid matching candidates; the composition ratio of candidates and the strength of auxiliary supervision are still continuously controlled by subsequent reliability status. The retained candidates collectively form a candidate group. The size of the candidate group is denoted as and satisfy .when When the training image does not contain a real target, skip the candidate reliability aggregation and auxiliary supervision calculation of the main detection branch; when the training image does not contain a real target, skip the candidate group construction and auxiliary supervision calculation of that image. Each auxiliary candidate in the algorithm is associated with the same real target. and main query Establish a clear correspondence. This primary-secondary correspondence is used for subsequent auxiliary supervised training, i.e., aggregating candidate groups to generate auxiliary supervised targets to supervise the primary query m(g), forcing the primary query to learn the beneficial information from the auxiliary candidates.
[0031] Background candidates that do not match the target for replication do not enter the candidate reliability aggregation and main detection branch auxiliary supervision, but only participate in background classification training; candidates that have been matched but have not passed the validity judgment after matching still participate in the original classification loss and localization loss of the auxiliary prediction branch, but do not enter the candidate reliability aggregation and main detection branch auxiliary supervision.
[0032] S3, for each auxiliary candidate in the candidate group, calculates its reliability status based on its classification confidence, positioning quality, prediction consistency with the corresponding main query, true target scale, and the importance of the airport functional area where the true target is located. The airport functional area factor is uniformly denoted as a target-level variable. Small target scale factors are uniformly denoted as target-level variables. Both are compensated solely through regional-scale conditional terms. Participation in compensation, this compensation item follows Increase and increase, and in When it approaches 0, it approaches 0; regional positioning penalty item With positioning quality It decreases and then increases.
[0033] Next, we will explain in detail each calculation item of the candidate group.
[0034] The airport functional area factor is calculated based on the overlap between the actual target bounding box and the airport functional area mask. Let... This represents the number of valid pixels covered by the actual target bounding box, and the coordinates of the nth pixel are ( , If the airport functional area factor is calculated as follows:
[0035]
[0036] In the formula, The preset region importance coefficient is determined by the category value of the airport functional area mask; when the category number of the region to which the nth pixel belongs is k, r0, r1, r2 or r3 corresponding to k is taken, where k∈{0,1,2,3}.
[0037] When the actual target spans two or more airport functional areas, the above calculation can form a continuous regional factor based on the proportion of each area within the target frame, avoiding abrupt changes in regional classification caused by relying solely on the target's center point. Alternatively, the airport functional area factor can be determined using the area where the bottom center point of the target frame is located, the category with the largest regional proportion, or a pre-generated continuous risk mask.
[0038] Calculate the small target scale factor based on the area of the real target relative to the image. :
[0039]
[0040] In the formula, For the input image, This is a scale adjustment parameter. When the actual target area decreases, The small target scale factor does not determine the candidate reliability on its own, but rather constitutes a conditional compensation term together with the airport functional area factor, classification confidence, and positioning quality.
[0041] For candidate groups The i-th auxiliary candidate in the list is assigned to the true target category. The predicted probability is defined as the classification confidence level. The intersection-union ratio (IUU) between the predicted bounding box and the ground truth bounding box is defined as the localization quality. :
[0042]
[0043] To reflect whether the auxiliary candidate and the main detection branch are consistent in their judgment of the same real target, the consistency of the main and auxiliary predictions is calculated based on the overlap of the bounding boxes of the auxiliary candidate and the main query and the similarity of their category distributions. :
[0044]
[0045] in, The category probability distribution output by the main query m(g); The predicted bounding box output by the main query m(g) is used as the main detection branch location result and classification result, respectively. The normalized main-auxiliary prediction consistency value is between 0 and 1; the more consistent the main and auxiliary branches are in judging the candidate location and category, the greater the main-auxiliary prediction consistency.
[0046] Therefore, by comprehensively considering classification confidence, positioning quality, consistency between primary and secondary predictions, small target scale, and airport functional area information, the reliability state of the i-th auxiliary candidate is calculated. :
[0047]
[0048] In the formula, For the Sigmoid function, For bias terms, each The parameters are non-negative weights. The weighting coefficients for the classification confidence term. To determine the weighting coefficients for the quality items, The weighting coefficients for the consistency term between primary and secondary predictions. For the regional-scale conditional compensation term, the weighting coefficients are... The weighting coefficients for the regional positioning penalty term. The regional-scale condition compensation term in the unified reliability formula varies with... Increase and increase, and in When it approaches 0, it approaches 0; the regional positioning penalty item follows. The value is reduced by increasing it, which is used to decrease the reliability of positioning offset candidates such as pavement markings, light reflections, or pavement textures. Due to the use of Sigmoid mapping, .
[0049] S4. Generate an adaptive threshold within the same candidate group based on the mean and dispersion of the reliability status, calculate the candidate soft gating value, and perform competitive weight redistribution on multiple auxiliary candidates within the candidate group according to the reliability status to obtain the relative weight within the group and the effective weight of the candidate group.
[0050] The reliability status serves as a common input for subsequent candidate gating, weight competition within candidate groups, and auxiliary supervision of the main detection branch. It does not construct independent candidate scores for different stages, thereby ensuring that the candidate screening results and the intensity of auxiliary supervision have a consistent control basis.
[0051] Among them, for the non-empty candidate groups involved in the calculation The number of candidates satisfies Calculate the mean and variance of the reliability within each group:
[0052]
[0053]
[0054] Discreteness is used to describe the reliability differences among multiple auxiliary candidates corresponding to the same real target. An adaptive threshold within each candidate group is generated based on the group's own reliability distribution, and a soft gating value is calculated for each auxiliary candidate.
[0055]
[0056] In the formula, This is the dispersion adjustment coefficient. For candidate soft-gated temperatures, the cutoff function is defined as follows: .when When increased, the soft gating value Monotonically increasing; when When decreasing, It decreases monotonically. This mapping is continuously differentiable, which reduces the risk of gradient interruption compared to the hard thresholding method.
[0057] To avoid situations where the reliability of all candidates within a candidate group is low, the normalized weights still ensure that the candidate group participates in auxiliary supervision with full strength. Therefore, the effective weights of the candidate groups are calculated based on the mean reliability of each candidate group. And based on candidate soft gating values Relative weights within the reliability state calculation group :
[0058]
[0059] In the formula, The effective threshold for candidate groups This is the candidate group's gate temperature. Because... and If the value is greater than 0, the denominator of the relative weights within a group is strictly greater than 0; the sum of the relative weights within each group is 1. It is only used to determine the proportion of candidates in the aggregated auxiliary targets within the group. This is only used to control the overall strength of the auxiliary supervision for this candidate group. The aforementioned candidate aggregation, stability activation, and gating constraints together constitute the auxiliary supervision for reliability control.
[0060] S5, using the redistributed weights to aggregate auxiliary candidates, obtains classification auxiliary targets and localization auxiliary targets for the main detection branch. The effective weights of the candidate groups are then used to uniformly control classification auxiliary supervision, localization auxiliary supervision, and query discrimination constraints for densely adjacent targets. This allows the effective weights of the candidate groups to uniformly control the classification auxiliary targets and localization auxiliary targets, thereby performing classification and localization supervision on the corresponding main query. Specifically, the exponential moving average of batch average reliability is calculated; when this value is continuous... When the number of training batches exceeds the reliability trigger threshold, the auxiliary supervision is enabled.
[0061] In this embodiment, auxiliary candidates are aggregated based on their relative weights within a group to obtain the classification auxiliary target corresponding to the true target g. and positioning auxiliary targets :
[0062]
[0063]
[0064] When aggregating and locating targets, all bounding boxes are represented using uniform normalized center coordinates and width and height. The aggregated classification targets, aggregated localization targets, and effective weights of candidate groups used for auxiliary supervision in the main detection branch all undergo stopping gradients; therefore, this auxiliary supervision does not backpropagate to the auxiliary prediction branch. (Intra-group relative weights) Decision No. The composition ratio of auxiliary candidates in the aggregated classification target and aggregated localization target; the effective weight of candidate groups. The strength of the overall aggregation objective's supervision over the main detection branch is determined. Auxiliary candidates, serving as auxiliary objectives during training, do not accept back-current gradients from the auxiliary supervision terms of the main detection branch.
[0065] Then, the effective weights of candidate groups are used to control the classification and localization supervision of the main query by the aggregation auxiliary target. Let... This indicates the number of non-empty candidate groups in the current training batch, and the non-empty candidate groups are renumbered. When calculating relative entropy, the classification auxiliary target is used as the target distribution of the stopping gradient, and the main query category distribution is used as the supervised distribution. The classification auxiliary loss, localization auxiliary loss, and their combination are calculated as follows; when... At that time, both the classification auxiliary loss and the localization auxiliary loss are defined as 0:
[0066]
[0067]
[0068]
[0069] in, This is a bounding box loss function used to measure the difference between the bounding box predicted by the main query and the localization assistance target. To supervise the classification auxiliary loss of the corresponding main query category probability distribution with the aggregation classification target, To aggregate the localization target and supervise the prediction bounding box of the corresponding main query, the localization auxiliary loss is used. The non-negative weighting coefficients for the classification auxiliary loss are... To locate the non-negative weighting coefficients of the auxiliary loss, The relative entropy between category distributions is represented by the bounding box loss, which can be L1 loss, IoU loss, or GIoU loss. The effective weights of candidate groups generated from the same reliability state and the relative weights within the groups simultaneously determine the composition and strength of the auxiliary targets, allowing the redistribution results within the candidate groups to directly enter the classification and localization learning of the main detection branch.
[0070] Second embodiment.
[0071] Building upon the first embodiment, to address the issue of densely packed personnel and vehicles on the tarmac and the potential for confusion between the query features of adjacent targets of the same type, this embodiment constructs a set of difficult adjacent target pairs during the training annotation process of the target detection network. Two real targets whose normalized center distance is less than a preset center distance threshold, or whose expanded ground truth bounding boxes (with unchanged centers and expanded width and height) have an intersection-union ratio greater than 0, belong to the same category or a preset easily confused category pair, and whose corresponding candidate groups are not empty, are added to this set of difficult adjacent target pairs. Furthermore, for each of these difficult adjacent target pairs, the product of the effective weights of the two candidate groups is used as the discrimination constraint weight. The main query discrimination constraint term is calculated based on the amount by which the cosine similarity of the two main query features corresponding to the two real targets exceeds a preset similarity interval.
[0072] In detail, the center distance between two real targets is defined as the Euclidean distance between their center points divided by the diagonal length of the input image, and the resulting normalized distance is between 0 and 1. The center distance threshold is calculated by multiplying the width and height of the bounding box by an expansion factor while keeping the center of the true bounding box unchanged. Obtain the expanded true bounding box. When the normalized center distance between two true targets is less than... If the intersection-union ratio (IU) of two extended ground truth boxes is greater than 0, and they belong to the same category or are pre-configured easily confused category pairs in the training annotations, and the candidate groups corresponding to the two ground truth objects are not empty, then add them to the set of hard neighbor object pairs; let Indicates the quantity of the target pair, the first Each target is denoted as .
[0073] For difficult adjacent target pairs, the effective weights of their respective candidate groups are used to control the main query's distinguishing constraints:
[0074]
[0075] In the formula, The query feature vector output by the main query m(g_k) corresponding to the true target g_k in the k-th hard neighboring target pair is... This is the query feature vector output by the main query m(h_k) corresponding to the real target h_k in the k-th hard neighboring target pair; the two main query features correspond to two different real targets. The range of values for the cosine similarity interval is given. The weights for the discrimination constraints on target pairs are... As the product increases, the strength of the distinguishing constraint increases accordingly; as the product approaches zero, the distinguishing constraint of the corresponding target pair approaches zero. At that time, Defined as 0.
[0076] Third embodiment.
[0077] Based on the first or second embodiment, in order to avoid prematurely passing supervision to the main detection branch when the auxiliary candidates are not yet stable in the early stage of training, the reliability states of all auxiliary candidates included in all non-empty candidate groups in the current training batch are averaged to obtain the batch average reliability, and the batch average reliability is subjected to an exponential moving average; when the exponential moving average is higher than the reliability trigger threshold for a preset number of consecutive training batches, auxiliary supervision of the main detection branch is enabled.
[0078] In some exemplary embodiments, let Indicates the first The total number of auxiliary candidates contained in all non-empty candidate groups in each training batch. When, calculate the batch average reliability of that batch and perform an exponential moving average; when At that time, the exponential moving average is not updated, and the consecutive satisfaction count for this batch is set to 0.
[0079]
[0080] In the formula, Let represent the reliability state of the i-th auxiliary candidate participating in the statistics in the t-th training batch. This is the exponential moving average of the batch average reliability obtained up to the (t-1)th training batch. Let be the arithmetic mean of the reliability states of all auxiliary candidate states that participated in the statistics in the t-th training batch. The coefficient of the exponential moving average. As a reliability trigger threshold, A positive integer. Initialize. .for Training batches, when season Otherwise ;when When Set to 0. Define indicator function. The value is 1 if the condition is met, and 0 otherwise. The auxiliary supervision activation flag is calculated according to the following formula and locked after activation using the flag from the previous batch:
[0081]
[0082] The main branch detection loss and the one-to-many detection loss of the auxiliary prediction branch together constitute the basic training objective. The overall joint optimization loss is:
[0083]
[0084] In the formula, The main branch detection loss includes the classification loss and localization loss of the main detection branch. The one-to-many detection loss for the auxiliary prediction branch includes the classification loss and localization loss of the auxiliary prediction branch itself. , and Non-negative loss weights; The auxiliary supervision enable flag defined in the foregoing embodiments. At that time, basic predictive capabilities are established solely from the detection losses of the main and auxiliary branches; Subsequently, the aggregation auxiliary target and the adjacent query differentiation constraint both begin to apply to the main detection branch.
[0085] For example, the regional importance coefficient is taken as , , , The typical scope of experience applicable to civil aviation airport inspection training tasks is as follows: ; ; ; ; ; , and All ranged from 0.1 to 0.5; ; and All ranged from 0.05 to 0.5; and All ranged from 0.4 to 0.8; ; ; ; ; ; , , , and All range from 0.5 to 2.0; and All range from 0.5 to 1.5; ; Of course, the values mentioned above are determined based on experimental results on the validation set, and are merely illustrative examples.
[0086] After training, the parameters of the backbone network, encoder, and main detection branch are saved. During the inference phase, one-to-many auxiliary prediction branches, candidate reliability aggregation, intra-group gating redistribution, and auxiliary supervision are disabled. Only the main detection branch outputs the target category, bounding box, and confidence score. Therefore, the unified reliability control mechanism during training does not increase the computational cost of auxiliary branches during deployment. In some optional examples, during the inference phase, the image of the UAV to be detected is input into the backbone network to extract multi-scale features, which are then encoded by a Transformer and fed into the main detection branch, directly outputting the target category, bounding box, and confidence score.
[0087] According to any of the above embodiments of this application, in the application of civil aviation airport inspection, a drone equipped with a visible light gimbal camera can collect images of the runway, taxiway and apron according to a preset route. The ground station or edge inference terminal loads the trained main detection branch model to detect workers, support vehicles, FOD, cones, lights and temporary construction facilities, and transmits the detection frame, target category and confidence level to the inspection terminal for manual review, inspection record or abnormal prompt.
[0088] Airport functional area masks can be manually annotated from training images, or generated based on image templates when the camera field of view and inspection routes are relatively fixed. It is not required to project each candidate onto the airport electronic map in real time. The area mask is mainly used to form the airport scene-related reliability state during the training phase. Even if the area mask is not input during the inference phase, the main detection branch can still complete the target detection independently.
[0089] For runways and runway protection areas, airport functional area factors With small target scale factor Joint participation in regional-scale condition compensation; the compensation intensity is continuously controlled by both classification confidence and positioning quality, and the compensation term approaches zero when the product of the two approaches zero. Regional positioning penalties increase as positioning quality decreases, used to suppress positioning offset candidates caused by pavement markings and reflections. For taxiway areas, personnel, vehicle, and debris candidates participate in reliability aggregation based on their corresponding airport functional area factors; for apron areas, one-to-many candidates for the same personnel or vehicle form a single aggregated auxiliary target through intra-group competition, and densely adjacent targets reduce mutual confusion through query differentiation constraints; for ordinary areas, regional factors are calculated according to coefficients. Sure.
[0090] The UAV dense small target detection training system based on reliability gating according to an embodiment of this application includes at least one processor, a memory, and program instructions stored in the memory and executable by the processor; the program instructions are used to implement the UAV dense small target detection training method based on reliability gating.
[0091] When the processor executes the program instructions, it implements the image and region mask acquisition module, the dual-branch target prediction module, the one-to-one and one-to-many matching module, the candidate group construction module, the reliability state calculation module, the adaptive gating and intra-group weight redistribution module, the aggregation auxiliary target generation module, the unified gating auxiliary supervision module, the reliability stability triggering module, the joint optimization module, and the main detection branch model output module.
[0092] The image and region mask acquisition module outputs spatially aligned training images, bounding boxes, and region masks; the dual-branch object prediction module outputs the main query prediction and auxiliary candidate predictions; and the matching and candidate group construction module outputs the main query correspondence. And candidate groups that pass the validity determination after matching. The reliability status calculation module outputs the reliability status. The gating and redistribution module outputs soft gating values. Relative weights within a group and candidate group effective weights .in, Used to generate aggregate auxiliary targets, The reliability stabilization trigger module is used to control the auxiliary loss intensity of candidate groups and outputs an enable flag based on the exponential moving average of batch reliability. The joint optimization module in When training is complete, the main detection branch model output module saves the main detection branch parameters and does not call the training-period auxiliary prediction, reliability aggregation, and stability triggering modules during deployment and inference.
[0093] Without changing the above continuous control relationship, the airport functional areas can be expanded to include more categories such as the edge of the parking stand, fire lanes, and training vehicle passage areas; the importance of the area can be replaced by the discrete coefficients with continuous risk values; other equivalent loss functions can be used for classification and localization auxiliary losses; and the main detection branch and auxiliary prediction branch can also be replaced by other target detection structures with one-to-one and one-to-many matching relationships.
[0094] The above description is merely a preferred embodiment of the present invention. It should be understood that the present invention is not limited to the forms disclosed herein and should not be construed as excluding other embodiments. It can be used in various other combinations, modifications, and environments, and can be altered within the scope of the concept described herein through the above teachings or related technologies or knowledge. Modifications and variations made by those skilled in the art that do not depart from the spirit and scope of the present invention should be within the protection scope of the appended claims.
Claims
1. A training method for detecting dense small targets using unmanned aerial vehicles (UAVs) based on reliability gating, characterized in that, Includes the following steps: Acquire training images of civil aviation airport inspections, target annotations, and multi-category airport functional area masks corresponding to the image space, and perform synchronous transformations on the image, target bounding boxes, and area masks when performing spatial data augmentation; The training image is input into the object detection network; wherein, the main detection branch of the object detection network determines a unique main query for each real target through one-to-one matching, and the auxiliary prediction branch of the object detection network determines auxiliary candidates for the same real target through one-to-many matching; For each auxiliary candidate in the candidate group, calculate the reliability status within the same candidate group; An adaptive threshold is generated within the same candidate group based on the mean and dispersion of the reliability status. Candidate soft gating values are calculated. Based on the candidate soft gating values and the reliability status within the candidate group, multiple auxiliary candidates within the candidate group are weighted and reallocated. The auxiliary candidates are aggregated using the redistributed weights to obtain classification auxiliary targets and localization auxiliary targets for the main detection branch. The classification auxiliary targets and localization auxiliary targets are then uniformly controlled by the effective weights of the candidate groups to perform classification supervision and localization supervision on the corresponding main query.
2. The training method for dense small target detection in UAVs based on reliability gating according to claim 1, characterized in that, The target detection network includes a backbone network, a multi-scale feature extraction structure, a Transformer encoder, a main detection branch, and an auxiliary prediction branch; The main detection branch is used to generate the target category, bounding box, and query features for the final deployment model; The auxiliary prediction branch only works during the training phase, for cases containing The number of training images of real targets is set to be no less than [number] auxiliary queries. ,in It is an integer not less than 1; For the g-th real target, the main detection branch obtains the unique corresponding main query m(g) through bipartite graph matching; The auxiliary prediction branch calculates the matching cost between the i-th auxiliary candidate and the g-th real target based on the classification cost, bounding box distance cost, and intersection-union ratio cost. The calculation formula is as follows: In the formula, Indicates the first The class probability distribution of the auxiliary candidate outputs. This indicates its predicted bounding box. and They represent the first The category labels and ground truth bounding boxes of each real target; Represents the classification cost function. , and These are the non-negative matching weights; Copy each real target This forms an auxiliary matching target set, and bipartite graph matching is performed. For the matched number Auxiliary candidates for replicating the target of the real target, ranked according to classification confidence level. and positioning quality Validity determination after matching.
3. The training method for dense small target detection in UAVs based on reliability gating according to claim 2, characterized in that, For candidate groups The i-th auxiliary candidate in the list is assigned to the true target category. The predicted probability is defined as the classification confidence level. The intersection-union ratio (IUU) between the predicted bounding box and the ground truth bounding box is defined as the localization quality. : 。 4. The training method for dense small target detection in UAVs based on reliability gating according to claim 3, characterized in that, According to classification confidence level and positioning quality The validity determination after matching includes: when The candidate is retained at that time, where, This is the classification confidence threshold used for determining the validity of the match. This is the positioning quality threshold used for determining the validity of a match. Retain candidates to form a candidate group The size of the candidate group is denoted as and satisfy ; when When the training image does not contain a real target, skip the candidate reliability aggregation and auxiliary supervision calculation of the main detection branch; when the training image does not contain a real target, skip the candidate group construction and auxiliary supervision calculation of the image. Candidate Group Each auxiliary candidate in the algorithm is associated with the same real target. and main query Establish corresponding relationships.
5. The UAV dense small target detection training method based on reliability gating according to claim 4, characterized in that, For each auxiliary candidate in the candidate group, the reliability status within the same candidate group is calculated based on its classification confidence, positioning quality, prediction consistency with the corresponding main query, true target scale, and the importance of the functional area of the airport where the true target is located. Calculate the reliability state of the i-th auxiliary candidate. : In the formula, For the Sigmoid function, For bias terms, each The parameters are non-negative weights. The weighting coefficients for the classification confidence term. To determine the weighting coefficients for the quality items, The weighting coefficients for the consistency term between primary and secondary predictions. For the regional-scale conditional compensation term, the weighting coefficients are... The weighting coefficient for the regional location penalty term. Airport functional area factor, For small target scale factors, Consistency between primary and secondary forecasts.
6. The training method for dense small target detection in UAVs based on reliability gating according to claim 5, characterized in that, The airport functional area factor is calculated as follows: In the formula, Indicates the number of valid pixels covered by the actual target bounding box, ( , () represents the coordinates of the nth pixel. The preset region importance coefficient is determined by the category value of the airport functional area mask; when the category number of the region to which the nth pixel belongs is k, take r0, r1, r2 or r3 corresponding to k, where k∈{0,1,2,3}; Small target scale factor : In the formula, For the input image, This is the scaling adjustment parameter.
7. The training method for dense small target detection in UAVs based on reliability gating according to claim 6, characterized in that, An adaptive threshold is generated within the same candidate group based on the mean and dispersion of the reliability status. Candidate soft-gating values are calculated. Based on these candidate soft-gating values and the reliability status within the candidate group, a weight reallocation of multiple auxiliary candidates within the candidate group is performed, including: For non-empty candidate groups involved in the calculation The number of candidates satisfies And calculate the mean reliability within each group. and variance ; An adaptive threshold within each candidate group is generated based on its own reliability distribution, and a soft gating value is calculated for each auxiliary candidate: In the formula, This is the dispersion adjustment coefficient. For candidate soft-gated temperatures, the cutoff function is defined as follows: ;when When increased, the soft gating value Monotonically increasing; when When decreasing, Monotonically decreasing; Calculate the effective weight of the candidate group based on the mean reliability of the candidate group. And based on candidate soft gating values Relative weights within the reliability state calculation group : In the formula, The effective threshold for candidate groups, The candidate group's gate temperature.
8. The training method for dense small target detection in UAVs based on reliability gating according to claim 7, characterized in that, The auxiliary candidates are aggregated using the reassigned weights to obtain classification auxiliary targets and localization auxiliary targets for the main detection branch. The effective weights of the candidate groups are then used to uniformly control the classification auxiliary targets and localization auxiliary targets to perform classification supervision and localization supervision on the corresponding main query, including: Auxiliary candidates are aggregated based on their relative weights within the group to obtain the classification auxiliary target corresponding to the true target g. and positioning auxiliary targets : In the formula, Indicates the first The class probability distribution of the auxiliary candidate outputs. This indicates its predicted bounding box. For candidate group size, Relative weights within the group make This indicates the number of non-empty candidate groups in the current training batch, and the non-empty candidate groups are renumbered. ; when At that time, both the classification auxiliary loss and the localization auxiliary loss are defined as 0: in, This is a bounding box loss function used to measure the difference between the bounding box predicted by the main query and the localization assistance target. To classify auxiliary losses, To locate auxiliary losses, The non-negative weighting coefficients for the classification auxiliary loss are... To locate the non-negative weighting coefficients of the auxiliary loss, Represents the relative entropy between category distributions. The category probability distribution output by the main query m(g); The predicted bounding box output by the main query m(g).
9. The training method for dense small target detection in UAVs based on reliability gating according to claim 1, characterized in that, The training process for the object detection network also includes: Construct a set of difficult adjacent target pairs by adding two real targets whose normalized center distance is less than a preset center distance threshold, or whose center remains unchanged but whose width and height are expanded respectively, have an intersection-union ratio greater than 0, belong to the same category or a preset easily confused category, and whose corresponding candidate groups are not empty, to the set of difficult adjacent target pairs. For each difficult adjacent target pair, the product of the effective weights of the two candidate groups is used as the distinguishing constraint weight. The main query distinguishing constraint term is calculated based on the amount by which the cosine similarity of the two main query features corresponding to the two real targets exceeds the preset similarity interval. The reliability status of the auxiliary candidates in the current training batch is averaged to obtain the batch average reliability, and an exponential moving average is performed on the batch average reliability. When the exponential moving average is higher than the reliability trigger threshold for a preset number of consecutive training batches, auxiliary supervision of the main detection branch is enabled.
10. A training system for detecting dense small targets in unmanned aerial vehicles (UAVs) based on reliability gating, comprising a processor, a memory, and program instructions stored in the memory and executable by the processor, characterized in that, The program instructions are used to implement the UAV dense small target detection training method based on reliability gating as described in any one of claims 1-9.
