A complex scene target detection method based on model dynamic combination

Abstract

A complex scene target detection method based on a model dynamic joint mechanism, which uses an RT-DETR model for preliminary detection, retains more potential targets through dynamic threshold adjustment, and projects the generated detection frame to the feature space of an improved YOLOv12 model through double-modal feature mapping; the improved YOLOv12 model integrates an SEAM attention module and a repulsion loss function to enhance the feature representation and positioning compactness of the occluded target. Then, the model joint mechanism is used to process the preliminary results of the two models: through difficult example mining and online learning, the missed detection targets are supplemented and the RT-DETR model is optimized; through dynamic weight distribution and hierarchical fusion decision tree based on scene complexity, the detection results of the two models are intelligently fused. Finally, an improved non-maximum suppression algorithm is applied for post-processing to reduce false deletion. The present application effectively solves the problems of target missed detection, false detection and inaccurate positioning in complex scenes, and significantly improves the recall rate and accuracy of detection.

Description

Technical Field

[0001] This application relates to the field of target detection technology, specifically to a method for target detection in complex scenes based on dynamic model fusion. Background Technology

[0002] With breakthroughs in computer vision technology, object detection, as a core task, has been deeply applied in fields such as autonomous driving, intelligent monitoring, and medical imaging. In the field of object detection in complex scenes, the RT-DETR (Real-Time DETR) algorithm and the YOLO series of algorithms constitute the current mainstream technical framework. Although these algorithms have made progress in specific scenarios, they still face some technical bottlenecks in object detection in complex scenes. Lack of dynamic adaptability: RT-DETR uses a fixed confidence threshold for box selection. In scenarios where the target scale changes drastically or the occlusion rate exceeds 60%, the fixed threshold mechanism leads to a 37%-52% increase in the false negative rate (supported by experimental data).

[0003] Limitations of feature representation: Although YOLOv12 enhances feature extraction capabilities by improving the backbone network, the cross-scale fusion of the feature pyramid is insufficient when dealing with dense occlusion (overlap area > 40%) and small targets (pixel area < 32×32), resulting in a 18%-23% decrease in mAP (mean accuracy).

[0004] Overlapping box handling defects: When the target density in the detection scene is greater than 0.8 targets / m², traditional loss functions (such as GIoU) ​​cannot effectively constrain the distribution of predicted boxes, resulting in an increase in the number of overlapping boxes by 2.1-3.4 times.

[0005] Limitations of the NMS algorithm: When processing dense bounding boxes, the false deletion rate of the standard NMS increases exponentially with the confidence level. When the number of targets in the scene exceeds 50, the false deletion rate can reach 12%-19%, significantly affecting the completeness of the detection.

[0006] Therefore, in target detection in complex scenarios, a trade-off between speed and accuracy is often necessary. While the RT-DETR algorithm has a high recall rate, its accuracy is insufficient; and while the YOLO algorithm has a high accuracy, it still suffers from false negatives and missed detections in complex scenarios. Summary of the Invention

[0007] This application provides a method for object detection in complex scenes based on dynamic model fusion, which can solve the problem that existing technologies often require a trade-off between speed and accuracy in object detection in complex scenes. Although the RT-DETR algorithm has a high recall rate, its accuracy is insufficient; while the YOLO algorithm has a high accuracy, it still suffers from missed detections and false detections in complex scenes.

[0008] In a first aspect, embodiments of this application provide a method for target detection in complex scenes based on dynamic model fusion, comprising: The RT-DETR model is used to perform preliminary detection on the input image to generate a first detection result set with several detection boxes having location, category, and confidence. The first detection result set is mapped and aligned to the feature map space of the YOLOv12 model, and then the second detection result set is generated using the YOLOv12 model. The fusion weights are adjusted based on the scene complexity of the input image; then, the first detection result set and the second detection result set are fused according to the hierarchical fusion strategy and the adjusted fusion weights to output the final detection result.

[0009] Preferably, the first detection result set and the second detection result set are fused according to the hierarchical fusion strategy and the adjusted fusion weights, specifically including the following steps: The detection boxes pointing to the same target in the first and second detection result sets are merged. The specific steps for fusing detection boxes pointing to the same target are as follows: If the first joint confidence of the two detection boxes is greater than the preset confidence value, both will be retained after verification. If the first joint confidence of the two detection boxes is not greater than the preset confidence value, the cross-union ratio (CUP) of the two detection boxes is calculated; when the CUP is lower than the first threshold, the detection boxes of the second detection result set are adopted; when the CUP is between the first threshold and the second threshold, the fusion weight is used as a coefficient to perform weighted fusion of the two detection boxes.

[0010] Preferably, the specific steps for verifying the two detection boxes are as follows: Perform the following three verification strategies and obtain three verification results; Determine whether the intersection-union ratio of the detection box and the detection box predicted based on the historical trajectory is greater than a third threshold; A lightweight classifier is used to perform secondary classification on the image region corresponding to the detection box, and it is determined whether the classification confidence is higher than the first confidence value. Evaluate whether the region concentration of the detection box in the model's self-attention heatmap is higher than a preset concentration. If all three verification results meet their corresponding conditions, the verification is successful; otherwise, it fails.

[0011] Preferably, the following steps are included before fusing the first detection result set and the second detection result set: The detection boxes in the first detection result set and the second detection result set are compared to identify the detection boxes that were missed by the RT-DETR model, and the missed detection boxes are marked as hard cases; The YOLOv12 model is used to detect hard cases, and the detection results are then added to the first detection result set to update the first detection result set; The labeled difficult examples form the training dataset, which is then fed back into the RT-DETR model for incremental learning to update the RT-DETR model parameters.

[0012] Preferably, adjusting the fusion weights based on the scene complexity of the input image specifically includes the following steps: During the current evaluation period, the preset test dataset is divided into multiple test subsets with different complexity levels based on the scene complexity of the input image; All test subsets were run in both the RT-DETR model and the YOLOv12 model to obtain the test results for each test subset in both models. The F1 score at different complexity levels is calculated based on the test results of each test subset in the RT-DETR model and the YOLOv12 model; Obtain the weights of the RT-DETR model from the previous evaluation period, and then calculate the mean of the F1 score for the current evaluation period; based on the weights of the RT-DETR model from the previous evaluation period and the mean, and combined with the double exponential smoothing algorithm formula, calculate the weights of the new RT-DETR model. The formula for the double exponential smoothing algorithm is as follows: ; Where γ is the trend sensitivity coefficient; The weights of the newly calculated RT-DETR model for the current evaluation period are: α is the current performance weight, used to control the degree of influence of the RT-DETR performance on the weight update in the current evaluation period; The mean F1 score of the RT-DETR model for the current evaluation period; The weights of the RT-DETR model in the previous evaluation period; The mean F1 score of the RT-DETR model in the previous evaluation period; The weights of the new YOLOv12 model are obtained by subtracting the weights of the new RT-DETR model from 1.

[0013] Preferably, the mean of the F1 score for the current evaluation period is calculated. The calculation formula is: Where M represents the scene complexity level, Scene weight coefficient ( ); The calculation of the F1 score at a complexity level includes the following steps: obtaining the recall of the RT-DETR model test results and the precision of the YOLOv12 model at the current complexity level; calculating the F1 score based on the precision and recall, combined with the F1 calculation formula; the F1 calculation formula is: F1=2×(Precision×Recall) / (Precision+Recall); where Precision is the precision and Recall is the recall.

[0014] Preferably, the input image is initially detected using the RT-DETR model, specifically including the following steps: Full-image feature extraction is performed on the input image, and several detection boxes are generated using the formula for calculating the number of detection boxes. The position and category of each detection box are then obtained. The formula for calculating the number of detection boxes is as follows: Where E×W is the input resolution, S is the sampling step size, σ is the scene complexity adjustment coefficient, and N is the number of detection boxes; The entropy value of the input image is calculated in real time; based on the baseline confidence threshold, the maximum and minimum values ​​of historical entropy values, and the entropy value, a real-time confidence threshold is calculated; the formula for calculating the real-time confidence threshold is: ;in As the baseline threshold, To adjust the coefficient, and These are the maximum and minimum historical entropy values, respectively. This is the real-time confidence threshold; The detection boxes of the RT-DETR model are filtered using a real-time confidence threshold to obtain a set of detection boxes; For each detection box in the set of detection boxes, a second joint confidence score is calculated. Then, a second filtering is performed using the second joint confidence score to obtain a first detection result set of several detection boxes with location, category, and confidence score. The formula for calculating the second joint confidence score is: ;in For classification confidence, IoU is the intersection-union ratio of the predicted bounding box and its nearest ground truth bounding box. This is the balance coefficient.

[0015] Preferably, the YOLOv12 model is an improved YOLOv12 model, in which a SEAM module is integrated after the Neck layer and before the Head layer; the SEAM module is used to perform weighted fusion of the output feature map of the Neck layer and output it to the Head layer by the parallel working separate attention submodule and enhanced attention submodule. The separate attention submodule is used to weight the input feature map in the spatial dimension to highlight the salient region of the occluded object in space; the enhanced attention submodule is used to weight the input feature map in the channel dimension to enhance the feature response of the occluded object in the channel. The improved YOLOv12 model combines the repulsion loss function and the original YOLOv12 loss function during the training phase, and optimizes its model parameters through the backpropagation algorithm. The repulsion loss function is used to penalize the overlap between the predicted bounding box and the surrounding ground truth bounding boxes. for:

[0016] in, The number of prediction boxes, and These are the i-th and j-th prediction boxes, respectively. For prediction boxes and The degree of overlap between them This is the smoothed L1 loss function.

[0017] Preferably, the improved YOLOv12 model is used in the inference phase to map the first detection result set to the information in the feature graph space of the improved YOLOv12 model for inference, so as to obtain the preliminary detection box; The confidence of overlapping detection boxes is attenuated using a non-maximum suppression algorithm with confidence attenuation factor to obtain a second detection result set.

[0018] Preferably, mapping and aligning the first detection result set to the feature map space of the YOLOv12 model specifically includes: Obtain the sampling step size of the RT-DETR model and the sampling step size of the YOLOv12 model to calculate the dynamic scale compensation factor; The dynamic scale compensation factor is used to scale each detection box in the first detection result set to correct the mapping deviation caused by the difference in model scale. The first detection result set after correction is projected from the feature space of the RT-DETR model to the feature map space of the YOLOv12 model to complete the mapping alignment; Among them, the dynamic scale compensation factor is The formula for calculating the dynamic scale compensation factor is as follows: δ is the scale difference sensitivity coefficient. and These are the sampling step sizes for the RT-DETR model and the YOLOv12 model, respectively. This is the preset maximum sampling step size.

[0019] The beneficial effects of the technical solutions provided in this application include: The RT-DETR model, with its global attention mechanism, performs coarse detection with high recall, ensuring that no potential targets are missed, and it excels at detecting difficult targets such as blurry or small-sized objects. Subsequently, the detection boxes generated by RT-DETR are mapped and aligned to the feature space of the YOLOv12 model using a dynamic scale compensation factor β through a feature transfer module, guiding YOLOv12 to perform fine-tuning detection only on high-value candidate regions. The improved YOLOv12 model integrates a SEAM attention module after its Neck layer, which enhances the feature representation ability of occluded and small targets through spatial and channel attention weighted fusion, thereby outputting more compact and accurate detection boxes. This collaborative mechanism of RT-DETR proposal and YOLOv12 refinement, through bimodal feature mapping and dynamic alignment, successfully combines the high recall of the former with the high precision of the latter, resulting in a significant improvement in average accuracy compared to a single model.

[0020] The system calculates image entropy in real time through an input image scene complexity quantification module and dynamically adjusts the confidence threshold of RT-DETR based on this entropy value to retain more potential targets. Simultaneously, it divides the test set into subsets of different levels according to scene complexity, evaluates the F1 scores of both models separately, and dynamically updates the fusion weights of the two models using a double exponential smoothing algorithm. This mechanism can automatically perceive the complexity of the current scene; for example, it increases the weight of YOLOv12 (because its SEAM module enhances occlusion features) when occlusion is severe, and relies on the results of RT-DETR when high recall is required. This fusion strategy based on scene complexity quantification and dynamic weight allocation gives the system context awareness and adaptive decision-making capabilities, significantly improving its detection robustness in varied and complex environments.

[0021] When fusing detections using a hierarchical fusion strategy, the system first calculates the joint confidence of the two bounding boxes. If the confidence exceeds a preset threshold, triple verification (spatial, semantic, and attention verification) is performed. If the confidence does not reach the threshold, the system chooses to adopt the YOLOv12 result or perform weighted fusion based on the intersection-over-union (IoU). This strategy combines hard example mining and online learning mechanisms, enabling real-time supplementation of missed targets in RT-DETR and feedback for training. This allows for the intelligent removal of inconsistent predictions while retaining valid detections. This hierarchical, rule-driven fusion decision effectively reduces false positives, false negatives, and overlapping localization issues in complex scenarios. Attached Figure Description

[0022] Figure 1 This is a flowchart illustrating the complex scene target detection method based on model dynamic fusion proposed in this application; Figure 2 This is a flowchart illustrating the framework of the SEAM module in this application. Detailed Implementation

[0023] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0024] First, some of the technical terms used in this application will be explained to help those skilled in the art understand this application.

[0025] RT-DETR algorithm: As an end-to-end detector based on the Transformer architecture, it directly regresses the target bounding box through an integrated encoder-decoder structure, abandoning the traditional non-maximum suppression (NMS) post-processing process. This design gives it a significant advantage in scenarios with stringent real-time requirements, especially in terms of detection speed and recall.

[0026] YOLO series algorithms: As a representative of the single-stage detection paradigm, YOLO achieves a balance between speed and accuracy through a gridded prediction mechanism. The latest iteration, YOLOv12, continues to optimize the feature extraction network and classification head architecture, further enhancing its detection performance in general scenarios.

[0027] Image entropy: It measures the complexity of texture and the amount of information in an image.

[0028] Predicted bounding box: The RT-DETR algorithm assumes that there is a bounding box for the target.

[0029] Ground truth bounding boxes: Manually labeled bounding boxes representing the actual locations of the target objects in the dataset. For a given predicted bounding box, the system searches for the ground truth bounding box with the highest overlap (i.e., the largest IoU) among all ground truth bounding boxes. This "nearest neighbor" does not refer to physical proximity, but rather to the ground truth label that is closest and most closely matched in terms of box overlap. To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0030] While the aforementioned algorithms have made progress in specific scenarios, they still face some technical bottlenecks in object detection in complex scenes: Lack of dynamic adaptability: RT-DETR uses a fixed confidence threshold for box selection. In scenarios where the target scale changes drastically or the occlusion rate exceeds 60%, the fixed threshold mechanism leads to a 37%-52% increase in the false negative rate (supported by experimental data).

[0031] Limitations of feature representation: Although YOLOv12 enhances feature extraction capabilities by improving the backbone network, the cross-scale fusion of the feature pyramid is insufficient when dealing with dense occlusion (overlap area > 40%) and small targets (pixel area < 32×32), resulting in a 18%-23% decrease in mAP (mean accuracy).

[0032] Overlapping box handling defects: When the target density in the detection scene is greater than 0.8 targets / m², traditional loss functions (such as GIoU) ​​cannot effectively constrain the distribution of predicted boxes, resulting in an increase in the number of overlapping boxes by 2.1-3.4 times.

[0033] Limitations of the NMS algorithm: When processing dense bounding boxes, the false deletion rate of the standard NMS increases exponentially with the confidence level. When the number of targets in the scene exceeds 50, the false deletion rate can reach 12%-19%, significantly affecting the completeness of the detection.

[0034] In object detection in complex scenes, existing technologies often need to strike a trade-off between speed and accuracy. While the RT-DETR algorithm boasts high recall, its accuracy is insufficient; and although the YOLO algorithm has high accuracy, it still suffers from false negatives and missed detections in complex scenes. Therefore, how to combine the advantages of both algorithms to improve the performance of multi-object detection in complex scenes has become a hot research topic.

[0035] In a first aspect, embodiments of this application provide a complex scene target detection system based on dynamic model fusion, which includes an RT-DETR model module, a feature transfer module, a YOLOv12 model module, a fusion module, and an input image scene complexity quantification module. The RT-DETR model module is used to perform preliminary detection on the input image using the RT-DETR model to generate a first set of detection results with several detection boxes having location, category, and confidence. The feature transfer module is used to map and align the first detection result set to the feature map space of the YOLOv12 model; The YOLOv12 model module is used to generate a second set of detection results using the YOLOv12 model; The input image scene complexity quantification module is used to quantify scene complexity. The fusion module is used to adjust the fusion weights based on the scene complexity of the input image, and then fuse the first detection result set and the second detection result set according to the hierarchical fusion strategy and the adjusted fusion weights to output the final detection result.

[0036] The RT-DETR model in the RT-DETR model module has the following design: In complex scenes with significant differences in target scale and severe occlusion, traditional fixed thresholds are prone to causing missed detections or false alarms. This application quantifies scene complexity by calculating the entropy value of the input image. Assess the complexity of the scenario. This represents the pixel grayscale probability. The higher the entropy value, the more complex the scene.

[0037] The input image scene complexity quantification module calculates the entropy value of the input image in real time; based on the baseline confidence threshold, the maximum and minimum values ​​of historical entropy values, and the entropy value, it calculates the real-time confidence threshold; the formula for calculating the real-time confidence threshold is as follows; ; in As the baseline confidence threshold, To adjust the coefficient, and These are the extreme values ​​of entropy in historical data. This is the real-time confidence threshold; the more complex the scenario, the lower the threshold, in order to retain more potential targets.

[0038] The real-time confidence threshold is used for the initial detection results of RT-DETR, based on the adjusted threshold. Perform secondary filtering, retaining only those with a confidence level higher than [value missing]. The detection box.

[0039] To further improve detection reliability, the RT-DETR model introduces a second joint confidence score based on the classification score and the IoU:

[0040] in For classification confidence, IoU is the intersection-union ratio of the predicted bounding box and its nearest ground truth bounding box. The optimal value for the balancing coefficients is determined through grid search. The second joint confidence score considers both classification accuracy and localization precision to reduce false detections.

[0041] The following design addresses the shortcomings of YOLOv12 in occluded target detection: The YOLOv12 model is an improved YOLOv12 model. The improved YOLOv12 model integrates a SEAM module after the Neck layer and before the Head layer. The SEAM module is used to perform weighted fusion of the output feature map of the Neck layer through the parallel working separate attention submodule and enhanced attention submodule, and then output it to the Head layer. After processing by the SEAM module, the output feature map is input into the YOLOv12 Head layer for subsequent classification and regression tasks.

[0042] The separate attention submodule employs a spatial attention mechanism to weight the input feature map spatially, highlighting the salient regions of occluded objects. Specifically, it generates a spatial attention weight map using convolutional layers and pooling layers, and then multiplies it element-wise with the input feature map to obtain the separate attention feature map.

[0043] The enhanced attention submodule employs a channel attention mechanism to weight the input feature map along the channel dimension, thereby enhancing the feature responses of occluded objects in each channel. Specifically, operations such as global average pooling and fully connected layers can be used to generate a channel attention weight map, which is then multiplied element-wise with the input feature map to obtain the enhanced attention feature map.

[0044] The fusion strategy for the separate attention submodule and the enhanced attention submodule is as follows: the separate attention feature map and the enhanced attention feature map are fused together, which can be done by addition, concatenation, or other fusion methods to obtain the final output feature map. The fused feature map contains both the spatially significant region information of the occluded object and enhances its feature response in the channel, thereby improving the model's ability to detect occluded objects.

[0045] The improved YOLOv12 model combines the repulsion loss function and the original YOLOv12 loss function during the training phase, and optimizes its model parameters through backpropagation. The aim is to reduce the overlap between predicted and ground truth bounding boxes, guiding the model to generate more compact and accurate predicted boxes. Specifically, the training phase uses a labeled dataset containing occluded targets. The loss function consists of the original YOLOv12 loss function (such as classification loss and localization loss) and the introduced repulsion loss function. Through backpropagation, the model parameters are optimized, enabling the model to learn more robust feature representations.

[0046] The repulsion loss function is used to penalize the overlap between the predicted bounding box and the surrounding ground truth bounding boxes. The repulsion loss function is:

[0047] in, The number of prediction boxes, and These are the i-th and j-th prediction boxes, respectively. For prediction boxes and The degree of overlap between them This is the smoothed L1 loss function.

[0048] The improved YOLOv12 model is used in the inference phase to map the first detection result set to information in the feature graph space of the improved YOLOv12 model for inference, so as to obtain the preliminary detection box. The confidence of overlapping detection boxes is attenuated using a non-maximum suppression algorithm with confidence attenuation factor to obtain a second detection result set.

[0049] Among them, the modified Non-Maximum Suppression (NMS) algorithm introduces a confidence decay factor:

[0050] Where μ is the attenuation coefficient (μ=2.0), which naturally attenuates the confidence of detection boxes with high overlap, effectively reducing the false deletion rate.

[0051] The feature transfer module projects the detection boxes generated by RT-DETR onto the YOLOv12 feature map space through a bimodal feature mapping module. A dynamic scale compensation factor β is introduced, and the following mapping relationship ensures feature alignment accuracy:

[0052] Where δ is the scale difference sensitivity coefficient (0.1≤δ≤0.3). and These are the sampling step sizes for the RT-DETR model and the YOLOv12 model, respectively. The maximum sampling step size is preset. This design ensures that the feature mapping error does not exceed 0.5 pixels, guaranteeing the alignment accuracy of subsequent detection boxes.

[0053] By using feature propagation, YOLOv12 can extract and compute features only on the detection boxes generated by RT-DETR (which usually only occupy a small part of the entire image area), thus avoiding repeated and time-consuming computations on invalid regions such as the background of the entire image.

[0054] This allows the powerful, improved YOLOv12 model to be integrated without significantly increasing the overall system latency, ensuring the real-time performance of the joint model and directly enhancing the detection capabilities for small and occluded targets. The candidate boxes provided by RT-DETR indicate the areas that YOLOv12 needs to focus on.

[0055] YOLOv12 performs a "magnifying glass" examination of these designated regions on its high-resolution feature maps, which can activate the SEAM attention mechanism to enhance the response to weak, occluded features in both space and channels.

[0056] This is equivalent to giving YOLOv12 a "hint" so that it can discover small targets that are easily overlooked or confused at the full-scale image, as well as severely occluded targets. Many targets that RT-DETR "feel but are uncertain about" are finally confirmed through YOLOv12's refinement of the detection boxes.

[0057] The "feature transfer optimization" sub-step precisely maps the RT-DETR proposal to the YOLOv12 feature space, ensuring that the detection boxes output by the two models are based on the same reference system in coordinates. This is a prerequisite for subsequent operations such as IOU calculation and box fusion.

[0058] Secondly, a method for object detection in complex scenes based on dynamic model fusion specifically includes the following steps: Step 100: Use the RT-DETR model to perform preliminary detection on the input image to generate a first detection result set with several detection boxes having location, category, and confidence. Step 200: Map and align the first detection result set to the feature map space of the YOLOv12 model, and then use the YOLOv12 model to generate the second detection result set; Step 300: Adjust the fusion weights based on the scene complexity of the input image; then, according to the hierarchical fusion strategy and the adjusted fusion weights, fuse the first detection result set and the second detection result set to output the final detection result.

[0059] The RT-DETR model, with its global attention mechanism, performs coarse detection with high recall, ensuring that no potential targets are missed, and it excels at detecting difficult targets such as blurry or small-sized objects. Subsequently, the detection boxes generated by RT-DETR are mapped and aligned to the feature space of the YOLOv12 model using a dynamic scale compensation factor β through a feature transfer module, guiding YOLOv12 to perform fine-tuning detection only on high-value candidate regions. The improved YOLOv12 model integrates a SEAM attention module after its Neck layer, which enhances the feature representation ability of occluded and small targets through spatial and channel attention weighted fusion, thereby outputting more compact and accurate detection boxes. This collaborative mechanism of RT-DETR proposal and YOLOv12 refinement, through bimodal feature mapping and dynamic alignment, successfully combines the high recall of the former with the high precision of the latter, resulting in a significant improvement in average accuracy compared to a single model.

[0060] The system calculates image entropy in real time through an input image scene complexity quantification module and dynamically adjusts the confidence threshold of RT-DETR based on this entropy value to retain more potential targets. Simultaneously, it divides the test set into subsets of different levels according to scene complexity, evaluates the F1 scores of both models separately, and dynamically updates the fusion weights of the two models using a double exponential smoothing algorithm. This mechanism can automatically perceive the complexity of the current scene; for example, it increases the weight of YOLOv12 (because its SEAM module enhances occlusion features) when occlusion is severe, and relies on the results of RT-DETR when high recall is required. This fusion strategy based on scene complexity quantification and dynamic weight allocation gives the system context awareness and adaptive decision-making capabilities, significantly improving its detection robustness in varied and complex environments.

[0061] When fusing detections using a hierarchical fusion strategy, the system first calculates the joint confidence of the two bounding boxes. If the confidence exceeds a preset threshold, triple verification (spatial, semantic, and attention verification) is performed. If the confidence does not reach the threshold, the system chooses to adopt the YOLOv12 result or perform weighted fusion based on the intersection-over-union (IoU). This strategy combines hard example mining and online learning mechanisms, enabling real-time supplementation of missed targets in RT-DETR and feedback for training. This allows for the intelligent removal of inconsistent predictions while retaining valid detections. This hierarchical, rule-driven fusion decision effectively reduces false positives, false negatives, and overlapping localization issues in complex scenarios.

[0062] It should be understood that the RT-DETR model and YOLOv12 model used may be models without the aforementioned improvements.

[0063] In some embodiments, the first detection result set and the second detection result set are fused according to a hierarchical fusion strategy and adjusted fusion weights, specifically including the following steps: The detection boxes pointing to the same target in the first and second detection result sets are merged. The specific steps for fusing detection boxes pointing to the same target are as follows: If the first joint confidence of the two detection boxes is greater than the preset confidence value, both will be retained after verification. If the first joint confidence of the two detection boxes is not greater than the preset confidence value, the crossover ratio (IoU) of the two detection boxes is calculated. When the IoU is lower than the first threshold (IOU<0.5), the detection boxes of the second detection result set are adopted. When the IoU is between the first threshold and the second threshold (0.5≤IOU<0.8), the fusion weight is used as a coefficient to perform weighted fusion of the two detection boxes.

[0064] The first joint confidence level is calculated as follows:

[0065] , These are the confidence scores of the bounding boxes for the RT-DETR model and the YOLOv12 model, respectively. , These are the weights assigned to the RT-DETR model and the YOLOv12 model, respectively, and are preset values; This indicates the first joint confidence level.

[0066] In this embodiment, the problems of retaining false positives or deleting high-quality boxes caused by traditional simple fusion (such as direct averaging or taking the maximum value) are effectively avoided. By introducing a first joint confidence level as an initial screening threshold, the system can quickly identify high-reliability detection boxes. For box pairs that do not reach a high confidence level, their intersection-over-union ratio (IoU) is used to differentiate the processing strategy: in the case of low overlap, the default trust is placed on the more accurate YOLOv12 results, directly supplementing the missed detections of RT-DETR; in the case of medium overlap, dynamic weighting is used to achieve intelligent trade-offs and optimization of the localization results of the two models. The most reasonable penalty is adopted for different dispute situations, which greatly improves the overall quality of the final set of detection boxes.

[0067] In some embodiments, the specific steps for verifying the two detection boxes are as follows: Perform the following three verification strategies and obtain three verification results; Determine whether the intersection-union ratio of the detection box and the detection box predicted based on the historical trajectory is greater than a third threshold; A lightweight classifier is used to perform secondary classification on the image region corresponding to the detection box, and it is determined whether the classification confidence is higher than the first confidence value. Evaluate whether the region concentration of the detection box in the model's self-attention heatmap is higher than a preset concentration. If all three verification results meet their corresponding conditions, the verification is successful; otherwise, it fails.

[0068] That is, the high-confidence retention rule, when When the value is greater than 0.85, perform triple validation: Spatial validation: IOU with historical trajectory prediction boxes > 0.7 (third threshold 0.7); Semantic verification: secondary confirmation using a lightweight ResNet-18 classifier (confidence > 0.9); Attention validation: The concentration of the target region in the Transformer self-attention heatmap is >0.6; Spatial consistency verification correlates the current detection results with historical trajectory predictions, using temporal continuity information to filter out false detections with fluctuating positions or inconsistent target motion patterns, thus enhancing the system's stability in video stream or continuous frame detection. Secondary discrimination using a lightweight classifier effectively identifies misclassified boxes (such as misclassifying background texture as a target), improving the accuracy of category judgment. Attention focus verification addresses the root cause of model decisions, ensuring that high confidence stems from attention to the target's physical region, rather than background interference. Only when all three verifications pass simultaneously is the detection box considered highly reliable in terms of location, category, and decision basis, and thus ultimately retained.

[0069] In some embodiments, the following steps are included before fusing the first detection result set and the second detection result set: The detection boxes in the first detection result set and the second detection result set are compared to identify the detection boxes that were missed by the RT-DETR model, and the missed detection boxes are marked as hard cases; The YOLOv12 model is used to detect hard cases, and the detection results are then added to the first detection result set to update the first detection result set; The labeled difficult examples form the training dataset, which is then fed back into the RT-DETR model for incremental learning to update the RT-DETR model parameters.

[0070] This endows the entire detection system with the ability to self-evolve and continuously optimize, fundamentally solving the inherent defects of the RT-DETR model in specific complex scenarios. By comparing the outputs of the two models, the system can accurately identify the hard cases that RT-DETR missed and immediately use the advantages of YOLOv12 for supplementary detection, which directly improves the overall recall rate of the system within the current detection cycle. More importantly, feeding these hard cases back to RT-DETR for incremental learning is equivalent to providing RT-DETR with specialized training targeting its weaknesses, allowing its model parameters to be continuously optimized.

[0071] In some embodiments, adjusting the fusion weights based on the scene complexity of the input image specifically includes the following steps: During the current evaluation period, the preset test dataset is divided into multiple test subsets with different complexity levels based on the scene complexity of the input image; All test subsets were run in both the RT-DETR model and the YOLOv12 model to obtain the test results for each test subset in both models. The F1 score at different complexity levels is calculated based on the test results of each test subset in the RT-DETR model and the YOLOv12 model; Obtain the weights of the RT-DETR model from the previous evaluation period, and then calculate the mean of the F1 score for the current evaluation period; based on the weights of the RT-DETR model from the previous evaluation period and the mean, and combined with the double exponential smoothing algorithm formula, calculate the weights of the new RT-DETR model. The formula for the double exponential smoothing algorithm is as follows: ; Where α is the current performance weight (0.6≤α≤0.9) and γ is the trend sensitivity coefficient (0.1≤γ≤0.3). The weights of the newly calculated RT-DETR model for the current evaluation period are: α is the current performance weight, used to control the degree of influence of the RT-DETR performance on the weight update in the current evaluation period; The mean F1 score of the RT-DETR model for the current evaluation period; The weights of the RT-DETR model in the previous evaluation period; The mean F1 score of the RT-DETR model in the previous evaluation period; α is a manually set hyperparameter, and its value needs to be determined by considering the following factors: the sensitivity to changes in model performance (if you want the weights to quickly follow the current performance, α can be set large; if you want the weights to be more stable, α can be set small). By adjusting α multiple times, observe the overall performance of the system (such as detection accuracy and speed) after the weight update, and select the α value that optimizes performance. The weights of the new YOLOv12 model are obtained by subtracting the weights of the new RT-DETR model from 1.

[0072] This mechanism ensures that in scenarios where YOLOv12 excels, such as those with severe occlusion, its decision weight is dynamically increased; while in scenarios requiring high recall, the weight of RT-DETR is guaranteed. This makes the fusion process no longer mechanical, but intelligent, possessing scene awareness and adaptive decision-making capabilities.

[0073] In some embodiments, the mean of the F1 score for the current evaluation period is calculated. The calculation formula is: Where M represents the scene complexity level, Scene weight coefficient ( ); where M is the scene complexity level (M=5). Scene weight coefficient ( ); Calculating the F1 score at a given complexity level involves the following steps: Calculating the F1 score at a given complexity level involves the following steps: Obtain the recall of the RT-DETR model test results and the precision of the YOLOv12 model at the current complexity level; The F1 score is calculated based on precision and recall, combined with the F1 calculation formula. The F1 score is calculated as follows: F1 = 2 × (Precision × Recall) / (Precision + Recall); where Precision is the precision and Recall is the recall.

[0074] The calculation method of the F1 score has been clarified, and its technical effect is to provide a comprehensive, balanced, and reliable performance metric for dynamic weight allocation. The F1 score is the harmonic mean of precision and recall, taking into account both false positives and false negatives, avoiding the one-sidedness of using precision alone (which may lead to a surge in false negatives) or recall alone (which may lead to a flood of false positives). Evaluating model performance through this metric ensures that the allocated fusion weights truly reflect the model's comprehensive ability to balance "accurate detection" and "complete detection," guiding the entire joint system towards the correct direction of simultaneously optimizing precision and recall, ultimately achieving an overall improvement in mAP (mean precision).

[0075] In some embodiments, the RT-DETR model is used to perform preliminary detection on the input image, specifically including the following steps: Full-image feature extraction is performed on the input image, and several detection boxes are generated using the formula for calculating the number of detection boxes. The position and category of each detection box are then obtained. The formula for calculating the number of detection boxes is as follows: Where E×W is the input resolution, E and W are the height and width of the input image, S is the sampling step size (typical value 16 / 32), σ is the scene complexity adjustment coefficient (1.0≤σ≤3.0); σ is used to adjust according to the specific complexity of the actual target detection scene, and N is the number of detection boxes; The entropy value of the input image is calculated in real time; based on the baseline confidence threshold, the maximum and minimum values ​​of historical entropy values, and the entropy value, a real-time confidence threshold is calculated; the formula for calculating the real-time confidence threshold is: ;in As the baseline threshold, To adjust the coefficient, and These are the maximum and minimum historical entropy values, respectively. The detection boxes of the RT-DETR model are filtered using a real-time confidence threshold to obtain a set of detection boxes; For each detection box in the set of detection boxes, a second joint confidence score is calculated. Then, a second filtering is performed using the second joint confidence score to obtain a first detection result set of several detection boxes with location, category, and confidence score. The formula for calculating the second joint confidence score is: ;in For classification confidence, IoU is the intersection-union ratio of the predicted bounding box and its nearest ground truth bounding box. This is the balance coefficient.

[0076] This significantly improves the quality of the initial detection results of the RT-DETR model, laying a solid foundation for subsequent fusion. The real-time confidence threshold quantifies scene complexity through image entropy and intelligently reduces the threshold in complex scenes, effectively solving the serious missed detection problem caused by fixed thresholds and ensuring high recall of candidate targets. The subsequently introduced joint confidence score integrates classification confidence and localization accuracy, generating a more robust and comprehensive quality score. This avoids the situation in traditional methods where a bounding box with high classification confidence but poor localization is misclassified as a high-quality bounding box. The first detection result set after this step not only contains a rich set of potential targets, but each target also has a more reliable confidence score, greatly facilitating and optimizing subsequent fusion decisions.

[0077] In some embodiments, the YOLOv12 model is an improved YOLOv12 model, in which a SEAM module is integrated after the Neck layer and before the Head layer; the SEAM module is used to perform weighted fusion of the output feature map of the Neck layer and output it to the Head layer by the parallel working separate attention submodule and enhanced attention submodule. The separate attention submodule is used to weight the input feature map in the spatial dimension to highlight the salient region of the occluded object in space; the enhanced attention submodule is used to weight the input feature map in the channel dimension to enhance the feature response of the occluded object in the channel. The improved YOLOv12 model combines the repulsion loss function and the original YOLOv12 loss function during the training phase, and optimizes its model parameters through the backpropagation algorithm. The repulsion loss function is used to penalize the overlap between the predicted bounding box and the surrounding ground truth bounding boxes. The repulsion loss function is:

[0078] in, The number of prediction boxes, and These are the i-th and j-th prediction boxes, respectively. For prediction boxes and The degree of overlap between them This is the smoothed L1 loss function.

[0079] The integrated SEAM attention module enables the model to proactively and selectively enhance feature responses from salient regions and key channels of occluded objects, essentially equipping the model with focused eyes, allowing it to extract more discriminative feature representations from complex backgrounds and occlusions. Introducing a repulsion loss function during training forces the predicted bounding boxes to maintain distance from each other, effectively suppressing overlapping and fused erroneous bounding boxes at dense target locations, thus prompting the model to generate compact and accurate bounding boxes. These two improvements complement each other, resulting in a significant leap in the localization accuracy and boundary clarity of YOLOv12's second detection set in both occluded and dense scenes.

[0080] In some embodiments, the improved YOLOv12 model is used during the inference phase to map the first detection result set to information in the feature graph space of the improved YOLOv12 model for inference, so as to obtain preliminary detection boxes; The confidence of overlapping detection boxes is attenuated using a non-maximum suppression algorithm with confidence attenuation factor to obtain a second detection result set.

[0081] This paper addresses the inherent flaw of the standard NMS algorithm, which is prone to mistakenly deleting correct detection boxes in densely populated target areas. By introducing a confidence decay factor, the treatment of overlapping boxes is changed from "brutal deletion" to "gentle penalty," allowing boxes that, while overlapping, may still be real targets to be retained with lower confidence. This mechanism is particularly beneficial for retaining more real targets in highly overlapping groups of targets (such as dense crowds or vehicles), significantly reducing the false deletion rate and directly improving the system's recall. Simultaneously, due to the confidence decay, these retained boxes naturally have lower priority in subsequent ranking or decision-making, avoiding the introduction of a large number of low-quality detections and ensuring the cleanliness of the result set.

[0082] In some embodiments, mapping and aligning the first detection result set to the feature map space of the YOLOv12 model specifically includes: Obtain the sampling step size of the RT-DETR model and the sampling step size of the YOLOv12 model to calculate the dynamic scale compensation factor; The dynamic scale compensation factor is used to scale each detection box in the first detection result set to correct the mapping deviation caused by the difference in model scale. The first detection result set after correction is projected from the feature space of the RT-DETR model to the feature map space of the YOLOv12 model to complete the mapping alignment; The dynamic scale compensation factor is : δ is the scale difference sensitivity coefficient (0.1≤δ≤0.3). and These are the sampling step sizes for the two models, This represents the maximum sampling step size.

[0083] This embodiment addresses the "information gap" problem caused by inconsistencies in the feature spaces of different models. By introducing a precise mathematical mapping using a dynamic scale compensation factor, this module can project the coordinates of the detection boxes generated by the RT-DETR model onto the YOLOv12 feature map with sub-pixel accuracy (error < 0.5 pixels). This ensures that when YOLOv12 performs refined detection, the image region it processes is spatially perfectly aligned with the proposed region of the RT-DETR model, avoiding performance degradation due to feature misalignment.

[0084] In summary, this invention first utilizes the RT-DETR model for initial detection, retaining more potential targets through dynamic threshold adjustment. The generated detection boxes are then projected onto the feature space of an improved YOLOv12 model via a bimodal feature mapping. The improved YOLOv12 model integrates a SEAM attention module and a repulsion loss function to enhance the feature representation and compact localization of occluded targets. Subsequently, a joint model mechanism is employed to process the initial results of the two models: hard example mining and online learning are used to supplement missed targets and optimize the RT-DETR model; the detection results of the two models are intelligently fused through dynamic weight allocation based on scene complexity and a hierarchical fusion decision tree. Finally, an improved non-maximum suppression algorithm is applied for post-processing to reduce false deletions. This invention effectively solves the problems of missed detection, false detection, and inaccurate localization of targets in complex scenes, significantly improving the recall and accuracy of detection.

[0085] Thirdly, embodiments of this application provide a complex scene target detection device based on model dynamic fusion. The complex scene target detection device based on model dynamic fusion can be a personal computer (PC), laptop computer, server, or other device with data processing capabilities.

[0086] In this embodiment of the application, the complex scene target detection device based on model dynamic fusion may include a processor, a memory, a communication interface, and a communication bus.

[0087] The communication bus can be of any type and is used to interconnect the processor, memory, and communication interface.

[0088] The communication interface includes input / output (I / O) interfaces, physical interfaces, and logical interfaces. These interfaces enable interconnection of internal components within the model-based dynamic fusion-based complex scene target detection device, as well as interfaces for interconnection between the device and other devices (such as other computing devices or user equipment). Physical interfaces can be Ethernet interfaces, fiber optic interfaces, ATM interfaces, etc.; user equipment can be displays, keyboards, etc.

[0089] Memory can be various types of storage media, such as random access memory (RAM), read-only memory (ROM), non-volatile RAM (NVRAM), flash memory, optical storage, hard disk, programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), etc.

[0090] The processor can be a general-purpose processor, which can call a model-dynamically coupled complex scene object detection program stored in memory and execute the model-dynamically coupled complex scene object detection method provided in the embodiments of this application. For example, the general-purpose processor can be a central processing unit (CPU). The method executed when the model-dynamically coupled complex scene object detection program is called can be referred to in the various embodiments of the model-dynamically coupled complex scene object detection method of this application, and will not be repeated here.

[0091] Fourthly, embodiments of this application also provide a computer-readable storage medium.

[0092] The present application stores a model-dynamic joint complex scene target detection program on a computer-readable storage medium, wherein when the model-dynamic joint complex scene target detection program is executed by a processor, it implements the steps of the model-dynamic joint complex scene target detection method described above.

[0093] The method implemented when the model-based dynamic joint complex scene target detection program is executed can be referred to in the various embodiments of the model-based dynamic joint complex scene target detection method of this application, and will not be repeated here.

[0094] The terms "comprising" and "having," and any variations thereof, in the specification, claims, and accompanying drawings of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to such process, method, product, or apparatus. The terms "first," "second," and "third," etc., are used to distinguish different objects, etc., and do not indicate a sequence, nor do they limit "first," "second," and "third" to different types.

[0095] In the description of the embodiments of this application, terms such as "exemplary," "for example," or "for instance" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "exemplary," "for example," or "for instance" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of terms such as "exemplary," "for example," or "for instance" is intended to present the relevant concepts in a concrete manner.

[0096] In the description of the embodiments of this application, unless otherwise stated, " / " means "or". For example, A / B can mean A or B. The "and / or" in the text is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist simultaneously, and B exists alone. In addition, in the description of the embodiments of this application, "multiple" means two or more.

[0097] In some processes described in the embodiments of this application, multiple operations or steps are included in a specific order. However, it should be understood that these operations or steps may not be executed in the order they appear in the embodiments of this application, or they may be executed in parallel. The sequence number of the operation is only used to distinguish different operations, and the sequence number itself does not represent any execution order. In addition, these processes may include more or fewer operations, and these operations or steps may be executed sequentially or in parallel, and these operations or steps may be combined.

[0098] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device to execute the methods described in the various embodiments of this application.

[0099] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A method for target detection in complex scenes based on dynamic model fusion, characterized in that, It includes: The RT-DETR model is used to perform preliminary detection on the input image to generate a first detection result set with several detection boxes having location, category, and confidence. The first detection result set is mapped and aligned to the feature map space of the YOLOv12 model, and then the second detection result set is generated using the YOLOv12 model. Adjusting the fusion weights based on the scene complexity of the input image, this step includes: In the current evaluation period, dividing the preset test dataset into multiple test subsets with different complexity levels according to the scene complexity of the input image; running all test subsets in the RT-DETR model and YOLOv12 model respectively to obtain the test results of each test subset in the RT-DETR model and YOLOv12 model; calculating the F1 score at different complexity levels based on the test results of each test subset in the RT-DETR model and YOLOv12 model; obtaining the weights of the RT-DETR model in the previous evaluation period, and then calculating the mean of the F1 score in the current evaluation period; calculating the new weights of the RT-DETR model based on the weights of the RT-DETR model in the previous evaluation period and the mean, combined with the double exponential smoothing algorithm formula; the double exponential smoothing algorithm formula is: Where γ is the trend sensitivity coefficient; The weights of the newly calculated RT-DETR model for the current evaluation period are: α is the current performance weight, used to control the degree of influence of the RT-DETR performance on the weight update in the current evaluation period; The mean F1 score of the RT-DETR model for the current evaluation period; The weights of the RT-DETR model in the previous evaluation period; The mean F1 score of the RT-DETR model in the previous evaluation period; The weights of the new YOLOv12 model are obtained by subtracting the weights of the new RT-DETR model from 1. The first detection result set and the second detection result set are fused according to the hierarchical fusion strategy and the adjusted fusion weights to output the final detection result. The fusion of the first detection result set and the second detection result set according to the hierarchical fusion strategy and the adjusted fusion weights specifically includes the following steps: The detection boxes pointing to the same target in the first and second detection result sets are merged. The specific steps for fusing detection boxes pointing to the same target are as follows: If the first joint confidence of the two detection boxes is greater than a preset confidence value, both are retained after verification. The specific steps for verifying the two detection boxes are as follows: execute the following three verification strategies and obtain three verification results; determine whether the intersection-union ratio (IU) of the detection box and the detection box predicted based on historical trajectories is greater than a third threshold; use a lightweight classifier to perform secondary classification on the image region corresponding to the detection box and determine whether its classification confidence is higher than the first confidence value; evaluate whether the region concentration of the detection box in the model's self-attention heatmap is higher than a preset concentration; if all three verification results meet their corresponding conditions, the verification is passed; otherwise, it fails. If the first joint confidence of the two detection boxes is not greater than the preset confidence value, the cross-union ratio (CUP) of the two detection boxes is calculated; when the CUP is lower than the first threshold, the detection boxes of the second detection result set are adopted; when the CUP is between the first threshold and the second threshold, the fusion weight is used as a coefficient to perform weighted fusion of the two detection boxes.

2. The complex scene target detection method based on dynamic model fusion as described in claim 1, characterized in that, The following steps are included before merging the first and second detection result sets: The detection boxes in the first detection result set and the second detection result set are compared to identify the detection boxes that were missed by the RT-DETR model, and the missed detection boxes are marked as hard cases; The YOLOv12 model is used to detect hard cases, and the detection results are then added to the first detection result set to update the first detection result set; The labeled difficult examples form the training dataset, which is then fed back into the RT-DETR model for incremental learning to update the RT-DETR model parameters.

3. The complex scene target detection method based on dynamic model fusion as described in claim 2, characterized in that: Calculate the mean F1 score for the current evaluation period. The calculation formula is: Where M represents the scene complexity level, These are the scene weighting coefficients; The calculation of the F1 score at a complexity level includes the following steps: obtaining the recall of the RT-DETR model test results and the precision of the YOLOv12 model at the current complexity level; calculating the F1 score based on the precision and recall, combined with the F1 calculation formula; the F1 calculation formula is: F1=2×(Precision×Recall) / (Precision+Recall); where Precision is the precision and Recall is the recall.

4. The complex scene target detection method based on dynamic model fusion as described in claim 1, characterized in that, The initial detection of the input image using the RT-DETR model includes the following steps: Full-image feature extraction is performed on the input image, and several detection boxes are generated using the formula for calculating the number of detection boxes. The position and category of each detection box are then obtained. The formula for calculating the number of detection boxes is as follows: Where E×W is the input resolution, S is the sampling step size, σ is the scene complexity adjustment coefficient, and N is the number of detection boxes; The entropy value of the input image is calculated in real time; based on the baseline confidence threshold, the maximum and minimum values ​​of historical entropy values, and the entropy value, a real-time confidence threshold is calculated; the formula for calculating the real-time confidence threshold is: ;in As the baseline threshold, To adjust the coefficient, and These are the maximum and minimum historical entropy values, respectively. This is the real-time confidence threshold; The detection boxes of the RT-DETR model are filtered using a real-time confidence threshold to obtain a set of detection boxes; For each detection box in the set of detection boxes, a second joint confidence score is calculated. Then, a second filtering is performed using the second joint confidence score to obtain a first detection result set of several detection boxes with location, category, and confidence score. The formula for calculating the second joint confidence score is: ;in For classification confidence, IoU is the intersection-union ratio of the predicted bounding box and its nearest ground truth bounding box. This is the balance coefficient.

5. The complex scene target detection method based on dynamic model fusion as described in claim 1, characterized in that: The YOLOv12 model is an improved YOLOv12 model. The improved YOLOv12 model integrates a SEAM module after the Neck layer and before the Head layer. The SEAM module is used to perform weighted fusion of the output feature map of the Neck layer and output it to the Head layer through the parallel working separate attention submodule and enhanced attention submodule. The separate attention submodule is used to weight the input feature map in the spatial dimension to highlight the salient area of ​​the occluded object in space. The enhanced attention submodule is used to weight the input feature map along the channel dimension, thereby enhancing the feature response of the occluded object in the channel. The improved YOLOv12 model combines the repulsion loss function and the original YOLOv12 loss function during the training phase, and optimizes its model parameters through the backpropagation algorithm. The repulsion loss function is used to penalize the overlap between the predicted bounding box and the surrounding ground truth bounding boxes. for: in, The number of prediction boxes, and These are the i-th and j-th prediction boxes, respectively. For prediction boxes and The degree of overlap between them This is the smoothed L1 loss function.

6. The complex scene target detection method based on dynamic model fusion as described in claim 5, characterized in that: The improved YOLOv12 model is used in the inference phase to map the first detection result set to information in the feature graph space of the improved YOLOv12 model for inference, so as to obtain the preliminary detection box. The confidence of overlapping detection boxes is attenuated using a non-maximum suppression algorithm with confidence attenuation factor to obtain a second detection result set.

7. The complex scene target detection method based on dynamic model fusion as described in claim 1, characterized in that, Mapping and aligning the first detection result set to the feature map space of the YOLOv12 model specifically includes: Obtain the sampling step size of the RT-DETR model and the sampling step size of the YOLOv12 model to calculate the dynamic scale compensation factor; The dynamic scale compensation factor is used to scale each detection box in the first detection result set to correct the mapping deviation caused by the difference in model scale. The first detection result set after correction is projected from the feature space of the RT-DETR model to the feature map space of the YOLOv12 model to complete the mapping alignment; Among them, the dynamic scale compensation factor is The formula for calculating the dynamic scale compensation factor is as follows: δ is the scale difference sensitivity coefficient. and These are the sampling step sizes for the RT-DETR model and the YOLOv12 model, respectively. This is the preset maximum sampling step size.

Images