Novel traffic scene target detection evaluation method and system

Abstract

The application discloses a novel traffic scene target detection evaluation method and system, and relates to the technical field of target detection model performance evaluation. The application obtains the prediction result of a target detection model, determines true positive TP or false positive FP based on the matching relationship between the prediction frame and the real label, and respectively calculates the position weight factor and the scale weight factor. Then, the application obtains PSTP / PSFP by weighting TP / FP, constructs a position-scale weighted PR curve, and integrates to obtain the position-scale weighted average precision PSAP. The application can significantly improve the evaluation sensitivity of edge regions and small-scale high-risk targets, support scene adaptation and security level linkage, and make the evaluation results more suitable for the actual safety needs of automatic driving.

Description

Technical Field

[0001] This invention relates to the field of target detection model performance evaluation technology, and in particular to a novel target detection evaluation method and system for traffic scenarios. Background Technology

[0002] In applications such as autonomous driving, advanced driver assistance systems (ADAS), and intelligent traffic monitoring, performance evaluation of object detection models is a core element in ensuring the reliability and safety of perception systems. Currently, Mean Accuracy (AP) and its multi-scale extensions are widely used as standard evaluation metrics for object detection algorithms. These metrics, based on Intersection over Union (IoU) thresholds and class matching, classify prediction results into true positives (TP) or false positives (FP), and quantify the overall model performance using the area under the precision-recall (PR) curve. This paradigm has achieved good results in general vision tasks and has become the mainstream evaluation standard in academia and industry.

[0003] However, in high-safety-requirement traffic scenarios, traditional AP-type metrics reveal significant limitations: First, they treat all TP and FP with equal weight, ignoring the actual risk differences corresponding to different spatial locations and target scales. For example, small-scale targets located at the edge of the image or at a distance, if missed or falsely detected, can easily cause serious traffic accidents, and their detection value should be significantly higher than that of large targets in the center of the image. Second, existing metrics lack the ability to perceive scene context and cannot dynamically adjust the evaluation focus according to road type, weather conditions, or system safety level. Third, traditional PR curves only reflect global performance and are difficult to provide fine-grained insights into the model's performance on key difficult subsets (such as small targets and edge targets). Therefore, while existing evaluation methods can measure the overall detection capability of the model, they cannot effectively reflect its reliability in dealing with high-risk scenarios in real traffic environments, leading to a disconnect between evaluation results and actual deployment needs, and hindering the targeted optimization and safety certification of perception systems.

[0004] Therefore, there is an urgent need to propose a novel target detection and evaluation method and system that can explicitly model position sensitivity and scale sensitivity, assign higher evaluation weights to high-risk targets, and support adaptive adjustments based on traffic scenario characteristics and safety requirements, thereby generating performance indicators that are closer to actual safety needs and providing credible, detailed, and interpretable evaluation basis for key applications such as autonomous driving. Summary of the Invention

[0005] In view of this, the present invention proposes a novel target detection and evaluation method and system for traffic scenarios. By introducing a dynamic weighting mechanism based on target location and scale, true positives and false positives are assigned differentiated values, and a position-scale weighted average accuracy (PSAP) evaluation index is constructed. This solves the technical problem that traditional AP indices cannot distinguish the detection performance of high-risk targets in traffic scenarios, leading to a disconnect between evaluation results and actual safety requirements.

[0006] This invention provides a novel method for target detection and evaluation in traffic scenes, comprising the following steps: S1. Obtain the detection results of the target detection model on the traffic scene image. The detection results include multiple predicted target boxes and their corresponding categories and confidence scores. S2. Match the predicted target boxes with the ground truth bounding boxes, and determine each predicted target box as a true positive (TP) or a false positive (FP) based on the intersection-union threshold and class consistency. S3. For each predicted target box, calculate the position weight factor and scale weight factor for each predicted target box, wherein the position weight factor is determined based on the distance from the center point of the predicted target box to the center of the image, and the scale weight factor is determined based on the proportion of the area of ​​the predicted target box to the total area of ​​the image. S4. Based on the location weighting factor and scale weighting factor, each TP and FP is weighted to obtain weighted true positive PSTP and weighted false positive PSFP. S5. Based on all weighted true positive PSTPs and weighted false positive PSFPs, calculate the location-scale weighted precision and location-scale weighted recall, and construct the location-scale weighted PR curve; S6. Integrate the position-scale weighted PR curve to obtain the position-scale weighted average accuracy (PSAP), which serves as the comprehensive performance evaluation index of the target detection model in traffic scenarios.

[0007] Furthermore, the position weighting factor The calculations include: Let the image width be W and the height be H. The coordinates of the predicted target box center are ( , If ), then the normalized Manhattan distance d is: ; And restrict d to the interval [0,1]; when At that time, position weight factor ; when At that time, position weight factor ,in The preset boundary sensitivity threshold is β>1, which is the boundary risk amplification factor.

[0008] Furthermore, the scale weighting factor The calculations include: Let the predicted target box area be The total area of ​​the image is Normalized area ; like Then the scale weight factor ; like Then the scale weight factor ,in The threshold for determining small targets.

[0009] Furthermore, in the calculation of the weighted true positive PSTP and weighted false positive PSFP, the position weight coefficient... With scale weight coefficient Dynamically configured based on traffic scenario type: In highway or rural road scenarios, set ; In urban intersections or densely populated pedestrian areas, set up .

[0010] Furthermore, the formula for calculating the location-scale weighted recall is as follows: The total weight of the true labels in the denominator is only calculated for true target boxes that meet any of the following conditions: the normalized area is less than the small target determination threshold or the normalized Manhattan distance from the center of the true target box to the center of the image is greater than the preset boundary sensitivity threshold; the weight of the other true target boxes is 1.

[0011] Furthermore, in the process of constructing the location-scale weighted PR curve, only prediction results with a confidence level higher than a preset confidence threshold τ are retained for PSTP and PSFP calculations, and τ is dynamically set according to the security level of the application scenario: In the evaluation of Level 4 autonomous driving systems ; In the evaluation of driver assistance systems, .

[0012] Furthermore, the method also includes: For the same target detection model, the position-scale weighted average accuracy (PSAP) is calculated on multiple different traffic sub-scene datasets to obtain a scene-specific PSAP set. The scene-specific PSAP set is weighted and averaged to generate a comprehensive location-scale weighted average precision PSAP, wherein the weight of each sub-scene is determined by the frequency of occurrence or the accident risk level of the sub-scene in actual deployment.

[0013] Furthermore, the present invention also provides a novel traffic scene target detection and evaluation system, comprising: a detection result acquisition module, used to acquire multiple predicted target boxes, their categories, and confidence scores output by the target detection model for a traffic scene image; a matching determination module, used to match the predicted target boxes with ground truth bounding boxes, and mark each predicted target box as a true positive (TP) or a false positive (FP) based on an intersection-union (IU) threshold and category consistency; and a weight calculation module, used to calculate a position weight factor and a scale weight factor for each predicted target box, wherein the position weight factor is determined based on the distance from the center point of the predicted target box to the center of the image, and the scale weight factor is... The weighting factor is determined based on the proportion of the area of ​​the predicted target box to the total area of ​​the image; the weighted statistics module is used to weight each true positive (TP) and false positive (FP) according to the location weighting factor and the scale weighting factor to obtain weighted true positive (PSTP) and weighted false positive (PSFP); the index construction module is used to calculate the location-scale weighted precision and location-scale weighted recall based on all weighted true positive (PSTP) and weighted false positive (PSFP), construct the location-scale weighted precision (PR) curve, and obtain the location-scale weighted average precision (PSAP) by integrating the curve, which serves as the comprehensive performance evaluation index of the target detection model in traffic scenarios.

[0014] The present invention has the following advantages over the prior art: By calculating the normalized Manhattan distance from the center of the predicted bounding box to the image center and employing a piecewise function, targets near the image edge receive higher PSTP rewards when correctly detected, and larger PSFP penalties for missed or false detections, thus enhancing the sensitivity to assessing high-risk targets. This effectively amplifies the performance differences in the model's detection in extremely edge regions, avoiding the problem of traditional AP models masking critical missed detections due to equal weighting.

[0015] By applying exponential weights only to small targets with normalized areas less than a threshold, while assigning unit weights to large targets, evaluation resources are concentrated on challenging samples that truly impact driving safety, such as distant vehicles and small traffic signs, thus focusing performance metrics on small-scale challenging samples. Experiments show that this design can clearly distinguish the performance gap between RetinaNet and YOLOv8 in highway scenarios, which is difficult to reflect with traditional mAP.

[0016] By dynamically configuring location and scale weight coefficients based on traffic scenario types, the same model can automatically match the assessment focus to the actual risk distribution in different road environments, achieving scenario-adaptive capabilities for the assessment logic. For example, in urban intersection assessment, the system pays more attention to pedestrians crossing at the edge; in highway scenarios, it pays more attention to cars in the distance, thus generating personalized assessment results that are strongly correlated with the deployment scenario.

[0017] By weighting only the ground truth (GT) of difficult samples in the denominator of the recall rate and dynamically setting the confidence threshold in conjunction with the safety level, the PSAP score is ensured to directly reflect the model's ability to cover high-risk targets at high confidence levels. This enhances the safety orientation and decision support value of the evaluation index, and provides more instructive quantitative basis for the selection of perception modules, algorithm iteration, and functional safety certification of autonomous driving systems. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart illustrating the overall process architecture of an embodiment of the present invention. Figure 2 This is a diagram illustrating the calculation process of PSAP in an embodiment of the present invention; Figure 3 This is a position-scale weighted PR curve representing an embodiment of the present invention; Figure 4 This shows the performance difference trend of mAP and mPSAP for different target detectors in embodiments of the present invention. Detailed Implementation

[0020] The technical solutions of the present invention will be clearly and completely described below with reference to the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention. It should be understood that the specific embodiments described herein are only used to explain the present invention, and not to limit the present invention.

[0021] like Figure 1 As shown, this invention provides a novel target detection and evaluation method for traffic scenes, comprising the following steps: S1. Obtain the detection results of the target detection model on the traffic scene image. The detection results include multiple predicted target boxes and their corresponding categories and confidence scores. In one implementation, the traffic scene images are derived from real road video frames captured by vehicle-mounted forward-view cameras, surround-view systems, or roadside monitoring equipment, covering typical scenes such as urban roads, highways, and intersections, and including challenging conditions such as changes in lighting, weather interference, and complex obstructions.

[0022] Specifically, the traffic scene image is input into a pre-trained object detection model, such as Faster R-CNN, YOLOv5, RetinaNet, or DETR, and the object detection model outputs a set of detection results. Each prediction result It includes predicted bounding boxes, class labels, and classification confidence scores. The predicted bounding boxes are... or The system includes: location and scale information in a formal representation; category labels such as "pedestrian," "vehicle," "traffic sign," and "bicycle," etc., all predefined categories; and a classification confidence score, a real number between 0 and 1, reflecting the model's confidence in the predicted category. The detection result set Q serves as the original input for subsequent matching decisions and weighted evaluations, and its completeness and diversity directly affect the representativeness and robustness of the evaluation metrics of this invention.

[0023] S2. Match the predicted target boxes with the ground truth bounding boxes, and determine each predicted target box as a true positive (TP) or a false positive (FP) based on the intersection-union threshold and class consistency. In one implementation, the matching process processes predicted bounding boxes one by one based on classification confidence from high to low, and combines class consistency and spatial overlap as dual criteria to ensure the accuracy and robustness of TP / FP determination. The determination process is as follows: Figure 2 As shown, the specific steps include: Specifically, firstly, the prediction result set Q obtained in step S1 is post-processed with non-maximum suppression (NMS) to remove redundant prediction boxes on the same target; then, according to the classification confidence... Iterate through each prediction box from highest to lowest. The following judgment process is executed: Confidence level screening: If If the predicted bounding box is selected, it will be retained for subsequent matching; otherwise, it will be discarded. Category matching: Check predicted boxes The category label is consistent with the category of any unmatched true bounding box (GT); if they are inconsistent, it is determined to be a false positive (FP), and its weighted value (PSFP) is calculated. Intersection over Union (IoU) determination: If the categories are the same, then calculate... The crossover ratio I between the GT and the GT; if If the result is positive, it is determined to be a true positive TP, and the GT is marked as a match; otherwise, it is determined to be an FP and the PSFP is calculated.

[0024] Through the above process, all predicted boxes are accurately classified as TP or FP. TP only comes from predictions that are correctly classified and have high spatial overlap, while FP covers cases of misclassification, localization bias, or missed detection.

[0025] S3. For each predicted target box, calculate the position weight factor and scale weight factor for each predicted target box, wherein the position weight factor is determined based on the distance from the center point of the predicted target box to the center of the image, and the scale weight factor is determined based on the proportion of the area of ​​the predicted target box to the total area of ​​the image. In one implementation, to more realistically reflect the differences in the impact of different spatial locations and target scales on driving safety in traffic scenarios, this invention designs a nonlinear, segmented, scene-aware two-factor weighting mechanism.

[0026] First, calculate the location weighting factor. Obtain the bounding box coordinates of the current predicted target box and calculate its center point. , ), and combined with the width W and height H of the input image, calculate its normalized Manhattan distance d to the image center: The normalized distance ranges from [0,2], but in practice, since the target is usually located inside the image, the effective range is constrained to [0,1]. If d>1, it is truncated to 1.

[0027] Subsequently, a piecewise function is used to perform a nonlinear mapping on d, when When the target is located in the central region of the image or at a general edge, the risk increases relatively slowly, and an exponential function is used to smoothly increase the weight. ;when When this occurs, it indicates that the target has entered the edge of a high-risk region, at which point the risk increases sharply. A linear amplification function is then used to further enhance the weights. Where β>1 is the boundary risk amplification factor, The preset boundary sensitivity threshold can be dynamically adjusted according to the camera's field of view or road type.

[0028] Secondly, calculate the scale weight factor. Calculate the width w and height h of the predicted bounding box to obtain the area of ​​the predicted bounding box. Then divide by the total area of ​​the image. The normalized area is obtained. .

[0029] Considering that not all objectives are equally difficult, this invention only applies extra weight to small-scale objectives that truly impact security: if Assuming If the target area is 0.02, meaning it's less than 2% of the total image area, then an exponential function is used to assign it a higher weight, resulting in a scale weight factor. ;like For conventional targets that are relatively large and easy to detect, the scale weight is set to 1. No additional weighting is applied, among which The threshold for determining small targets.

[0030] The aforementioned location and scale weighting factors serve as the core input for weighted calculations. Predictions closer to the image boundary and smaller targets receive higher PSTP rewards when correctly detected, and larger PSFP penalties when missed or falsely detected. This guides the evaluation metrics to focus on the most critical and vulnerable perception scenarios in the autonomous driving system.

[0031] S4. Based on the location weighting factor and scale weighting factor, each TP and FP is weighted to obtain the weighted true positive PSTP and weighted false positive PSFP; in the calculation of the weighted true positive PSTP and weighted false positive PSFP, the location weighting coefficient... With scale weight coefficient Dynamically configure based on traffic scenario type: In highway or rural road scenarios, set... In urban intersections or densely populated pedestrian areas, set up .

[0032] In one implementation, this step is a key step in risk perception assessment, which replaces the equal weight assumption of TP=1 and FP=1 in traditional AP by integrating spatial location sensitivity and target scale sensitivity into a unified weighted score.

[0033] Specifically, for each predicted target box that has been identified as TP or FP First, obtain the position weight factor calculated in S3. and scale weight factor Subsequently, two configurable weighting coefficients are introduced: position weighting coefficient. With scale weight coefficient Both satisfy This is used to adjust the relative importance of location risk and scale risk in the comprehensive assessment. Weighted value Calculate using the following formula: ,like If the result is a true positive TP, then the corresponding weighted true positive PSTP is: ,like If the false positive FP is , then its corresponding weighted false positive PSFP is . In particular, and It is not a fixed value, but rather dynamically configured based on the type of traffic scenario to match the main safety concerns in different road environments. In highway or rural road scenarios, distant small targets are the main source of risk, therefore, it is set... This allows the scale factor to dominate the weighting; in urban intersections or densely populated pedestrian areas, pedestrians crossing at the edges and non-motorized vehicles suddenly cutting in are high-risk factors, therefore, a scale factor is set... This enhances sensitivity to the marginality of a location.

[0034] S5. Based on all weighted true positive PSTPs and weighted false positive PSFPs, calculate the location-scale weighted precision and location-scale weighted recall, and construct the location-scale weighted PR curve; In one implementation, this step aims to transform the weighted statistics of the S4 output into quantifiable evaluation metrics and to achieve a refined characterization of model performance by constructing a novel PR curve. For example... Figure 3 As shown, the horizontal axis of the curve represents the location-scale weighted recall rate. The vertical axis represents the position-scale weighted accuracy. Its form directly reflects the model's overall performance on high-risk targets.

[0035] The formula for calculating the location-scale weighted recall is as follows: The total weight of the true labels in the denominator is only calculated for true target boxes that meet any of the following conditions: the normalized area is less than the small target determination threshold or the normalized Manhattan distance from the center of the true target box to the center of the image is greater than the preset boundary sensitivity threshold; the weight of the other true target boxes is 1.

[0036] During the construction of the location-scale weighted PR curve, only prediction results with confidence levels higher than a preset confidence threshold τ are retained for PSTP and PSFP calculations, and τ is dynamically set according to the security level of the application scenario. In the evaluation of Level 4 autonomous driving systems ; In the evaluation of driver assistance systems, .

[0037] Specifically, firstly, all prediction results are sorted by classification confidence level. The bounding boxes are sorted from highest to lowest confidence level, and only those with a confidence level higher than a preset confidence threshold τ are retained for subsequent calculations. This threshold τ is not fixed but dynamically set based on the security level of the application scenario. In the evaluation of L4 autonomous driving systems, due to the extremely high requirements for perception reliability, τ≥0.7 is set, and only high-confidence predictions are considered. In the evaluation of driver assistance systems, a certain number of false alarms can be accepted to improve recall rates, so τ∈[0.3,0.5].

[0038] Subsequently, at each confidence cutoff point k, the weighted statistics of the first k predictions are accumulated and the index is calculated according to the following formula: (1). Location-scale weighted accuracy : The numerator is the cumulative weighted true positive count, and the denominator is the cumulative weighted total number of tests, reflecting how many of the results detected at high confidence level are truly effective high-risk targets.

[0039] (2) Location-scale weighted recall : Where the denominator is the total weight of all real-world labeled targets. However, not all ground truth boxes are weighted: the position-scale weight of the j-th ground truth bounding box is calculated only if it satisfies any of the following conditions, and its normalized area is... That is, the normalized Manhattan distance from the small target or its center to the image center. That is, located in a high-risk borderline area; for regular GT that does not meet the above conditions, its weight .

[0040] This design ensures that the recall denominator focuses on the set of difficult samples that truly impact safety, avoiding the dilution of sensitivity to critical scenarios by large targets. All ( , After connecting and smoothing the points, we get the following: Figure 3 The position-scale weighted PR curve shown is compared to the traditional PR curve. The fluctuations of this curve in the low recall region better reflect the model's stable detection capability for high-risk targets. Its overall upward or rightward shift indicates that the model performs better in traffic-critical scenarios.

[0041] S6. Integrate the position-scale weighted PR curve to obtain the position-scale weighted average accuracy (PSAP), which serves as the comprehensive performance evaluation index of the target detection model in traffic scenarios.

[0042] In one implementation, such as Figure 3 As shown, the position-scale weighted PR curve constructed in S5 is transformed into a single, comparable, and quantifiable numerical indicator, namely the position-scale weighted average accuracy (PSAP), which is used to comprehensively characterize the overall performance of the target detection model in key traffic scenarios.

[0043] Specifically, PSAP is calculated by numerically integrating the area under the location-scale weighted PR curve. The system supports two mainstream integration strategies, which can be flexibly selected according to evaluation needs, including: 11-point interpolation: Sample at 11 equally spaced points (0.0, 0.1, 0.2, ..., 1.0) on the recall axis, and take all samples that satisfy the criteria at each recall value r. The highest accuracy in predictions Let p(r) be the value of PSAP. Then PSAP is calculated as follows: This method is computationally stable and suitable for comparing different models laterally.

[0044] Full-point integration method: directly integrating all discrete points on the PR curve ( , Monotonicity is applied to ensure that precision does not increase with recall, and then the area under the curve is calculated using the trapezoidal rule or the cumulative rectangle method. ,in This method preserves more details and is suitable for fine-grained analysis of the model's performance in high-confidence regions.

[0045] The obtained PSAP value ranges from 0 to 1, with a higher value indicating a higher model risk. Since the PSAP calculation basis has implemented differentiated weighting for difficult samples through S3–S5, it reflects the safety and reliability of the model in real traffic environments better than the traditional mAP.

[0046] Furthermore, to support multi-category or multi-scenario assessments, this invention is extended to mPSAP, which calculates the PSAP for all target categories separately and then averages the results; or assesses different sub-scenarios separately and then fuses the results according to accident risk weights. For example... Figure 4 As shown, there are significant differences in the performance ranking of multiple object detection models under mAP and mPSAP, indicating that mPSAP can effectively identify models that are masked under traditional indicators and perform better in key scenarios, providing more instructive decision-making basis for the selection and optimization of autonomous driving perception systems.

[0047] Furthermore, the method also includes: calculating the location-scale weighted average precision (PSAP) for the same target detection model on multiple different traffic sub-scenario datasets to obtain a scenario-specific PSAP set; performing a weighted average on the scenario-specific PSAP set to generate a comprehensive location-scale weighted average precision (PSAP), wherein the weight of each sub-scenario is determined by the frequency of occurrence or accident risk level of the sub-scenario in actual deployment.

[0048] In one implementation, to more comprehensively and realistically reflect the overall reliability of the target detection model in complex traffic environments, this invention introduces a multi-sub-scene refined evaluation and weighted fusion mechanism. This mechanism overcomes the limitations of traditional single-dataset evaluation, enabling the final index to dynamically adapt to the risk distribution of the actual deployment environment.

[0049] Specifically, the test data is first divided into several traffic sub-scenario datasets. Typical sub-scenarios include, but are not limited to: highways (dense small targets at a distance); urban intersections (frequent pedestrian crossings at the edge); low-light roads at night (low visibility, high false detection rate); rainy and foggy weather scenarios (blurred images, unclear target outlines); schools / residential areas (high risk such as residents suddenly entering), etc.

[0050] Subsequently, the complete evaluation process described in S1–S6 was executed independently for each sub-scene dataset, and the corresponding scene-specific PSAP values ​​were calculated to form a scene-specific PSAP set. .

[0051] Next, assign a scene weight to each sub-scene i. The weights are not set equally, but are dynamically determined based on at least one of the following actual deployment factors, such as frequency of occurrence, the proportion of mileage or time of a certain type of scenario in the daily driving of a vehicle through vehicle data records or high-precision map statistics; accident risk level: referring to the accident statistics released by the traffic management department or the hazard analysis results in the ISO 21448 standard, higher weights are given to high-risk scenarios.

[0052] Finally, the overall position-scale weighted average accuracy (PSAP) is calculated using the following formula: The weights satisfy the normalization condition. The Position-Scale Weighted Average Accuracy (PSAP) not only retains the ability to sensitively assess edges and small targets in each sub-scenario, but also makes the final score closer to real-world safety requirements through risk-oriented weighted fusion. This differentiated assessment result can directly guide model selection, sensor configuration, and functional safety verification strategies, significantly improving the scenario adaptability and risk controllability of autonomous driving systems.

[0053] Furthermore, the present invention also provides a novel traffic scene target detection and evaluation system, comprising: a detection result acquisition module, used to acquire multiple predicted target boxes, their categories, and confidence scores output by the target detection model for a traffic scene image; a matching determination module, used to match the predicted target boxes with ground truth bounding boxes, and mark each predicted target box as a true positive (TP) or a false positive (FP) based on an intersection-union (IU) threshold and category consistency; and a weight calculation module, used to calculate a position weight factor and a scale weight factor for each predicted target box, wherein the position weight factor is determined based on the distance from the center point of the predicted target box to the center of the image, and the scale weight factor is... The weighting factor is determined based on the proportion of the area of ​​the predicted target box to the total area of ​​the image; the weighted statistics module is used to weight each true positive (TP) and false positive (FP) according to the location weighting factor and the scale weighting factor to obtain weighted true positive (PSTP) and weighted false positive (PSFP); the index construction module is used to calculate the location-scale weighted precision and location-scale weighted recall based on all weighted true positive (PSTP) and weighted false positive (PSFP), construct the location-scale weighted precision (PR) curve, and obtain the location-scale weighted average precision (PSAP) by integrating the curve, which serves as the comprehensive performance evaluation index of the target detection model in traffic scenarios.

[0054] like Figure 4 As shown, on the same set of traffic scenario test data, the mAP values ​​of multiple target detection models are relatively similar, making it difficult to reflect their actual performance differences on high-risk targets (such as edge pedestrians and small-scale obstacles). However, after adopting the mPSAP index proposed in this invention, the performance of each model shows a significant difference. This result indicates that traditional mAP, by treating all targets with equal weights, masks the differences in the models' capabilities on key difficult samples; while mPSAP, through a position-scale weighting mechanism, effectively amplifies the evaluation sensitivity to high-risk scenarios, thus providing a more safety-guided performance criterion for autonomous driving systems.

[0055] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A novel target detection and evaluation method for traffic scenes, characterized in that, Includes the following steps: S1. Obtain the detection results of the target detection model on the traffic scene image. The detection results include multiple predicted target boxes and their corresponding categories and confidence scores. S2. Match the predicted target boxes with the ground truth bounding boxes, and determine each predicted target box as a true positive (TP) or a false positive (FP) based on the intersection-union threshold and class consistency. S3. For each predicted target box, calculate the position weight factor and scale weight factor for each predicted target box, wherein the position weight factor is determined based on the distance from the center point of the predicted target box to the center of the image, and the scale weight factor is determined based on the proportion of the area of ​​the predicted target box to the total area of ​​the image. S4. Based on the location weighting factor and scale weighting factor, each TP and FP is weighted to obtain weighted true positive PSTP and weighted false positive PSFP. S5. Based on all weighted true positive PSTPs and weighted false positive PSFPs, calculate the location-scale weighted precision and location-scale weighted recall, and construct the location-scale weighted PR curve; S6. Integrate the position-scale weighted PR curve to obtain the position-scale weighted average accuracy (PSAP), which serves as the comprehensive performance evaluation index of the target detection model in traffic scenarios.

2. The method as described in claim 1, characterized in that, The position weight factor The calculations include: Let the image width be W and the height be H. The coordinates of the predicted target box center are ( , If ), then the normalized Manhattan distance d is: ； And restrict d to the interval [0,1]; when At that time, position weight factor ; when At that time, position weight factor ,in The preset boundary sensitivity threshold is β>1, which is the boundary risk amplification factor.

3. The method as described in claim 1, characterized in that, The scale weight factor The calculations include: Let the predicted target box area be The total area of ​​the image is Normalized area ; like Then the scale weight factor ; like Then the scale weight factor ,in The threshold for determining small targets.

4. The method as described in claim 1, characterized in that, In the calculation of the weighted true positive PSTP and weighted false positive PSFP, the position weight coefficient... With scale weight coefficient Dynamically configured based on traffic scenario type: In highway or rural road scenarios, set ; In urban intersections or densely populated pedestrian areas, set up .

5. The method as described in claim 1, characterized in that, The formula for calculating the location-scale weighted recall is as follows: The total weight of the true labels in the denominator is only calculated for true target boxes that meet any of the following conditions: the normalized area is less than the small target determination threshold or the normalized Manhattan distance from the center of the true target box to the center of the image is greater than the preset boundary sensitivity threshold; the weight of the other true target boxes is 1.

6. The method as described in claim 1, characterized in that, During the construction of the location-scale weighted PR curve, only prediction results with confidence levels higher than a preset confidence threshold τ are retained for PSTP and PSFP calculations, and τ is dynamically set according to the security level of the application scenario. In the evaluation of Level 4 autonomous driving systems ; In the evaluation of driver assistance systems, .

7. The method as described in claim 1, characterized in that, The method further includes: For the same target detection model, the position-scale weighted average precision (PSAP) is calculated on multiple different traffic sub-scene datasets to obtain a scene-specific PSAP set. The scene-specific PSAP set is weighted and averaged to generate a comprehensive location-scale weighted average precision PSAP, wherein the weight of each sub-scene is determined by the frequency of occurrence or the accident risk level of the sub-scene in actual deployment.

8. A novel target detection and evaluation system for traffic scenes, characterized in that, include: The detection result acquisition module is used to acquire multiple predicted target boxes, their categories, and confidence scores output by the target detection model for traffic scene images. The matching and determination module is used to match the predicted target boxes with the ground truth bounding boxes, and to mark each predicted target box as a true positive (TP) or a false positive (FP) based on the intersection-union threshold and class consistency. The weight calculation module is used to calculate the position weight factor and scale weight factor for each predicted target box, wherein the position weight factor is determined based on the distance from the center point of the predicted target box to the center of the image, and the scale weight factor is determined based on the proportion of the area of ​​the predicted target box to the total area of ​​the image. The weighted statistics module is used to weight each true positive TP and false positive FP according to the location weight factor and scale weight factor to obtain weighted true positive PSTP and weighted false positive PSFP. The index construction module is used to calculate the location-scale weighted precision and location-scale weighted recall based on all weighted true positive PSTP and weighted false positive PSFP, construct the location-scale weighted PR curve, and obtain the location-scale weighted average precision PSAP by integrating the curve, which serves as the comprehensive performance evaluation index of the target detection model in traffic scenarios.

Images