Road spray video detection model self-evolution method and system

Abstract

The present application relates to a kind of road throw video detection model self-evolution method and system;The method comprises: production alarm data automatic backflow, multi-dimensional feature signature extraction and misclassification automatic pre-classification, hierarchical review and difficult negative sample mining, active learning sample selection, incremental training, degradation defense, gray verification and hot loading online;The system is used to execute the above method.The present application constructs complete automation closed loop from production data to model evolution;In the review link of closed loop, the automatic pre-classification of misclassification is realized by multi-dimensional feature signature, and different types of misclassification are accurately identified, so as to reduce the workload of manual review;The misclassification data after classification is used as difficult negative sample, together with misclassification type label, to drive incremental training, to realize the pertinence of model to specific misclassification type Correction;With the dual safety mechanism of degradation defense and gray verification, prevent model degradation.

Description

Technical Field

[0001] This invention relates to a self-evolving method and system for video detection models of road spills, belonging to the fields of artificial intelligence and intelligent transportation. Background Technology

[0002] In existing technologies, road debris video detection systems based on deep learning take road monitoring video streams as the core input and rely on the strong feature extraction capabilities of deep learning algorithms to efficiently extract key features of debris. Combined with efficient detection algorithms such as the YOLO series, they can achieve real-time detection, accurate positioning, and timely early warning of road debris, effectively making up for the shortcomings of traditional manual inspections, such as low efficiency and high missed detection rate, and providing technical support for road safety maintenance and emergency response.

[0003] However, existing systems face the following technical bottlenecks in actual deployment:

[0004] On the one hand, the false detection identification is crude and cannot accurately distinguish the false detection type. The existing system generates a large number of candidate targets in the initial detection stage, including a large number of false detections. Common sources of false detection include: (1) Lane lines and road markings: White or yellow road markings are easily identified as foreign objects in the background difference, especially when the lighting changes; (2) Guardrail reflection: The bright reflective area produced by the metal guardrail under sunlight or vehicle headlights is similar in shape to small fragments; (3) Road shadows: The shadows cast by vehicles, bridges and signs appear as "newly appearing dark areas" in the background difference, which are easily misjudged; (4) Edge noise: Due to the camera installation angle and lens distortion, the edge of the image is prone to continuous small change areas.

[0005] Some existing systems use conventional detection models such as YOLO / SSD, which can only output confidence and category, and do not have the ability to identify specific false detection types. Although some existing systems have false detection filtering schemes, these schemes often have the following shortcomings: (1) Relying solely on confidence thresholds: Such schemes reduce false detections by raising the threshold, but this also increases missed detections and cannot accurately distinguish between "false detections of different reasons"; (2) Historical false alarm database comparison: Such schemes (such as CN119863770A) compare the current detection results with the features in the historical false alarm database, and exclude those that match. However, such methods rely on the accumulation and maintenance of the false alarm database, and cannot immediately identify newly emerging false detection types, and are slow to adapt to new scenarios; (3) Single-dimensional filtering: Such schemes only filter by area or shape thresholds and cannot distinguish between targets that are similar in shape but different in essence, such as thin plastic films and thin lane lines.

[0006] On the other hand, the model does not evolve after deployment, and the false detection data is not utilized. Specifically, there are: (1) The model is not updated after deployment: Most existing systems have fixed models after deployment. As the environment changes (such as road repair, seasonal changes, and facility replacement), their detection effect gradually declines; (2) Iteration depends on external teams: After users report that the effect has declined, the data needs to be sent to the algorithm supplier for retraining, which is time-consuming and costly; (3) Production data is not utilized: The alarm data (including correct detection and false detection) generated daily in the production environment contains rich scene information, but it is not converted into training resources, especially the false detection data, which accurately reflects the cognitive bias of the model, but is simply discarded instead of being used to correct the model; (4) Lack of safe online deployment mechanism: Directly replacing the entire model is risky, and the new model may actually be worse in some scenarios. In addition, although the existing MLOps technology can iterate models for cloud AI services, it does not target traffic video edge deployment scenarios, nor does it involve the accurate classification of false detection types.

[0007] In summary, the main technical bottleneck of existing systems in practical deployment lies in their inability to accurately identify false detection types for targeted filtering, and their inability to systematically feed back identified false detection data into the training process to drive model evolution. This results in false detection filtering and model iteration being two separate processes. The applicant has conducted research and development to address these technical problems and has achieved results, which are now being applied for in this patent application. Summary of the Invention

[0008] The main objective of this invention is to address the problems existing in the prior art by providing a self-evolving method for detecting road debris in video surveillance. This method automatically converts production environment detection data into training resources, achieves automatic pre-classification of false positives and accurate mining of difficult negative samples through multi-dimensional feature signatures, and drives the continuous evolution of the detection model through incremental training, degradation defense, and grayscale verification. A corresponding system is also provided.

[0009] The technical solution of this invention to solve its technical problem is as follows:

[0010] A self-evolutionary method for detecting road debris in video models includes the following steps:

[0011] Step 1: Automatic Feedback of Production Alarm Data: When the video detection model for road debris in the production environment triggers a debris alarm each time, the production alarm data is automatically saved and stored in the queue to be reviewed.

[0012] Step 2: Multi-dimensional feature signature extraction and automatic pre-classification of false detections: The spilled objects in each returned production alarm data are taken as candidate targets; multi-dimensional feature signatures are automatically extracted for each candidate target, and the false detection type is automatically predicted according to the preset classification rule chain.

[0013] Step 3: Graded review and mining of difficult negative samples: Based on the results obtained in Step 1 and Step 2, the candidate targets are graded and difficult negative samples are mined out.

[0014] Step 4: Actively learn sampling: Actively mine high-value samples from unlabeled videos through uncertainty sampling and unexpected sampling;

[0015] Step 5, Incremental Training: Incremental training is automatically triggered according to preset conditions; during training, the model is adjusted based on the current best road litter video detection model. The training data is a combination of historical accumulated data and newly labeled data, including difficult negative samples and high-value samples.

[0016] Step 6, Degradation Defense: Automatically perform degradation evaluation on the post-trained detection models, prevent post-trained detection models that fail the evaluation from going online, and use post-trained detection models that pass the evaluation as detection models that pass the degradation defense.

[0017] Step 7, Gray-scale verification and hot-loading deployment: The detection model with degradation defense will be verified in gray-scale, and the detection model that passes the gray-scale verification will be hot-loaded and deployed to the production testing service without interruption.

[0018] The further improved technical solution of this invention is as follows:

[0019] Preferably, in the first step, the production alarm data includes: alarm screenshots, before and after video clips, confidence level of the spilled object, type of spilled object, timestamp, camera number, and environmental metadata; wherein, the alarm screenshots contain the location of the spilled object detection box, the confidence level of the spilled object is high confidence, medium confidence, or low confidence, and the environmental metadata includes lighting conditions and weather conditions.

[0020] Preferably, in the second step, the execution flow for extracting multi-dimensional feature signatures is as follows:

[0021] For each candidate target, the following dimensional features are automatically extracted as the multi-dimensional feature signature of that candidate target:

[0022] S1. Extract shape dimension features; shape dimension features include: solidity, elongation, and area; where solidity is the ratio of the candidate target area to its convex hull area, reflecting the degree of filling of the candidate target; elongation is the ratio of the long side to the short side of the circumscribed rectangle of the candidate target, reflecting the elongation of the candidate target; area is the pixel area of ​​the candidate target contour.

[0023] S2. Extract color dimension features; color dimension features include: main saturation, average brightness, and color contrast with the road surface; where, main saturation is the median saturation of the candidate target area; average brightness is the average brightness of the candidate target area; and color contrast with the road surface is calculated by taking the road surface area with a preset margin extended outward from the bounding rectangle of the candidate target as the reference area and calculating the absolute value of the average brightness difference between the candidate target area and the reference area.

[0024] S3. Extract spatial dimension features; the spatial dimension feature is the distance from the edge of the image; the distance from the edge of the image is the shortest distance from the center of the candidate target to the four edges of the image.

[0025] S4. Extract the temporal stability dimension; the temporal stability dimension is the feature stability; the feature stability is calculated by dividing the standard deviation of the brightness value at the same position of the candidate target by the mean within several frames after the candidate target first appears, and then subtracting the normalized standard deviation by 1.

[0026] More preferably, in S1, the shape dimension features also include: aspect ratio and roundness; wherein, the aspect ratio is the aspect ratio of the bounding rectangle of the candidate target; and the roundness is calculated based on the area and perimeter of the candidate target, reflecting the degree to which the candidate target is close to a circle.

[0027] In S2, the color dimension features also include the dominant hue; the dominant hue is the median value of the pixel hue of the candidate target region in the HSV color space.

[0028] Preferably, in the second step, the execution process for automatically predicting the false detection type according to the preset classification rule chain is as follows: the multi-dimensional feature signature of the candidate target is checked one by one according to the preset classification rule chain to see if it matches; if a matching rule is found, the check stops, the candidate target is predicted as a candidate target of the corresponding false detection type according to the matching rule, and the false detection type label is recorded in the production alarm data corresponding to the candidate target; if no matching rule is found after the check, the candidate target is regarded as a candidate target that is not predicted as a false detection.

[0029] The preset classification rule chain includes lane line exclusion rules, guardrail reflection exclusion rules, shadow exclusion rules, and edge noise exclusion rules arranged in sequence. The criteria for lane line exclusion are: extension length greater than the first threshold, solidity greater than the second threshold, main saturation less than the third threshold, and average brightness greater than the fourth threshold. The criteria for guardrail reflection exclusion are: distance from the edge of the image less than the fifth threshold, extension length greater than the sixth threshold, average brightness greater than the seventh threshold, and feature stability less than the eighth threshold. The criteria for shadow exclusion are: color contrast with the road surface less than the ninth threshold, main saturation less than the tenth threshold, area greater than the eleventh threshold, and average brightness less than the twelfth threshold. The criteria for edge noise exclusion are: distance from the edge of the image less than the thirteenth threshold, or area less than the fourteenth threshold.

[0030] Candidate targets are checked against each of the above rules in sequence. If any rule is matched, the target is pre-judged as a false detection of the corresponding type, and the pre-judged false detection type label is recorded in the corresponding production alarm data.

[0031] More preferably, the specific criteria for lane line exclusion rules are: extension length greater than 5.0, solidity greater than 0.8, main saturation less than 30 and average brightness greater than 180; the specific criteria for guardrail reflection exclusion rules are: distance from the edge of the image less than 50 pixels, extension length greater than 3.0, average brightness greater than 200 and feature stability less than 0.7; the specific criteria for shadow exclusion rules are: color contrast with the road surface less than 20, main saturation less than 20, area greater than a preset threshold and average brightness less than 120; the specific criteria for edge noise exclusion rules are: distance from the edge of the image less than 20 pixels, or area less than the minimum area for effective detection.

[0032] Preferably, the execution flow of the third step is as follows:

[0033] T1. The candidate targets that are predicted to be false detections are handed over to the labeler for confirmation or correction of the prediction results; if the candidate target is confirmed by the labeler to be a false detection, the production alarm data corresponding to the candidate target is marked as a difficult negative sample.

[0034] T2. For candidate targets that are not pre-judged as false detections, if the confidence level of the candidate target is high, the candidate target is pre-labeled as "correctly detected" and submitted to the annotator for confirmation or correction; if the confidence level of the candidate target is medium or low, the candidate target is submitted to the annotator for a complete and comprehensive review. When the candidate target is verified by the annotator as a false detection and a false detection type label is marked, the false detection type label is recorded in the production alarm data corresponding to the candidate target, and the production alarm data is marked as a difficult negative sample.

[0035] Preferably, the execution flow of the fourth step is as follows:

[0036] V1. Uncertainty Sampling: Periodically run the current road spill video detection model on the production video to identify frames with confidence levels near the decision boundary;

[0037] V2, Unexpected Sampling: Identifies frames where the detection results change abruptly between consecutive frames;

[0038] Frames obtained from V1 and V2 are automatically added to the priority labeling queue as high-value samples, and then labeled by labelers for future use.

[0039] Preferably, in the fifth step, the preset condition is to automatically trigger incremental training when any of the following conditions are met: the amount of newly labeled data reaches a preset threshold, the time since the last training exceeds a preset time, or the detection index deteriorates beyond a preset value.

[0040] More preferably, in the fifth step, difficult negative samples are used as high-weighted negative examples in incremental training; corresponding data augmentation strategies are selected according to the false detection type label, including: for lane line false detection labels, randomly superimposing white or yellow linear interference objects in the training image to simulate the appearance of lane lines under different lighting conditions; for guardrail reflection false detection labels, adding random preset high-brightness spots and applying brightness perturbation to the image edge area to simulate the metallic reflection of guardrails; for shadow false detection labels, superimposing preset low-contrast dark irregular areas in the image to simulate the shadows cast by bridges and signs; for edge noise false detection labels, applying preset small geometric deformations and noise to the image edges to simulate edge artifacts caused by lens distortion.

[0041] Preferably, in step six, the execution flow of degradation defense is as follows:

[0042] W1. Use a fixed validation set to evaluate the trained detection model and obtain core metrics; core metrics include mAP, precision, and recall.

[0043] W2. Compare each core indicator with the currently used detection model; based on each core indicator of the currently used detection model, set the tolerance range for each core indicator.

[0044] W3. If any core metric of the post-trained detection model falls below the corresponding tolerance range mentioned above, the post-trained detection model will be automatically prevented from going online, and the reason for degradation will be recorded in the training log for subsequent diagnosis. If all core metrics of the post-trained detection model are not lower than the corresponding tolerance range mentioned above, the post-trained detection model will be regarded as a detection model that has passed the degradation defense.

[0045] Preferably, in step seven, the execution flow for grayscale verification and hot-loading deployment is as follows:

[0046] Y1. Use the detection model that has passed the degradation defense as the new model, and use the current detection model as the old model; select a few cameras to deploy the new model, and retain the old model for the remaining cameras;

[0047] Y2. After a preset grayscale run time, compare the detection results of the new model and the old model on the same video stream.

[0048] Y3. If the false alarm rate or false negative rate exceeds the corresponding preset threshold, the cameras selected in Y1 will be automatically rolled back to the old model; if the false alarm rate and false negative rate do not exceed the corresponding preset threshold, the new model will pass the grayscale verification.

[0049] Y4. After the new model that has passed the gray-scale verification is manually confirmed, it will be fully launched online.

[0050] Y5. Push the new model file to each preset node. Each preset node will automatically hot-load the new model after detecting changes in the model file, ensuring uninterrupted production and testing services throughout the process.

[0051] This invention also provides:

[0052] A self-evolving system for road litter video detection models, employing the self-evolving method for road litter video detection models described above, comprises the following modules:

[0053] The automatic production alarm data feedback module is used to perform automatic feedback of production alarm data.

[0054] The multi-dimensional feature signature extraction and automatic pre-classification of false detection modules are used to perform multi-dimensional feature signature extraction and automatic pre-classification of false detection.

[0055] The tiered review and hard negative sample mining module is used to perform tiered review and hard negative sample mining.

[0056] The active learning sampling module is used to perform active learning sampling.

[0057] The incremental training module is used to perform incremental training;

[0058] The degradation defense module is used to perform degradation defense.

[0059] The grayscale verification and hot-load deployment module is used to perform grayscale verification and hot-load deployment.

[0060] Compared with existing technologies, this invention constructs a complete automated closed loop from production data to model evolution. In the closed-loop review process, multi-dimensional feature signatures are used to achieve automatic pre-classification of false detections, accurately identifying different types of false detections (lane lines, guardrail reflections, shadows, edge noise), thereby reducing the workload of manual review. The classified false detection data is used as hard negative samples, along with the false detection type labels, to drive incremental training, enabling the model to correct specific false detection types. It has a dual security mechanism of degradation defense and grayscale verification to prevent model degradation. Attached Figure Description

[0061] Figure 1 This is a schematic diagram of the main process of Embodiment 1 of the present invention.

[0062] Figure 2 This is a schematic diagram of extracting multi-dimensional feature signatures in Embodiment 1 of the present invention.

[0063] Figure 3 This is a comparison chart of the typical characteristic values ​​of the four false detection types and actual debris fragments in Embodiment 1 of the present invention.

[0064] Figure 4 This is a schematic diagram illustrating the effect of the upper limit of brightness condition in the shadow exclusion rule of Embodiment 1 of the present invention.

[0065] Figure 5 This is a graph showing the improvement in model accuracy with iteration rounds in Embodiment 2 of the present invention. Detailed Implementation

[0066] The present invention will be further described in detail below with reference to the embodiments. However, the present invention is not limited to the examples given.

[0067] Example 1

[0068] The core innovation of the self-evolutionary method for the road litter video detection model in this embodiment is: to construct a fully automatic closed loop of "automatic feedback of production alarm data → multi-dimensional feature signature false detection pre-classification → hierarchical review and mining of difficult negative samples → incremental training → degradation defense → grayscale verification → hot loading and online deployment". Among them, the use of multi-dimensional feature signatures to achieve automatic and accurate identification of false detection types provides high-quality difficult negative samples and their false detection type labels for the closed loop, enabling incremental training to specifically correct the specific cognitive biases of the model.

[0069] like Figure 1 As shown, the self-evolutionary method for road debris video detection model in this embodiment specifically includes the following steps:

[0070] Step 1: Automatic Feedback of Production Alarm Data

[0071] When the video detection model for road debris in a production environment triggers a debris alarm each time, it automatically saves the production alarm data, which includes: alarm screenshot (including the location of the debris detection box), previous and subsequent video clips, the confidence level of the debris, the type of debris, timestamp, camera number, and environmental metadata (lighting conditions, weather conditions, etc.). The confidence level of the debris is either high confidence, medium confidence, or low confidence. The production alarm data is then stored in a queue awaiting review.

[0072] Step 2: Multi-dimensional feature signature extraction and automatic pre-classification of false detections:

[0073] Spilled material from each returned production alarm data point is considered as a candidate target. For each candidate target, multi-dimensional feature signatures are automatically extracted, and the false detection type is automatically predicted according to a pre-defined classification rule chain. This step is performed before manual review, providing auxiliary decision-making basis for subsequent tiered review.

[0074] (1) The execution process for extracting multi-dimensional feature signatures is as follows:

[0075] like Figure 2 As shown, for each candidate target, the following dimensional features are automatically extracted as the multi-dimensional feature signature of the candidate target (in this embodiment, it is a four-dimensional feature signature).

[0076] (i) Extract shape dimension features, specifically including: aspect ratio, solidity, circularity, elongation, and area.

[0077] Among them, aspect ratio is the aspect ratio of the bounding rectangle of the candidate target; solidity is the ratio of the area of ​​the candidate target to the area of ​​its convex hull, reflecting the degree of filling of the candidate target; roundness is calculated based on the area and perimeter of the candidate target, reflecting the degree to which the candidate target is close to a circle; elongation is the ratio of the long side to the short side of the bounding rectangle of the candidate target, reflecting the elongation of the candidate target; and area is the pixel area of ​​the candidate target outline.

[0078] (ii) Extract color dimension features, specifically including: dominant hue, dominant saturation, mean value, and color contrast with the road surface.

[0079] Among them, the dominant color is the median value of the pixel color of the candidate target area in the HSV color space; the dominant saturation is the median value of the saturation of the candidate target area; the average brightness is the average brightness of the candidate target area; and the color contrast with the road surface is calculated by taking the road surface area with a preset margin extended outward from the outer rectangle of the candidate target as the reference area and calculating the absolute value of the average brightness difference between the candidate target area and the reference area.

[0080] (iii) Extract spatial dimension features, specifically: edge distance; the edge distance is the shortest distance from the center of the candidate target to the four edges of the image.

[0081] (iv) Extract the temporal stability dimension, specifically: feature stability. Feature stability is calculated by dividing the standard deviation of the brightness value at the same location of the candidate target by the mean within several frames after the candidate target first appears, and then subtracting the normalized standard deviation from 1. The features of real debris remain stable between frames (feature stability is close to 1), while the guardrail reflection flickers as vehicles pass by (feature stability is low).

[0082] (2) The execution process for automatically predicting the false detection type according to the preset classification rule chain is as follows:

[0083] The multi-dimensional feature signature of the candidate target is checked against the preset classification rule chain one by one to see if it matches. If a matching rule is found, the check stops and the candidate target is pre-judged as a candidate target of the corresponding false detection type according to the matching rule. The false detection type label is recorded in the production alarm data corresponding to the candidate target. If no matching rule is found after the check, the candidate target is regarded as a candidate target that is not pre-judged as a false detection.

[0084] The preset classification rule chain includes lane line exclusion rules, guardrail reflection exclusion rules, shadow exclusion rules, and edge noise exclusion rules arranged in sequence; in actual debugging and verification, the typical characteristic values ​​of the four false detection types and real debris fragments are as follows: Figure 3 As shown.

[0085] (i) Lane line exclusion rules:

[0086] The physical reason for this rule: lane lines are thin, bright, and low-saturation road markings.

[0087] The criteria for this rule are: high elongation (greater than the first threshold), high solidity (greater than the second threshold), low principal saturation (less than the third threshold), and high average brightness (greater than the fourth threshold); specifically: elongation greater than 5.0, solidity greater than 0.8, principal saturation less than 30, and average brightness greater than 180.

[0088] Through actual debugging and verification, under the above conditions, setting the elongation threshold to 5.0 can distinguish between lane lines (whose elongation is usually 8-15) and thin plastic films (whose elongation is usually 2-4, because the film will curl up and cause uneven width).

[0089] (ii) Rules for excluding reflective railings:

[0090] The physical cause of this rule is that the guardrail reflection appears at the edge of the image as thin, bright spots that flicker as vehicles pass by.

[0091] The criteria for this rule are: close to the edge of the screen (less than the fifth threshold), high elongation (greater than the sixth threshold), extremely high average brightness (greater than the seventh threshold), and low feature stability (less than the eighth threshold); specifically: distance from the edge of the screen less than 50 pixels, elongation greater than 3.0, average brightness greater than 200, and feature stability less than 0.7.

[0092] Through actual debugging and verification, under the above conditions, setting the threshold of feature stability to 0.7 is the key parameter. The feature stability of guardrail reflection is usually between 0.3 and 0.6 (because the reflection changes with the passage of vehicles), while the feature stability of real debris is usually between 0.8 and 1.0. This dimension is the decisive feature for distinguishing between "continuous debris" and "intermittent reflection".

[0093] (iii) Shadow exclusion rules:

[0094] The physical cause of this rule is that shadows are low-contrast, low-saturation, and large areas of dark color.

[0095] The criteria for this rule are: low color contrast with the road surface (less than the ninth threshold), low main saturation (less than the tenth threshold), large area (greater than the eleventh threshold), and low average brightness (less than the twelfth threshold); specifically: color contrast with the road surface is less than 20, main saturation is less than 20, area is greater than the preset threshold, and average brightness is less than 120.

[0096] Actual testing and verification showed that, under the above conditions, although the brightness of the shadow was lower than that of the road surface, the color contrast between the shadow and the road surface was very small, usually less than 20, while the color contrast between the actual spilled debris and the road surface was usually greater than 30; Figure 4 As shown, setting an upper limit for average brightness can prevent high-brightness light-colored debris fragments (such as white plastic film) from being misjudged as shadows. Only true dark shadows will be excluded, and light-colored debris fragments will be preserved.

[0097] (iv) Edge noise exclusion rules:

[0098] The physical cause of this rule is: lens distortion at the edges of the image and small, persistent changes caused by the mounting angle.

[0099] The criteria for this rule are: the distance from the edge of the image is extremely small (less than the thirteenth threshold), or the area is extremely small (less than the fourteenth threshold); specifically, the distance from the edge of the image is less than 20 pixels, or the area is less than the minimum area for effective detection.

[0100] Candidate targets are checked against the above rules in sequence. If any rule is matched, it is predicted as a false detection of the corresponding type. The predicted false detection type label is recorded in the corresponding production alarm data. The specific labels are: lane_line, guardrail_glare, shadow, and edge_noise.

[0101] Step 3: Tiered Review and Difficult Negative Sample Mining

[0102] Based on the results obtained in the first and second steps, the candidate targets are classified and processed, and difficult negative samples are extracted from them.

[0103] The execution flow for this step is as follows:

[0104] (1) The candidate targets that are predicted to be false detections are handed over to the labelers for confirmation or correction of the prediction results (the labelers only need to confirm or correct them simply); if the candidate target is confirmed by the labelers to be a false detection, the production alarm data (including false detection type labels) corresponding to the candidate target is marked as a difficult negative sample;

[0105] (2) For candidate targets that are not pre-judged as false detections, (i) if the confidence level of the candidate target is high confidence, the candidate target is pre-labeled as "correct detection" and handed over to the annotator for confirmation or correction (only simple confirmation or correction by the annotator is required); (ii) if the confidence level of the candidate target is medium confidence or low confidence, the candidate target is handed over to the annotator for a complete and comprehensive review. When the candidate target is verified by the annotator as a false detection and a false detection type label is marked, the false detection type label is recorded in the production alarm data corresponding to the candidate target, and the production alarm data (including the false detection type label) is marked as a difficult negative sample.

[0106] The key point of this step is to label false positives confirmed manually by the annotators as difficult negative samples, i.e., cases where the road litter video detection model identifies them as litter but they are not. Unlike existing traditional solutions, this step adds structured exclusion reason labels to each difficult negative sample (i.e., false positive type labels given by the second step's prediction or manually annotated in this step). These labels can not only be used for statistical analysis but also directly guide data augmentation strategies for subsequent incremental training. Based on the applicant's practical experience, in actual project iterations, difficult negative samples have a far greater value in improving the road litter video detection model than ordinary positive samples. The false positives of the detection model are essentially "cognitive biases" in certain scenarios. Only by correcting these easily confused scenarios with the correct answers can the bias be accurately eliminated. Ordinary positive samples only reinforce the model's existing capabilities and have limited help in eliminating biases.

[0107] Step 4: Actively learn sample selection:

[0108] The process of proactively identifying high-value samples from unlabeled videos is as follows:

[0109] (1) Uncertainty sampling: Periodically run the current road litter video detection model on the production video to identify frames with confidence levels near the decision boundary, which are cases where the model is "unsure".

[0110] (2) Unexpected sampling: Identify frames where the detection results change abruptly between frames (such as being detected in the previous frame but lost in the next frame), thereby locating the weak links in the model.

[0111] The frames obtained in the above execution process are automatically added to the priority labeling queue as high-value samples, and then labeled by the labelers for future use.

[0112] Step 5: Incremental Training

[0113] Incremental training is automatically triggered when any of the following conditions are met:

[0114] (1) The amount of newly labeled data reaches the preset threshold;

[0115] (2) More than the preset time has passed since the last training session;

[0116] (3) The detection index deteriorates beyond the preset value.

[0117] The training is based on a fine-tuned model for detecting road litter videos, and the training data is a combination of historical accumulated data and newly labeled data, including difficult negative samples and high-value samples.

[0118] The key points of this step are: using difficult negative samples as high-weighted negative examples in incremental training; and selecting corresponding data augmentation strategies based on the false detection type labels. Specifically, for the lane line false detection label, randomly superimposing white or yellow linear interference objects into the training images to simulate the appearance of lane lines under different lighting conditions; for the guardrail reflection false detection label, adding randomly preset high-brightness spots and applying brightness perturbations to the image edge areas to simulate the metallic reflection of guardrails; for the shadow false detection label, superimposing preset low-contrast dark irregular areas into the image to simulate the shadows cast by bridges and signs; and for the edge noise false detection label, applying preset small geometric deformations and noise to the image edges to simulate edge artifacts caused by lens distortion. The specific mapping relationship between false detection type labels and data augmentation strategies is shown in the table below.

[0119]

[0120] This allows incremental training to specifically correct cognitive biases of the detection model for specific scenarios, rather than indiscriminately increasing training data.

[0121] Step 6, Degenerate Defense:

[0122] After the incremental training in step five is completed, a degradation assessment is performed automatically. The specific execution process is as follows:

[0123] (1) Use a fixed validation set to evaluate the trained detection model and obtain core metrics such as mAP, precision, and recall;

[0124] (2) Compare each of the core indicators with the current detection model (i.e., the current production model); and set the tolerance range for each core indicator according to the core indicators of the current detection model.

[0125] (3) If any core indicator of the post-training detection model is lower than the corresponding tolerance range mentioned above, the post-training detection model will be automatically prevented from going online, and the reason for degradation will be recorded in the training log for subsequent diagnosis; if all core indicators of the post-training detection model are not lower than the corresponding tolerance range mentioned above, the post-training detection model will be used as a detection model that passes the degradation defense.

[0126] Step 7: Gray-scale verification and hot-loading deployment:

[0127] The sixth step involves gray-scale verification using a degradation defense detection model. The specific execution process is as follows:

[0128] (1) Select a number of cameras to deploy the detection model that passed the degradation defense in step 6, i.e. the new model; the remaining cameras retain the original detection model, i.e. the old model;

[0129] (2) Run grayscale for a preset time and compare the detection results of the new model and the old model on the same video stream;

[0130] (3) If the false alarm rate or false negative rate exceeds the corresponding preset threshold, the old model will be automatically rolled back to the selected cameras in (1); if the false alarm rate and false negative rate do not exceed the corresponding preset threshold, the new model will pass the grayscale verification.

[0131] (4) After the new model that has passed the grayscale verification is manually confirmed, it will be fully launched online;

[0132] (5) Push the new model file to each preset node. After each preset node detects the change in the model file, it automatically hot-loads the new model, ensuring uninterrupted production and testing services throughout the process.

[0133] Example 2

[0134] This embodiment uses a highway project as an example to illustrate the typical operation process of the method in Embodiment 1.

[0135] The specific details of this embodiment are as follows:

[0136] 1. Initial state before running:

[0137] An initial YOLO detection model was trained on 936 manually labeled images, with an initial mAP50 of 0.46. After deployment to the production environment, the model generated approximately 50-100 alerts per day.

[0138] 2. First iteration:

[0139] (1) Data feedback: The system automatically collects alarm data over a period of time. Each data item includes alarm screenshots, confidence scores, video clips, etc.

[0140] (2) Feature signature pre-classification: The system extracts a four-dimensional feature signature for each candidate target and makes a prediction based on the rule chain. Typical case: A bright candidate target is located on the right edge of the screen. Its feature signature is: length = 4.2, brightness = 235, edge distance = 35 pixels, time stability = 0.45. It hits the guardrail reflection rule and is automatically predicted as a false detection of the "guardrail_glare" class. Another candidate target is located in the center of the road. Its length = 12.3, solidity = 0.92, saturation = 18, brightness = 210. It hits the lane line rule and is predicted as a false detection of the "lane_line" class.

[0141] (3) Tiered review: The annotator confirms or corrects the prediction results. The prediction accuracy rate is about 85%, which greatly reduces the workload of annotation. Confirmed false positives, along with the type label, are marked as difficult negative samples.

[0142] (4) Incremental training: New labeled data (including difficult negative samples) are merged with the original data and fine-tuned based on the current model. Difficult negative samples are used as high-weight negative examples. The system automatically adds targeted data augmentation based on the label of the reason for exclusion - edge highlight enhancement is added for reflective guardrails, and low contrast enhancement is added for shadows.

[0143] (5) Degradation defense and gray-scale verification: The new model was evaluated on a fixed validation set. The core indicators were all better than the current model. After passing the degradation evaluation, it entered gray-scale operation. After running gray-scale for one week on 3 cameras and the performance was stable, it was fully launched.

[0144] 3. Subsequent iterations:

[0145] After multiple rounds of iterative optimization, combined with the effects of model architecture optimization, hyperparameter tuning, and closed-loop correction of difficult negative samples, the model's mAP50 gradually increased from the initial 0.46 to 0.80. Figure 5 The table below shows typical indicators for each stage:

[0146]

[0147] The aforementioned accuracy improvement is the result of a combination of factors, including model architecture selection, hyperparameter tuning, training data expansion, and closed-loop correction using difficult negative samples. The closed-loop mechanism of this invention ensures that false detection data from the production environment can be systematically fed back into the training process, working in conjunction with other optimization methods to continuously improve the model's detection accuracy.

[0148] Example 3

[0149] This embodiment is a scenario example of precise filtering of feature signatures and collection of difficult negative samples.

[0150] The specific details of this embodiment are as follows:

[0151] Scenario 1: Lane line filtering

[0152] In a camera feed, background subtraction detected a candidate target located in the center of the road. The system extracted a four-dimensional feature signature: stretch = 12.3, solidity = 0.92, saturation = 18, and brightness = 210. Inputting the classification rule chain, it matched the lane line rule (stretch greater than 5.0, solidity greater than 0.8, saturation less than 30, and brightness greater than 180), and was pre-classified as a "lane_line" false positive. After confirmation by the labeler, this case was added to the training set as a difficult negative sample, labeled "lane_line".

[0153] Scene 2: Guardrail Reflective Filtering

[0154] A bright candidate target was detected on the right edge of the image. Feature signature: edge distance = 35 pixels, elongation = 4.2, brightness = 235, temporal stability = 0.45. It matches the guardrail reflection rule (edge ​​distance less than 50, elongation greater than 3.0, brightness greater than 200, and stability less than 0.7), and is pre-classified as a false positive of the "guardrail_glare" type. After confirmation, it is added to the training set.

[0155] Scenario 3: Correct Preservation of Real Fragments

[0156] A dark fragment appeared on the road surface. Feature signature: Length = 1.8, Solidity = 0.65, Saturation = 45, Brightness = 85, Color Contrast = 42, Edge Distance = 280 pixels, Stability = 0.93. Four rules were failed in sequence: lane line not satisfied (insufficient length), guardrail reflectivity not satisfied (long edge distance and high stability), shadow not satisfied (contrast greater than 20), edge noise not satisfied (long distance). All rules passed, proceeding to normal review process; the annotator confirmed it as a correct detection.

[0157] Scenario 4: Light-colored fragment protection

[0158] A white plastic film fell onto the road. Characteristic signature: Saturation = 12, Brightness = 195, Color Contrast = 15, Area = 180 pixels. During shadow rule checks: Contrast and saturation are both less than 20, but Brightness = 195 is greater than the brightness limit of 120, therefore the shadow rule was not matched. The brightness limit condition in the shadow rule successfully protected this light-colored spill from being mistakenly detected.

[0159] Example 4

[0160] This embodiment is an example of a degradation defense model that prevents defects.

[0161] The specific details of this embodiment are as follows:

[0162] In one iteration, the labeled data exhibited a systematic bias (some false positives were incorrectly labeled as correct). When the new model was evaluated on a fixed validation set, its accuracy abnormally decreased (from 0.82 to 0.71). The degradation defense mechanism detected that the accuracy exceeded the tolerance range and automatically prevented the model from going live. The system recorded the cause of the degradation and issued an alert. After technicians investigated and found the labeling bias, they cleaned up the incorrectly labeled data, retrained the model, and restored it to normal.

[0163] Example 5

[0164] This example demonstrates how grayscale verification can detect problems and trigger a rollback.

[0165] The specific details of this embodiment are as follows:

[0166] After passing degradation defense, the new model entered grayscale validation and was deployed to run on three cameras. During grayscale validation, a high false positive rate was found at night (in infrared scenes, guardrail reflections were misidentified as debris). The grayscale validation step automatically rolled back the old model after detecting that the false positive rate exceeded a threshold. Simultaneously, screenshots of false positives from guardrail reflections at night were added as new difficult negative samples (labeled "guardrail_glare") to the next round of training data, creating a positive feedback loop for continuous improvement.

[0167] As can be seen from the above embodiments, the present invention has the following characteristics:

[0168] 1. Closed-loop self-evolution: It constructs a complete automated closed loop from production data to model evolution, and the accuracy continues to improve after the model is deployed without relying on external teams.

[0169] 2. Accurate classification of false positives: Through multi-dimensional feature signatures and classification rule chains, the system can automatically identify four common false positive types: lane lines, guardrail reflections, shadows, and edge noise, reducing the workload of manual review.

[0170] 3. Targeted error correction: Difficult negative samples carry false detection type labels into the training, enabling incremental training to perform targeted data augmentation for specific false detection types. The error correction efficiency is much higher than that of indiscriminate data augmentation.

[0171] 4. Dual security guarantee: Degradation defense plus grayscale verification provide double protection to prevent defective models from affecting production when they go online.

[0172] 5. Zero-downtime iteration: The hot-loading mechanism enables model updates to continue without interrupting the detection service.

[0173] 6. Improved annotation efficiency: Feature signature prediction assists in hierarchical review, and proactive learning selects samples to prioritize the annotation of the most valuable data.

[0174] In addition to the embodiments described above, the present invention may have other implementations. All technical solutions formed by equivalent substitution or equivalent transformation fall within the protection scope claimed by the present invention.

Claims

1. A self-evolutionary method for road litter video detection model, characterized in that, Includes the following steps: Step 1: Automatic Feedback of Production Alarm Data: When the video detection model for road debris in the production environment triggers a debris alarm each time, the production alarm data is automatically saved. Store production alarm data in the pending review queue; Step 2: Multi-dimensional feature signature extraction and automatic pre-classification of false detections: The spilled objects in each returned production alarm data are taken as candidate targets; multi-dimensional feature signatures are automatically extracted for each candidate target, and the false detection type is automatically predicted according to the preset classification rule chain. Step 3: Graded review and mining of difficult negative samples: Based on the results obtained in Step 1 and Step 2, the candidate targets are graded and difficult negative samples are mined out. Step 4: Actively learn sampling: Actively mine high-value samples from unlabeled videos through uncertainty sampling and unexpected sampling; Step 5, Incremental Training: Incremental training is automatically triggered according to preset conditions; During training, the model is adjusted based on the current best road litter video detection model. The training data is a combination of historical accumulated data and newly labeled data, including difficult negative samples and high-value samples. Step 6, Degradation Defense: Automatically perform degradation evaluation on the post-trained detection models, prevent post-trained detection models that fail the evaluation from going online, and use post-trained detection models that pass the evaluation as detection models that pass the degradation defense. Step 7, Gray-scale verification and hot-loading deployment: The detection model with degradation defense will be verified in gray-scale, and the detection model that passes the gray-scale verification will be hot-loaded and deployed to the production testing service without interruption.

2. The self-evolutionary method for road debris video detection model according to claim 1, characterized in that, In the first step, the production alarm data includes: alarm screenshots, before and after video clips, confidence level of the spilled object, type of spilled object, timestamp, camera number, and environmental metadata. Among them, the alarm screenshots contain the location of the spilled object detection box, the confidence level of the spilled object is high confidence, medium confidence, or low confidence, and the environmental metadata includes lighting conditions and weather conditions.

3. The self-evolutionary method for detecting road debris video according to claim 2, characterized in that, in the second step, the execution flow for extracting multi-dimensional feature signatures is as follows: For each candidate target, the following dimensional features are automatically extracted as the multi-dimensional feature signature of that candidate target: S1. Extract shape dimension features; shape dimension features include: solidity, elongation, and area; where solidity is the ratio of the candidate target area to its convex hull area, reflecting the degree of filling of the candidate target; elongation is the ratio of the long side to the short side of the circumscribed rectangle of the candidate target, reflecting the elongation of the candidate target; area is the pixel area of ​​the candidate target contour. S2. Extract color dimension features; color dimension features include: main saturation, average brightness, and color contrast with the road surface; where, main saturation is the median saturation of the candidate target area; average brightness is the average brightness of the candidate target area; and color contrast with the road surface is calculated by taking the road surface area with a preset margin extended outward from the bounding rectangle of the candidate target as the reference area and calculating the absolute value of the average brightness difference between the candidate target area and the reference area. S3. Extract spatial dimension features; the spatial dimension feature is the distance from the edge of the image; the distance from the edge of the image is the shortest distance from the center of the candidate target to the four edges of the image. S4. Extract the temporal stability dimension; the temporal stability dimension is the feature stability; the feature stability is calculated by dividing the standard deviation of the brightness value at the same position of the candidate target by the mean within several frames after the candidate target first appears, and then subtracting the normalized standard deviation by 1.

4. The self-evolutionary method for road debris video detection model according to claim 3, characterized in that, In S1, the shape dimension features also include: aspect ratio and roundness; where aspect ratio is the ratio of the width to the height of the bounding rectangle of the candidate target; roundness is calculated based on the area and perimeter of the candidate target, reflecting the degree to which the candidate target is close to a circle. In S2, the color dimension features also include the dominant hue; the dominant hue is the median value of the pixel hue of the candidate target region in the HSV color space.

5. The self-evolutionary method for road debris video detection model according to claim 3, characterized in that, In the second step, the execution process for automatically predicting the false detection type according to the preset classification rule chain is as follows: the multi-dimensional feature signature of the candidate target is checked one by one according to the preset classification rule chain to see if it matches; if a matching rule is found, the check stops, the candidate target is predicted as a candidate target of the corresponding false detection type according to the matching rule, and the false detection type label is recorded in the production alarm data corresponding to the candidate target; if no matching rule is found after the check, the candidate target is regarded as a candidate target that is not predicted as a false detection. The preset classification rule chain includes lane line exclusion rules, guardrail reflection exclusion rules, shadow exclusion rules, and edge noise exclusion rules arranged in sequence. The criteria for lane line exclusion are: extension length greater than the first threshold, solidity greater than the second threshold, main saturation less than the third threshold, and average brightness greater than the fourth threshold. The criteria for guardrail reflection exclusion are: distance from the edge of the image less than the fifth threshold, extension length greater than the sixth threshold, average brightness greater than the seventh threshold, and feature stability less than the eighth threshold. The criteria for shadow exclusion are: color contrast with the road surface less than the ninth threshold, main saturation less than the tenth threshold, area greater than the eleventh threshold, and average brightness less than the twelfth threshold. The criteria for edge noise exclusion are: distance from the edge of the image less than the thirteenth threshold, or area less than the fourteenth threshold. Candidate targets are checked against each of the above rules in sequence. If any rule is matched, the target is pre-judged as a false detection of the corresponding type, and the pre-judged false detection type label is recorded in the corresponding production alarm data.

6. The self-evolutionary method for road debris video detection model according to claim 5, characterized in that, The specific criteria for excluding lane lines are: extension length greater than 5.0, solidity greater than 0.8, main saturation less than 30, and average brightness greater than 180. The specific criteria for excluding guardrail reflections are: distance from the edge of the image less than 50 pixels, extension length greater than 3.0, average brightness greater than 200, and feature stability less than 0.

7. The specific criteria for excluding shadows are: color contrast with the road surface less than 20, main saturation less than 20, area greater than the preset threshold, and average brightness less than 120. The specific criteria for excluding edge noise are: distance from the edge of the image less than 20 pixels, or area less than the minimum area for effective detection.

7. The self-evolutionary method for road litter video detection model according to claim 5, characterized in that, The execution process for the third step is as follows: T1. The candidate targets that are predicted to be false detections are handed over to the labeler for confirmation or correction of the prediction results; if the candidate target is confirmed by the labeler to be a false detection, the production alarm data corresponding to the candidate target is marked as a difficult negative sample. T2. For candidate targets that are not pre-judged as false detections, if the confidence level of the candidate target is high, the candidate target is pre-labeled as "correctly detected" and submitted to the annotator for confirmation or correction; if the confidence level of the candidate target is medium or low, the candidate target is submitted to the annotator for a complete and comprehensive review. When the candidate target is verified by the annotator as a false detection and a false detection type label is marked, the false detection type label is recorded in the production alarm data corresponding to the candidate target, and the production alarm data is marked as a difficult negative sample.

8. The self-evolutionary method for road debris video detection model according to claim 7, characterized in that, The execution process for step four is as follows: V1. Uncertainty Sampling: Periodically run the current road spill video detection model on the production video to identify frames with confidence levels near the decision boundary; V2, Unexpected Sampling: Identifies frames where the detection results change abruptly between consecutive frames; Frames obtained from V1 and V2 are automatically added to the priority labeling queue as high-value samples, and then labeled by labelers for future use.

9. The self-evolution method for a video detection model of road litter as described in claim 8, characterized in that, in the fifth step, incremental training is automatically triggered when any of the following conditions are met: the amount of newly labeled data reaches a preset threshold, the time since the last training exceeds a preset time, or the detection index deteriorates beyond a preset value.

10. The self-evolutionary method for road debris video detection model according to claim 9, characterized in that, In the fifth step, difficult negative samples are used as high-weighted negative examples in incremental training; corresponding data augmentation strategies are selected according to the false detection type label, including: for lane line false detection label, white or yellow linear interference objects are randomly superimposed on the training image to simulate the appearance of lane lines under different lighting conditions; for guardrail reflection false detection label, random preset high-brightness spots are added to the image edge area and brightness perturbation is applied to simulate the metallic reflection of guardrails; for shadow false detection label, preset low-contrast dark irregular areas are superimposed on the image to simulate the shadows cast by bridges and signs; for edge noise false detection label, preset small geometric deformations and noise are applied to the image edges to simulate edge artifacts caused by lens distortion.

11. The self-evolutionary method for road litter video detection model according to claim 9, characterized in that, in step six, the execution flow of degradation defense is as follows: W1. Use a fixed validation set to evaluate the trained detection model and obtain core metrics; core metrics include mAP, precision, and recall. W2. Compare each core indicator with the current detection model; based on the core indicators of the current detection model, set the tolerance range for each core indicator. W3. If any core metric of the post-trained detection model falls below the corresponding tolerance range mentioned above, the post-trained detection model will be automatically prevented from going online, and the reason for degradation will be recorded in the training log for subsequent diagnosis. If all core metrics of the post-trained detection model are not lower than the corresponding tolerance range mentioned above, the post-trained detection model will be regarded as a detection model that has passed the degradation defense.

12. The self-evolutionary method for a video detection model of road debris according to claim 11, Its characteristic is that, in step seven, the execution flow of grayscale verification and hot-loading deployment is as follows: Y1. Use the detection model that has passed the degradation defense as the new model, and use the current detection model as the old model; select a few cameras to deploy the new model, and retain the old model for the remaining cameras; Y2. After a preset grayscale run time, compare the detection results of the new model and the old model on the same video stream. Y3. If the false alarm rate or false negative rate exceeds the corresponding preset threshold, the cameras selected in Y1 will be automatically rolled back to the old model; if the false alarm rate and false negative rate do not exceed the corresponding preset threshold, the new model will pass the grayscale verification. Y4. After the new model that has passed the gray-scale verification is manually confirmed, it will be fully launched online. Y5. Push the new model file to each preset node. Each preset node will automatically hot-load the new model after detecting changes in the model file, ensuring uninterrupted production and testing services throughout the process.

13. A self-evolving system for video detection of road debris, characterized in that, The self-evolutionary method for detecting road debris video according to any one of claims 1 to 12, the system comprising the following modules: The automatic production alarm data feedback module is used to perform automatic feedback of production alarm data. The multi-dimensional feature signature extraction and automatic pre-classification of false detection modules are used to perform multi-dimensional feature signature extraction and automatic pre-classification of false detection. The tiered review and hard negative sample mining module is used to perform tiered review and hard negative sample mining. The active learning sampling module is used to perform active learning sampling. The incremental training module is used to perform incremental training; The degradation defense module is used to perform degradation defense. The grayscale verification and hot-load deployment module is used to perform grayscale verification and hot-load deployment.

Images