Tunnel spray intelligent detection method based on double inhibition and spatiotemporal consistency

Abstract

The application discloses a tunnel throwing object intelligent detection method based on double inhibition and space-time consistency, and belongs to the technical field of tunnel throwing object intelligent detection. In order to solve the problem of insufficient stability of tunnel throwing object detection, the application collects tunnel monitoring video streams or picture sequences to obtain input source data; a plurality of frames of samples are extracted according to a preset time interval, image processing is carried out to generate a reference image without dynamic target shielding; dynamic target and shadow inhibition is carried out on a current image to be detected to obtain a first inhibited image corresponding to the current image to be detected; ROI (Region of Interest) filtering processing is carried out to obtain an ROI image retaining a core area of a lane, candidate area extraction based on static linear interference elimination is carried out to obtain a candidate area set; through double constraints of spatial stability and time continuity, event level determination is carried out to generate a throwing object event result. The application is suitable for automatic detection of road surface throwing objects under fixed angle tunnel monitoring conditions.

Description

Technical Field

[0001] This invention belongs to the field of intelligent detection technology for tunnel debris, specifically relating to an intelligent detection method for tunnel debris based on dual inhibition and spatiotemporal consistency. Background Technology

[0002] Tunnel scenarios are characterized by limited lighting, uneven brightness distribution, strong reflections from vehicle headlights, high vehicle speeds, long-term continuous video feeds, and relatively fixed background structures. The presence of debris such as cargo fragments, tire scraps, household garbage, and fallen vehicle parts on the road surface can easily trigger traffic accidents such as tire blowouts, emergency maneuvers, and rear-end collisions. Therefore, a debris detection method capable of continuous, stable operation with low false alarm rates under fixed camera conditions is needed.

[0003] Most existing road debris detection solutions are derived from open road scenarios, typically employing methods such as single-layer target detection, fixed background subtraction, rectangular region of interest filtering, and simple area threshold screening. These solutions lack targeted handling for specific interferences such as headlight spots in tunnels, vehicle shadows, lane lines on curves, edges of maintenance walkways, and obstructions from fixed equipment. This results in high false alarm rates, poor stability, and difficulty in meeting the requirements for continuous 24 / 7 online operation. Summary of the Invention

[0004] This invention aims to address the problems of insufficient stability and high long-term operating costs in tunnel debris detection, and proposes an intelligent tunnel debris detection method based on dual inhibition and spatiotemporal consistency.

[0005] To achieve the above objectives, the present invention provides the following technical solution:

[0006] A smart detection method for tunnel debris based on dual inhibition and spatiotemporal consistency includes the following steps:

[0007] S1. Acquire tunnel monitoring video streams or image sequences to obtain input source data;

[0008] S2. Extract multiple frames of samples from the input source data obtained in step S1 at preset time intervals, and perform image processing to generate a reference image Ref without dynamic target occlusion;

[0009] S3. For the current image to be detected in the input source data obtained in step S1, use the Ref obtained in step S2 to perform dynamic target and shadow suppression to obtain the first suppression image corresponding to the current image to be detected.

[0010] S4. Perform ROI region of interest filtering on the first suppression image obtained in step S3 to obtain an ROI image that retains the core lane region;

[0011] S5. For the ROI image of the core area of ​​the retained lane obtained in step S4, use the Ref obtained in step S2 to extract candidate regions based on static linear interference removal to obtain a set of candidate regions.

[0012] S6. For the candidate region set obtained in step S5, perform event-level determination by combining spatial stability and temporal persistence constraints to generate the spill event result;

[0013] S7. Perform validity monitoring on the reference image Ref obtained in step S2 without dynamic target occlusion. When the average brightness shift or the proportion of dynamically replaced areas in the Ref exceeds the set threshold, execute step S2 to regenerate the Ref.

[0014] Furthermore, the specific implementation method of step S2 includes the following steps:

[0015] S2.1. For the input source data obtained in step S1, extract K detection sample frames at a preset time interval, where K ranges from 3 to 6 frames.

[0016] S2.2. For the detection sample frames in step S2.1, dynamic target detection is performed using a target detection model. The target detection model includes a first detection model for fast detection and a second detection model for compensated detection, which respectively detect vehicles, pedestrians and non-motorized vehicles to obtain a set of detection boxes. Each detection box is then expanded outward by a preset expansion amount to obtain an expanded detection box to cover the vehicle body edge, shadow and trailing area.

[0017] S2.3. Select one detection sample frame processed in step S2.2 as the base frame, and determine the area covered by the expanded detection box in the base frame as the area to be repaired r;

[0018] S2.4. For the remaining detection sample frames processed in step S2.2, extract the corresponding positions of the regions to be repaired r obtained in step S2.3, and use them as candidate patches P. r ;

[0019] S2.5. For the candidate patches obtained in step S2.4, score each candidate patch based on a combination of brightness similarity, contrast similarity, and edge matching degree, and extract the candidate patch with the highest score.

[0020] S2.6. Select the candidate patch with the highest score for each region to be repaired and perform edge fusion to generate a reference image Ref without dynamic target occlusion.

[0021] Furthermore, the specific implementation method of step S3 includes the following steps:

[0022] S3.1. For the current image to be detected in the input source data obtained in step S1, the first detection model is used for detection to obtain the first target mask M. t (1) ;

[0023] S3.2. For the current image to be detected, set M t (1) The pixels in the covered area are replaced with pixels at the same position in the reference image Ref, which is free from dynamic target occlusion, to obtain the intermediate image I. t (1) ;

[0024] S3.3. Use the second detection model to process the intermediate image I t (1) Perform detection to obtain the second target mask M t (2) ;

[0025] S3.4. For the intermediate image, M t (2) The covered area is then replaced with the corresponding area in Ref, resulting in the first suppression image I. t d .

[0026] Furthermore, the specific implementation method of step S4 includes the following steps:

[0027] S4.1. Read the ROI configuration file containing the coordinates of the polygon vertices and generate a binary mask M with the same dimensions as the image. r M r The internal pixel value is 1, and the external pixel value is 0;

[0028] S4.2. Compare the first suppression image obtained in step S3 with M r Perform pixel-wise multiplication to obtain the ROI image I that preserves the core lane region. t r .

[0029] Furthermore, the specific implementation method of step S5 includes the following steps:

[0030] S5.1. Grayscale conversion and normalization: Convert the ROI image of the core area of ​​the retained lane obtained in step S4 and the Ref obtained in step S2 into grayscale images respectively, and then perform normalization processing to obtain the normalized grayscale image of the current frame and the normalized grayscale image of the reference image.

[0031] S5.2. Adaptive Differentiation: Calculate the absolute difference between the normalized grayscale image of the current frame and the normalized grayscale image of the reference image, and generate an adaptive threshold θ based on the brightness offset. t θ t As the absolute difference diagram Dt The binarization threshold is determined; then based on D t With θ t The comparison results are binarized to obtain the difference foreground binary map F. t ;

[0032] S5.3. Static linear feature extraction: Perform probabilistic Hough linear transformation on the difference foreground binary map obtained in step S5.2 to extract the set of line segments corresponding to lane lines and road markings;

[0033] S5.4. Segment Mask Removal: Generate a segment mask L based on the segment set. t Linear static interference is removed from the difference foreground binary image to obtain the static interference removal result image Clean. t ;

[0034] S5.5. Morphological Closure Operation and Connected Component Analysis: Cleaning the Result of Static Interference Removal t Perform a closing operation to extract the connected components and calculate their area, center point, and circumscribed rectangle;

[0035] S5.6. For the connected components extracted in step S5.5, retain those whose area is within a preset range and whose overlap ratio with the fixed ignore region is lower than the overlap ratio threshold ρ0, to obtain the candidate region set C. t .

[0036] Furthermore, the specific implementation method of step S6 includes the following steps:

[0037] S6.1. Spatial Consistency Filtering: Maintenance length is N w For the historical center point cache, for each candidate region in the candidate region set obtained in step S5, count the number of nearest neighbors of the center point of the candidate region that satisfy the spatial distance constraint within the historical center point cache.

[0038] S6.2. Timing consistency check: Maintenance length is N w A Boolean sliding window is used to record whether there are candidate regions that pass the spatial consistency filter in each frame, and the number of frames that have candidate regions that pass the spatial consistency filter within the Boolean sliding window is counted as the number of valid frames.

[0039] S6.3. Set spatial consistency threshold and temporal consistency threshold, and judge the statistical results obtained in steps S6.1 and S6.2 respectively. If the results are greater than the spatial consistency threshold and temporal consistency threshold, generate a spill event and output the timestamp, candidate box position and visual evidence frame.

[0040] Furthermore, the specific implementation method of step S7 includes the following steps:

[0041] S7.1. Reference Image Validity Monitoring: Within the ROI corresponding to the core region of the retained lane, calculate the normalized grayscale image N of the current frame. t Normalized grayscale image N compared to the reference image ref Average brightness offset ξ between t And the proportion η of the dynamic replacement region of the union of the first and second layer region replacement masks. t ;

[0042] S7.2. Trigger update when ξ t or η t Continuous M v When the number of frames exceeds the corresponding threshold, a mismatch in the reference image is determined, triggering the re-execution of step S2, where M v The threshold for the number of consecutive mismatched frames;

[0043] S7.3. Set the condition for condition-triggered disk writing to be that the current frame is determined to be a valid event frame and the candidate region set C t When not empty, execute the writing of screenshots, logs, and event structured data to disk.

[0044] The beneficial effects of this invention are:

[0045] This invention discloses an intelligent tunnel debris detection method based on dual suppression and spatiotemporal consistency. First, a reusable reference image is constructed using unobstructed areas from multiple frames of samples. Then, dual suppression is implemented around this reference image. The first suppression removes dynamic targets and their shadow interference, while the second suppression eliminates static linear interference and fixed ignored areas. Next, small-sized stationary foreign objects are extracted at the candidate region level, and event-level determination is completed based on their spatial stability and temporal persistence. Finally, long-term online operation capability is ensured through reference image validity monitoring and conditional triggering output. Thus, this invention forms a dedicated detection chain revolving around "reference image construction—dual suppression—candidate region extraction—spatiotemporal consistency event determination—long-term operation optimization," suitable for automatic detection of road debris under fixed-view tunnel monitoring conditions.

[0046] This invention presents an intelligent detection method for tunnel debris based on dual inhibition and spatiotemporal consistency, proposing a "constructed reference image" generation mechanism. Unlike directly using a single-frame background image or continuously updating the background model, this invention generates a highly reliable reference image suitable for tunnel scenarios through multi-frame sampling, dynamic target detection and expansion, candidate patch extraction, patch quality scoring, and optimal patch stitching. This image can be reused in subsequent detection processes, thus transforming the background reference from a static input object into a computationally constructed technical object.

[0047] This invention presents an intelligent tunnel debris detection method based on dual suppression and spatiotemporal consistency, proposing a "dual suppression" structure centered around a reference image. The first layer of suppression targets vehicles, pedestrians, non-motorized vehicles, and shadow trails, removing dynamic interference through two layers of detection and region replacement. The second layer of suppression targets lane lines, road markings, and fixed neglected areas, removing static interference through linear feature masks and neglected area filtering, thereby suppressing dynamic and static interference separately within the same detection link.

[0048] This invention presents an intelligent detection method for tunnel debris based on dual suppression and spatiotemporal consistency, proposing an event-level judgment mechanism based on the physical characteristics of the debris. Utilizing the objective characteristics of real debris being relatively stable in spatial location and continuously existing in time, and through joint constraints of spatial and temporal consistency, single-frame candidate regions are elevated to event-level output, significantly suppressing false alarms caused by instantaneous light spots, noise, and short-term occlusion.

[0049] The present invention discloses an intelligent detection method for tunnel debris based on dual inhibition and spatiotemporal consistency. It proposes a reference image validity monitoring and condition-triggered output mechanism, which reduces invalid reconstruction and invalid disk writing while ensuring the adaptability of the reference image, thereby enhancing the long-term operational feasibility of the scheme. Attached Figure Description

[0050] Figure 1 This is a flowchart of an intelligent detection method for tunnel debris based on dual inhibition and spatiotemporal consistency, as described in this invention. Detailed Implementation

[0051] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only for explaining the invention and are not intended to limit the invention; that is, the described specific embodiments are merely a part of the embodiments of the invention, and not all of them. The components of the specific embodiments of the invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations, and the invention may also have other embodiments.

[0052] Therefore, the following detailed description of specific embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected specific embodiments of the invention. All other specific embodiments obtained by those skilled in the art based on these specific embodiments without inventive effort are within the scope of protection of this invention.

[0053] To further understand the invention's content, features, and effects, the following specific embodiments are provided, along with accompanying drawings. Figure 1Detailed explanation is as follows:

[0054] Example 1:

[0055] A smart detection method for tunnel debris based on dual inhibition and spatiotemporal consistency includes the following steps:

[0056] S1. Acquire tunnel monitoring video streams or image sequences to obtain input source data;

[0057] S2. Extract multiple frames of samples from the input source data obtained in step S1 at preset time intervals, and perform image processing to generate a reference image Ref without dynamic target occlusion;

[0058] Furthermore, the specific implementation method of step S2 includes the following steps:

[0059] S2.1. For the input source data obtained in step S1, extract K detection sample frames at a preset time interval, where K ranges from 3 to 6 frames.

[0060] Furthermore, the preset time interval is denoted as ΔT, which can be 0.5-3s; the base frame is preferably the middle time frame among the K sample frames. When an auxiliary sample frame is still covered by the extended detection box at the corresponding position of the area to be repaired, or when the candidate patch exceeds the image boundary, the candidate patch corresponding to the auxiliary sample frame is discarded and does not participate in the quality scoring of the area to be repaired.

[0061] S2.2. For the detection sample frames in step S2.1, dynamic target detection is performed using a target detection model. The target detection model includes a first detection model for fast detection and a second detection model for compensated detection, which respectively detect vehicles, pedestrians and non-motorized vehicles to obtain a set of detection boxes. Each detection box is then expanded outward by a preset expansion amount to obtain an expanded detection box to cover the vehicle body edge, shadow and trailing area.

[0062] Furthermore, the target detection model includes a first detection model and a second detection model. The first detection model, denoted as D1, is a lightweight dynamic target detection model for fast full-image screening of the current frame to be detected, preferably using a YOLO11n structure. The second detection model, denoted as D2, is a high-recall dynamic target detection model for residual target compensation detection in the intermediate image after the first-layer region replacement, preferably using a YOLO11s structure. Both D1 and D2 detect vehicles, pedestrians, and non-motorized vehicles. The input is a scaled and normalized RGB image, and the output is a set of detection boxes B={(x1,y1,x2,y2,c,s)}, where x1, y1, x2, and y2 represent the target bounding box coordinates, c represents the target category label, and s represents the target confidence score. Preferably, D1 uses a first parameter set biased towards real-time performance, and D2 uses a second parameter set biased towards recall. The first and second parameter sets differ in at least one of the following: input resolution, confidence threshold, non-maximum suppression threshold, or minimum target size constraint. Preferably, the confidence threshold τ1 of D1 is set to 0.35-0.50, and the confidence threshold τ2 of D2 is set to 0.20-0.35, so that the second detection model has a higher recall rate for small-sized, edge-located, partially occluded, and shadow-adjacent targets. The dual-model structure of D1 and D2 allows the initial rapid detection and residual target compensation detection to be optimized for different target distributions and real-time requirements, unlike the approach of using a single model to handle both tasks simultaneously.

[0063] Furthermore, each sample frame is first input into the first detection model D1 to perform fast full-image detection, resulting in the first set of detection boxes B. k (1) Then, using the second parameter set with high resolution and low threshold, the same sample frame is input into the second detection model D2 to perform compensated detection, resulting in the compensated detection box set B. k (2) D1 is a lightweight single-stage YOLO11n target detection network, used to prioritize the detection of vehicles, pedestrians, and non-motorized vehicles with large areas and clear outlines; D2 is a high-recall compensation YOLO11s detection network, used to compensate for small-sized targets, image edge targets, partially occluded targets, and residual dynamic targets with low contrast to the background that were missed in the first round of detection. Detection boxes with confidence scores satisfying s≥τ1 or s≥τ2 are retained respectively, and the two sets of detection boxes are merged to form the detection box set B of the sample frames. k Then, each detection frame is expanded outward by a preset expansion amount to cover the vehicle body edge, shadow and trailing area;

[0064] Preferably, the set of detection boxes B of the sample frame k From the first set of detection frames B k (1) With the compensated detection box set Bk (2) The result is obtained by taking the union of the sets. For the detection boxes in the union that have an intersection-union ratio greater than a preset threshold and the same category label, those with higher confidence can be retained to avoid repeated expansion.

[0065] Furthermore, the detection box can be expanded in the following form: ;

[0066] Where n represents the detection box number; ExBox n This represents the nth detection box after expansion; x n (1) y n (1) These represent the x and y coordinates of the top-left corner of the original detection box, respectively; x n (2) y n (2) These represent the x and y coordinates of the lower right corner of the original detection frame, respectively; d x d y represents the horizontal expansion and vertical expansion respectively; W and H represent the width and height of the input image respectively; max(·) and min(·) represent taking the larger value and taking the smaller value respectively.

[0067] S2.3. Select one detection sample frame processed in step S2.2 as the base frame, and determine the area covered by the expanded detection box in the base frame as the area to be repaired r;

[0068] S2.4. For the remaining detection sample frames processed in step S2.2, extract the corresponding positions of the regions to be repaired r obtained in step S2.3, and use them as candidate patches P. r ;

[0069] S2.5. For the candidate patches obtained in step S2.4, score each candidate patch based on a combination of brightness similarity, contrast similarity, and edge matching degree, and extract the candidate patch with the highest score.

[0070] Furthermore, the following scoring method is preferred:

[0071] ;

[0072] ;

[0073] ;

[0074] ;

[0075] ;

[0076] Where r represents the r-th region to be repaired in the base frame; k represents the k-th candidate patch corresponding to the r-th region to be repaired; Q rk α represents the overall quality score of the k-th candidate patch for the r-th region to be patched; L α C α E These represent the weighting coefficients for the brightness, contrast, and edge parameters, respectively, and the sum of the three is 1; L rk Indicates brightness similarity; C rk Indicates contrast similarity; E rk Indicates edge matching degree; μ Br μ represents the average gray value of the r-th region to be repaired in the base frame. Prk σ represents the average grayscale value of the k-th candidate patch at the corresponding position; Br σ represents the standard deviation of the grayscale value of the r-th region to be repaired in the base frame; Prk N represents the standard deviation of the grayscale value at the corresponding position of the k-th candidate patch; ∩rk N represents the number of overlapping edge pixels between the r-th region to be repaired and the k-th candidate patch in the edge map of the base frame; Br N represents the number of edge pixels in the r-th region to be repaired in the base frame; Prk This represents the number of edge pixels at the corresponding position of the k-th candidate patch; ε1, ε2, and ε3 represent positive constants to prevent the denominator from being zero; k * Indicates that Q rk The candidate patch number that yields the maximum value; max(·) indicates taking the larger value; argmax k This indicates the number of the independent variable that maximizes the objective function.

[0077] S2.6. Select the candidate patch with the highest score for each area to be repaired and perform edge fusion to generate a reference image Ref without dynamic target occlusion;

[0078] Furthermore, during stitching, a fusion weight W is applied to the transition region between the optimal patch and the base frame edge. r (u,v) is used to reduce the visibility of the seam; when the reference image cache exists and passes the validity check, the subsequent detection steps can directly reuse the reference image.

[0079] S3. For the current image to be detected in the input source data obtained in step S1, use the Ref obtained in step S2 to perform dynamic target and shadow suppression to obtain the first suppression image corresponding to the current image to be detected.

[0080] Furthermore, the specific implementation method of step S3 includes the following steps:

[0081] S3.1. For the current image to be detected in the input source data obtained in step S1, the first detection model is used for detection to obtain the first target mask M. t (1) ;

[0082] Furthermore, the first detection model D1 performs fast full-image detection on the current frame. D1 is a lightweight single-stage object detection network based on YOLO11n, employing a first parameter set biased towards real-time performance to prioritize the detection of vehicles, pedestrians, and non-motorized vehicles with larger areas and clearer outlines. Detection boxes with a confidence level not lower than the threshold τ1 are rasterized into a first object mask Mt. (1) ;

[0083] S3.2. For the current image to be detected, set M t (1) The pixels in the covered area are replaced with pixels at the same position in the reference image Ref, which is free from dynamic target occlusion, to obtain the intermediate image I. t (1) ;

[0084] S3.3. Use the second detection model to process the intermediate image I t (1) Perform detection to obtain the second target mask M t (2) ;

[0085] Furthermore, the second detection model D2 is used to analyze the intermediate image I. t (1) Residual target compensation detection is performed, where D2 is a YOLO11s high recall compensation detection network, using a second parameter set biased towards recall; vehicle, pedestrian, and non-motorized vehicle detection boxes with confidence levels not lower than the threshold τ2 are rasterized into a second target mask M. t (2) It is used to compensate for small-sized targets, image edge targets, partially occluded targets, and residual dynamic targets with low contrast to the background that were missed by the first layer of detection.

[0086] Furthermore, let B be the set of detection boxes output by the first detection model D1. t (1) ={b ti (1) The set of detection boxes output by the second detection model D2 is B. t (2) ={b ti (2)}, where each detection box b=(x1,y1,x2,y2,c,s). D1 performs fast detection on the current frame or sample frame using the first parameter set, and D2 performs compensated detection under a high recall configuration in the intermediate image or the same frame using the second parameter set; the detection boxes that meet the confidence threshold are expanded and rasterized to obtain the first target mask M. t (1) With the second target mask M t (2) .

[0087] S3.4. For the intermediate image, M t (2) The covered area is then replaced with the corresponding area in Ref, resulting in the first suppression image I. t d ;

[0088] Furthermore, the following pixel replacement calculation is used:

[0089]

[0090]

[0091] Where t represents the current frame number to be detected; u and v represent pixel coordinates; M t (1) (u,v) and M t (2) (u,v) represent the values ​​of the replacement masks in the first and second layers at pixel (u,v), respectively, and can be either 0 or 1; t (u,v) represents the pixel value at pixel (u,v) of the current frame to be detected; I t (1) (u,v) represents the pixel value at pixel (u,v) in the intermediate image after the first layer region replacement; I t d (u,v) represents the pixel value at pixel (u,v) of the first-level suppressed image after the second-level region replacement; Ref(u,v) represents the pixel value at pixel (u,v) of the reference image.

[0092] S4. Perform ROI region of interest filtering on the first suppression image obtained in step S3 to obtain an ROI image that retains the core lane region;

[0093] Furthermore, the specific implementation method of step S4 includes the following steps:

[0094] S4.1. Read the ROI configuration file containing the coordinates of the polygon vertices and generate a binary mask M with the same dimensions as the image. r M rThe internal pixel value is 1, and the external pixel value is 0;

[0095] Furthermore, the ROI configuration file uses a pixel coordinate sequence [(x1,y1),(x2,y2),…,(xp,yp)] stored in vertex order, with each vertex arranged in clockwise or counterclockwise order and closed at both ends; when the configuration file contains multiple polygonal regions, the union of each polygonal region is taken to generate the ROI mask M. r .

[0096] S4.2. Compare the first suppression image obtained in step S3 with M r Perform pixel-wise multiplication to obtain the ROI image I that preserves the core lane region. t r .

[0097] Furthermore, the calculation formula is as follows:

[0098]

[0099] Among them, I t r (u,v) represents the pixel value at pixel (u,v) in the ROI image; I t d (u,v) represents the pixel value at pixel (u,v) in the first-level suppressed image; M r (u,v) represents the value of the ROI mask at pixel (u,v), which can be 0 or 1; t represents the current frame number to be detected; u and v represent pixel coordinates; the superscript r indicates that the mask and image belong to the ROI filtering step. The advantage of using polygonal ROIs is that they can adapt to curves, slopes, and irregular lane boundaries, avoiding the invalid background introduced by rectangular ROIs.

[0100] S5. For the ROI image of the core area of ​​the retained lane obtained in step S4, use the Ref obtained in step S2 to extract candidate regions based on static linear interference removal to obtain a set of candidate regions.

[0101] Furthermore, the specific implementation method of step S5 includes the following steps:

[0102] S5.1. Grayscale conversion and normalization: Convert the ROI image of the core area of ​​the retained lane obtained in step S4 and the Ref obtained in step S2 into grayscale images respectively, and then perform normalization processing to obtain the normalized grayscale image of the current frame and the normalized grayscale image of the reference image.

[0103] S5.2. Adaptive Differentiation: Calculate the absolute difference between the normalized grayscale image of the current frame and the normalized grayscale image of the reference image, and generate an adaptive threshold θ based on the brightness offset. t θ tAs the absolute difference diagram D t The binarization threshold is determined; then based on D t With θ t The comparison results are binarized to obtain the difference foreground binary map F. t ;

[0104] S5.3. Static linear feature extraction: Perform probabilistic Hough linear transformation on the difference foreground binary map obtained in step S5.2 to extract the set of line segments corresponding to lane lines and road markings;

[0105] Furthermore, the minimum line length of the probabilistic Hough linear transform is denoted as L. min The maximum fracture tolerance is denoted as G. max Only line segments that intersect the lane extension direction within a preset angle range are retained as static linear features, and each line segment is expanded by a preset width wL to generate a line segment mask L. t This covers the actual width of lane lines and road markings.

[0106] S5.4. Segment Mask Removal: Generate a segment mask L based on the segment set. t Linear static interference is removed from the difference foreground binary image to obtain the static interference removal result image Clean. t ;

[0107] S5.5. Morphological Closure Operation and Connected Component Analysis: Cleaning the Result of Static Interference Removal t Perform a closing operation to extract the connected components and calculate their area, center point, and circumscribed rectangle;

[0108] S5.6. For the connected components extracted in step S5.5, retain those whose area is within a preset range and whose overlap ratio with the fixed ignore region is lower than the overlap ratio threshold ρ0, to obtain the candidate region set C. t .

[0109] Furthermore, the following calculation method is preferred:

[0110] ;

[0111] ;

[0112] ;

[0113] ;

[0114] ;

[0115] ;

[0116] ;

[0117] ;

[0118] In the formula, t represents the current frame number to be detected; u and v represent pixel coordinates; R t (u,v), G t (u,v), B t (u, v) represent the ROI image I, respectively. t r The red, green, and blue channel values ​​at pixel (u,v); R ref (u,v), G ref (u,v), B ref (u,v) represent the red, green, and blue channel values ​​of the reference image Ref at pixel (u,v), respectively; Y t (u,v) and Y ref (u,v) represent grayscale values ​​respectively; N t (u,v) and N ref (u,v) represent the normalized gray values; Y t min Y t max Y ref min Y ref max These represent the minimum and maximum values ​​of the corresponding grayscale image within the ROI, respectively; D t (u,v) represents the value of the absolute difference image at pixel (u,v); θ t This represents the adaptive difference threshold for the current frame and is used as the absolute difference map D. t The binarization threshold is θ0; θ0 represents the base difference threshold; λ represents the brightness offset adjustment coefficient; μ t and μ ref N t With N ref Average gray value within the ROI; F t (u,v) represents the value at pixel (u,v) in the difference foreground binary image; L t (u,v) represents the value of the linear interference mask at pixel (u,v); Clean t (u,v) represents the value at pixel (u,v) in the static interference removal result image; ε4 represents a positive constant to prevent the denominator from being zero.

[0119] In the connected component filtering stage, let the j-th connected component be C. tj Its area is A tj , and the q-th fixed neglect region Z q The overlap ratio is:

[0120] ;

[0121] Among them, C tj A represents the j-th connected component in the t-th frame; tj Represents the connected component C tj The area of ​​Z; q Represents the q-th fixed ignored region; ρ tjq Represents the connected component C tj With fixed neglect region Z q The overlap ratio; Area(·) represents area operation; ε5 represents a positive constant to prevent the denominator from being zero; A min and A max These represent the minimum and maximum area thresholds for the candidate regions, respectively; ρ0 represents the overlap ratio threshold; C t This represents the final set of candidate regions that are retained.

[0122] S6. For the candidate region set obtained in step S5, perform event-level determination by combining spatial stability and temporal persistence constraints to generate the spill event result;

[0123] The input for this step is the candidate region set C. t The output is a standardized event result. Since real spilled material usually has a relatively stable position in a short period of time, while headlight flashes, noise spots, and momentary reflections do not have this characteristic, this step achieves event-level determination through the dual constraints of "spatial stability + temporal persistence".

[0124] Furthermore, the specific implementation method of step S6 includes the following steps:

[0125] S6.1. Spatial Consistency Filtering: Maintenance length is N w For the historical center point cache, for each candidate region in the candidate region set obtained in step S5, count the number of nearest neighbors of the center point of the candidate region that satisfy the spatial distance constraint within the historical center point cache.

[0126] Furthermore, the center point ctj of the current candidate region is taken as the arithmetic mean of the coordinates of the upper left corner and the lower right corner of the bounding rectangle of the candidate region; if the current candidate region and multiple historical candidate regions simultaneously satisfy the spatial distance constraint, then the historical candidate region with the smallest Euclidean distance is selected to participate in the nearest neighbor statistics.

[0127] S6.2. Timing consistency check: Maintenance length is N w A Boolean sliding window is used to record whether there are candidate regions that pass the spatial consistency filter in each frame, and the number of frames that have candidate regions that pass the spatial consistency filter within the Boolean sliding window is counted as the number of valid frames.

[0128] S6.3. Set spatial consistency threshold and temporal consistency threshold, and judge the statistical results obtained in steps S6.1 and S6.2 respectively. If the results are greater than the spatial consistency threshold and temporal consistency threshold, generate a spill event and output the timestamp, candidate box position and visual evidence frame.

[0129] Furthermore, the following determination method is preferred:

[0130]

[0131] k tj =1, when n tj ≥K s Otherwise k tj =0

[0132] b t =1, when Σ j=1 Mt k tj ≥1; otherwise b t =0

[0133]

[0134] e t =1, when h t ≥K t Otherwise e t =0

[0135] Where t represents the current frame number to be detected; j represents the candidate region number in the current frame; s represents the historical frame number; m represents the candidate region number in the historical frame; N w M represents the length of the spatiotemporal consistency determination window. s c represents the number of candidate regions in the s-th frame; tj c represents the coordinates of the center point of the j-th candidate region in the t-th frame; sm d represents the coordinates of the center point of the m-th candidate region in the s-th frame; ||·||2 represents the Euclidean norm; d s Indicates the spatial distance threshold; Ind(·) represents the indicator function, which takes the value 1 when the condition within the parentheses is true, and 0 otherwise; n tj K represents the number of nearest neighbors of the current candidate region that satisfy the spatial distance constraint within the historical window; s Indicates the spatial consistency threshold; k tj A flag indicating whether a candidate region has passed the spatial consistency determination; b t A frame-level flag indicating whether the current frame contains candidate regions that pass spatial consistency determination; h t K represents the number of valid frames within the time window. t Indicates the timing consistency threshold; e tThis is an event flag indicating whether the current frame constitutes a valid spill event. When e t When =1, the event number, frame number, timestamp, candidate box coordinates, candidate box size, and labeled evidence frame are output; if necessary, short segments of several frames before and after the event can be output simultaneously for verification.

[0136] S7. Perform validity monitoring on the reference image Ref obtained in step S2 without dynamic target occlusion. When the average brightness shift or the proportion of dynamically replaced areas in the Ref exceeds the set threshold, execute step S2 to regenerate the Ref.

[0137] Furthermore, the specific implementation method of step S7 includes the following steps:

[0138] S7.1. Reference Image Validity Monitoring: Within the ROI corresponding to the core region of the retained lane, calculate the normalized grayscale image N of the current frame. t Normalized grayscale image N compared to the reference image ref Average brightness offset ξ between t And the proportion η of the dynamic replacement region of the union of the first and second layer region replacement masks. t ;

[0139] S7.2. Trigger update when ξ t or η t Continuous M v When the number of frames exceeds the corresponding threshold, a mismatch in the reference image is determined, triggering the re-execution of step S2, where M v The threshold for the number of consecutive mismatched frames;

[0140] Furthermore, the average brightness offset threshold is denoted as ξ0, the dynamic replacement region proportion threshold is denoted as η0, and the number of consecutive mismatched frames is denoted as M. v When ξ t Continuous M v Frame satisfies ξ t >ξ0, or η t Continuous M v Frames satisfy η t When η > 0, the current reference image cache is determined to be mismatched. After triggering the update, the old reference image cache is marked as invalid, and the cache is updated atomically after the new reference image is generated, in order to avoid cross-use of old and new reference images during the detection process.

[0141] S7.3. Set the condition for condition-triggered disk writing to be that the current frame is determined to be a valid event frame and the candidate region set C t When not empty, execute the writing of screenshots, logs, and event structured data to disk.

[0142] Furthermore, the calculation formula is as follows:

[0143]

[0144]

[0145] Where, ξ t η represents the average brightness offset between the reference image and the current scene. t N represents the ratio of the union area of ​​the first and second layer region replacement masks to the area of ​​the ROI; t N represents the normalized grayscale image of the current frame within the ROI; ref This represents the normalized grayscale image of the reference image within the ROI; Mean (u,v)∈ROI (·) indicates averaging the pixels within the ROI; Area(·) indicates area calculation; M t (1) and M t (2) These represent the first and second layer region replacement masks, respectively; ROI represents the region of interest; t represents the current frame number to be detected; M v This represents the threshold for the number of consecutive mismatched frames. This design reduces invalid reconstructions and invalid disk writes while ensuring adaptability.

[0146] Further application examples of this embodiment are illustrated below:

[0147] This embodiment is used to illustrate the parameter configuration method of the present invention and does not constitute a limitation on the scope of protection. Preferred parameters are as follows:

[0148] 1. The number of sample frames K is 3-6, preferably 5; the expansion amount d x d y 20-40 pixels can be selected respectively, with 30 pixels being preferred;

[0149] 2. The optimal weight for patch scoring is α. L =0.40、α C =0.30、α E =0.30;

[0150] 3. The basic differential threshold θ0 can be 55-80, preferably 68; the brightness offset adjustment coefficient λ can be 12-25, preferably 18;

[0151] 4. The candidate region area threshold can be set to A. min =40 pixels, A max =600 pixels; the threshold for the overlap ratio of the ignored area, ρ0, can be set to 0.50;

[0152] 5. Spatiotemporal consistency window length N w The value can be 5-8, with 5 being the preferred value; spatial distance threshold d s The resolution can be 40-60 pixels, with 50 pixels being preferred; the spatial nearest neighbor threshold K sThe threshold value can be 3; the time-series decision threshold K t 3 is acceptable;

[0153] 6. In a preferred embodiment, the first detection model D1 employs the YOLO11n lightweight single-stage target detection network, with an input size preferably of 640×640, a confidence threshold preferably of 0.35-0.50, and a non-maximum suppression threshold preferably of 0.45-0.60. This model is used to quickly complete the first round of full-image detection of vehicles, pedestrians, and non-motorized vehicles at the edge. The second detection model D2 employs the YOLO11s high-recall compensation detection network, with an input size preferably of 960×960 or using slice inference. The confidence threshold is preferably of 0.20-0.35, and the non-maximum suppression threshold is preferably of 0.35-0.50. This model is used to re-detect the targets missed in the first round on the image after the first-layer region replacement. D1 and D2 use the same category labeling system but employ different network capacities, input resolutions, threshold parameters, and training sample ratios. D2 adds samples of small targets, edge targets, partially occluded targets, and targets near shadows to improve compensation detection capabilities.

[0154] 7. Preferably, the ROI configuration file is stored in the form of polygon vertex pixel coordinates; the minimum line length L of the probabilistic Hough linear transformation. min Available resolution: 30-60 pixels; maximum fracture tolerance: G max The width of the line segment mask expansion can be 8-15 pixels, and the width wL can be 3-8 pixels.

[0155] 8. Preferably, in the reference image mismatch determination, the average brightness offset threshold ξ0 can be taken as 0.10-0.18, the dynamic replacement region proportion threshold η0 can be taken as 0.12-0.25, and the number of consecutive mismatch frames M v 3-6 is acceptable.

[0156] The technical effects of this embodiment are as follows:

[0157] 1. By generating reference images from multiple frames, scoring candidate patches for quality, and stitching together the best patch, the problem of unstable background in traditional single frames is avoided, thus improving the reliability and reusability of the differential reference.

[0158] 2. By replacing the dynamic target region centered on the reference image, it can simultaneously cover vehicles, pedestrians, non-motorized vehicles and shadow trails, significantly reducing false alarms caused by dynamic targets;

[0159] 3. The second layer of suppression, consisting of static linear feature mask, fixed ignore area and area filtering, significantly reduces false detections caused by the inherent static structure of the tunnel;

[0160] 4. By combining spatial consistency and temporal consistency constraints, the physical characteristics of the relatively stable and continuous existence of the spilled object position can be utilized to upgrade single-frame candidates to event-level output, reducing false alarms caused by headlight flicker, transient noise and short-term occlusion.

[0161] 5. By monitoring the validity of reference images and using condition-triggered disk writing, the system significantly reduces invalid disk writing and maintenance costs during long-term operation, making it suitable for continuous operation 24 / 7.

[0162] 6. By coupling reference image construction, dual suppression, spatiotemporal consistency and long-term operation optimization mechanism into a unified technical link, a special detection scheme suitable for fixed-view tunnel monitoring scenarios is formed.

[0163] It should be noted that relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0164] Although this application has been described above with reference to specific embodiments, various modifications can be made and components can be replaced with equivalents without departing from the scope of this application. In particular, as long as there is no structural conflict, the features in the specific embodiments disclosed in this application can be combined with each other in any way. The lack of an exhaustive description of these combinations in this specification is merely for the sake of brevity and resource conservation. Therefore, this application is not limited to the specific embodiments disclosed herein, but includes all technical solutions falling within the scope of the claims.

Claims

1. A method for intelligent detection of tunnel debris based on dual inhibition and spatiotemporal consistency, characterized in that, Includes the following steps: S1. Acquire tunnel monitoring video streams or image sequences to obtain input source data; S2. Extract multiple frames of samples from the input source data obtained in step S1 at preset time intervals, and perform image processing to generate a reference image Ref without dynamic target occlusion; S3. For the current image to be detected in the input source data obtained in step S1, use the Ref obtained in step S2 to perform dynamic target and shadow suppression to obtain the first suppression image corresponding to the current image to be detected. S4. Perform ROI region of interest filtering on the first suppression image obtained in step S3 to obtain an ROI image that retains the core lane region; S5. For the ROI image of the core area of ​​the retained lane obtained in step S4, use the Ref obtained in step S2 to extract candidate regions based on static linear interference removal to obtain a set of candidate regions. S6. For the candidate region set obtained in step S5, perform event-level determination by combining spatial stability and temporal persistence constraints to generate the spill event result; S7. Perform validity monitoring on the reference image Ref obtained in step S2 without dynamic target occlusion. When the average brightness shift or the proportion of dynamically replaced areas in the Ref exceeds the set threshold, execute step S2 to regenerate the Ref.

2. The intelligent detection method for tunnel debris based on dual inhibition and spatiotemporal consistency according to claim 1, characterized in that, The specific implementation method of step S2 includes the following steps: S2.

1. For the input source data obtained in step S1, extract K detection sample frames at a preset time interval, where K ranges from 3 to 6 frames. S2.

2. For the detection sample frames in step S2.1, dynamic target detection is performed using a target detection model. The target detection model includes a first detection model for fast detection and a second detection model for compensated detection, which respectively detect vehicles, pedestrians and non-motorized vehicles to obtain a set of detection boxes. Each detection box is then expanded outward by a preset expansion amount to obtain an expanded detection box to cover the vehicle body edge, shadow and trailing area. S2.

3. Select one detection sample frame processed in step S2.2 as the base frame, and determine the area covered by the expanded detection box in the base frame as the area to be repaired r; S2.

4. For the remaining detection sample frames processed in step S2.2, extract the corresponding positions of the regions to be repaired r obtained in step S2.3, and use them as candidate patches P. r ; S2.

5. For the candidate patches obtained in step S2.4, score each candidate patch based on a combination of brightness similarity, contrast similarity, and edge matching degree, and extract the candidate patch with the highest score. S2.

6. Select the candidate patch with the highest score for each region to be repaired and perform edge fusion to generate a reference image Ref without dynamic target occlusion.

3. The intelligent detection method for tunnel spills based on dual inhibition and spatiotemporal consistency according to claim 2, characterized in that, The specific implementation method of step S3 includes the following steps: S3.

1. For the current image to be detected in the input source data obtained in step S1, the first detection model is used for detection to obtain the first target mask M. t (1) ; S3.

2. For the current image to be detected, set M t (1) The pixels in the covered area are replaced with pixels at the same position in the reference image Ref, which is free from dynamic target occlusion, to obtain the intermediate image I. t (1) ; S3.

3. Use the second detection model to process the intermediate image I t (1) Perform detection to obtain the second target mask M t (2) ; S3.

4. For the intermediate image, M t (2) The covered area is then replaced with the corresponding area in Ref, resulting in the first suppression image I. t d .

4. The intelligent detection method for tunnel spills based on dual inhibition and spatiotemporal consistency according to claim 3, characterized in that, The specific implementation method of step S4 includes the following steps: S4.

1. Read the ROI configuration file containing the coordinates of the polygon vertices and generate a binary mask M with the same dimensions as the image. r M r The internal pixel value is 1, and the external pixel value is 0; S4.

2. Compare the first suppression image obtained in step S3 with M r Perform pixel-wise multiplication to obtain the ROI image I that preserves the core lane region. t r .

5. The intelligent detection method for tunnel spills based on dual inhibition and spatiotemporal consistency according to claim 4, characterized in that, The specific implementation method of step S5 includes the following steps: S5.

1. Grayscale conversion and normalization: Convert the ROI image of the core area of ​​the retained lane obtained in step S4 and the Ref obtained in step S2 into grayscale images respectively, and then perform normalization processing to obtain the normalized grayscale image of the current frame and the normalized grayscale image of the reference image. S5.

2. Adaptive Differentiation: Calculate the absolute difference between the normalized grayscale image of the current frame and the normalized grayscale image of the reference image, and generate an adaptive threshold θ based on the brightness offset. t θ t As the absolute difference diagram D t The binarization threshold is determined; then based on D t With θ t The comparison results are binarized to obtain the difference foreground binary map F. t ; S5.

3. Static linear feature extraction: Perform probabilistic Hough linear transformation on the difference foreground binary map obtained in step S5.2 to extract the set of line segments corresponding to lane lines and road markings; S5.

4. Segment Mask Removal: Generate a segment mask L based on the segment set. t Linear static interference is removed from the difference foreground binary image to obtain the static interference removal result image Clean. t ; S5.

5. Morphological Closure Operation and Connected Component Analysis: Cleaning the Result of Static Interference Removal t Perform a closing operation to extract the connected components and calculate their area, center point, and circumscribed rectangle; S5.

6. For the connected components extracted in step S5.5, retain those whose area is within a preset range and whose overlap ratio with the fixed ignore region is lower than the overlap ratio threshold ρ0, to obtain the candidate region set C. t .

6. The intelligent detection method for tunnel spills based on dual inhibition and spatiotemporal consistency according to claim 5, characterized in that, The specific implementation method of step S6 includes the following steps: S6.

1. Spatial Consistency Filtering: Maintenance length is N w For the historical center point cache, for each candidate region in the candidate region set obtained in step S5, count the number of nearest neighbors of the center point of the candidate region that satisfy the spatial distance constraint within the historical center point cache. S6.

2. Timing consistency check: Maintenance length is N w A Boolean sliding window is used to record whether there are candidate regions that pass the spatial consistency filter in each frame, and the number of frames that have candidate regions that pass the spatial consistency filter within the Boolean sliding window is counted as the number of valid frames. S6.

3. Set spatial consistency threshold and temporal consistency threshold, and judge the statistical results obtained in steps S6.1 and S6.2 respectively. If the results are greater than the spatial consistency threshold and temporal consistency threshold, generate a spill event and output the timestamp, candidate box position and visual evidence frame.

7. The intelligent detection method for tunnel spills based on dual inhibition and spatiotemporal consistency according to claim 6, characterized in that, The specific implementation method of step S7 includes the following steps: S7.

1. Reference Image Validity Monitoring: Within the ROI corresponding to the core region of the retained lane, calculate the normalized grayscale image N of the current frame. t Normalized grayscale image N compared to the reference image ref Average brightness offset ξ between t And the proportion η of the dynamic replacement region of the union of the first and second layer region replacement masks. t ; S7.

2. Trigger update when ξ t or η t Continuous M v When the number of frames exceeds the corresponding threshold, a mismatch in the reference image is determined, triggering the re-execution of step S2, where M v The threshold for the number of consecutive mismatched frames; S7.

3. Set the condition for condition-triggered disk writing to be that the current frame is determined to be a valid event frame and the candidate region set C t When not empty, execute the writing of screenshots, logs, and event structured data to disk.

Images