A roadside camera-based road litter detection method

Abstract

The application relates to a roadside camera-based road litter detection method, the core of the method is that, aiming at the modeling accuracy problem of a traditional Gaussian mixture model background modeling method on a large-scale target, a multi-scale adaptive road background modeling method is designed based on an image pyramid and an adaptive background difference method, the background modeling accuracy of the original method under different video resolutions is improved, on the other hand, aiming at the problem of insufficient visual feature recognition accuracy of the litter, a multi-level feature screening method of the litter is constructed based on visual features and motion features. The scheme designed by the application has strong scale adaptability and environmental adaptability, can effectively reduce the false alarm and missing report rate of the litter, and improves the detection accuracy of the litter visual detection system.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of intelligent transportation and information perception technology, and mainly relates to a road litter detection method based on a roadside monitoring camera. BACKGROUND

[0002] Among the many factors that induce traffic accidents, the risk of litter is relatively large, and it is difficult to find in time, which can easily induce serious traffic accidents such as vehicle rollover and vehicle out of control, endangering the safety of the driver and passengers.

[0003] The existing litter identification methods are mainly divided into two categories, namely manual screening and visual event identification system. The manual screening method has a large amount of labor, serious information lag, high omission and false report rate, and is difficult to support the needs of accident prevention and control. Compared with manual screening, the visual event identification system has great advantages in detection efficiency and detection range. However, due to the complex and diverse litter and detection environment, the detection accuracy of the existing system still needs to be improved.

[0004] The existing litter detection method basically follows the mode of background modeling and foreground target discrimination. In the background modeling part, the commonly used background modeling methods are mean method, Vibe background model method and Gaussian mixture model method, among which the Gaussian mixture model method has higher detection accuracy and wider application range. Patent CN111339824A discloses a litter extraction method based on this model. However, since the operation object of this method is each pixel, it is difficult to obtain accurate foreground target position when the target size is too large. In the foreground target discrimination part, the common ones are visual feature discrimination method and deep learning model discrimination method. Patent CN111523536A discloses a deep learning litter discrimination model based on Faster-RCNN, however, due to the diversity of litter, only using supervised learning to establish the discrimination model has high false report rate and poor detection accuracy. SUMMARY

[0005] In order to improve the accuracy of the existing road litter detection algorithm in background modeling and foreground discrimination, the present application formulates the following technical scheme:

[0006] Firstly, the existing Gaussian mixture model background establishment method is improved, the image pyramid is introduced as the image input of the Gaussian mixture model, the modeling ability of the Gaussian mixture model for different scale background pictures is improved, so that the stable and reliable road background model is obtained;Secondly, the background difference method is used to obtain the foreground target of the current video frame relative to the road background model, in order to avoid the generation of the hole phenomenon, the image expansion and corrosion algorithm is used to process the foreground target area twice, and the more complete foreground target is obtained;Thirdly, the tracking algorithm is used to track the foreground target, the motion trajectory characteristics of the foreground target in a certain time sequence are obtained, the general target detection deep learning algorithm is used to distinguish the tracked foreground target, and the common targets such as vehicles, pedestrians and non-motor vehicles are filtered out;Finally, the region contrast feature, motion trajectory feature and tracking time length and other information are used to distinguish the remaining foreground target twice, the influence of tree shadow and vehicle shadow is reduced, and the image position of the litter is obtained.

[0007] The specific steps of the method are as follows:

[0008] Step one, obtaining the video stream of the roadside camera, establishing a multi-scale road background model for distinguishing the foreground moving target;

[0009] Substep one: collecting the traffic flow video of the roadside camera, marking the original resolution picture set of the video stream as the first picture set Φ1, using the downsampling algorithm to solve the pyramid image set of the first picture set Φ1, and obtaining Φ2, Φ3,..., Φ N , respectively. i Wherein, Φ N is the i-th picture set, N is a constant, generally 3 or 4, under the premise of ensuring the definition of the down-sampled picture, and the output frame Φ of the video stream is obtained.

[0010] Substep two: based on the video stream output frame Φ obtained in substep one, the parameter information of the Gaussian mixture model is initialized, in order to reduce the motion target interference in the parameter initialization process, the number of Gaussian mixture models is set as K, the multi-frame average value method is used as the initialization parameter, and the number of multi-frame is T K , the pixel value X t in the output frame Φ at t time is calculated as follows:

[0011]

[0012]

[0013] In order to improve the execution speed of the algorithm initialization process, only the first Gaussian distribution function is allocated with the weight w1=1 / K, and the mean value μ i, variance σ i and weight w i are all 0;

[0014] Sub-step three: set the updating method of Gaussian mixture model, sort all pixel points corresponding to the Gaussian model according to w i,t-1 / σ i,t-1 , and use formula (3) in order until the matching Gaussian component is found:

[0015] |X t -μ i,t-1 |<δσ i,t-1 (3), Where δ is a set threshold, if the corresponding matching component cannot be found in all Gaussian distribution models, the last Gaussian distribution model in the sorting is re-established by using the method in sub-step two, if the current pixel point matches the Gaussian model, then the related parameters of the first matching Gaussian distribution are updated, and the calculation formula is as follows:

[0016] w i,t =(1-α)w i,t-1 +α(1-c′) (4)

[0017] μ i,t =(1-ρ i,t-1 )μ i,t-1 +ρ i,t-1 X t (5)

[0018]

[0019] ρ i,t-1 =α / w i,t-1 (7)

[0020] c′=c·α (8)

[0021] Where α is the learning rate of the model, the greater α is, the faster the updating speed of the background model is; ρ i,t-1 is the parameter learning rate of the model, and c is the prior coefficient, and except for the first matching Gaussian distribution, the remaining Gaussian distributions only update the weight, and the calculation formula is as follows:

[0022] w i,t =(1-α)w i,t-1 +αc′ (9)

[0023] Sub-step four: set the background model generation method of Gaussian mixture model, sort K Gaussian models according to the weight, and select the first B t gaussian models as the basis of the background model, and the calculation formula of B t is as follows:

[0024]

[0025] where c f is the background threshold value, used to describe the maximum Gaussian component proportion of the foreground target.

[0026] Sub-step five: obtain the road background model, and update the background pixel value of the pixel point at time t The calculation formula is as follows:

[0027]

[0028] Step two: according to the road background model in step one and the output frame, an adaptive background difference method is designed to obtain a moving foreground target; for the background image generated by the Gaussian mixture model The image I to be detected t (x, y), the binary image F of the moving foreground target t (x, y) is calculated as follows:

[0029]

[0030] where S is a set threshold value used to describe the difference between the background and the foreground, λ is a set integer constant, m and n are the length and width of the binary image respectively, and the introduction of the integer constant method can avoid the problem of fixed threshold value misclassification and reduce the adverse effects of light; in order to further reduce the interference noise in the foreground image, based on the image morphological algorithm, the binary image is subjected to noise suppression and elimination. First, the isolated noise points are eliminated by using the erosion operation, and then the dilation operation is performed to fill the edge holes of the moving target, so that a more complete foreground target is obtained.

[0031] Step three: using an edge extraction algorithm, the two-dimensional image boundary position of the candidate thrown object is obtained, a general target detection deep learning algorithm is used to filter the foreground moving target, common vehicles, pedestrians and non-motor vehicles are excluded, a tracking algorithm is used to track the remaining foreground target, and the motion trajectory of the candidate thrown object is obtained;

[0032] Sub-step one: using the Canny edge detection algorithm to obtain the binary image F of the moving foreground target t (x, y) in the target boundary position, for the edge point set Δ(x, y) of the target, the two-dimensional image boundary position of the target The calculation formula is as follows:

[0033]

[0034] wherein and respectively are the left upper corner boundary horizontal position vector and vertical position vector of the candidate target set of the thrown object, and respectively are the right lower corner boundary horizontal position vector and vertical position vector of the corresponding target.

[0035] Substep two: the image pyramid in H is detected by using a general target detection deep learning algorithm, when the target is determined to be a common vehicle, pedestrian and non-motor vehicle, the target is discarded, and the remaining target position H is obtained i , i is in (0, E), wherein E is the number of remaining targets.

[0036] Substep three: the remaining foreground target is tracked by using a visual target tracking algorithm, the candidate quantity of tracking is the image centroid position and image boundary position of the target, and the motion trajectory Γ of the remaining foreground target in the video signal is obtained i .

[0037] Step four: whether the foreground target belongs to the thrown object is comprehensively judged by using the target region contrast feature, target motion trajectory feature and tracking duration, and the final thrown object image position is output; the core of the step is that, firstly, the corresponding image block J i in the original video frame is obtained according to the target position H i output in step three, i the image block contrast between J i and the image block at the corresponding position in the background model is calculated and secondly, the target motion acceleration value Acc i is calculated according to the target motion trajectory, i thirdly, the tracking duration ET i of the target before being covered by the background model is counted, and finally, whether the target belongs to the thrown object is comprehensively judged, and the calculation formula is as follows:

[0038]

[0039] wherein γ and ε are respectively a contrast threshold and an existing duration threshold, when P i is 1, the target belongs to the thrown object, and when P i is 0, the target is discarded.

[0040] Compared with the prior art, the present application has the following advantages:

[0041] 1) the thrown object foreground target extraction method disclosed in the present application based on the image pyramid, Gaussian mixture model and adaptive background difference method effectively improves the modeling ability of the original Gaussian mixture model for large-size targets, improves the background modeling precision of the Gaussian mixture model under different video resolutions, and due to the existence of adaptive parameters, the method has strong adaptability to light and other large-scale environmental changes.

[0042] 2) The candidate object multi-level screening method disclosed by the present application can quickly exclude common dynamic interference targets compared with traditional visual feature recognition, reduces the false alarm rate of the object, and can avoid common tree branch interference, further improving the detection accuracy of the object. BRIEF DESCRIPTION OF DRAWINGS

[0043] Figure 1 The overall technical scheme of the present application is shown in the figure. DETAILED DESCRIPTION

[0044] The technical scheme of the present application will be further described below in combination with the drawings and examples.

[0045] With the continuous deepening of the construction of metropolitan areas such as Beijing-Tianjin-Hebei, Yangtze River Delta, and Pearl River Delta, and the continuous growth of residents' travel demand, the development of transportation infrastructure such as expressways and express lanes has entered a fast lane. Among the many factors that induce traffic accidents, the danger of objects is relatively large, and it is difficult to find them in time, which can easily induce serious traffic accidents such as vehicle rollover and vehicle out of control, endangering the safety of drivers and passengers. Therefore, quickly identifying and removing objects on the road has become an important goal to ensure road safety.

[0046] Existing object identification methods are mainly divided into two categories, namely manual screening and visual event identification system. With the continuous expansion of the scale of highway network and the exponential growth of traffic monitoring cameras, the manual screening method is time-consuming and labor-intensive, and the information lag is serious, which is difficult to meet the needs of accident prevention and control. The visual event identification system is an automatic highway traffic event detection product, such as the highway video monitoring system launched by Hikvision and Yutai Technology. These systems can realize automatic online detection of common objects with the help of deep learning algorithms. However, due to the complexity and diversity of objects and detection environments, the detection accuracy of existing systems still needs to be improved.

[0047] The existing object detection method basically follows the mode of background modeling and foreground target discrimination. In the background modeling part, the commonly used background modeling methods include mean method, Vibe background model method and Gaussian mixture model method, wherein the Gaussian mixture model method has higher detection accuracy and wider application range, and the patent CN111339824A discloses a throwing object extraction method based on the model. However, since the operation object of the method is a pixel by pixel, when the target size is too large, it is difficult to obtain the accurate foreground target position. In the foreground target discrimination part, the common methods include visual feature discrimination method and deep learning model discrimination method, and the patent CN111523536A discloses a deep learning throwing object discrimination model based on Faster-RCNN. However, due to the diversity of throwing objects, the misreporting rate is high and the detection accuracy is poor when only the supervised learning method is used to establish the discrimination model.

[0048] Due to the influence of the sensing distance and resolution of the roadside camera, the existing algorithm has poor background modeling accuracy for variable scales, and due to the diversity of throwing objects, the deep learning method is difficult to determine the untrained throwing misclassification, and the missed detection and false reporting events occur frequently. How to improve the modeling accuracy of the road background model and improve the discrimination ability of the existing algorithm for the foreground throwing object has become a great challenge. The solution and method of the problem will greatly improve the practicability and reliability of the visual event recognition system and improve the recognition and early warning ability of the system for the throwing object.

[0049] To solve the precision problem of background modeling and foreground discrimination in the throwing object recognition process, the present application formulates the following technical scheme:

[0050] Firstly, the existing Gaussian mixture model background establishment method is improved, the image pyramid is introduced as the image input of the Gaussian mixture model, the modeling ability of the Gaussian mixture model for different scale background pictures is improved, and a stable and reliable multi-scale road background model is obtained. Secondly, the background difference method is used to obtain the foreground target of the current video frame relative to the road background model, in order to avoid the generation of the hole phenomenon, the image expansion and corrosion algorithm is used to process the foreground target area twice, and a more complete foreground target is obtained. Thirdly, the general target detection deep learning algorithm is used to discriminate the tracked foreground target, and the common targets such as vehicles, pedestrians and non-motor vehicles are filtered out, the tracking algorithm is used to track the non-traffic motion foreground target, and the motion trajectory features of the non-traffic motion foreground target in a certain time sequence are obtained. Finally, the region contrast feature, motion trajectory feature and tracking time length and other information are used to discriminate the remaining foreground target twice, the influence of tree shadow and vehicle shadow is reduced, the image position of the throwing object is obtained, and the overall scheme is shown in Figure 1 .

[0051] The method of the present application comprises the following specific steps:

[0052] Step one, obtain the video stream of the roadside camera, establish a multi-scale road background model for distinguishing foreground moving targets;

[0053] Sub-step one: collect the traffic flow video of the roadside camera, mark the original resolution picture set of the video stream as the first picture set Φ1, use the down-sampling algorithm to solve the pyramid image set of the first picture set Φ1, and obtain Φ2, Φ3,..., Φ N , respectively, where Φ i is the i-th picture set, N is a constant, which is generally 3 or 4 under the premise of ensuring the clarity of the down-sampled pictures, and the output frame Φ of the video stream is obtained as {Φ1, Φ2,..., Φ N}, taking a common 2K resolution camera as an example, the down-sampling step is 2, and N is 4, then the resolution of Φ1 is 2048×1080, the resolution of Φ2 is 1024×540, the resolution of Φ3 is 512×270, and the resolution of Φ4 is 256×135;

[0054] Sub-step two: based on the video stream output frame Φ obtained in sub-step one, initialize the parameter information of the Gaussian mixture model, in order to reduce the motion target interference in the parameter initialization process, set the number of Gaussian mixture models as K, which is usually 3 to 5, use the multi-frame average value method as the initialization parameter, and the number of multi-frame frames is T K , selecting a larger initial frame number can obtain a more stable initial background, but it takes a long time, generally 100 frames, for the pixel value X t in the output frame Φ at time t, the calculation formulas of the mean μ1 and the variance σ1 of the first Gaussian distribution function are as follows:

[0055]

[0056]

[0057] In order to improve the execution speed of the algorithm initialization process, only the first Gaussian distribution function is assigned a weight w1=1 / K, and the mean μ i , the variance σ i and the weight w i of the other Gaussian distributions are all 0;

[0058] Sub-step three: set the update mode of the Gaussian mixture model, sort all pixel points corresponding to the Gaussian model according to w i,t-1 / σ i,t-1 , and use formula (3) in order to find the matching Gaussian component:

[0059] |X t -μ i,t-1 |<δσ i,t-1 (3)

[0060] wherein, δ is a set threshold value, generally selected as 2.5, if a corresponding matching component cannot be found in all Gaussian distribution models, the method in sub-step two is used to re-establish the last Gaussian distribution model in the sorting, if the current pixel point matches the Gaussian model, the related parameters of the first matching Gaussian distribution are updated, and the calculation formula is as follows:

[0061] w i,t =(1-α)w i,t-1 +α(1-c′) (4)

[0062] μ i,t =(1-ρ i,t-1 )μ i,t-1 +ρ i,t-1 X t (5)

[0063]

[0064] ρ i,t-1 =α / w i,t-1 (7)

[0065] c′=c·α (8)

[0066] wherein, α is a learning rate of the model, the greater α is, the faster the updating speed of the background model is, and the commonly used learning is 0.05; ρ i,t-1 is a parameter learning rate of the model, c is a priori coefficient, generally selected as one of 0.1 to 50 according to the fluctuation of the background, and the rest of the Gaussian distributions only update the weight value, and the calculation formula is as follows:

[0067] w i,t =(1-α)w i,t-1 +αc′ (9)

[0068] Sub-step four: setting a background model generation method of the Gaussian mixture model, sorting K Gaussian models according to the weight value, and selecting the first B t gaussian models as the basis of the background model, and the calculation formula of B t is as follows:

[0069]

[0070] wherein, c f is a background threshold value, generally selected as 0.2, and used to describe the maximum Gaussian component proportion of the foreground target.

[0071] Sub-step five: obtaining a road background model, for the background pixel value of the t time updated pixel point The calculation formula is as follows:

[0072]

[0073] Step two, according to the road background model and the output frame in step one, an adaptive background difference method is designed to obtain the moving foreground target; for the background image generated by the Gaussian mixture model The image to be detected I t (x,y), the binary image F of the moving foreground target t The calculation formula of (x,y) is as follows:

[0074]

[0075] Wherein, S is a set threshold value for describing the difference between the background and the foreground, generally taking 0.2, λ is a set integer constant, m and n are the length and width of the binary image respectively, which are 2048 and 1080 respectively in a 2K image, the method of introducing an integer constant can avoid the problem of fixed threshold value misdivision and reduce the adverse effects of light; in order to further reduce the interference noise in the foreground image, based on the image morphological algorithm, the binary image is subjected to noise suppression and elimination. First, the isolated noise points are eliminated by using erosion operation, and then inflation operation is performed to fill the edge holes of the moving target, so that a more complete foreground target is obtained.

[0076] Step three, using edge extraction algorithm to obtain the two-dimensional image boundary position of the candidate throwing object, using general target detection deep learning algorithm to filter the foreground moving target, excluding common vehicles, pedestrians and non-motor vehicles, using tracking algorithm to track the remaining foreground target, and obtaining the motion trajectory of the candidate throwing object;

[0077] Substep one: using Canny edge detection algorithm to obtain the binary image F of the moving foreground target t (x,y) in the target boundary position, for the edge point set Δ(x,y) of the target, the two-dimensional image boundary position of the target The calculation formula of (x,y) is as follows:

[0078]

[0079] Wherein, And are the left upper corner boundary horizontal position vector and the vertical position vector of the candidate throwing object set respectively, And are the corresponding right lower corner boundary horizontal position vector and the vertical position vector.

[0080] Sub-step 2: Use a general object detection deep learning algorithm to detect the image pyramid within H. When a target is identified as a common vehicle, pedestrian, or non-motorized vehicle, discard the target and obtain the remaining target positions H. i ,i∈(0,E), where E is the number of remaining targets. To quickly obtain the category of the target, single-stage target detection algorithms such as YOLO can be used. Currently, the most commonly used ones are YOLOX-tiny and YOLOv5s.

[0081] Sub-step 3: Use a visual target tracking algorithm to visually track the remaining foreground targets. The candidate quantities to be tracked are the image centroid position and the image boundary position of the target, and obtain the motion trajectory Γ of the remaining foreground targets in the video signal. i The commonly used target tracking algorithm is DeepSort.

[0082] Step 4: Utilizing three different levels of features—target region contrast features, target motion trajectory features, and tracking duration—to comprehensively determine whether the foreground target belongs to a thrown object, and output the final image location of the thrown object; the core of this step lies in first determining the target location H output in Step 3. i Obtain the corresponding image block J in the original video frame. i Calculate J i Contrast with corresponding image blocks in the background model and Next, based on the target's trajectory, calculate the target's acceleration value Acc. i Secondly, the tracking time ET before the statistical target is covered by the background model. i Finally, a comprehensive judgment is made as to whether the target belongs to the category of scattered objects. The calculation formula is as follows:

[0083]

[0084] Where γ and ε are the contrast threshold and presence duration threshold, respectively. γ is generally selected as 0.3, and ε is adjusted according to the video bitrate. When the video refresh rate is 25Hz, it is generally taken as 50. i When P is 1, the target is a projectile object; when P is 1... i If the value is 0, discard the target.

Claims

1. A method for detecting road debris based on a roadside camera, characterized in that: The method includes the following steps: Step 1: Acquire video streams from roadside cameras and establish a multi-scale road background model to distinguish foreground moving targets; Step 2: Based on the road background model and output frame from Step 1, design an adaptive background subtraction method to obtain the moving foreground target; Step 3: Use the edge extraction algorithm to obtain the two-dimensional image boundary position of the candidate thrown object, use the general object detection deep learning algorithm to filter the foreground moving targets, exclude common vehicles, pedestrians and non-motorized vehicles, and use the tracking algorithm to track the remaining foreground targets to obtain the motion trajectory of the candidate thrown object. Step 4: Using the contrast features of the target area, the target motion trajectory features, and the tracking time, comprehensively determine whether the foreground target belongs to the thrown object, and output the final image position of the thrown object. The detailed steps for Step One are as follows: Sub-step 1: Acquire traffic flow video from roadside cameras, label the set of original resolution images from the video stream as the first image set Φ1, and use a downsampling algorithm to solve for the pyramid image set of the first image set Φ1, obtaining Φ2, Φ3, ..., Φ N , where Φ i Let N be the i-th image set, and N be an optional constant, which can be 3 or 4 while ensuring the clarity of the downsampled images. This yields the output frames Φ = {Φ1, Φ2, ..., Φ...} of the video stream. N }; Sub-step 2: Based on the video stream output frame Φ obtained in sub-step 1, initialize the parameter information of the Gaussian mixture model. To reduce moving target interference during parameter initialization, set the number of Gaussian mixture models to K, and use the multi-frame averaging method as the initialization parameter. The number of frames is T. K For the pixel value X in the output frame Φ at time t t The formulas for calculating the mean μ1 and variance σ1 of the first Gaussian distribution function are as follows: To improve the execution speed of the algorithm initialization process, only the first Gaussian distribution function is assigned a weight w1 = 1 / K, while the mean μ of other Gaussian distributions is used. i σ i and weight w i All are 0; Sub-step 3: Set the update method for the Gaussian mixture model, and update the Gaussian model for all pixels according to w. i,t-1 / σ i,t-1 Sort in the following manner, and apply formula (3) sequentially until a matching Gaussian component is found: |X t -m i,t-1 |<ds i,t-1 (3) Where δ is the set threshold, if no matching component can be found in any Gaussian distribution model, the last Gaussian distribution model in the sorting is rebuilt using the method in sub-step two. If the current pixel matches the Gaussian model, the relevant parameters of the first matching Gaussian distribution are updated. The calculation formula is as follows: w i,t =(1-α)w i,t-1 +α(1-c′) (4) m i,t =(1-ρ i,t-1 )m i,t-1 +r i,t-1 X t (5) r i,t-1 =a / w i,t-1 (7) c′=c·α (8) where α is the learning rate of the model, and the larger α is, the faster the background model is updated; ρ i,t-1 Let be the model's parameter learning rate, and c be the prior coefficients. Except for the first matching Gaussian distribution, the remaining Gaussian distributions only update the weights. The calculation formula is as follows: w i,t =(1-α)w i,t-1 +αc′ (9) Sub-step four: Define the background model generation method for the Gaussian mixture model, sort the K Gaussian models according to their weights, and select the top B... t A Gaussian model serves as the basis for the background model, B t The calculation formula is as follows: Where c f The background threshold is used to describe the proportion of the maximum Gaussian component of the foreground target; Sub-step 5: Obtain the road background model, and update the background pixel values ​​at time t. The calculation formula is as follows:

2. The method for detecting road debris based on a roadside camera according to claim 1, characterized in that: Step 2: Background image generated by Gaussian mixture model Image to be detected I t (x,y), the binarized image of the moving foreground target F t The formula for calculating (x,y) is as follows: Where S is the set threshold used to describe the difference between the background and the foreground, λ is the set global coefficient, and m and n are the length and width of the binarized image, respectively. The method of introducing the global coefficient avoids the problem of missegmentation by the fixed threshold and reduces the adverse effects of illumination. In order to further reduce the interference noise in the foreground image, noise suppression and elimination are performed on the binarized image based on image morphology algorithm. First, erosion operation is used to eliminate isolated noise points, and then dilation operation is performed to fill the edge holes of the moving target to obtain a more complete foreground target.

3. The method for detecting road debris based on a roadside camera according to claim 2, characterized in that: The detailed steps for step three are as follows: Sub-step 1: Obtain the binarized image F of the moving foreground target using the Canny edge detection algorithm. t The target boundary location in (x,y), for the target's edge point set Δ(x,y), the two-dimensional image boundary location of the target. The calculation formula is as follows: in, and These are the horizontal and vertical position vectors of the top-left boundary of the candidate projectile target set, respectively. and These are the corresponding horizontal and vertical position vectors of the lower right boundary, respectively; Sub-step 2: Use a general object detection deep learning algorithm to detect the image pyramid within H. When a target is identified as a common vehicle, pedestrian, or non-motorized vehicle, discard the target and obtain the remaining target positions H. i ,i∈(0,E), where E is the number of remaining targets; Sub-step 3: Use a visual target tracking algorithm to visually track the remaining foreground targets. The candidate quantities to be tracked are the image centroid position and the image boundary position of the target, and obtain the motion trajectory Γ of the remaining foreground targets in the video signal. i .

4. The method for detecting road debris based on a roadside camera according to claim 3, characterized in that: The detailed steps for step four are as follows: First, regarding the target position H output in step three... i Obtain the corresponding image block J in the original video frame. i Calculate J i Contrast with corresponding image blocks in the background model and Next, based on the target's trajectory, calculate the target's acceleration value Acc. i Secondly, the tracking time ET before the statistical target is covered by the background model. i Finally, a comprehensive judgment is made as to whether the target belongs to the category of scattered objects. The calculation formula is as follows: Where γ and ε are the contrast threshold and the duration threshold, respectively, when P i When P is 1, the target is a projectile object; when P is 1... i If the value is 0, discard the target.

Images