Traffic event detection method based on multi-modal edge fusion

Abstract

The present application relates to the technical field of traffic detection, and particularly relates to a traffic event detection method based on multi-modal edge fusion. The method comprises the following steps: S1, obtaining multi-modal perception data of a current road section at an edge computing node, wherein the multi-modal perception data comprises a visible light image sequence collected by a visual sensor and sparse point cloud data collected by a millimeter wave radar sensor; S2, performing target detection on the visible light image sequence to identify a dense vehicle queue area constituting a static traffic flow, and extracting accurate longitudinal distance values of a plurality of discrete vehicles in front of the dense vehicle queue area in the sparse point cloud data, wherein the accurate longitudinal distance values are taken as known spatial sampling points; S3, in response to a failure of visual depth estimation in the middle and rear regions of the dense vehicle queue area in the visible light image due to occlusion, constructing a one-dimensional variation function model based on lane line direction, and calculating a spatial correlation decay law under a current traffic flow state by using the known spatial sampling points.

Description

Technical Field

[0001] This invention relates to the field of traffic detection technology, and specifically to a traffic incident detection method based on multimodal edge fusion. Background Technology

[0002] Currently, the automatic detection of traffic incidents in intelligent transportation systems mainly relies on roadside or vehicle-mounted sensors for real-time perception of the road environment. In the daily operation of highways and urban expressways, traffic congestion caused by traffic accidents, road construction, or peak-hour traffic saturation is one of the most common types of events that have the greatest impact on traffic efficiency.

[0003] To promptly issue warnings to vehicles behind and guide traffic diversion, the system needs to accurately determine the position of the tail of the congestion queue in order to calculate the congestion length and assess its spread rate. Existing technical solutions for this requirement mainly fall into two categories. The first category is based on single-modal visual perception. It deploys high-definition cameras on roadside poles and uses deep learning object detection algorithms to identify and locate vehicles in the image, then combines lane line geometric constraints to calculate the longitudinal distance of vehicles in the real world. This method achieves relatively ideal detection results in scenarios with good lighting conditions, wide field of view, and sparse vehicles. However, in congested conditions, a large number of vehicles are closely packed together, and the image features of vehicles behind are largely obscured by vehicles in front. The depth estimation network of monocular vision almost fails due to the lack of effective texture parallax and motion parallax information, resulting in serious systematic bias in the output distance values. Simultaneously, the visual algorithm is extremely sensitive to weather and lighting conditions. In low-light conditions at night, rainy or foggy weather, or strong backlighting, vehicle edges are blurred, contrast decreases, and object detection boxes exhibit severe jitter or even missed detections, leading to highly unstable tail-end localization results. The second approach relies on active detection using millimeter-wave radar. It transmits frequency-modulated continuous waves and receives the reflected echoes from targets, utilizing the Doppler effect to accurately measure the radial distance and relative velocity of objects ahead. The advantage of radar lies in its immunity to illumination and most adverse weather conditions, achieving decimeter-level accuracy in ranging metallic objects. However, its inherent limitations are drastically amplified when facing dense, stationary convoys of dozens or even hundreds of vehicles.

[0004] First, millimeter-wave radar has limited angular resolution. When multiple targets at close range are located in the same radial direction or adjacent beams, the echo signal will generate severe multipath interference and point cloud aliasing. Radar processing algorithms struggle to distinguish densely packed vehicles as independent targets, often only outputting a blurry cluster point or simply filtering out vehicles behind as background noise. Second, even if the radar can detect some vehicles at the rear of the queue, due to the lack of a precise pixel-level correspondence with the visual image, the sparse point cloud coordinates alone cannot stably associate the detected point with a specific vehicle instance in the image, let alone determine the vehicle's exact position in the queue. Attempts have been made to fuse vision and radar, such as performing intersection-union matching (IoU) between the visual detection box and the radar point cloud projection at the decision layer, or directly concatenating the feature vectors of the two modalities at the feature layer before inputting them into the fusion network. These approaches improve detection robustness in scenarios with sparse targets and simple motion states, but their fusion logic essentially still involves weighted averaging or cross-verification of multi-source descriptions of the same target at the same time. They do not address the problem of permanent information loss due to physical occlusion or sensor limitations. When the visual image is completely obscured by a vehicle in front, or when radar point clouds are too dense to form effective clusters, the fusion system also faces the dilemma of having no data to fuse, and can only passively output the extrapolated prediction from the previous cycle or directly discard the perception results for that area. Summary of the Invention

[0005] The technical problem solved by this invention is to provide a traffic event detection method based on multimodal edge fusion. Under the condition that visible light images are severely obscured by densely packed vehicles and millimeter-wave radar point clouds cannot form effective independent targets due to multipath interference, the method can calculate the estimated physical position and corresponding estimated variance of each vehicle in the rear of the congestion queue along the lane line direction. When the estimated variance exceeds the limit, the method actively suppresses the output of high-precision alarms to avoid triggering false alarms based on unreliable positioning data.

[0006] The basic solution provided by this invention is a traffic incident detection method based on multimodal edge fusion, which includes the following steps: S1. Obtain multimodal perception data of the current road segment at the edge computing node. The multimodal perception data includes visible light image sequences collected by a visual sensor and sparse point cloud data collected by a millimeter-wave radar sensor. S2. Target detection is performed on the visible light image sequence to identify the dense vehicle queue area that constitutes the stationary traffic flow. At the same time, the precise longitudinal distance values ​​of several discrete vehicles in front of the corresponding dense vehicle queue area are extracted from the sparse point cloud data, and the precise longitudinal distance values ​​are used as known spatial sampling points. S3. In response to the failure of visual depth estimation in the middle and rear regions of dense vehicle queues in visible light images due to occlusion, a one-dimensional variation function model based on lane line direction is constructed, and the spatial correlation attenuation law under the current traffic flow is calculated using the known spatial sampling points. S4. Based on the one-dimensional variation function model, perform ordinary kriging interpolation on the interpolation position at the rear of the dense vehicle queue area, and output the estimated physical position and corresponding estimated variance of each vehicle in the area. S5. Determine whether the estimated variance exceeds a preset safety threshold. If it does not exceed the threshold, use the estimated physical location as the tail boundary of the dense vehicle queue to calculate the traffic congestion length or generate high-precision event coordinates. If it exceeds the threshold, generate an abnormal state identifier representing the fuzzy positioning and suppress the output of high-precision traffic event alarms based on the area.

[0007] The principle and advantages of this invention are as follows: After the edge computing node acquires visible light images and millimeter-wave radar point clouds, it does not directly input the two types of data into the black-box model. Instead, it first uses vision to identify the dense vehicle queue area caused by congestion, and then extracts the longitudinal distance values ​​of several vehicles that the radar can accurately measure at the front of the queue. These high-precision distance measurement points are used as known spatial sampling points. When it is necessary to know the position of vehicles at the rear of the queue that cannot be directly measured due to visual obstruction, the system constructs a one-dimensional variation function model along the lane line direction. By analyzing the correlation between the distance between known sampling points and the change in traffic flow density, the approximate spacing of vehicles at unknown positions is calculated. Subsequently, ordinary Kriging interpolation is used to not only provide the estimated position, but also output an estimated variance to quantify the reliability of the position. Finally, a safety threshold is set for traffic diversion: those with small variance are directly used as the queue tail boundary for calculating the congestion length, while those with large variance are marked as having ambiguous positioning and alarms are suppressed.

[0008] Compared to existing technologies, this solution has the advantage of not forcibly outputting an unreliable coordinate when vision fails, nor abandoning the perception of the queue tail position simply because radar cannot penetrate dense traffic flow. It utilizes spatial statistical laws to transform areas that cannot be directly measured into areas that can be quantified and estimated, and for the first time introduces the output of estimated variance in traffic event detection, enabling the system to distinguish which congestion length data are reliable and which require further confirmation. This proactive management of perceptual uncertainty avoids missed or false congestion warnings due to incorrect queue tail positioning, and has significant engineering practical value, especially in long-distance highway congestion scenarios.

[0009] Furthermore, S2 includes the following steps: S21. Perform target detection on the current frame of the visible light image sequence and extract the two-dimensional bounding boxes of all motor vehicles and their corresponding vehicle type labels; S22. Perform cross-union matching on the two-dimensional bounding boxes of adjacent frames, calculate the displacement vector of each target in the pixel coordinate system, determine the motor vehicles whose displacement vector amplitude is lower than the preset motion threshold as stationary vehicles, and mark the connected image region composed of continuous stationary vehicles as a dense vehicle queue region. S23. Project the sparse point cloud data onto the pixel plane of the visible light image through a pre-calibrated sensor extrinsic matrix, and filter out point cloud targets whose projection falls within a preset longitudinal range in front of the dense vehicle queue area. S24. Perform static attribute verification based on Doppler velocity on the selected point cloud targets, remove dynamic clutter with Doppler velocity absolute values ​​higher than the preset static threshold, and retain valid radar targets that meet the static conditions. S25. Extract the radial distance and azimuth angle of the effective radar target in the radar coordinate system, and use the radial distance as the accurate longitudinal distance value corresponding to the effective radar target, and use the set of the effective radar targets to form the known spatial sampling points.

[0010] A target detection network is run on the current frame of the visible light image to outline all motor vehicles and label their types, providing basic semantic information for identifying traffic participants. The position changes of the bounding boxes of the same target in adjacent frames are compared, and the displacement vector amplitude is calculated. When this amplitude is less than a preset motion threshold, the vehicle is identified as stationary, and continuous clusters of stationary vehicles are marked as dense vehicle queues. The key is to filter out truly stationary vehicle groups through motion information, eliminating interference from slow-moving vehicles. Using a pre-calibrated sensor extrinsic matrix, the 3D point cloud of the millimeter-wave radar is projected onto a 2D image plane, retaining only point cloud targets within a certain longitudinal range in front of the dense vehicle queue area. A secondary verification based on Doppler velocity is performed on the selected radar targets to eliminate dynamic clutter points that, although their projected positions match, are still moving, ensuring that the retained radar targets are completely consistent with their visually stationary state. The radial distance and azimuth of these effective radar targets are extracted, and the radial distance is used as the precise longitudinal distance value to form a set of known spatial sampling points.

[0011] The advantages lie in the establishment of a rigorous verification mechanism for stationary vehicles, where visual static assessment and radar static measurement mutually corroborate each other, greatly reducing the risk of sampling point contamination caused by single-mode misjudgment. Simultaneously, extracting only radar targets at the front of the queue, rather than the entire queue, avoids the inherent defects of radar in dense metal target scenarios, such as severe multipath interference and decreased ranging reliability, while also providing high-quality, low-noise one-dimensional range sequence samples for subsequent variogram modeling.

[0012] Furthermore, S3 includes the following steps: S31. Obtain the geometric information of the lane lines covered by the dense vehicle queue area, extract the longitudinal extension direction vector of the lane lines, and arrange the precise longitudinal distance values ​​of the known spatial sampling points into a one-dimensional spatial sequence along this direction vector. ,in This represents the precise longitudinal range value of the i-th effective radar target, where n is the total number of known spatial sampling points; S32, For multiple preset hysteresis distances Calculate the experimental semivariance value corresponding to each lag distance.

[0013] in This indicates that the distance difference is satisfied. The number of sampling point pairs, This represents the local traffic flow density characterization value corresponding to the radar target at a longitudinal distance z; S33. Based on multiple lag distances and the corresponding experimental semivariogram values, a weighted least squares fitting is performed using a preset theoretical variogram model to obtain the optimal fitting parameters of the theoretical variogram model. The theoretical variogram model includes one of a spherical model, an exponential model, or a Gaussian model. S34. The fitted theoretical variogram model is determined as the one-dimensional variogram model, which describes the quantitative relationship between vehicle spacing and spatial correlation decay under the current traffic flow state, i.e., the spatial correlation decay law.

[0014] First, the geometric orientation of the lane lines covered by the dense vehicle queue is obtained, and a series of precise longitudinal distance values ​​are arranged into a one-dimensional sequence according to the direction of lane line extension. Next, for a pre-defined set of distance intervals, i.e., lag distances, the experimental semivariogram value corresponding to each lag distance is calculated. This semivariogram is calculated by finding all sampling point pairs whose distance difference is approximately equal to the lag distance, and taking half the average of the squares of the differences in their corresponding local traffic flow density characterization values. The local traffic flow density characterization value can be defined based on radar reflection intensity or visual detection box density, reflecting the density of vehicles at a certain longitudinal position. Then, a scatter plot is constructed by combining the calculated lag distances and the experimental semivariogram values. A weighted least squares fit is performed using one of the spherical model, exponential model, or Gaussian model to obtain the optimal parameters of the theoretical model. The final fitted function is the variogram model describing the spatial correlation of the current congestion queue. This model implies that when the distance between two vehicles is small, their traffic flow density states are highly correlated; as the distance increases, the correlation gradually decreases until it becomes uncorrelated.

[0015] Compared to existing technologies, this method reduces a complex two-dimensional spatial interpolation problem to a one-dimensional statistical modeling problem along the lane direction, significantly reducing computational complexity and making it suitable for edge nodes with limited computing power. More importantly, the variation function model is dynamically fitted from real-time radar sampling points of the current road, rather than using preset fixed parameters. Therefore, it can adapt to different congestion patterns on different road segments. For example, the compactness of queues on urban expressways is very different from the looseness of queues on long downhill sections of highways. Dynamic fitting ensures the scenario-specific nature of subsequent interpolation results.

[0016] Furthermore, S4 includes the following steps: S41. Determine the interpolation position at the rear of the dense vehicle queue area. The interpolation position represents the longitudinal distance coordinate of the effective radar sampling point that cannot be obtained in the visual image due to occlusion. S42. Based on the one-dimensional variation function model, calculate the semivariance matrix between each pair of known spatial sampling points. and the position to be interpolated The semivariance vector between each spatial sampling point ,in:

[0017]

[0018] in, , These are the longitudinal distance values ​​between the i-th and j-th known spatial sampling points, respectively. Represents a one-dimensional variational function model; S43. Construct the ordinary Kriging equation system and solve for the Kriging weight coefficient vector. and the Lagrange coefficient μ:

[0019] S44. Using the Kriging weighting coefficient vector λ, perform a weighted linear combination of the local traffic flow density characterization values ​​corresponding to the known spatial sampling points to obtain the location to be interpolated. Estimated density characterization value at [location] :

[0020] in, Longitudinal distance Local traffic flow density characterization value at the location; S45. Calculate the interpolation position based on the Kriging weight coefficient vector and the semivariance vector. Estimated variance at location :

[0021] S46. Based on the pre-calibrated density-distance mapping relationship, the estimated density characterization value will be... Convert to the estimated physical location corresponding to the location to be interpolated.

[0022] A specific longitudinal coordinate for interpolation is determined at the rear of a dense vehicle queue. This coordinate corresponds to an area where radar sampling points cannot be directly obtained visually due to obstruction by the preceding vehicle. Using the obtained variogram model, the semivariogram matrix between each pair of known sampling points and the semivariogram vector between the interpolation location and each known sampling point are calculated. The semivariogram matrix describes the spatial dependency structure between known points, and the semivariogram vector describes the spatial dependency strength between unknown points and known points. A system of ordinary Kriging equations is constructed and solved to obtain a set of weighting coefficients and a Lagrange multiplier. The mathematical constraints of this system of equations guarantee the unbiasedness of the estimation results and the minimization of the estimation variance. The local traffic flow density characterization values ​​corresponding to the known sampling points are weighted and summed using the obtained weights to obtain the estimated density characterization value of the interpolation location. The variance of this estimate is calculated using the weighting coefficients and the semivariogram vector, serving as a quantitative indicator of the reliability of the interpolation results. The estimated density characterization value is converted into the actual physical distance through a pre-defined mapping relationship between density and distance.

[0023] Compared with existing technologies that directly cluster radar point clouds or average visual depth maps, the weight allocation is determined entirely by the spatial correlation of the data itself, rather than by artificially setting uniform weights or distance-inverse weights, and the estimation results are closer to the real traffic flow.

[0024] Furthermore, S5 includes the following steps: S51, Estimating the variance With preset safety threshold In comparison, if Then determine the position to be interpolated. The confidence level of the estimated physical location meets the preset requirements, and the estimated physical location is determined as the tail boundary position of the dense vehicle queue; S52. Calculate the congestion queue length under the current traffic flow state based on the queue tail boundary position and the starting position of the dense vehicle queue area, and output the congestion queue length and the queue tail boundary position as traffic event detection results; S53, if Then determine the position to be interpolated. If the location is in an ambiguous state, generate a corresponding abnormal status identifier; S54: In response to the abnormal state identifier, suppress the output of high-precision traffic event alarm based on the location to be interpolated, and report the fuzzy positioning state to the cloud collaboration layer to trigger the auxiliary verification mechanism.

[0025] The estimated variance of the output is compared with a pre-set safety threshold to classify the perceived result of the interpolation location into two states: reliable and ambiguous. When the estimated variance is less than or equal to the safety threshold, it indicates that the known sampling points have a sufficiently strong constraint on the interpolation location, and the confidence level of the interpolation result meets the preset requirements. At this time, the system formally determines the estimated physical location as the tail boundary of the dense vehicle queue, and calculates the actual length of the current congestion queue in conjunction with the starting position of the queue area. This length and the tail position are output as the traffic event detection result for use in traffic guidance screen information dissemination or traffic light timing adjustment. When the estimated variance is greater than the safety threshold, it indicates that the known sampling points are too far from the interpolation location or the spatial correlation has been almost completely attenuated, and the forced output of the location coordinates may have a large error. At this time, the system determines that the location is in an ambiguous state, generates an abnormal state identifier, actively suppresses the high-precision alarm output based on this area, and reports the ambiguous location event itself to the cloud collaboration layer, triggering the background manual review or drone patrol confirmation mechanism. Compared with existing technologies, the advantage is that it breaks the black-and-white output mode of traditional detection algorithms. Existing technologies, regardless of confidence level, can only provide a single, definitive coordinate, leading to frequent false alarms under harsh operating conditions and wasting management time. This solution introduces a variance threshold to proactively identify and isolate unreliable sensing results, ensuring the immediate availability of high-confidence data while preventing low-confidence data from contaminating the event log database. This transparent handling of uncertainty significantly reduces the false alarm rate and manual verification workload of traffic monitoring systems, making it particularly suitable for long-distance, large-scale trunk highway traffic incident monitoring scenarios.

[0026] Furthermore, S54 includes the following steps: S541. The edge computing node packages the abnormal state identifier, the corresponding longitudinal distance coordinates of the position to be interpolated, the estimated variance value, and the current timestamp into a positioning fuzzy event message, and reports it to the cloud collaboration layer through the wireless communication link. S542. After receiving the positioning ambiguity event message, the cloud collaboration layer automatically retrieves the occupancy status of the pre-configured drone hangars for the road segment. If there are idle drones, an inspection route is generated along the lane line of the dense vehicle queue area. The starting point of the route is the sampling point with the largest longitudinal distance among the known spatial sampling points, and the ending point of the route is the position 50 meters forward from the position to be interpolated. The drone is then instructed to perform a video image acquisition task according to the route. S543. The video footage transmitted back by the drone is pushed to the monitoring terminal interface of the cloud collaboration layer, allowing managers to manually confirm the actual position of the tail of the formation. The manual confirmation result is transmitted back to the cloud collaboration layer through the confirmation control on the interface and stored in the historical sample database. Attached Figure Description

[0027] Figure 1 This is a schematic diagram of an embodiment of the present invention. Detailed Implementation

[0028] The following detailed description illustrates the specific implementation method: The basic implementation examples are as follows: Figure 1 As shown: The traffic incident detection method based on multimodal edge fusion includes the following steps: S1. Obtain multimodal perception data of the current road segment at the edge computing node. The multimodal perception data includes visible light image sequences collected by a visual sensor and sparse point cloud data collected by a millimeter-wave radar sensor.

[0029] Specifically, on a key monitoring section of a six-lane highway, an 8-megapixel high-definition visible light camera and a 77GHz millimeter-wave radar are installed on roadside pillars. The camera acquires a sequence of visible light images within a 200-meter range at a rate of 25 frames per second, while the millimeter-wave radar simultaneously scans the same area at a rate of 20 frames per second, outputting sparse point cloud data including radial distance, azimuth angle, Doppler velocity, and reflection cross-section. The camera and radar are connected via Ethernet to an edge computing node deployed at the base of the pillar. This node is equipped with an 8-core ARM architecture processor and a neural network acceleration chip with a computing power of 32 TOPS. A time synchronization service runs on the edge computing node, using the IEEE 1588 precise time protocol to assign a unified timestamp to both the camera image frames and the radar scan frames, ensuring that the deviation between the two data sets at the time of acquisition does not exceed 5 milliseconds. The edge computing node temporarily stores the received visible light image sequence in a memory buffer, and simultaneously parses the millimeter-wave radar point cloud data into a structured target list. Each target record contains five fields: target ID, radial distance, azimuth angle, radial velocity, and echo intensity. S2: Target detection is performed on the visible light image sequence to identify dense vehicle queue regions constituting stationary traffic flow. Simultaneously, the precise longitudinal distance values ​​of several discrete vehicles at the front of the corresponding dense vehicle queue regions are extracted from the sparse point cloud data, and these precise longitudinal distance values ​​are used as known spatial sampling points.

[0030] S2 includes the following steps: S21. Perform target detection on the current frame of the visible light image sequence and extract the two-dimensional bounding boxes of all motor vehicles and their corresponding vehicle type labels; S22. Perform cross-union matching on the two-dimensional bounding boxes of adjacent frames, calculate the displacement vector of each target in the pixel coordinate system, determine the motor vehicles whose displacement vector amplitude is lower than the preset motion threshold as stationary vehicles, and mark the connected image region composed of continuous stationary vehicles as a dense vehicle queue region. S23. Project the sparse point cloud data onto the pixel plane of the visible light image through a pre-calibrated sensor extrinsic matrix, and filter out point cloud targets whose projection falls within a preset longitudinal range in front of the dense vehicle queue area. S24. Perform static attribute verification based on Doppler velocity on the selected point cloud targets, remove dynamic clutter with Doppler velocity absolute values ​​higher than the preset static threshold, and retain valid radar targets that meet the static conditions. S25. Extract the radial distance and azimuth angle of the effective radar target in the radar coordinate system, and use the radial distance as the accurate longitudinal distance value corresponding to the effective radar target, and use the set of the effective radar targets to form the known spatial sampling points.

[0031] Specifically, the edge computing node inputs the current visible light image frame into a lightweight pruned YOLOv8 object detection network. This network's inference time per frame on the edge acceleration chip is approximately 12 milliseconds. The network outputs two-dimensional bounding boxes and vehicle type labels for all vehicles in the image, including passenger cars, trucks, and large buses. Each bounding box is represented by four values: the top-left pixel coordinates, width, and height. For example, in a certain frame, the network detects a truck in the near-end region of the image; the bounding box is located at pixel coordinates 480-720 and 320-560, labeled as a truck.

[0032] Edge computing nodes perform intersection-union (IoU) matching on all bounding boxes detected in the current frame with those in the previous frame, with an IoU threshold of 0.3. For the same target that successfully matches, the magnitude of its pixel displacement vector between the two frames is calculated. A preset motion threshold is that the vertical movement of the image does not exceed 3 pixels per frame. Vehicles with displacement vector magnitudes below this threshold are classified as stationary vehicles. For example, the displacement vector magnitudes of the aforementioned truck in five consecutive frames are 1.2 pixels, 0.8 pixels, 1.5 pixels, 1.0 pixels, and 0.9 pixels, respectively, all less than 3 pixels, and are therefore classified as stationary vehicles. Subsequently, a connected component analysis algorithm is used to aggregate the bounding boxes of stationary vehicles appearing consecutively in the image into a connected image region, which is marked as a dense vehicle queue region. The vertical pixel span of this region extends from row 120 to row 600.

[0033] Edge computing nodes load a pre-calibrated sensor extrinsic parameter matrix, which contains three rotation parameters and three translation parameters to map the three-dimensional point coordinates in the radar coordinate system to the image pixel plane. The sparse point cloud data acquired in S1 is projected point-by-point to obtain the corresponding pixel row and column number of each radar detection point in the image. The pre-defined longitudinal range at the front of the dense vehicle queue area is defined as the region closest to the camera in the image, i.e., the region with pixel row numbers between 120 and 240, corresponding to an actual longitudinal distance of approximately 30 to 60 meters. Point cloud targets whose projection falls within this range are selected. For example, a radar point, after projection, falls in pixel row 150 and column 400, within the aforementioned front range, and is retained; another radar point, after projection, falls in row 500, exceeding the front range, and is discarded.

[0034] For each selected point cloud target, its corresponding Doppler velocity value is read. The preset static threshold is that the absolute value of the radial velocity does not exceed 0.5 m / s. Point cloud targets with an absolute Doppler velocity higher than 0.5 m / s are classified as dynamic clutter and discarded, retaining only point cloud targets with an absolute Doppler velocity less than or equal to 0.5 m / s as valid radar targets. For example, a point cloud target with a Doppler velocity of 0.2 m / s is classified as a valid target; another point cloud target with a Doppler velocity of 8.5 m / s is classified as a moving vehicle and discarded.

[0035] Extract the radial range and azimuth values ​​of each valid radar target in the radar coordinate system. For example, the radial ranges of the three retained valid radar targets are 42.5 meters, 48.3 meters, and 56.7 meters, and the azimuth values ​​are -1.2 degrees, 0.8 degrees, and +1.5 degrees, respectively. Use the radial range values ​​directly as the precise longitudinal range values ​​corresponding to the target. These three precise longitudinal range values ​​(42.5 meters, 48.3 meters, and 56.7 meters) and their order in the lane line direction together constitute the set of known spatial sampling points finally output in step S2.

[0036] S3. In response to the failure of visual depth estimation in the middle and rear regions of dense vehicle queues in visible light images due to occlusion, a one-dimensional variation function model based on lane line direction is constructed, and the spatial correlation attenuation law under the current traffic flow is calculated using the known spatial sampling points.

[0037] S3 includes the following steps: S31. Obtain the geometric information of the lane lines covered by the dense vehicle queue area, extract the longitudinal extension direction vector of the lane lines, and arrange the precise longitudinal distance values ​​of the known spatial sampling points into a one-dimensional spatial sequence along this direction vector. ,in This represents the precise longitudinal range value of the i-th effective radar target, where n is the total number of known spatial sampling points; S32, For multiple preset hysteresis distances Calculate the experimental semivariance value corresponding to each lag distance.

[0038] in This indicates that the distance difference is satisfied. The number of sampling point pairs, This represents the local traffic flow density characterization value corresponding to the radar target at a longitudinal distance z; S33. Based on multiple lag distances and the corresponding experimental semivariogram values, a weighted least squares fitting is performed using a preset theoretical variogram model to obtain the optimal fitting parameters of the theoretical variogram model. The theoretical variogram model includes one of a spherical model, an exponential model, or a Gaussian model. S34. The fitted theoretical variogram model is determined as the one-dimensional variogram model, which describes the quantitative relationship between vehicle spacing and spatial correlation decay under the current traffic flow state, i.e., the spatial correlation decay law.

[0039] Specifically, the edge computing node obtains the geometric information of the second lane (middle lane) covered by the dense vehicle queue area from the high-precision map module or the visual lane detection module, and extracts the longitudinal extension direction vector of the lane line. This direction vector, in meters, represents the change in lateral and vertical coordinates of the lane line for every 1 meter of longitudinal extension. The precise longitudinal distance values ​​of the known spatial sampling points obtained in S2 (42.5 meters, 48.3 meters, and 56.7 meters) are arranged sequentially along the direction vector to form a one-dimensional spatial sequence. The first element of the sequence is the sampling point at 42.5 meters, the second element is the sampling point at 48.3 meters, and the third element is the sampling point at 56.7 meters, with a total of n equal to 3.

[0040] Preset a set of hysteresis distances Used to calculate spatial correlation. In this embodiment, five lag distances are set as 5 meters, 10 meters, 15 meters, 20 meters, and 25 meters. For each lag distance, all sampling point pairs in the one-dimensional spatial sequence with a distance difference approximately equal to that lag distance are traversed. For example, for a lag distance of 10 meters, sampling point pairs in the sequence with a distance difference between 9 meters and 11 meters include points near 42.5 meters and 52.5 meters, points near 48.3 meters and 58.3 meters, etc. Local traffic flow density characterization value. Defined as the number of valid targets detected by the radar within a 2-meter radius before and after a longitudinal distance z. For example, if one target is detected within a 2-meter radius before and after a distance of 42.5 meters, The value is 1 at 42.5 meters; and 1 at 48.3 meters. For each pair of sampling points, the square of the difference between their density characterization values ​​is calculated, summed, and divided by twice the number of sampling point pairs to obtain the experimental semivariance value corresponding to that lag distance.

[0041] Five two-dimensional data points were constructed using the five lag distances (5m, 10m, 15m, 20m, and 25m) and their corresponding five experimental semivariogram values. An exponential model was selected as the theoretical variogram model, containing three parameters to be fitted: sill value, range, and nugget value. Weighted least squares was used to fit the five data points, with the weights determined by the number of sampling pairs corresponding to each lag distance. After the fitting iterations converged, the optimal fitting parameters were obtained: sill value of 1.85, range of 42.0m, and nugget value of 0.12.

[0042] The fitted exponential model was determined as a one-dimensional variogram model applicable to the current traffic flow pattern. The mathematical meaning of this model is: when the longitudinal distance between two stationary vehicles is less than 42 meters, their traffic flow density states have a strong spatial correlation, and the correlation is stronger the closer the distance; when the distance exceeds 42 meters, the correlation decays to a negligible level. This correlation decay law represents the spatial structure characteristics of the current congestion queue, which is used to guide the subsequent allocation of interpolation weights.

[0043] S4. Based on the one-dimensional variation function model, perform ordinary kriging interpolation on the interpolation position at the rear of the dense vehicle queue area, and output the estimated physical position and corresponding estimated variance of each vehicle in the area.

[0044] S4 includes the following steps: S41. Determine the interpolation position at the rear of the dense vehicle queue area. The interpolation position represents the longitudinal distance coordinate of the effective radar sampling point that cannot be obtained in the visual image due to occlusion. S42. Based on the one-dimensional variation function model, calculate the semivariance matrix between each pair of known spatial sampling points. and the position to be interpolated The semivariance vector between each spatial sampling point ,in:

[0045]

[0046] in, , These are the longitudinal distance values ​​between the i-th and j-th known spatial sampling points, respectively. Represents a one-dimensional variational function model; S43. Construct the ordinary Kriging equation system and solve for the Kriging weight coefficient vector. and the Lagrange coefficient μ:

[0047] S44. Using the Kriging weighting coefficient vector λ, perform a weighted linear combination of the local traffic flow density characterization values ​​corresponding to the known spatial sampling points to obtain the location to be interpolated. Estimated density characterization value at [location] :

[0048] in, Longitudinal distance Local traffic flow density characterization value at the location; S45. Calculate the interpolation position based on the Kriging weight coefficient vector and the semivariance vector. Estimated variance at location :

[0049] S46. Based on the pre-calibrated density-distance mapping relationship, the estimated density characterization value will be... Convert to the estimated physical location corresponding to the location to be interpolated.

[0050] Specifically, determine the position to be interpolated. In this embodiment, due to obstruction by the vehicle in front, the area from line 240 in the visual image towards the depth of the image cannot be used to extract valid sampling points from the radar point cloud. Operators need to know the approximate location of a vehicle at the rear of the queue, approximately 120 meters from the camera, and its specific coordinates. Therefore, the longitudinal distance coordinates to be interpolated are... It is set to 120 meters.

[0051] Using a defined one-dimensional variogram model, calculate the semivariogram matrix Γ between each pair of known spatial sampling points, and the semivariogram vector between the interpolation location 120 meters and each known sampling point. The sampling points are known to be located at 42.5 meters, 48.3 meters, and 56.7 meters. The distance between 42.5 meters and 48.3 meters is calculated to be 5.8 meters. Substituting this into the variogram model, the corresponding semivariogram value is filled into the first row, second column, and second row, first column of the matrix; this value is 0.34. This process is repeated to fill the 3x3 symmetric matrix. The distance between 120 meters and 42.5 meters is calculated to be 77.5 meters. Substituting this into the model, the semivariogram value is 1.86, which is filled into the first element of the vector. The distance between 120 meters and 48.3 meters is 71.7 meters, corresponding to a semivariogram value of 1.84; the distance between 120 meters and 56.7 meters is 63.3 meters, corresponding to a semivariogram value of 1.79. This results in a 3x1 semivariogram vector.

[0052] Construct a system of ordinary kriging equations. The system consists of four equations: the first three correspond to unbiased constraints for three known sampling points, and the fourth equation is a normalization constraint where the sum of the weights equals 1. The coefficient matrix of the system is formed by expanding the semivariogram matrix obtained in step S42 by one row and one column and padding it with 1s and 0s. The right-hand side is formed by padding the semivariogram vector with a 1. Solve the four-variable linear system of equations using the Cholsky decomposition method at the edge computing nodes, obtaining three kriging weight coefficients of 0.22, 0.31, and 0.47, with a Lagrange multiplier of 0.08.

[0053] Using the obtained weighting coefficients of 0.22, 0.31, and 0.47, a weighted linear combination of the local traffic flow density characterization values ​​from the three known sampling points was performed. The density characterization values ​​at 42.5 meters, 48.3 meters, and 56.7 meters were all 1. The weighted summation was 0.22 multiplied by 1 plus 0.31 multiplied by 1 plus 0.47 multiplied by 1, which equals 1.00. Therefore, the estimated density characterization value at the interpolation location of 120 meters was 1.00.

[0054] Using weighting coefficients of 0.22, 0.31, and 0.47, semivariance vectors of 1.86, 1.84, and 1.79, and a Lagrange multiplier of 0.08, the estimated variance of the interpolation location is calculated. The variance is calculated as 0.22 multiplied by 1.86, 0.31 multiplied by 1.84, 0.47 multiplied by 1.79, and 0.08, totaling approximately 1.91.

[0055] Edge computing nodes load a density-distance mapping table pre-established through calibration experiments. This table is created by recording the correspondence between radar-measured local density values ​​and actual vehicle distances under different congestion scenarios, forming a lookup table. For example, a density value of 1 corresponds to a typical vehicle distance of approximately 8.5 meters. Based on the estimated density value of 1.00, the table is looked up to obtain a vehicle distance of 8.5 meters at this location. Combining lane line geometry information, the density value at the interpolation location of 120 meters is converted into an estimated physical location: a stationary vehicle exists near 120 meters, with a longitudinal coordinate of approximately 120.0 meters, and this location is estimated through spatial interpolation.

[0056] S5. Determine whether the estimated variance exceeds a preset safety threshold. If it does not exceed the threshold, use the estimated physical location as the tail boundary of the dense vehicle queue to calculate the traffic congestion length or generate high-precision event coordinates. If it exceeds the threshold, generate an abnormal state identifier representing the fuzzy positioning and suppress the output of high-precision traffic event alarms based on the area.

[0057] S5 includes the following steps: S51, Estimating the variance With preset safety threshold In comparison, if Then determine the position to be interpolated. The confidence level of the estimated physical location meets the preset requirements, and the estimated physical location is determined as the tail boundary position of the dense vehicle queue; S52. Calculate the congestion queue length under the current traffic flow state based on the queue tail boundary position and the starting position of the dense vehicle queue area, and output the congestion queue length and the queue tail boundary position as traffic event detection results; S53, if Then determine the position to be interpolated. If the location is in an ambiguous state, generate a corresponding abnormal status identifier; S54: In response to the abnormal state identifier, suppress the output of high-precision traffic event alarm based on the location to be interpolated, and report the fuzzy positioning state to the cloud collaboration layer to trigger the auxiliary verification mechanism.

[0058] Specifically, in this embodiment, the preset safety threshold is 2.5. This threshold is set based on the following: through historical data analysis, when the estimated variance is below 2.5, the deviation between the interpolation result and the actual manual measurement value is usually within ±12 meters, meeting the accuracy requirements for congestion length estimation in highway traffic control. The output estimated variance of 1.91 is compared with the safety threshold of 2.5. Since 1.91 is less than 2.5, the confidence level of the estimated physical location at the interpolation position of 120 meters is determined to meet the preset requirements. This estimated physical location at 120 meters is then formally determined as the tail boundary position of the dense vehicle queue.

[0059] The starting position of the dense vehicle queue area, i.e., the head of the queue, has been precisely determined by radar in step S2 as the first stationary vehicle 42.5 meters away from the camera. The tail boundary position is 120 meters away, as determined in step S51. The length of the congested queue under the current traffic flow is calculated as 120.0 meters minus 42.5 meters, resulting in 77.5 meters. Based on the standard passenger car length of 4.5 meters, it is estimated that there are approximately 17 stationary vehicles in this congested queue. The edge computing node packages the congested queue length of 77.5 meters, the tail boundary position of 120.0 meters, and the timestamp into a traffic event detection result, uploads it to the traffic monitoring center platform via the 4G wireless communication module, and displays it on the map of the corresponding road segment as a red heat bar on the monitoring screen.

[0060] In the normal process of this embodiment, steps S53 and S54 are not triggered because the estimated variance of 1.91 is less than the safety threshold of 2.5. To illustrate the complete logic, a scenario for triggering an abnormal state is added: If in another detection cycle, the number of known sampling points is too small, for example, only one, and this sampling point is located at 45 meters, while the interpolation position is still 120 meters, the two are 75 meters apart. At this time, the estimated variance calculated in step S45 is 5.23, which is greater than the safety threshold of 2.5. Based on this, the edge computing node determines that the interpolation position of 120 meters is in a state of positioning ambiguity and generates an abnormal state identification record. This record contains three fields: the abnormal type is "tail positioning ambiguity", the occurrence time is the current timestamp, and the longitudinal range of the affected area is 80 meters to 150 meters.

[0061] In response to the abnormal status indicator, the edge computing node suppresses the output of high-precision traffic event alarms based on the interpolation location of 120.0 meters within the current detection cycle. Specifically, it does not write 120 meters as the queue tail boundary into the regular event record, nor does it push specific congestion length values ​​to the information board. Instead, it reports the event itself (the location is ambiguous) and its metadata to the cloud collaboration layer via an encrypted link. Upon receiving the report, the cloud collaboration layer automatically creates a work order for review and pushes it to the terminal application of the on-duty monitor. The work order suggests paying attention to the congestion situation in the section from K12 + 300 meters to K12 + 450 meters, as the current automatic detection confidence level is low. Simultaneously, if there is an automated drone hangar on the roadside, the cloud can issue a command to trigger a drone to conduct an automatic patrol along this section, transmitting real-time video footage for manual confirmation.

[0062] S54 includes the following steps: S541. The edge computing node packages the abnormal state identifier, the corresponding longitudinal distance coordinates of the position to be interpolated, the estimated variance value, and the current timestamp into a positioning fuzzy event message, and reports it to the cloud collaboration layer through the wireless communication link. S542. After receiving the positioning ambiguity event message, the cloud collaboration layer automatically retrieves the occupancy status of the pre-configured drone hangars for the road segment. If there are idle drones, an inspection route is generated along the lane line of the dense vehicle queue area. The starting point of the route is the sampling point with the largest longitudinal distance among the known spatial sampling points, and the ending point of the route is the position 50 meters forward from the position to be interpolated. The drone is then instructed to perform a video image acquisition task according to the route. S543. The video footage transmitted back by the drone is pushed to the monitoring terminal interface of the cloud collaboration layer, allowing managers to manually confirm the actual position of the tail of the formation. The manual confirmation result is transmitted back to the cloud collaboration layer through the confirmation control on the interface and stored in the historical sample database.

[0063] The above are merely embodiments of the present invention. Commonly known structures and characteristics are not described in detail here. Those skilled in the art are aware of all common technical knowledge in the field prior to the application date or priority date, are aware of all existing technologies in that field, and have the ability to apply conventional experimental methods prior to that date. Those skilled in the art can, under the guidance of this application, improve and implement this solution in combination with their own capabilities. Some typical known structures or methods should not be obstacles for those skilled in the art to implement this application. It should be noted that those skilled in the art can make several modifications and improvements without departing from the structure of the present invention. These should also be considered within the scope of protection of the present invention, and will not affect the effectiveness of the implementation of the present invention or the practicality of the patent. The scope of protection claimed in this application should be determined by the content of its claims, and the specific embodiments described in the specification can be used to interpret the content of the claims.

Claims

1. A traffic incident detection method based on multi-modal edge fusion, characterized in that: Includes the following steps: S1. Obtain multimodal perception data of the current road segment at the edge computing node. The multimodal perception data includes visible light image sequences collected by a visual sensor and sparse point cloud data collected by a millimeter-wave radar sensor. S2. Target detection is performed on the visible light image sequence to identify the dense vehicle queue area that constitutes the stationary traffic flow. At the same time, the precise longitudinal distance values ​​of several discrete vehicles in front of the corresponding dense vehicle queue area are extracted from the sparse point cloud data, and the precise longitudinal distance values ​​are used as known spatial sampling points. S2 includes the following steps: S21. Perform target detection on the current frame of the visible light image sequence and extract the two-dimensional bounding boxes of all motor vehicles and their corresponding vehicle type labels; S22. Perform cross-union matching on the two-dimensional bounding boxes of adjacent frames, calculate the displacement vector of each target in the pixel coordinate system, determine the motor vehicles whose displacement vector amplitude is lower than the preset motion threshold as stationary vehicles, and mark the connected image region composed of continuous stationary vehicles as a dense vehicle queue region. S23. Project the sparse point cloud data onto the pixel plane of the visible light image through a pre-calibrated sensor extrinsic matrix, and filter out point cloud targets whose projection falls within a preset longitudinal range in front of the dense vehicle queue area. S24. Perform static attribute verification based on Doppler velocity on the selected point cloud targets, remove dynamic clutter with Doppler velocity absolute values ​​higher than the preset static threshold, and retain valid radar targets that meet the static conditions. S25. Extract the radial distance and azimuth angle of the effective radar target in the radar coordinate system, and use the radial distance as the accurate longitudinal distance value corresponding to the effective radar target, and use the set of effective radar targets to form the known spatial sampling points; S3. In response to the failure of visual depth estimation in the middle and rear regions of dense vehicle queues in visible light images due to occlusion, a one-dimensional variation function model based on lane line direction is constructed, and the spatial correlation attenuation law under the current traffic flow is calculated using the known spatial sampling points. S4. Based on the one-dimensional variation function model, perform ordinary kriging interpolation on the interpolation position at the rear of the dense vehicle queue area, and output the estimated physical position and corresponding estimated variance of each vehicle in the area. S5. Determine whether the estimated variance exceeds a preset safety threshold. If it does not exceed the threshold, use the estimated physical location as the tail boundary of the dense vehicle queue to calculate the traffic congestion length or generate high-precision event coordinates. If it exceeds the threshold, generate an abnormal state identifier representing the fuzzy positioning and suppress the output of high-precision traffic event alarms based on the area.

2. The traffic incident detection method based on multimodal edge fusion according to claim 1, characterized in that: S3 includes the following steps: S31. Obtain the geometric information of the lane lines covered by the dense vehicle queue area, extract the longitudinal extension direction vector of the lane lines, and arrange the precise longitudinal distance values ​​of the known spatial sampling points into a one-dimensional spatial sequence along this direction vector. ,in This represents the precise longitudinal range value of the i-th effective radar target, where n is the total number of known spatial sampling points; S32, For multiple preset hysteresis distances Calculate the experimental semivariance value corresponding to each lag distance. in This indicates that the distance difference is satisfied. The number of sampling point pairs, This represents the local traffic flow density characteristic value corresponding to the radar target at a longitudinal distance z, which can be calculated from radar reflection intensity or visual detection box density. Let be the longitudinal distance value of the i-th known sampling point; S33. Based on multiple lag intervals and their corresponding experimental semivariogram values, a weighted least squares fitting is performed using a pre-defined theoretical variation function model, with the number of sampling points corresponding to each lag interval as the basis. As weights, the optimal fitting parameters of the theoretical variation function model are obtained, wherein the theoretical variation function model includes one of the spherical model, the exponential model, or the Gaussian model; S34. The fitted theoretical variogram model is determined as the one-dimensional variogram model, which describes the quantitative relationship between vehicle spacing and spatial correlation decay under the current traffic flow state, i.e., the spatial correlation decay law.

3. The traffic incident detection method based on multimodal edge fusion according to claim 2, characterized in that: S4 includes the following steps: S41. Determine the interpolation position at the rear of the dense vehicle queue area. The interpolation position represents the longitudinal distance coordinate of the effective radar sampling point that cannot be obtained in the visual image due to occlusion. S42. Based on the one-dimensional variation function model, calculate the semivariance matrix between each pair of known spatial sampling points. and the position to be interpolated The semivariance vector between each spatial sampling point ,in: in, , Let be the longitudinal distance values ​​of the i-th and j-th known spatial sampling points, respectively. Represents a one-dimensional variation function model; S43. Construct the ordinary Kriging equation system and solve for the Kriging weight coefficient vector. and the Lagrange coefficient μ: S44. Using the Kriging weighting coefficient vector λ, perform a weighted linear combination of the local traffic flow density characterization values ​​corresponding to the known spatial sampling points to obtain the location to be interpolated. Estimated density characterization value at [location] : in, Longitudinal distance Local traffic flow density characterization value at the location; S45. Calculate the interpolation position based on the Kriging weight coefficient vector and the semivariance vector. Estimated variance at location : S46. Based on the pre-calibrated density-distance mapping relationship, the estimated density characterization value will be... Convert to the estimated physical location corresponding to the location to be interpolated.

4. The traffic incident detection method based on multimodal edge fusion according to claim 3, characterized in that: S5 includes the following steps: S51, Estimating the variance With preset safety threshold In comparison, if Then determine the position to be interpolated. The confidence level of the estimated physical location meets the preset requirements, and the estimated physical location is determined as the tail boundary position of the dense vehicle queue; S52. Calculate the congestion queue length under the current traffic flow state based on the queue tail boundary position and the starting position of the dense vehicle queue area, and output the congestion queue length and the queue tail boundary position as traffic event detection results; S53, if Then determine the position to be interpolated. If the location is in an ambiguous state, generate a corresponding abnormal status identifier; S54: In response to the abnormal state identifier, suppress the output of high-precision traffic event alarm based on the location to be interpolated, and report the fuzzy positioning state to the cloud collaboration layer to trigger the auxiliary verification mechanism.

5. The traffic incident detection method based on multimodal edge fusion according to claim 4, characterized in that: S54 includes the following steps: S541. The edge computing node packages the abnormal state identifier, the corresponding longitudinal distance coordinates of the position to be interpolated, the estimated variance value, and the current timestamp into a positioning fuzzy event message, and reports it to the cloud collaboration layer through the wireless communication link. S542. After receiving the positioning ambiguity event message, the cloud collaboration layer automatically retrieves the occupancy status of the pre-configured drone hangars for the road segment. If there are idle drones, an inspection route is generated along the lane line of the dense vehicle queue area. The starting point of the route is the sampling point with the largest longitudinal distance among the known spatial sampling points, and the ending point of the route is the position 50 meters forward from the position to be interpolated. The drone is then instructed to perform a video image acquisition task according to the route. S543. The video footage transmitted back by the drone is pushed to the monitoring terminal interface of the cloud collaboration layer, allowing managers to manually confirm the actual position of the tail of the formation. The manual confirmation result is transmitted back to the cloud collaboration layer through the confirmation control on the interface and stored in the historical sample database.

Images