A vehicle outer dimension dynamic detection system based on multi-sensor asynchronous fusion

Abstract

The application relates to the technical field of traffic, and particularly discloses a vehicle external dimension dynamic detection system based on multi-sensor asynchronous fusion, which aims to solve the technical problems of low static detection efficiency, high deployment cost and great maintenance difficulty of a synchronous trigger scheme, insufficient detection precision of a single sensor, and lack of visual evidence information in the prior art. The system comprises: a horizontal laser radar subsystem, which is used for performing head scanning on a vehicle entering a detection area at a high frame rate, extracting and tracking a bumper corner feature point, and generating millimeter-level precision distance reference data based on a coordinate offset; and a vertical laser radar subsystem, which is used for performing transverse cross-section scanning when the vehicle passes through a preset light curtain area, and generating a continuous slice data set containing vehicle height and vehicle width information. The application can realize millimeter-level precision dynamic detection of the external dimension of the vehicle in a high-speed passing scene, the detection efficiency is significantly improved compared with a static detection scheme, and the law enforcement and evidence collection requirements are met.

Description

Technical Field

[0001] This invention belongs to the field of transportation technology, specifically relating to a dynamic detection system for vehicle outline dimensions based on asynchronous fusion of multiple sensors. Background Technology

[0002] With the continuous deepening of intelligent transportation infrastructure and the enforcement of vehicle overload control, the dynamic and accurate detection of vehicle external dimensions has become a key technical link in the highway safety management system. In practical application scenarios, highway entrances, overload detection stations, and vehicle passage checkpoints have strict compliance requirements for vehicle length, width, height, and other external parameters. Traditional detection methods are gradually revealing obvious limitations when facing complex and ever-changing traffic conditions.

[0003] In practical inspection applications, single-sensor solutions are limited by their inherent physical characteristics, making it difficult to simultaneously meet the comprehensive requirements of multiple technical indicators. While light curtain-type inspection devices possess a certain measurement speed, they cannot acquire detailed texture information of the vehicle surface, resulting in a lack of intuitive visualization and interpretation of the inspection results, making it difficult to provide effective evidence support in subsequent law enforcement and evidence collection stages. Although static laser scanning solutions can acquire relatively fine spatial point cloud data, their reliance on the spatial movement of mechanical structures to complete the scanning coverage leads to long inspection times and low traffic efficiency. In high-traffic scenarios such as highways, this can easily cause vehicle congestion, significantly limiting the system's practical application effectiveness.

[0004] More importantly, the core bottleneck of existing technologies in the field of multi-sensor fusion lies in the challenge of spatiotemporal alignment of asynchronous data. In traffic scenarios, vehicles are in continuous motion, and the operating clock domains of different sensors are independent. Even with so-called synchronous triggering mechanisms, it is difficult to achieve true and strict clock synchronization due to differences in signal processing delays within each sensor and inconsistencies in mechanical response characteristics. When a horizontal lidar performs longitudinal scanning at a frame rate of 60Hz, there will inevitably be a phase deviation between the vertical cross-sectional sampling and the horizontal distance measurement on the time axis. This temporal uncertainty directly leads to errors in spatial position mapping, resulting in distortion of the 3D reconstruction model. Limited by the processing capabilities of existing algorithms, this asynchronous temporal error is often simply ignored or only a coarse timestamp compensation strategy is used, with compensation accuracy only reaching the sub-second level, which cannot meet the accuracy requirements for millimeter-level size detection under high-speed vehicle traffic conditions. When the vehicle speed reaches 80km / h or above, the vehicle can move a distance of more than 2 meters within 100 milliseconds. Such a magnitude of spatial displacement is enough to cause serious misjudgment in the detection of external dimensions. The resulting missed or false detections not only affect the fairness of traffic law enforcement, but also pose a potential threat to the safe operation of infrastructure such as roads and bridges. Summary of the Invention

[0005] This application provides a vehicle outline dimension dynamic detection system based on multi-sensor asynchronous fusion, which aims to solve the technical problems in the prior art such as low static detection efficiency, high deployment cost and maintenance difficulty of synchronous triggering scheme, insufficient detection accuracy of single sensor and lack of visual evidence information.

[0006] The first aspect of this application provides a vehicle outline dimension dynamic detection system based on multi-sensor asynchronous fusion, including: The horizontal lidar subsystem is used to scan the front of vehicles entering the detection area at a high frame rate, extract and track the feature points of the bumper corners, calculate the coordinate offset between adjacent scanning cycles, and generate distance reference data with millimeter-level accuracy based on the coordinate offset. The vertical lidar subsystem is used to perform lateral cross-section scanning at a high frame rate when a vehicle passes through a preset light curtain area, generating a continuous slice dataset containing vehicle height and width information. The industrial camera subsystem is used to acquire high-resolution video streams of vehicles passing through the detection area and obtain vehicle exterior texture information; The edge computing unit is used to receive and process the distance reference data output by the horizontal lidar subsystem, the slice dataset output by the vertical lidar subsystem, and the video stream output by the industrial camera subsystem, to achieve spatiotemporal alignment of asynchronous data from multiple sensors, complete the reconstruction of the three-dimensional point cloud model of the vehicle outline, and perform pixel-level visual fusion of the three-dimensional point cloud model with the video stream data collected by the industrial camera subsystem. The over-limit determination and handling module is used to calculate the vehicle outline size parameters based on the 3D point cloud model of the vehicle outline reconstructed by the edge computing unit, compare the calculated vehicle outline size parameters with the preset over-limit threshold, determine whether the vehicle exceeds the limit, and trigger a local audio-visual alarm device when the determination result is over-limit, generate an evidence package containing vehicle image, detection parameters, passage time and passage location, and upload the evidence package to a preset cloud server.

[0007] In one optional implementation, the horizontal lidar subsystem includes a lidar sensor, a feature extraction module, and a coordinate offset calculation module; the lidar sensor is used to continuously scan the front end of the vehicle at a frame rate of 60 Hz; the feature extraction module is used to extract bumper corner feature points from the raw point cloud data collected by the lidar sensor; the coordinate offset calculation module is used to calculate the position coordinate difference of the same feature point in adjacent scanning cycles, and generate a distance reference with millimeter-level accuracy based on the position coordinate difference.

[0008] In one optional implementation, the vertical lidar subsystem includes a vertically mounted lidar sensor and a slice generation module; the lidar sensor is used to perform lateral cross-sectional scanning of vehicles passing through the light curtain area at a frame rate of 60 Hz; the slice generation module is used to integrate the data obtained from each lateral cross-sectional scan into slice data containing vehicle height and width information, and arrange the slice data obtained from multiple consecutive scans in chronological order to form a continuous slice dataset.

[0009] In one optional implementation, the edge computing unit includes a data receiving module, a spatiotemporal alignment module, a 3D reconstruction module, and a visual fusion module. The data receiving module asynchronously receives raw data from the horizontal LiDAR subsystem, the vertical LiDAR subsystem, and the industrial camera subsystem. The spatiotemporal alignment module uses a distance reference generated by the horizontal LiDAR subsystem as a time reference to spatiotemporally align the slice dataset of the vertical LiDAR subsystem with the distance reference. The 3D reconstruction module reconstructs a 3D point cloud model of the vehicle outline based on the aligned slice dataset. The visual fusion module jointly calibrates the reconstructed 3D point cloud model with the video stream data acquired by the industrial camera to achieve pixel-level visual fusion.

[0010] In one optional implementation, the spatiotemporal alignment module is specifically used to: assign a corresponding time tag to each slice of data generated by the vertical lidar subsystem based on the timestamp information contained in the distance reference output by the horizontal lidar subsystem; calculate the spatial position mapping relationship between the slice data and the distance reference based on the time interval between adjacent horizontal lidar scanning cycles and the vehicle traffic speed; and project the slice data onto a unified coordinate system based on the distance reference according to the spatial position mapping relationship.

[0011] In one optional implementation, the 3D reconstruction module is specifically used to: read the spatiotemporally aligned continuous slice dataset; extract vehicle outline edge points from each slice data; connect the outline edge points in adjacent slices to generate the skeleton line of the vehicle outline; calculate the dimensions of the vehicle outline in the length direction based on the skeleton line and the distance reference data; and comprehensively calculate the dimensions of the vehicle outline in the height and width directions with the dimensions in the length direction to generate a complete 3D point cloud model.

[0012] In one optional implementation, the visual fusion module is specifically used to: pre-establish a joint calibration relationship between the lidar coordinate system and the industrial camera coordinate system; based on the joint calibration relationship, project the three-dimensional coordinate points in the three-dimensional point cloud model onto the pixel coordinate system of the video frame acquired by the industrial camera; and overlay the projected three-dimensional point cloud with the corresponding video frame to generate a fused visual image.

[0013] In one optional implementation, the over-limit determination and handling module includes a size calculation unit, an over-limit determination unit, an alarm unit, and an evidence generation unit. The size calculation unit is used to extract the vehicle's length, height, and width dimensions from the 3D point cloud model. The over-limit determination unit is used to compare the extracted length, height, and width dimensions with preset length, height, and width limits, respectively, and determines that an over-limit is being imposed when any dimension exceeds the corresponding limit. The alarm unit is used to trigger a local audible and visual alarm device to issue an alarm signal when an over-limit is determined. The evidence generation unit is used to capture an image of the vehicle's exterior captured by an industrial camera when an over-limit is determined, and associate the detected size parameters, detection time, and detection location information to generate a complete evidence package.

[0014] In one optional implementation, the evidence generation unit is further configured to store the generated evidence package to a local storage device and upload the evidence package to a preset cloud server via a network communication interface for subsequent law enforcement evidence collection.

[0015] In one optional implementation, the over-limit determination and handling module further includes a data storage unit, which is used to store the vehicle's passage data, including passage time, passage location and detected size parameters, into a local database when the determination result is that the vehicle does not exceed the limit, for subsequent data query and statistical analysis.

[0016] The vehicle outline dimension dynamic detection system based on multi-sensor asynchronous fusion provided in this application achieves high-precision dynamic detection of vehicle outline dimensions through the collaborative operation of a horizontal LiDAR subsystem and a vertical LiDAR subsystem. The horizontal LiDAR subsystem scans the front of the vehicle at a high frame rate, generating a distance reference with millimeter-level accuracy through feature point tracking and coordinate offset calculation, solving the spatiotemporal alignment problem of asynchronous data from multiple sensors and achieving accurate ranging without complex synchronization triggering devices. After spatiotemporally aligning the continuous slice dataset generated by the vertical LiDAR subsystem with the horizontal distance reference, a 3D point cloud model of the vehicle outline can be efficiently reconstructed, balancing detection efficiency and reconstruction accuracy. The high-resolution video stream acquired by the industrial camera subsystem is pixel-level visually fused with the 3D point cloud model, significantly improving the visualization effect of the detection results and the evidentiary validity of over-limit evidence collection. The over-limit judgment and handling module realizes closed-loop management of the entire process from detection and judgment to alarm, evidence collection, and control.

[0017] Compared with the prior art, the advantages and positive effects of the present invention are as follows: 1. The detection accuracy and efficiency are significantly improved. By adopting a 60 Hz high frame rate scanning combined with asynchronous spatiotemporal alignment technology, the dynamic detection of vehicle outline dimensions with millimeter-level accuracy is achieved in high-speed traffic scenarios, and the detection efficiency is more than three times higher than that of static detection schemes.

[0018] 2. Convenient deployment and maintenance, no need for high-precision synchronous triggering devices, reducing equipment deployment costs and maintenance difficulty. The system can be flexibly adapted to various application scenarios such as highway inspection stations, vehicle inspection stations, and urban road checkpoints.

[0019] 3. Enhanced evidence collection effectiveness: By integrating laser point cloud data with visual texture information from industrial cameras, the generated evidence package contains complete information such as time, location, size parameters, and vehicle exterior images, meeting the law enforcement evidence collection needs for overload control.

[0020] 4. Wide range of applicable scenarios: The system is compatible with different vehicle types and traffic speed requirements, and can simultaneously meet the detection needs of various application scenarios such as highway overload detection, vehicle annual inspection, and urban traffic management, providing reliable technical support for intelligent traffic management. Attached Figure Description

[0021] Figure 1 This is a schematic diagram of the overall technical architecture of a vehicle outline dimension dynamic detection system based on asynchronous fusion of multiple sensors, according to an embodiment of this application.

[0022] Figure 2 This is a schematic diagram of the core principle framework for spatiotemporal alignment and 3D point cloud reconstruction based on multi-sensor asynchronous data in embodiments of this application.

[0023] Figure 3 This is a logical flowchart of the collaborative acquisition of horizontal and vertical lidar subsystems based on an embodiment of this application.

[0024] Figure 4 This is a schematic diagram illustrating the relationship between the 3D point cloud model and the pixel-level visual fusion of the video stream from the industrial camera in the edge computing unit based on an embodiment of this application.

[0025] Figure 5 This is a flowchart illustrating the process of determining limits and generating evidence packages based on embodiments of this application. Detailed Implementation

[0026] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0027] In the accompanying drawings, the size of constituent elements, the thickness of layers, or areas may sometimes be exaggerated for clarity. Therefore, any implementation of this disclosure is not necessarily limited to the dimensions shown in the drawings, and the shapes and sizes of the components in the drawings do not reflect true proportions. Furthermore, the drawings schematically illustrate ideal examples, and any implementation of this disclosure is not limited to the shapes or values ​​shown in the drawings.

[0028] Reference Figure 1 , Figure 1 This is a schematic diagram of the overall technical architecture of a vehicle outline dimension dynamic detection system based on multi-sensor asynchronous fusion proposed in an embodiment of this application. Figure 1 As shown, the system includes: a horizontal lidar subsystem, a vertical lidar subsystem, an industrial camera subsystem, an edge computing unit, and an over-limit judgment and handling module.

[0029] The horizontal LiDAR subsystem performs high-frequency frame rate scanning of the vehicle's front end as it enters the detection area, extracts and tracks bumper corner feature points, calculates coordinate offsets between adjacent scanning cycles, and generates millimeter-level precision distance reference data based on these offsets. The vertical LiDAR subsystem performs high-frequency frame rate lateral cross-sectional scanning as the vehicle passes through a preset light curtain area, generating a continuous slice dataset containing vehicle height and width information. The industrial camera subsystem acquires high-resolution video streams of vehicles passing through the detection area, obtaining vehicle exterior texture information. The edge computing unit receives and processes the distance reference data output by the horizontal LiDAR subsystem, the slice dataset output by the vertical LiDAR subsystem, and the video stream output by the industrial camera subsystem. This enables spatiotemporal alignment of asynchronous data from multiple sensors, reconstructs a 3D point cloud model of the vehicle's outline, and performs pixel-level visual fusion of the 3D point cloud model with the video stream data acquired by the industrial camera subsystem. The over-limit determination and handling module is used to calculate the vehicle outline size parameters based on the 3D point cloud model of the vehicle outline reconstructed by the edge computing unit, compare the calculated vehicle outline size parameters with the preset over-limit threshold, determine whether the vehicle exceeds the limit, and trigger a local audio-visual alarm device when the determination result is over-limit, generate an evidence package containing vehicle image, detection parameters, passage time and passage location, and upload the evidence package to a preset cloud server.

[0030] In this embodiment, high-precision dynamic detection of vehicle outline dimensions is achieved through the collaborative operation of the horizontal and vertical lidar subsystems. The horizontal lidar subsystem scans the front of the vehicle at a high frame rate, generating a distance reference with millimeter-level accuracy through feature point tracking and coordinate offset calculation. This solves the problem of spatiotemporal alignment of asynchronous data from multiple sensors, enabling accurate ranging without the need for complex synchronization triggering devices. After spatiotemporally aligning the continuous slice dataset generated by the vertical lidar subsystem with the horizontal distance reference, a 3D point cloud model of the vehicle outline can be efficiently reconstructed, balancing detection efficiency and reconstruction accuracy. The high-resolution video stream acquired by the industrial camera subsystem is fused with the 3D point cloud model at the pixel level, significantly improving the visualization effect of the detection results and the evidentiary validity of over-limit evidence collection. The over-limit judgment and handling module realizes a closed-loop management of the entire process from detection and judgment to alarm, evidence collection, and control.

[0031] In one embodiment, this application also provides a vehicle outline dimension dynamic detection system based on multi-sensor asynchronous fusion. In this system, the horizontal lidar subsystem includes a lidar sensor, a feature extraction module, and a coordinate offset calculation module. The lidar sensor is used to continuously scan the front end of the vehicle at a frame rate of 60 Hz. The feature extraction module is used to extract bumper corner feature points from the raw point cloud data collected by the lidar sensor. The coordinate offset calculation module is used to calculate the position coordinate difference of the same feature point in adjacent scanning cycles and generate a distance reference with millimeter-level accuracy based on the position coordinate difference.

[0032] In this embodiment, the lidar sensor of the horizontal lidar subsystem continuously scans the front end of a vehicle entering the detection area at a high frame rate of 60 Hz, acquiring raw point cloud data of the vehicle's front face. The feature extraction module, based on a pre-trained corner detection model, automatically identifies and extracts bumper corner feature points from the raw point cloud data. These feature points are typically located at the intersection of the bumper below the license plate and the wheel arch, possessing stable geometric features that facilitate continuous tracking throughout the entire passage. The coordinate offset calculation module records the three-dimensional spatial position of the same feature point in adjacent scanning cycles. By calculating the position coordinate difference between adjacent cycles, the displacement of the vehicle relative to the detection baseline is obtained. This displacement is the distance reference data, with an accuracy down to the millimeter level. For example, suppose the three-dimensional coordinates of a feature point in the first scanning cycle are (X1, Y1, Z1) and the three-dimensional coordinates in the second scanning cycle are (X2, Y2, Z2). Then the coordinate offset between adjacent scanning cycles is (ΔX, ΔY, ΔZ) = (X2-X1, Y2-Y1, Z2-Z1), where ΔX is the distance offset along the vehicle's travel direction, and its value directly reflects the vehicle's travel distance.

[0033] In this embodiment, the horizontal lidar subsystem continuously tracks the positional changes of feature points at the bumper corners, enabling it to acquire real-time information about the vehicle's forward distance during passage. This distance information serves as the temporal and spatial reference for the entire detection system, providing a unified reference system for aligning the slice data of the subsequent vertical lidar subsystem. Compared to traditional synchronous triggering schemes, this system eliminates the need for complex hardware synchronization devices, reducing equipment deployment costs and maintenance complexity while ensuring detection accuracy.

[0034] In one embodiment, this application also provides a vehicle outline dimension dynamic detection system based on multi-sensor asynchronous fusion. In this system, the vertical lidar subsystem includes a vertically mounted lidar sensor and a slice generation module. The lidar sensor is used to perform lateral cross-sectional scanning of vehicles passing through the light curtain area at a frame rate of 60 Hz. The slice generation module is used to integrate the data obtained from each lateral cross-sectional scan into slice data containing vehicle height and width information, and arrange the slice data obtained from multiple consecutive scans in chronological order to form a continuous slice dataset.

[0035] In this embodiment, the vertical lidar subsystem employs a vertically mounted lidar sensor, whose scanning plane is perpendicular to the vehicle's travel direction. When the vehicle passes through a preset light curtain area at a certain speed, the vertical lidar sensor performs continuous transverse cross-sectional scans of the vehicle at a frame rate of 60 Hz. Each scan acquires point cloud data of the vehicle on the same cross-section, which contains the vehicle's contour information at that cross-section. The slice generation module processes the data obtained from each transverse cross-sectional scan, extracting the vehicle's top and bottom edges and left and right edges at that cross-section to obtain the vehicle's height and width information. Specifically, the slice data includes the coordinates of the highest point (the height of the vehicle's top from the ground), the coordinates of the leftmost point, and the coordinates of the rightmost point (the boundary in the vehicle's width direction) at the cross-section, as well as the timestamp information of the cross-section. This timestamp is used for alignment with the distance reference generated by the horizontal lidar subsystem.

[0036] In this embodiment, as the vehicle continuously passes through the light curtain area, the vertical LiDAR subsystem generates a series of continuous slice data. These slice data are arranged in chronological order to form a continuous slice dataset. Each slice in the continuous slice dataset corresponds to a transverse cross-section of the vehicle. By combining all the slice data, the contour changes of the vehicle along its length can be completely depicted. It should be noted that since the scanning of the vertical LiDAR sensor and the horizontal LiDAR subsystem are independent of each other and operate under different time sequences, there is a temporal and spatial deviation between the slice data collected by the vertical LiDAR subsystem and the distance reference generated by the horizontal LiDAR subsystem. This requires spatiotemporal alignment processing in the subsequent edge computing unit.

[0037] In one embodiment, this application also provides a vehicle outline dynamic detection system based on multi-sensor asynchronous fusion. In this system, the edge computing unit includes a data receiving module, a spatiotemporal alignment module, a 3D reconstruction module, and a visual fusion module. The data receiving module is used to asynchronously receive raw data from the horizontal LiDAR subsystem, the vertical LiDAR subsystem, and the industrial camera subsystem. The spatiotemporal alignment module is used to spatiotemporally align the slice dataset of the vertical LiDAR subsystem with the distance reference generated by the horizontal LiDAR subsystem, using the distance reference generated by the horizontal LiDAR subsystem as the time reference. The 3D reconstruction module is used to reconstruct a 3D point cloud model of the vehicle outline based on the aligned slice dataset. The visual fusion module is used to jointly calibrate the reconstructed 3D point cloud model with the video stream data acquired by the industrial camera to achieve pixel-level visual fusion.

[0038] In this embodiment, the edge computing unit is the core of the entire system's data processing, responsible for receiving, processing, and fusing raw data from three different sensors. The data receiving module employs asynchronous reception, independently receiving data from the horizontal LiDAR subsystem, the vertical LiDAR subsystem, and the industrial camera subsystem. After entering the data receiving module, the three data streams are respectively placed into their corresponding data buffer queues for subsequent processing. The data receiving module assigns a timestamp to each data stream, recording the moment the data arrives at the edge computing unit, which is used for subsequent time-series analysis and data correlation.

[0039] In this embodiment, the spatiotemporal alignment module uses the distance reference generated by the horizontal LiDAR subsystem as the time reference and spatiotemporally aligns the slice dataset of the vertical LiDAR subsystem with this distance reference. The specific alignment principle will be described in detail in subsequent embodiments. The 3D reconstruction module reads the spatiotemporally aligned continuous slice dataset and reconstructs a 3D point cloud model of the vehicle outline based on the edge point information in the slice data. The visual fusion module pre-establishes a joint calibration relationship between the LiDAR coordinate system and the industrial camera coordinate system. Based on this joint calibration relationship, it projects the 3D coordinate points in the 3D point cloud model onto the pixel coordinate system of the video frames acquired by the industrial camera, achieving pixel-level visual fusion and generating a fused visualization image.

[0040] In one embodiment, this application also provides a vehicle outline size dynamic detection system based on multi-sensor asynchronous fusion. In this system, the spatiotemporal alignment module is specifically used to: assign a corresponding time tag to each slice data generated by the vertical lidar subsystem according to the timestamp information contained in the distance reference output by the horizontal lidar subsystem; calculate the spatial position mapping relationship between the slice data and the distance reference based on the time interval between adjacent horizontal lidar scanning cycles and the vehicle speed; and project the slice data onto a unified coordinate system based on the distance reference according to the spatial position mapping relationship.

[0041] In this embodiment, the core function of the spatiotemporal alignment module is to solve the asynchronous data fusion problem between the horizontal and vertical lidar subsystems. Since the two subsystems scan using independent time sequences, there are time and phase differences between the data, requiring alignment using a specific algorithm. Specifically, firstly, based on the timestamp information in the distance reference, each slice of data is assigned a corresponding time label. This time label reflects the index of the vehicle's position in the horizontal distance reference sequence. For example, if a vehicle passes through the detection area at a speed of 5 meters per second, and the time interval between adjacent horizontal lidar scan cycles is 16.67 milliseconds (corresponding to a 60 Hz frame rate), then the distance the vehicle travels between adjacent scan cycles is approximately 8.35 millimeters. The first slice of data acquired by the vertical lidar subsystem corresponds to the moment the vehicle just enters the light curtain area; this moment corresponds to the first node in the horizontal distance reference sequence, and subsequent slices are mapped sequentially.

[0042] In this embodiment, the spatiotemporal alignment module calculates the spatial position mapping relationship between the slice data and the distance reference based on the time interval between adjacent horizontal lidar scanning cycles and the vehicle speed. The specific calculation formula is as follows: Let the time interval between adjacent horizontal lidar scanning cycles be Δt, and the vehicle speed be v, then the spatial displacement between adjacent scanning cycles is Δs = v × Δt. For the i-th slice data collected by the vertical lidar subsystem, its corresponding vehicle position can be represented as Pi = P0 + i × Δs, where P0 is the initial position corresponding to the vehicle entering the light curtain area. Substituting this position into a unified coordinate system based on the distance reference, spatial position alignment is achieved. After spatiotemporal alignment processing, the slice dataset of the vertical lidar subsystem and the distance reference of the horizontal lidar subsystem are consistent in time and space, providing an accurate data foundation for subsequent 3D reconstruction.

[0043] In one embodiment, this application also provides a vehicle outline size dynamic detection system based on multi-sensor asynchronous fusion. In this system, the three-dimensional reconstruction module is specifically used for: reading a spatiotemporally aligned continuous slice dataset; extracting vehicle outline edge points from each slice data; connecting the outline edge points in adjacent slices to generate a skeleton line of the vehicle outline; calculating the vehicle outline size in the length direction based on the skeleton line and the distance reference data; and comprehensively calculating the vehicle outline size in the height and width directions with the length direction to generate a complete three-dimensional point cloud model.

[0044] In this embodiment, the implementation process of the 3D reconstruction module includes the following steps: First, read the continuous slice dataset after spatiotemporal alignment processing. Each slice in this dataset contains vehicle contour information at the corresponding cross-section. Second, extract the vehicle contour edge points from each slice, including the highest, lowest, leftmost, and rightmost points at that cross-section. Third, connect the contour edge points in adjacent slices to generate the skeleton line of the vehicle outline. This skeleton line describes the orientation of the vehicle's outer surface in 3D space. Fourth, based on the skeleton line and distance reference data, calculate the dimensions of the vehicle outline in the length direction (i.e., the direction of travel), specifically the distance between the first and last slice positions. Fifth, combine the dimensional information in the height and width directions to generate a complete 3D point cloud model.

[0045] In this embodiment, the vehicle's height is calculated as follows: extract the highest point from all slice data, subtract the lowest point (i.e., the ground reflection point) from the maximum value to obtain the vehicle's total height. The width is estimated by averaging or taking the maximum width value from all slice data. The length is calculated by subtracting the spatial position corresponding to the first slice from the spatial position corresponding to the last slice to obtain the vehicle's length. By combining the dimensions in these three directions, the complete three-dimensional dimensional parameters of the vehicle's outline can be obtained.

[0046] Reference Figure 2 , Figure 2 This is a schematic diagram illustrating the core principle framework of spatiotemporal alignment of multi-sensor asynchronous data and 3D point cloud reconstruction, as shown in an embodiment of this application. Figure 2As shown, the spatiotemporal alignment and 3D point cloud reconstruction of asynchronous multi-sensor data mainly includes four core stages: data synchronization, slice alignment, skeleton line generation, and point cloud fusion. The data synchronization stage is responsible for temporally aligning the data from the horizontal and vertical lidar subsystems; the slice alignment stage projects the slice data into a unified spatial coordinate system; the skeleton line generation stage generates the skeleton lines of the vehicle outline based on the aligned slice data; and the point cloud fusion stage expands the skeleton lines into a complete 3D point cloud model.

[0047] In one embodiment, this application also provides a vehicle outline dimension dynamic detection system based on multi-sensor asynchronous fusion. In this system, the visual fusion module is specifically used to: pre-establish a joint calibration relationship between the lidar coordinate system and the industrial camera coordinate system; based on the joint calibration relationship, project the three-dimensional coordinate points in the three-dimensional point cloud model onto the pixel coordinate system of the video frame acquired by the industrial camera; and overlay the projected three-dimensional point cloud with the corresponding video frame to generate a fused visualization image.

[0048] In this embodiment, the joint calibration of the LiDAR coordinate system and the industrial camera coordinate system is a prerequisite for achieving pixel-level visual fusion. The joint calibration method can employ either the calibration board method or the motion calibration method, specifically including the following steps: First, a calibration board with a specific pattern is placed within the detection area. The calibration board contains known three-dimensional feature points. Then, the LiDAR point cloud data from the calibration board and the image data from the industrial camera are acquired simultaneously. Next, a feature matching algorithm is used to establish the correspondence between the three-dimensional feature points and the two-dimensional pixels. Finally, the transformation matrix between the two coordinate systems is solved using the least squares method or the PnP algorithm. This transformation matrix represents the joint calibration relationship in this system.

[0049] In this embodiment, based on a pre-established joint calibration relationship, each 3D coordinate point (X, Y, Z) in the 3D point cloud model is projected onto the pixel coordinate system of the video frame acquired by the industrial camera to obtain the pixel coordinates (u, v) of that point in the image. The projection formula is: s·[u, v, 1]^T = K·[R|t]·[X, Y, Z, 1]^T, where K is the intrinsic parameter matrix of the industrial camera, [R|t] is the extrinsic parameter matrix (including the rotation matrix R and the translation vector t), and s is the scale factor. The 3D point cloud obtained by projection is superimposed on the corresponding video frame to form a fused visualization image. This image simultaneously contains the depth information of the laser point cloud and the texture information of the industrial camera, significantly improving the visualization effect of the detection results.

[0050] Reference Figure 3 , Figure 3 This is a logical flow diagram illustrating the collaborative acquisition of data by a horizontal lidar subsystem and a vertical lidar subsystem, as shown in an embodiment of this application. Figure 3As shown, the horizontal LiDAR subsystem scans the front of the vehicle at a frame rate of 60 Hz, extracts feature points at the bumper corners, calculates coordinate offsets, and generates distance reference data. The vertical LiDAR subsystem also performs a lateral cross-sectional scan of the vehicle at a frame rate of 60 Hz, generating slice data. The data streams from both subsystems are fed in parallel into the edge computing unit, where spatiotemporal alignment and 3D reconstruction are performed, ultimately generating a 3D point cloud model of the vehicle's outline.

[0051] Reference Figure 4 , Figure 4 This is a schematic diagram illustrating the relationship between a 3D point cloud model and a pixel-level visual fusion of an industrial camera video stream in an edge computing unit, as shown in an embodiment of this application. Figure 4 As shown, the edge computing unit first receives a 3D point cloud model and a video stream captured by an industrial camera. Through a joint calibration relationship, it projects the 3D point cloud onto the pixel coordinate system of the video frame. Then, it overlays the projected 3D point cloud with the video frame to generate a fused visualization image. This visualization image clearly displays the vehicle's 3D contours and surface texture, providing an intuitive basis for over-limit detection.

[0052] In one embodiment, this application also provides a vehicle outline dimension dynamic detection system based on multi-sensor asynchronous fusion. In this system, the over-limit judgment and handling module includes a dimension calculation unit, an over-limit judgment unit, an alarm unit, and an evidence generation unit. The dimension calculation unit is used to extract the vehicle length, height, and width dimension data from the three-dimensional point cloud model. The over-limit judgment unit is used to compare the extracted vehicle length, height, and width dimension data with preset length limits, height limits, and width limits, respectively. When any dimension data exceeds the corresponding limit, it is judged as an over-limit. The alarm unit is used to trigger a local audible and visual alarm device to issue an alarm signal when an over-limit is determined. The evidence generation unit is used to capture an image containing the vehicle's exterior captured by an industrial camera when an over-limit is determined, and associate the detected dimension parameters, detection time, and detection location information to generate a complete evidence package.

[0053] In this embodiment, the over-limit determination and handling module implements a complete handling process from size calculation, over-limit determination to alarm evidence collection. The size calculation unit automatically extracts the vehicle's external dimensions from the 3D point cloud model, including vehicle length, vehicle height, and vehicle width. The specific extraction method is as follows: vehicle length is the distance between the first and last slice positions; vehicle height is the difference between the highest and lowest points in all slices; vehicle width is the maximum (or average) difference between the left and right edge points in all slices.

[0054] In this embodiment, the over-limit determination unit compares the extracted dimensional data with preset limits to determine whether the vehicle exceeds the limits. The preset limits can be set according to different application scenarios. For example, in a highway over-limit detection scenario, the limits are 4 meters in height, 2.5 meters in width, and 18 meters in length; in a vehicle annual inspection scenario, the limits are set according to relevant standards. When any dimensional data exceeds the corresponding limit, the over-limit determination unit immediately outputs the over-limit determination result and sends it to the alarm unit and the evidence generation unit.

[0055] In this embodiment, upon receiving the over-limit determination result, the alarm unit immediately triggers the local audible and visual alarm device, issuing an audible and visual alarm signal to alert on-site personnel. The alarm device may include an audible siren and a warning light, distinguishing different over-limit types through different alarm modes. Upon receiving the over-limit determination result, the evidence generation unit automatically captures an image of the vehicle's exterior taken by the industrial camera at the current moment, and associates it with the vehicle's detected size parameters, detection time, and detection location information to generate a complete evidence package. The evidence package includes at least one vehicle image (which may include images of the front, side, and rear of the vehicle), the vehicle's actual size parameters (specific measurements of vehicle length, height, and width), the detection time (accurate to the second), the detection location (which can be determined via GPS coordinates or a preset location marker), and the over-limit determination result (which item(s) exceeded the limit).

[0056] In one embodiment, this application also provides a vehicle outline size dynamic detection system based on multi-sensor asynchronous fusion. In this system, the evidence generation unit is further used to store the generated evidence package to a local storage device and upload the evidence package to a preset cloud server through a network communication interface for subsequent law enforcement evidence collection.

[0057] In this embodiment, the evidence package generated by the evidence generation unit is first stored on a local storage device, which can be a solid-state drive or a redundant disk array to ensure data security and reliability. Simultaneously, the evidence generation unit uploads the evidence package to a preset cloud server via a network communication interface. The cloud server can be a private cloud of the law enforcement agency or an authorized public cloud service. The uploaded evidence package is encrypted to ensure data security during transmission. After receiving the evidence package, the cloud server can use it for subsequent law enforcement evidence collection, and law enforcement personnel can view the detailed contents of the evidence package with exclusive access permissions.

[0058] In one embodiment, this application also provides a vehicle outline dimension dynamic detection system based on multi-sensor asynchronous fusion. In this system, the over-limit judgment and handling module further includes a data storage unit, which is used to store the vehicle's passage data, including passage time, passage location and detected dimension parameters, into a local database when the judgment result is that the vehicle does not exceed the limit, for subsequent data query and statistical analysis.

[0059] In this embodiment, for vehicles whose assessment result indicates they are not exceeding the limits, the data storage unit of the over-limit assessment and handling module stores their passage data in a local database. The stored data includes the vehicle's passage time (accurate to the second), passage location (location identifier of the detection station), and the detected dimensional parameters (specific measurements of vehicle length, height, and width). This data can be used for subsequent data querying and statistical analysis, such as statistically analyzing the size distribution of vehicles passing through a certain period and the proportion of over-limit vehicles, providing data support for traffic management decisions.

[0060] Reference Figure 5 , Figure 5 This is a flowchart illustrating the process of determining limits and generating evidence packages, as shown in an embodiment of this application. Figure 5 As shown, after receiving the 3D point cloud model, the over-limit detection and handling module first performs size calculations, extracting the vehicle length, height, and width parameters. Then, it compares the calculated size parameters with a preset over-limit threshold. If it is determined to be over-limit, the alarm unit is triggered to issue an audible and visual alarm, and simultaneously, the evidence generation unit is triggered to generate an evidence package, which is then uploaded to the cloud server. If it is determined not to be over-limit, the passage data is stored in the local database. The entire handling process is automated, requiring no manual intervention, achieving efficient over-limit detection and evidence collection.

[0061] The proposed vehicle outline dimension dynamic detection system based on multi-sensor asynchronous fusion has the following significant advantages: Significantly improved detection accuracy and efficiency: Utilizing a 60Hz high frame rate scan combined with asynchronous spatiotemporal alignment technology, it achieves millimeter-level precision dynamic detection of vehicle outline dimensions in high-speed traffic scenarios, increasing detection efficiency by more than three times compared to static detection schemes. Convenient deployment and maintenance: The system eliminates the need for high-precision synchronous triggering devices, reducing equipment deployment costs and maintenance difficulty. It can flexibly adapt to various application scenarios such as highway inspection stations, vehicle inspection stations, and urban road checkpoints. Enhanced evidence collection effectiveness: By fusing laser point cloud data with industrial camera visual texture information, the generated evidence package includes complete information such as time, location, size parameters, and vehicle appearance images, meeting the law enforcement evidence collection needs for overload control. Wide applicability: The system is compatible with different vehicle types and traffic speed requirements, simultaneously meeting the detection needs of various application scenarios such as highway overload detection, vehicle inspection, and urban traffic management, providing reliable technical support for intelligent traffic management.

[0062] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0063] Those skilled in the art will understand that embodiments of this application can be provided as methods, apparatus, or computer program products. Therefore, embodiments of this application can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of this application can take the form of computer program products implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0064] This application describes embodiments with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0065] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0066] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal equipment, causing a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0067] Although preferred embodiments of the present application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present application.

[0068] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0069] The above provides a detailed description of the vehicle outline dimension dynamic detection system based on multi-sensor asynchronous fusion provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A dynamic detection system for vehicle outline dimensions based on multi-sensor asynchronous fusion, characterized in that, include: The horizontal lidar subsystem is used to scan the front of vehicles entering the detection area at a high frame rate, extract the feature points of the bumper corners, calculate the coordinate offset between adjacent scanning cycles, and generate distance reference data based on the coordinate offset. The vertical lidar subsystem is used to perform lateral cross-section scanning at a high frame rate when a vehicle passes through a preset light curtain area, generating a continuous slice dataset containing vehicle height and width information; the industrial camera subsystem is used to acquire high-resolution video streams of vehicles passing through the detection area. An edge computing unit is used to receive and asynchronously process the distance reference data, the slice dataset, and the video stream, realize the spatiotemporal alignment of asynchronous data from multiple sensors, complete the reconstruction of the three-dimensional point cloud model of the vehicle outline, and perform pixel-level visual fusion of the three-dimensional point cloud model and the video stream data. The over-limit judgment and handling module is used to calculate the vehicle's external dimension parameters based on the three-dimensional point cloud model, compare the dimension parameters with a preset over-limit threshold, determine whether the vehicle exceeds the limit, and trigger a local audio-visual alarm device, generate an evidence package containing vehicle images, detection parameters, passage time and passage location when the judgment result is over-limit, and upload the evidence package to a preset cloud server.

2. The vehicle outline dimension dynamic detection system according to claim 1, characterized in that, The horizontal lidar subsystem includes a lidar sensor, a feature extraction module, and a coordinate offset calculation module. The lidar sensor is used to continuously scan the front of the vehicle at a frame rate of 60 Hz. The feature extraction module is used to extract the corner feature points of the bumper from the raw point cloud data. The coordinate offset calculation module is used to calculate the position coordinate difference of the same feature point in adjacent scanning cycles and generate a distance reference with millimeter-level accuracy based on the position coordinate difference.

3. The vehicle outline dimension dynamic detection system according to claim 1, characterized in that, The vertical lidar subsystem includes a vertically mounted lidar sensor and a slice generation module; the lidar sensor is used to perform lateral cross-sectional scanning of vehicles passing through the light curtain area at a frame rate of 60 Hz; the slice generation module is used to integrate the data obtained from each lateral cross-sectional scan into slice data containing vehicle height and width information, and arrange them in chronological order to form a continuous slice dataset.

4. The vehicle outline dimension dynamic detection system according to claim 1, characterized in that, The edge computing unit includes a data receiving module, a spatiotemporal alignment module, a 3D reconstruction module, and a visual fusion module; the data receiving module is used to asynchronously receive raw data from the horizontal lidar subsystem, the vertical lidar subsystem, and the industrial camera subsystem; the spatiotemporal alignment module is used to spatiotemporally align the slice dataset with the distance reference using the distance reference as the time reference; The 3D reconstruction module is used to reconstruct a 3D point cloud model of the vehicle outline based on the aligned slice dataset; the visual fusion module is used to jointly calibrate the reconstructed 3D point cloud model with the video stream data to achieve pixel-level visual fusion.

5. The vehicle outline dimension dynamic detection system according to claim 4, characterized in that, The spatiotemporal alignment module assigns a corresponding time label to each slice data according to the timestamp information contained in the distance reference; calculates the spatial position mapping relationship between the slice data and the distance reference based on the time interval between adjacent scanning cycles and the vehicle traffic speed; and projects the slice data onto a unified coordinate system based on the distance reference according to the spatial position mapping relationship.

6. The vehicle outline dimension dynamic detection system according to claim 4, characterized in that, The 3D reconstruction module reads the spatiotemporally aligned continuous slice dataset and extracts the vehicle outline edge points from each slice data. Connect the contour edge points in adjacent slices to generate the skeleton line of the vehicle outline; Based on the skeleton lines and the distance reference data, the dimensions of the vehicle outline in the length direction are calculated; the dimensions of the vehicle outline in the height and width directions are combined with the dimensions in the length direction to generate a complete three-dimensional point cloud model.

7. The vehicle outline dimension dynamic detection system according to claim 4, characterized in that, The visual fusion module pre-establishes a joint calibration relationship between the lidar coordinate system and the industrial camera coordinate system; based on the joint calibration relationship, it projects the three-dimensional coordinate points in the three-dimensional point cloud model onto the pixel coordinate system of the video stream; and it overlays the projected three-dimensional point cloud with the corresponding video frames to generate a fused visual image.

8. The vehicle outline dimension dynamic detection system according to claim 1, characterized in that, The over-limit determination and handling module includes a size calculation unit, an over-limit determination unit, an alarm unit, and an evidence generation unit. The size calculation unit is used to extract the vehicle's length, height, and width dimensions from the 3D point cloud model. The over-limit determination unit is used to compare the extracted dimension data with preset length, height, and width limits, respectively, and determine that an over-limit is triggered when any dimension data exceeds the corresponding limit. The alarm unit is used to trigger a local audio-visual alarm device to issue an alarm signal when an over-limit is determined. The evidence generation unit is used to extract an image containing the vehicle's appearance from the video stream when an over-limit is determined, associate the detected dimension parameters, detection time, and detection location information, and generate a complete evidence package.

9. The vehicle outline dimension dynamic detection system according to claim 8, characterized in that, The evidence generation unit stores the generated evidence package in a local storage device and uploads the evidence package to a preset cloud server through a network communication interface.

10. The vehicle outline dimension dynamic detection system according to claim 1, characterized in that, The over-limit judgment and handling module also includes a data storage unit, which is used to store the vehicle's passage data, including passage time, passage location and detected size parameters, into a local database when the judgment result is that the vehicle does not exceed the limit, for subsequent data query and statistical analysis.

Images