Vehicle inspection method, apparatus and system, and computer-readable storage medium

Abstract

The present disclosure relates to the field of vehicle security inspection, and relates to a vehicle inspection method, apparatus and system, and a computer-readable storage medium. The vehicle inspection method comprises: while at least one vehicle under inspection is traveling through an inspection road section, using a multi-line laser radar to acquire raw point cloud data, and determining a laser scanning line to which each point cloud of each vehicle under inspection in the raw point cloud data belongs; on the basis of the raw point cloud data, acquiring vehicle point cloud data of each vehicle under inspection among the at least one vehicle under inspection; determining at least one of position information and speed information of each vehicle under inspection on the basis of the vehicle point cloud data; determining, from among the vehicle point cloud data, target point cloud data of each vehicle under inspection scanned by at least one laser line of which the vertical field of view is greater than a vertical field of view threshold; and determining position information of a loaded object of each vehicle under inspection on the basis of the target point cloud data of each vehicle under inspection.

Description

Vehicle testing methods, apparatus and systems, and computer-readable storage media

[0001] Cross-references to related applications

[0002] This disclosure is based on and claims priority to CN application No. 202411630347.4, filed on November 14, 2024, the contents of which are incorporated herein by reference in their entirety. Technical Field

[0003] This disclosure relates to the field of vehicle safety inspection, and in particular to vehicle inspection methods, devices and systems, and computer-readable storage media. Background Technology

[0004] In vehicle security inspection equipment operation, as a vehicle passes through the scanning channel, sensors need to capture the vehicle's position and speed information in real time. For container trucks, the position of the onboard container also needs to be monitored simultaneously. This critical position information is transmitted to the scanning inspection system, providing accurate triggering timing for peripheral equipment that performs X-ray scanning inspections, license plate recognition, container number recognition, and other procedures.

[0005] In related technologies, a single-line lidar is installed near the height of the vehicle's license plate to detect the position of the front and rear of the vehicle, and another single-line lidar is installed at a position where the height of the container body can be stably observed to detect the position of the front and rear surfaces of the container. Summary of the Invention

[0006] According to a first aspect of this disclosure, a vehicle detection method is provided, comprising: acquiring raw point cloud data using a multi-line lidar while at least one vehicle to be inspected is traveling on a detection road section, and determining the laser scan line to which each point cloud of each vehicle to be inspected belongs in the raw point cloud data; acquiring vehicle point cloud data of each of the at least one vehicle to be inspected based on the raw point cloud data; determining at least one of position information and vehicle speed information of each vehicle to be inspected based on the vehicle point cloud data; determining target point cloud data of each vehicle to be inspected obtained by scanning with at least one laser line whose vertical field of view is greater than a vertical field of view threshold from the vehicle point cloud data; and determining the position information of the load of each vehicle to be inspected based on the target point cloud data of each vehicle to be inspected.

[0007] According to a second aspect of this disclosure, a vehicle detection device is provided, comprising: a first acquisition module configured to acquire raw point cloud data using a multi-line lidar while at least one vehicle to be inspected is traveling on a detection road section, and to determine the laser scan line to which each point cloud of each vehicle to be inspected belongs in the raw point cloud data; a second acquisition module configured to acquire vehicle point cloud data of each of the at least one vehicle to be inspected based on the raw point cloud data; a first determination module configured to determine at least one of position information and vehicle speed information of each vehicle to be inspected based on the vehicle point cloud data; a second determination module configured to determine target point cloud data of each vehicle to be inspected obtained by scanning with at least one laser line whose vertical field of view is greater than a vertical field of view threshold from the vehicle point cloud data; and a third determination module configured to determine the position information of the load of each vehicle to be inspected based on the target point cloud data of each vehicle to be inspected.

[0008] According to a third aspect of this disclosure, a vehicle detection apparatus is provided, comprising: a memory; and a processor coupled to the memory, the processor being configured to execute the vehicle detection method described in any of the above embodiments based on instructions stored in the memory.

[0009] According to a fourth aspect of this disclosure, a vehicle detection system is provided, comprising: a vehicle detection device as described in any of the above embodiments; a multi-line lidar configured to scan at least one vehicle to be inspected traveling on a detection road section, obtain raw point cloud data of the at least one vehicle to be inspected, and send the raw point cloud data to the vehicle detection device.

[0010] According to a fifth aspect of this disclosure, a computer-readable storage medium is provided that stores computer program instructions thereon, which, when executed by a processor, implement the vehicle detection method described in any of the above embodiments.

[0011] According to a sixth aspect of this disclosure, a computer program product is provided, including computer instructions, wherein when executed by a processor, the computer instructions implement the vehicle detection method described in any of the above embodiments. Attached Figure Description

[0012] The accompanying drawings, which form part of this specification, illustrate embodiments of this disclosure and, together with the specification, serve to explain the principles of this disclosure.

[0013] This disclosure will become clearer with reference to the accompanying drawings and the following detailed description, wherein:

[0014] Figure 1 is a schematic flowchart illustrating a vehicle detection method according to some embodiments of the present disclosure;

[0015] Figure 2 is a schematic flowchart illustrating the process of obtaining vehicle point cloud data for each vehicle to be inspected according to some embodiments of the present disclosure.

[0016] Figure 3 is a schematic flowchart illustrating the process of determining the location information of the load of each vehicle to be inspected according to some embodiments of the present disclosure;

[0017] Figure 4 is a schematic diagram illustrating a multi-line lidar according to some embodiments of the present disclosure;

[0018] Figure 5 is a schematic diagram illustrating a multi-line lidar scanning of a vehicle under inspection according to some embodiments of the present disclosure;

[0019] Figure 6 is a schematic diagram illustrating point cloud data for determining the affiliation of each point cloud according to some embodiments of the present disclosure;

[0020] Figure 7 is a schematic diagram illustrating point cloud data of a vehicle to be inspected according to some embodiments of the present disclosure;

[0021] Figure 8 is a schematic diagram illustrating a tilted multi-line lidar according to some embodiments of the present disclosure;

[0022] Figure 9 is a schematic diagram illustrating a horizontally mounted multi-line lidar according to some embodiments of the present disclosure;

[0023] Figure 10 is a schematic diagram showing the original point cloud data before correction according to some embodiments of the present disclosure;

[0024] Figure 11 is a schematic diagram illustrating corrected point cloud data according to some embodiments of the present disclosure;

[0025] Figure 12 is a diagram illustrating the effect of multi-vehicle segmentation according to some embodiments of the present disclosure;

[0026] Figure 13 is a schematic flowchart illustrating a vehicle detection method according to some other embodiments of the present disclosure;

[0027] Figure 14 is a block diagram illustrating a vehicle detection device according to some embodiments of the present disclosure;

[0028] Figure 15 is a block diagram illustrating a vehicle detection device according to some other embodiments of the present disclosure;

[0029] Figure 16 is a block diagram illustrating a vehicle detection system according to some embodiments of the present disclosure;

[0030] Figure 17 is a block diagram illustrating a computer system for implementing some embodiments of the present disclosure. Detailed Implementation

[0031] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of the present disclosure.

[0032] At the same time, it should be understood that, for ease of description, the dimensions of the various parts shown in the accompanying drawings are not drawn according to actual scale.

[0033] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit this disclosure or its application or use.

[0034] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.

[0035] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values.

[0036] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0037] In related technologies, when detecting multiple vehicle types in the direction of travel, it is necessary to use two or more single-line lidars, which increases the complexity and cost of the system.

[0038] To address the aforementioned technical issues, this disclosure proposes a solution that can reduce the complexity and cost of vehicle inspection, improve the flexibility and accuracy of inspection, and achieve unified inspection of different vehicle models.

[0039] Figure 1 is a flowchart illustrating a vehicle detection method according to some embodiments of the present disclosure, including steps S110 to S150.

[0040] As shown in Figure 1, the vehicle detection method includes: Step S110, during the process of at least one vehicle to be inspected traveling on the detection road section, acquiring original point cloud data using a multi-line lidar, and determining the laser scanning line to which each point cloud of each vehicle to be inspected belongs in the original point cloud data, wherein the multi-line lidar is installed on the side of the detection road section and is used to scan at least one vehicle to be inspected; Step S120, acquiring vehicle point cloud data for each of the at least one vehicle to be inspected based on the original point cloud data; Step S130, determining at least one of position information and vehicle speed information for each vehicle to be inspected based on the vehicle point cloud data; Step S140, determining the target point cloud data of each vehicle to be inspected obtained by scanning with at least one laser line whose vertical field of view is greater than the vertical field of view threshold from the vehicle point cloud data; and Step S150, determining the position information of the load of each vehicle to be inspected based on the target point cloud data of each vehicle to be inspected.

[0041] In this embodiment, because the multi-line lidar has multiple laser lines in the vertical direction, its detection range in the vertical direction is wide. Regardless of whether at least one vehicle to be inspected on the detection section belongs to the same model or different models, the multi-line lidar can detect the point cloud data of any model of vehicle to be inspected. This allows for the determination of at least one of the position and speed information of any model of vehicle to be inspected, reducing the complexity and cost of vehicle detection, improving the flexibility and accuracy of detection, and enabling unified detection of different vehicles. In this disclosure, laser line and laser scanning line have the same meaning.

[0042] Furthermore, since the loads on the vehicles under inspection are typically quite high, the target point cloud data for each vehicle can be determined from the vehicle point cloud data, obtained by scanning with at least one laser line whose vertical field of view is greater than a threshold. This at least one laser line originates from a multi-line lidar. It is understood that by limiting the vertical field of view, the obtained target point cloud data is associated with the load of the vehicle under inspection. Therefore, based on the target point cloud data for each vehicle under inspection, the location information of the load can be determined. This reduces the amount of point cloud data processing required to determine the load's location, thereby reducing the overall complexity and cost of vehicle load detection and improving detection efficiency. Moreover, since the target point cloud data is obtained by scanning with at least one laser line whose vertical field of view is greater than a threshold, interference from point cloud data obtained by laser lines with a vertical field of view smaller than the threshold can be reduced, improving the accuracy of load detection.

[0043] In this disclosure, by using only a single multi-line lidar, the position and / or speed of the vehicle and the position of the load can be determined simultaneously based on a single acquisition of raw point cloud data. This further improves the detection efficiency of the vehicle and its load, enhances the flexibility of detection, and reduces the detection complexity and cost of the vehicle and its load.

[0044] On the other hand, single-line lidar has only one laser line in the vertical direction, while multi-line lidar has multiple laser lines in the vertical direction. This significantly expands the vertical monitoring range of multi-line lidar. Compared with single-line lidar solutions, the blind zone is significantly reduced, improving detection accuracy, reducing the stringent requirements on the installation position and tilt angle of the lidar, and improving the flexibility and adaptability of lidar installation.

[0045] In step S110, while at least one vehicle to be inspected is traveling through the detection section, raw point cloud data is acquired using a multi-line LiDAR, and the laser scan line to which each point in the raw point cloud data belongs is determined. The multi-line LiDAR is installed on the side of the detection section to scan at least one vehicle to be inspected. For example, the installation position of the multi-line LiDAR ensures that the vehicle to be inspected is scanned while traveling through the detection section. Since the multi-line LiDAR is installed on the side of the detection section, it can scan the vehicle as it enters the detection section. As the vehicle travels along the detection section, the multi-line LiDAR continuously scans it, obtaining the raw point cloud data of the vehicle. It should be noted that the raw point cloud data acquired using the multi-line LiDAR typically includes environmental point cloud data; when no vehicle to be inspected is passing by, the raw point cloud data only contains environmental point cloud data.

[0046] In some embodiments, the laser scan line to which each point cloud of each vehicle to be inspected belongs in the raw point cloud data can be determined in the following manner.

[0047] First, for each point cloud of each vehicle to be inspected in the original point cloud data, determine the ratio of the coordinate value of each point cloud on the vertical axis in the radar coordinate system with the multi-line lidar as the origin to the projection length of the line segment formed by each point cloud and the origin on the plane formed by the first horizontal axis and the second horizontal axis in the radar coordinate system.

[0048] For example, the coordinate value of each point cloud on the vertical axis in the radar coordinate system with the multi-line lidar as the origin represents the vertical distance of each point cloud to the plane formed by the first horizontal axis and the second horizontal axis.

[0049] For example, the vertical axis is the z-axis, the first horizontal axis is the x-axis, and the second horizontal axis is the y-axis. The z-axis is perpendicular to the rotation axis of the multi-line lidar, while the x-axis and y-axis are parallel to the rotation axis of the multi-line lidar.

[0050] Then, determine the difference between the ratio and the tangent of the vertical field of view of each of at least one laser line.

[0051] Finally, if the difference between the ratio of a point cloud and the tangent of one of the at least one laser line is within the specified range, the point cloud is determined to be point cloud data belonging to that laser line. For example, the difference range is -0.01 to 0.01. This difference range is only an example and does not constitute a specific limitation of this disclosure.

[0052] For each point cloud, if one of the differences among the corresponding differences lies within a certain range, the ratio can be considered approximately equal to the tangent of the vertical field of view corresponding to the difference within that range. This means each point cloud belongs to the original point cloud data obtained from scanning with at least one laser line. In this case, the angle between the line segment formed by each point cloud and the origin and the plane formed by the first and second horizontal axes is approximately equal to or equal to the vertical field of view of one of the laser lines. That is, the point cloud can be considered to originate from that single laser line.

[0053] For each point cloud, if there is no difference within the difference range, determine that each point cloud does not belong to the point cloud in the original point cloud data obtained by the corresponding laser line scan.

[0054] Through the above operations, the laser scan line to which each point in the original point cloud data belongs can be calculated.

[0055] In step S120, based on the original point cloud data, vehicle point cloud data for each of at least one vehicle to be inspected is obtained. For example, the original point cloud data can be processed by operations such as ground correction and conditional Euclidean clustering, which will be described later, to obtain vehicle point cloud data for each vehicle to be inspected.

[0056] For example, in some embodiments, the vehicle point cloud data of each of at least one vehicle to be inspected can be obtained based on the original point cloud data through steps S121 to S124 shown in Figure 2.

[0057] Figure 2 is a schematic flowchart illustrating the process of obtaining vehicle point cloud data for each vehicle to be inspected according to some embodiments of the present disclosure.

[0058] In step S121, ground point cloud data is obtained from the original point cloud data, and a ground plane model of the detection segment is constructed using the random sample consensus algorithm. In step S122, a rotation matrix is ​​determined based on the normal vector of the ground plane model and the unit direction vector of the vertical axis of the radar coordinate system, where the rotation matrix rotates the normal vector to the vertical axis. In step S123, the original point cloud data is ground-corrected using the rotation matrix to obtain corrected point cloud data. In step S124, point cloud data within the channel is obtained based on the corrected point cloud data, and the point cloud data within the channel is clustered using the conditional Euclidean clustering algorithm to obtain vehicle point cloud data for each vehicle to be inspected.

[0059] In some embodiments, when non-vehicle objects are obtained by clustering point cloud data within a channel, the duration of the non-vehicle objects' existence is monitored; if the duration exceeds a time threshold, an alert is issued. For example, when non-vehicle objects are obtained by clustering vehicle point cloud data, it is first detected whether the non-vehicle object is a person (i.e., intrusion detection). If the non-vehicle object is a person, it is determined whether the non-vehicle object continues to exist. If the non-vehicle object continues to exist, an intrusion alarm is triggered; if the non-vehicle object does not continue to exist, the duration of the non-vehicle object's existence is recorded.

[0060] In step S130, at least one of the position information and speed information of each vehicle to be inspected is determined based on the vehicle point cloud data. For example, the position and speed of the vehicle can be determined by analyzing the position and timestamp changes of the point cloud between consecutive frames in the vehicle point cloud data of each vehicle to be inspected, and based on the analysis results.

[0061] In step S140, target point cloud data for each vehicle under inspection is determined from the vehicle point cloud data, obtained by scanning with at least one laser line whose vertical field of view is greater than a vertical field of view threshold. This at least one laser line originates from a multi-line lidar. For example, the at least one laser line can be the laser line with the largest vertical field of view, or it can be the laser line with the largest vertical field of view (i.e., the laser line at the top layer of the scanning layer) and the laser line with the second largest vertical field of view (i.e., the laser line at the top layer of the scanning layer).

[0062] It should be noted that when the vehicle is carrying a container, the height range of the container is basically fixed. Therefore, for a multi-line lidar installed at a fixed height, its vertical field of view threshold is also basically fixed. This allows us to obtain point cloud data associated with, for example, the edges of the container (e.g., the front and rear edges). In this way, we can select point cloud data from a specific range of the container that can be observed relatively stably for analysis.

[0063] For example, when the vertical field of view threshold is 13 degrees and the maximum vertical field of view of the multi-line lidar is 15 degrees, at least one laser line is a laser line with a vertical field of view between 13 and 15 degrees.

[0064] In step S150, the location information of the load of each vehicle to be inspected is determined based on the target point cloud data of each vehicle. For example, the load includes a container-structured object, or it can be an object of other structures. For example, a container-structured object can be a shipping container, or it can be an object of other container structures. For example, the location information of the load can be determined by determining the front and / or rear surfaces of the container-structured object.

[0065] In some embodiments, the location information of the load of each vehicle to be inspected can be determined based on the target point cloud data of each vehicle to be inspected, as shown in Figure 3.

[0066] Figure 3 is a schematic flowchart illustrating the process of determining the location information of the load of each vehicle to be inspected according to some embodiments of the present disclosure.

[0067] As shown in Figure 3, determining the location information of the load on each vehicle to be inspected includes steps S151 to S152.

[0068] In step S151, based on the target point cloud data of each vehicle to be inspected, it is determined whether there are discontinuous features and / or corner features in the target point cloud data of each vehicle. Discontinuous features refer to situations where the distance between two discontinuous points in the target point cloud data is greater than a distance threshold. Corner features refer to situations where the curvature between points in the target point cloud data changes abruptly or drastically.

[0069] For example, at least one of the discontinuity features and corner features in target point cloud data can be determined using feature analysis or feature recognition. Specifically, corner features can be determined based on geometric analysis, such as analyzing the curvature and normal vectors between points; locations where these parameters change drastically may be corners, and simultaneously analyzing the neighborhood of points that may be corners. Alternatively, discontinuity features can be extracted using clustering algorithms to determine their presence. Furthermore, corner feature detection can be performed using trained models and deep learning.

[0070] In step S152, if there are discontinuous features and / or corner features in the target point cloud data of each vehicle to be inspected, the location information of the load of each vehicle to be inspected is determined according to the point cloud coordinates of the points where the discontinuous features and / or corner features exist.

[0071] In some embodiments, the vehicle detection method may further include determining at least one of the vehicle's front position, rear position, and speed based on vehicle point cloud data. Because multi-line LiDAR has a wide vertical detection range, it can adapt to the detection of vehicles of different sizes and models, exhibiting strong vehicle compatibility and greater versatility of the detection method. In this disclosure, by using only a single multi-line LiDAR, accurate position and speed measurements of vehicles of various models can be performed, while effectively monitoring the position of vehicle loads (e.g., onboard containers) at different heights. This ensures that during vehicle scanning, various sensors, such as area scan cameras and line scan cameras, can efficiently and accurately capture and process vehicle data based on the determined vehicle position and speed, as well as the position of the vehicle loads. For example, various sensors, such as area scan cameras and line scan cameras, can select appropriate triggering times based on the determined vehicle position and speed, and the position of the vehicle loads.

[0072] The vehicle detection methods in some embodiments of this disclosure will now be described with reference to Figures 4 to 13.

[0073] Figure 4 is a schematic diagram illustrating a multi-line lidar according to some embodiments of the present disclosure.

[0074] For example, when the detection section is set as the scanning channel of a vehicle safety inspection device, a multi-line LiDAR, similar to that shown in Figure 4, can be arranged on the side wall of the scanning channel. As the vehicle to be inspected passes through the scanning channel, all the scanning lines (also called laser lines) of the multi-line LiDAR can generate a vehicle contour point cloud. The multi-line LiDAR in Figure 4 is a 16-line LiDAR with a vertical detection field of view of ±15° and a vertical angular resolution of 2°. However, this disclosure can also use other models of multi-line LiDARs with different line counts, vertical field of view ranges, and vertical angular resolutions. Deploying a multi-line LiDAR on the side wall of the scanning channel is only one embodiment of this disclosure and does not limit the arrangement scenarios of this disclosure. For example, a pillar can be installed on the side of the road, and then the multi-line LiDAR can be installed on the pillar.

[0075] In some embodiments, these vehicle contour point clouds can be used for detecting the front and rear positions of the vehicle under inspection and calculating its speed.

[0076] In some embodiments, taking a container as an example, the topmost scan line, i.e., the +15 degree scan line, can be extracted from the scanning layer of the multi-line LiDAR. The detection height of this scan line varies with the distance between the multi-line LiDAR and the vehicle to be inspected. When the vehicle first enters the scanning channel, it is far from the multi-line LiDAR, and the detection height of the +15 degree scan line is high, allowing the multi-line LiDAR to initially acquire the position information of containers at higher heights. As the vehicle approaches the multi-line LiDAR, the detection height of the +15 degree scan line decreases, allowing the multi-line LiDAR to also acquire the position information of containers at lower heights. That is, the multi-line LiDAR can acquire the position information of the front and rear surfaces of the container at different heights. This position information can be determined based on the position of the intersection edge between the front (rear) surface of the container and the side facing the multi-line LiDAR. Through hierarchical analysis of the point cloud data, the position information of both the vehicle and the container can be acquired simultaneously. The hierarchical analysis here refers to using the target point cloud data obtained from the topmost scan line in the scanning layer to determine the location information of the vehicle-mounted container, while using the vehicle point cloud data from all scanning layers obtained from multi-line LiDAR scanning to determine the vehicle's location information.

[0077] Figure 5 is a schematic diagram illustrating a multi-line lidar scanning of a vehicle under inspection according to some embodiments of the present disclosure.

[0078] As shown in Figure 5, the vehicle to be inspected 52 enters the scanning channel. The multi-line lidar 51 emits a laser line from the side of the scanning channel (i.e., the side of the vehicle to be inspected 52) to the vehicle to be inspected, thereby scanning the vehicle to be inspected 52. For the +15 degree laser line, referring to Figure 5, it can be seen that as the vehicle to be inspected 52 moves in the scanning channel, this laser line can scan the container 53 loaded on the vehicle to be inspected 52.

[0079] For example, the method for determining the laser scan line to which each point cloud of each vehicle to be inspected belongs in the raw point cloud data of the vehicle to be inspected obtained from multi-line lidar scanning can be referred to Figure 6. Figure 6 is a schematic diagram illustrating the determination of the belonging of each point cloud according to some embodiments of the present disclosure. It should be noted that by determining the laser scan line to which each point cloud belongs, it is beneficial to extract the target point cloud data of each vehicle to be inspected subsequently.

[0080] As shown in Figure 6, the radar coordinate system with the multi-line lidar as the origin includes the x-axis, y-axis, and z-axis. For a point cloud with coordinates (xi, yi, zi), if the calculation result is as shown in formula (1), then the point cloud can be regarded as a point obtained by a +15 degree laser line scan.

[0081] By traversing the point cloud data and performing the calculations shown in formula (1), all points on the +15 degree laser line can be extracted, which is the topmost point cloud in Figure 6. For each other point cloud, the laser scan line to which each belongs can also be determined through the same processing method. Thus, the laser scan line to which all point cloud data belongs can be determined.

[0082] Let zi be the z-coordinate of the point in the point cloud with coordinates (xi, yi, zi). Let be the projection length of the line segment formed by the point with coordinates (xi,yi,zi) and the origin on the plane formed by the x-axis and y-axis.

[0083] Figure 7 is a schematic diagram illustrating point cloud data of a vehicle to be inspected according to some embodiments of the present disclosure. As shown in Figure 7, the vehicle point cloud data of the vehicle to be inspected includes point cloud data of the front of the vehicle 71, tires 72, container 73, and connecting parts 74 between the front of the vehicle and the container. The contour features of the container 73 on the scan line include obvious corners 731 (corner features) and / or curved gaps 732 (discontinuities) generated between the front of the container and the front of the vehicle. These features provide key information for accurately locating the container's position, and by comprehensively analyzing these features, the position of the container can be accurately determined.

[0084] In some embodiments, during actual product applications, due to limitations in scanning channel width and / or vehicle model detection requirements, multi-line LiDARs are installed in both horizontal and tilted configurations, with the latter being more common. When a multi-line LiDAR is installed at a tilt, the point cloud coordinate system inside the LiDAR is not parallel to the actual ground, thus increasing the complexity of point cloud data processing.

[0085] Figure 8 is a schematic diagram illustrating a tilted multi-line lidar according to some embodiments of the present disclosure.

[0086] Figure 9 is a schematic diagram illustrating a horizontally mounted multi-line lidar according to some embodiments of the present disclosure.

[0087] To address the technical problem of increased complexity in point cloud data processing due to tilted installation, this disclosure proposes ground correction of the raw point cloud data collected by the multi-line lidar before using the point cloud data of the vehicle under inspection.

[0088] For example, the RANSAC (Random Sample Consensus) algorithm can be used to accurately fit ground point clouds. The RANSAC algorithm builds a model by randomly selecting samples from the dataset and iterates continuously to find the model that best represents the entire dataset.

[0089] The process of obtaining a ground data model using the RANSAC algorithm can be referenced as follows, including steps one through four.

[0090] The first step is to filter out irrelevant point cloud data such as environment, walls, and buildings from the original point cloud data through point cloud filtering, and only select point cloud datasets containing the ground.

[0091] The second step is to randomly select a set of points from the selected point cloud dataset as initial interior points, and fit an initial planar model based on these points.

[0092] The third step is to calculate the distance from other points in the selected point cloud dataset to the fitted planar model, and to include points whose distance is less than a preset threshold into the inlier set.

[0093] Fourth, repeat steps two and three until a predetermined number of iterations is reached or the size of the inlier set reaches a certain threshold, ultimately determining the optimal ground plane model.

[0094] The ground plane model obtained by fitting the RANSAC algorithm is shown in equation (2). Ax+By+Cz+D=0 (2)

[0095] In formula (2), the normal vector of the ground plane model is (A, B, C). In the point cloud coordinate system, the unit direction vector of the z-axis is (0, 0, 1). To rotate the point cloud coordinate system to align with the ground, this disclosure calculates a rotation matrix that rotates the ground normal vector to the z-axis direction. The Rodrigues rotation formula can be used to calculate this rotation matrix R. In practical applications, it is usually not necessary to manually calculate the rotation matrix, as most mathematical and scientific computing libraries provide implementations of the Rodrigues formula. After rotating the point cloud data according to the rotation matrix, the ground point cloud data can be aligned with the radar coordinate system. Thus, ground correction can be performed on the original point cloud data to obtain corrected point cloud data, which facilitates subsequent filtering of point cloud data outside the scanning channel according to the scanning channel size.

[0096] In some embodiments, during the process of acquiring the ground plane model described above, raw point cloud data is acquired when there are no objects (e.g., no vehicles) in the scanning channel. This raw point cloud data only includes environmental point cloud data, and the point cloud data obtained after filtering is empty. Therefore, it is considered empty when determining the position and / or velocity information of each vehicle to be detected based on the vehicle point cloud data, as described later. This is because when objects are present in the channel, they obscure the ground, leading to inaccurate ground detection. In some embodiments, the acquisition of the ground plane model is performed only once during the initial system configuration phase. After obtaining a set of rotation matrix parameters, only these parameters need to be retrieved subsequently. This is because performing ground correction on all point cloud data would reduce the real-time performance of the vehicle detection method.

[0097] Figure 10 is a schematic diagram showing the original point cloud data before correction according to some embodiments of the present disclosure.

[0098] Figure 11 is a schematic diagram illustrating corrected point cloud data according to some embodiments of the present disclosure.

[0099] As shown in Figure 10, before correction, the horizontal plane of the radar coordinate system (the plane formed by the x-axis and y-axis) is not parallel to the ground 101. As shown in Figure 11, after correction, the horizontal plane of the radar coordinate system is parallel to the ground 111.

[0100] In this embodiment, after the ground correction of the original point cloud data is completed, it is convenient to use the radar coordinate system as a reference to detect the position of the vehicle and the position of the vehicle's cargo (such as a container).

[0101] In some embodiments, the corrected point cloud data can also be filtered to remove environmental point clouds. The aforementioned ground correction operation facilitates the point cloud filtering process in this embodiment. Thus, for example, only the point cloud data within the scanning channel can be retained from the corrected point cloud data.

[0102] In some embodiments, when multiple vehicles to be inspected exist in the scanning channel, a conditional Euclidean clustering algorithm is used to accurately segment the point cloud data of different vehicles. This involves performing conditional Euclidean clustering on the vehicle point cloud data or the point cloud data within the channel after ground correction and point cloud filtering. Conditional Euclidean clustering is a distance-based clustering method that classifies data points by calculating the Euclidean distance between them, ensuring that similar data points belong to the same cluster. In vehicle security inspections, maintaining a safe distance between vehicles is crucial to ensure the integrity of vehicle data collection. Therefore, in the conditional Euclidean clustering algorithm, the distance between data points in the vehicle's driving direction (i.e., the safe distance between vehicles in the driving direction) is set as a clustering constraint. When the distance between two points in the vehicle's driving direction exceeds a preset threshold, the algorithm automatically assigns these data points to different clusters.

[0103] Figure 12 is a diagram illustrating the effect of multi-vehicle segmentation according to some embodiments of the present disclosure.

[0104] As shown in Figure 12, the precise location of the point cloud data of different vehicles to be inspected can be determined by the conditional Euclidean clustering algorithm, namely the point cloud data of the first vehicle to be inspected 1201 and the second vehicle to be inspected 1202.

[0105] In some embodiments, the vehicle detection method includes at least one step of point cloud downsampling, ground correction, pass-through filtering, conditional Euclidean clustering, point cloud classification, non-vehicle target processing, vehicle target processing, and onboard container position detection. In some embodiments, the vehicle detection method includes all of the above steps.

[0106] In point cloud downsampling, after obtaining the original point cloud data, voxel filtering is first performed to reduce the number of laser points per unit volume, thereby improving the efficiency of subsequent data processing.

[0107] In ground calibration, the pre-saved rotation matrix parameters are read, and the downsampled original point cloud data is rotated to align with the ground and coordinate axes.

[0108] In pass-through filtering, a pass-through filter is applied to filter and retain point cloud data that is only in the detection channel (scanning channel) to obtain point cloud data within the channel.

[0109] In conditional Euclidean clustering, the conditional Euclidean clustering algorithm is performed on the point cloud data within the channel to group points that are close to each other into one class in order to determine the point cloud distribution within the scanned channel.

[0110] In point cloud classification, clustered point cloud data is categorized based on characteristics such as size and quantity to distinguish vehicles or other target objects. This yields vehicle point cloud data and non-vehicle point cloud data.

[0111] In the processing of non-vehicle targets, if the point cloud type is determined to be non-vehicle, the anti-intrusion detection process will be initiated. If the non-vehicle target is detected to continue to exist, an early warning will be issued.

[0112] In vehicle target processing, if the point cloud type is determined to be a vehicle body, at least one of the vehicle's position and speed information can be determined based on the vehicle point cloud data. For example, using the vehicle point cloud data of each vehicle to be inspected, at least one of the vehicle's position and speed information can be determined. Specifically, by analyzing the position and timestamp changes of the point cloud between consecutive frames in the vehicle point cloud data of each vehicle to be inspected, the vehicle's position and speed are determined based on the analysis results. The vehicle's displacement can be determined using at least one of the following methods: direct comparison of point cloud data using difference methods, feature matching methods, iterative nearest point algorithms, normal distribution transformation algorithms, and deep learning-based methods.

[0113] For example, in the position detection of a vehicle-mounted container, for the vehicle body point cloud, a scan line of +15 degrees can be extracted, and the contour change of the scan line can be detected along the direction from the rear to the front of the vehicle to identify the position of the front and rear surfaces of the vehicle-mounted container. That is, the target point cloud data is determined from the vehicle point cloud data, and the position information of the vehicle-mounted container is determined accordingly.

[0114] The specific implementation of the vehicle detection method in some embodiments of this disclosure will be described in detail below with reference to Figure 13.

[0115] Figure 13 is a schematic flowchart illustrating a vehicle detection method according to other embodiments of the present disclosure.

[0116] As shown in Figure 13, the vehicle detection method includes steps S210 to S225.

[0117] In step S210, raw point cloud data is acquired. For example, using a multi-line lidar installed on the side of the detection section (such as a scanning channel), at least one vehicle to be inspected traveling on the detection section is scanned to obtain vehicle point cloud data for at least one vehicle to be inspected. It is understood that in the absence of a vehicle to be inspected, the raw point cloud data only includes environmental point cloud data.

[0118] In step S211, a ground plane model of the detected road segment is constructed, that is, the ground parameters are fitted using the random sampling consensus algorithm to obtain the normal vector of the ground plane model as the ground parameters.

[0119] In step S212, the rotation matrix is ​​calculated based on the normal vector and the unit direction vector of the vertical axis of the radar coordinate system. The rotation matrix can be saved locally.

[0120] In step S213, the laser scan line to which each point cloud in the original point cloud data belongs is determined. At this time, only the affiliation of each point cloud is calculated and labeled, so that it can be extracted after obtaining the vehicle point cloud data later.

[0121] In step S214, the vehicle point cloud data is downsampled to obtain downsampled point cloud data. Steps S211, S212, S213, and S214 do not have a strict execution order and can be performed simultaneously. Those skilled in the art can set the order according to the circumstances.

[0122] In step S215, the rotation matrix calculated in step S212 (e.g., reading the locally stored rotation matrix) is used to perform ground correction on the downsampled point cloud data to obtain corrected point cloud data, so that the horizontal plane of the radar coordinate system is parallel to the ground.

[0123] In step S216, a point cloud filtering operation is performed on the corrected point cloud data to obtain filtered point cloud data. The point cloud filtering operation can filter environmental point clouds, improving vehicle detection accuracy. When the detection segment is a scanning channel, step S216 can acquire the point cloud data within the scanning channel.

[0124] In step S217, conditional Euclidean clustering is performed on the filtered point cloud data. Conditional Euclidean clustering uses the safe distance between vehicles in the driving direction as a condition to cluster the filtered point cloud data.

[0125] In step S218, the clustered point cloud data is classified to identify whether the clustered point cloud data belongs to the vehicle body.

[0126] If the clustered point cloud data belongs to the vehicle body, proceed with steps S219 to S221.

[0127] In step S219, the location and speed information of the vehicle to be inspected are determined based on the clustered point cloud data.

[0128] In step S220, target point cloud data is extracted from the clustered point cloud data. Target point cloud data, for example, is point cloud data of the vehicle to be inspected obtained by scanning with at least one laser line from a multi-line lidar whose vertical field of view is greater than a vertical field of view threshold.

[0129] In step S221, the location information of the load on the vehicle to be inspected is determined based on the target point cloud data of the vehicle to be inspected.

[0130] If the clustered point cloud data does not belong to the vehicle body, proceed with steps S222 to S225.

[0131] In step S222, personnel intrusion detection is performed.

[0132] In step S223, it is determined whether the intruding target continues to exist.

[0133] If the intruder persists, proceed to step S224, in which an intrusion alarm is triggered.

[0134] If the intrusion target is not continuously present, proceed to step S225, in which the duration of the target's presence is recorded, and the intrusion detection continues.

[0135] The embodiments shown in Figure 13 are merely detailed examples and do not constitute a specific limitation of this disclosure.

[0136] The technical solution disclosed herein has high measurement accuracy, strong anti-interference ability, and is not easily affected by noise points or missing data, thereby ensuring the stability and reliability of the overall detection effect.

[0137] In the above embodiments, a single multi-line lidar is used to achieve precise position and velocity measurement of vehicle targets within the scanning channel, ensuring efficient and accurate data acquisition of vehicles in dynamic environments. Using multi-line lidar for precise positioning of containers on vehicles allows for accurate capture of the containers' spatial location.

[0138] The vehicle detection methods in some embodiments of this disclosure have now been described.

[0139] It should be noted that in some of the described embodiments, a single laser scanning radar is used to accurately measure the position and speed of vehicles of various models, while effectively monitoring the position of the vehicle's cargo. However, in cases where the detection road (e.g., scanning channel) is long, more than one laser scanning radar can be used to increase the detection distance, for example, two laser scanning radars can be spaced apart in the vehicle's direction of travel. In this case, the vehicle detection method of this disclosure can still be implemented, the difference being that, before acquiring the vehicle point cloud data, the point cloud data from multiple multi-line laser radars are fused into a single point cloud data, and then the position information and / or vehicle speed information, as well as the position information of the vehicle's cargo, are confirmed; or, after determining the vehicle's position information and / or vehicle speed information, and the position information of the vehicle's cargo, the relevant information is fused at the processing level.

[0140] Figure 14 is a block diagram illustrating a vehicle detection apparatus according to some embodiments of the present disclosure.

[0141] As shown in Figure 14, the vehicle detection device 13 includes a first acquisition module 131, a second acquisition module 132, a first determination module 133, a second determination module 134, and a third determination module 135.

[0142] The first acquisition module 131 is configured to acquire raw point cloud data using a multi-line lidar while at least one vehicle to be inspected is traveling on the detection section, and to determine the laser scan line to which each point cloud of each vehicle to be inspected belongs in the raw point cloud data. The multi-line lidar is installed on the side of the detection section and is used to scan at least one vehicle to be inspected, for example, by performing step S110 as shown in Figure 1.

[0143] The second acquisition module 132 is configured to acquire vehicle point cloud data of at least one vehicle to be inspected based on the original point cloud data, for example, by performing step S120 as shown in Figure 1.

[0144] The first determining module 133 is configured to determine at least one of the position information and speed information of each vehicle to be inspected based on the vehicle point cloud data, for example, by performing step S130 as shown in Figure 1.

[0145] The second determining module 134 is configured to determine, from the vehicle point cloud data, the target point cloud data of each vehicle to be inspected obtained by scanning with at least one laser line whose vertical field of view is greater than a vertical field of view threshold, for example, by performing step S140 as shown in Figure 1. Here, at least one laser line comes from a multi-line lidar.

[0146] The third determining module 135 is configured to determine the location information of the load of each vehicle to be inspected based on the target point cloud data of each vehicle to be inspected, for example, by performing step S150 as shown in Figure 1.

[0147] Figure 15 is a block diagram illustrating a vehicle detection apparatus according to other embodiments of the present disclosure.

[0148] As shown in FIG15, the vehicle detection device 14 includes a memory 141 and a processor 142 coupled to the memory 141. The memory 141 is used to store instructions for executing embodiments of the vehicle detection method. The processor 142 is configured to execute the vehicle detection method in any of the embodiments of this disclosure based on the instructions stored in the memory 141.

[0149] Figure 16 is a block diagram illustrating a vehicle detection system according to some embodiments of the present disclosure.

[0150] As shown in Figure 16, the vehicle detection system 1 includes a vehicle detection device 161 and a multi-line lidar 162.

[0151] The vehicle detection device 161 is a vehicle detection device in any embodiment of this disclosure, configured to perform the vehicle detection method in any embodiment of this disclosure. For example, the vehicle detection device 161 is the vehicle detection device 13 in FIG. 14 or the vehicle detection device 14 in FIG. 15.

[0152] The multi-line lidar 162 is configured to scan at least one vehicle to be inspected traveling on the detection section, obtain raw point cloud data of at least one vehicle to be inspected, and send the raw point cloud data to the vehicle detection device 161. The multi-line lidar 162 is installed on the side of the detection section.

[0153] Figure 17 is a block diagram illustrating a computer system for implementing some embodiments of the present disclosure.

[0154] As shown in Figure 17, the computer system 170 can be represented in the form of a general computing device. The computer system 170 includes a memory 1710, a processor 1720, and a bus 1700 connecting different system components.

[0155] The memory 1710 may include, for example, system memory, non-volatile storage media, etc. The system memory may store, for example, an operating system, application programs, a boot loader, and other programs. The system memory may include volatile storage media, such as random access memory (RAM) and / or cache memory. The non-volatile storage media may store, for example, instructions for performing at least one embodiment of the vehicle detection method. Non-volatile storage media include, but are not limited to, disk storage, optical storage, flash memory, etc.

[0156] The processor 1720 can be implemented using a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete hardware components such as discrete gates or transistors. Accordingly, each module, such as the decision module and the determination module, can be implemented by executing instructions in the central processing unit (CPU) memory to perform the corresponding steps, or by implementing dedicated circuitry to perform the corresponding steps.

[0157] The Bus 1700 can use any of the various bus architectures available. For example, bus architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MCA) bus, and the Peripheral Component Interconnect (PCI) bus.

[0158] The computer system 170 may also include an input / output interface 1730, a network interface 1740, and a storage interface 1750. These interfaces 1730, 1740, and 1750, as well as the memory 1710 and processor 1720, can be connected via a bus 1700. The input / output interface 1730 provides a connection interface for input / output devices such as a monitor, mouse, and keyboard. The network interface 1740 provides a connection interface for various networked devices. The storage interface 1750 provides a connection interface for external storage devices such as floppy disks, USB flash drives, and SD cards.

[0159] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus, and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations thereof, can be implemented by computer-readable program instructions.

[0160] These computer-readable program instructions are provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable device to produce a machine, such that execution of the instructions by the processor produces means for implementing the functions specified in one or more boxes of the flowchart and / or block diagram.

[0161] These computer-readable program instructions may also be stored in a computer-readable storage medium. These instructions cause a computer to work in a particular manner to produce an article of manufacture, including instructions that implement the functions specified in one or more boxes in a flowchart and / or block diagram.

[0162] This disclosure may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects.

[0163] The vehicle detection method, apparatus, system, and computer-readable storage medium described in the above embodiments can reduce the complexity and cost of vehicle detection, improve the flexibility and accuracy of detection, and achieve unified detection for different vehicle models.

[0164] The vehicle detection method, apparatus, system, and computer-readable storage medium according to this disclosure have been described in detail above. To avoid obscuring the concept of this disclosure, some details known in the art have not been described. Those skilled in the art can fully understand how to implement the technical solutions disclosed herein based on the above description.

Claims

1. A vehicle inspection method, comprising: While at least one vehicle to be inspected is traveling on the inspection section, raw point cloud data is acquired using a multi-line lidar, and the laser scan line to which each point cloud of each vehicle to be inspected belongs is determined in the raw point cloud data. Based on the original point cloud data, obtain the vehicle point cloud data of each of the at least one vehicle to be inspected. Based on the vehicle point cloud data, at least one of the location information and vehicle speed information of each vehicle to be inspected is determined; From the vehicle point cloud data, determine the target point cloud data of each vehicle to be inspected, which is obtained by scanning at least one laser line with a vertical field of view greater than the vertical field of view threshold. Based on the target point cloud data of each vehicle to be inspected, the location information of the load of each vehicle to be inspected is determined.

2. The vehicle detection method according to claim 1, wherein, The step of determining the location information of the load of each vehicle to be inspected based on the target point cloud data of each vehicle includes: Based on the target point cloud data of each vehicle to be inspected, determine whether there are discontinuous features and / or corner features in the target point cloud data of each vehicle to be inspected. If there are discontinuous features and / or corner features in the target point cloud data of each vehicle to be inspected, the location information of the load of each vehicle to be inspected is determined based on the point cloud coordinates of the points where the discontinuous features and / or corner features exist.

3. The vehicle detection method according to claim 1, wherein, The process of determining the laser scan line to which each point cloud of each vehicle to be inspected belongs in the original point cloud data includes: For each point cloud of each vehicle to be inspected in the original point cloud data, determine the ratio of the coordinate value of each point cloud on the vertical axis in the radar coordinate system with the multi-line lidar as the origin to the projection length of the line segment formed by each point cloud and the origin on the plane formed by the first horizontal axis in the radar coordinate system. Determine the difference between the ratio and the tangent of the vertical field of view of each of the at least one laser line; If the difference between the ratio of a point cloud and the tangent of the vertical field of view of one of the at least one laser line is within the range of the difference, then the point cloud is determined to belong to the point cloud data emitted by that laser line.

4. The vehicle detection method according to claim 1, wherein, The step of obtaining the vehicle point cloud data of each of the at least one vehicles to be inspected based on the original point cloud data includes: Ground point cloud data is obtained from the original point cloud data, and a ground planar model of the detected road segment is constructed. The rotation matrix is ​​determined based on the normal vector of the ground plane model and the unit direction vector of the vertical axis of the radar coordinate system; Using the rotation matrix, ground correction is performed on the original point cloud data to obtain corrected point cloud data; Based on the corrected point cloud data, the point cloud data within the channel is obtained, and the point cloud data within the channel is clustered using the conditional Euclidean clustering algorithm to obtain the vehicle point cloud data for each vehicle to be inspected.

5. The vehicle detection method according to claim 4, wherein, The construction of the ground plan model of the detected road segment includes: A ground plane model of the detected road segment is constructed using the random sampling consensus algorithm.

6. The vehicle detection method according to claim 4, wherein, The rotation matrix causes the normal vector to rotate to the vertical axis.

7. The vehicle detection method according to claim 4 further includes: When non-vehicle objects are obtained by clustering the point cloud data in the channel, the duration of the existence of the non-vehicle objects is monitored. An early warning will be issued if the duration exceeds a time threshold.

8. The vehicle detection method according to claim 1, wherein, The location information of each vehicle to be inspected includes at least one of the front position and the rear position.

9. The vehicle detection method according to claim 1, wherein, The location information of the load on each vehicle to be inspected includes at least one of the locations of the front and rear cargo surfaces.

10. The vehicle detection method according to claim 1, wherein, The multi-line lidar is installed on the side of the detection section and is used to scan the at least one vehicle to be inspected.

11. The vehicle detection method according to any one of claims 1-10, wherein, The cargo includes cargo with a box-like structure.

12. A vehicle detection device, comprising: The first acquisition module is configured to acquire raw point cloud data using a multi-line lidar while at least one vehicle to be inspected is traveling on the inspection section, and to determine the laser scan line to which each point cloud of each vehicle to be inspected belongs in the raw point cloud data. The second acquisition module is configured to acquire vehicle point cloud data of each of the at least one vehicle to be inspected based on the original point cloud data. The first determining module is configured to determine at least one of the position information and speed information of each vehicle to be inspected based on the vehicle point cloud data. The second determining module is configured to determine, from the vehicle point cloud data, the target point cloud data of each vehicle to be inspected obtained by scanning with at least one laser line whose vertical field of view is greater than the vertical field of view threshold. The third determining module is configured to determine the location information of the load of each vehicle to be inspected based on the target point cloud data of each vehicle to be inspected.

13. A vehicle detection device, comprising: Memory; as well as A processor coupled to the memory, the processor being configured to execute the vehicle detection method as described in any one of claims 1 to 11 based on instructions stored in the memory.

14. A vehicle detection system, comprising: The vehicle detection device as described in claim 12 or 13; A multi-line lidar is configured to scan at least one vehicle to be inspected traveling on the detection section, obtain raw point cloud data of the at least one vehicle to be inspected, and send the raw point cloud data to the vehicle detection device.

15. A computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the vehicle detection method as claimed in any one of claims 1 to 11.

16. A computer program product comprising computer instructions, wherein the computer instructions, when executed by a processor, implement the vehicle detection method as described in any one of claims 1-11.

Images