A method for identifying vehicles in a service area based on 3D lidar data

Abstract

The present application discloses a method for identifying vehicles in a service area based on three-dimensional lidar data, which includes the following steps: collecting service area environmental information and constructing a colored base map line model including key ground features of the service area; constructing a local coordinate system and a world coordinate system; dividing the colored base map line model into regions, and arranging lidars at the safe area positions of the colored base map line model based on the priority of vehicle identification positions; classifying and real-time status identifying vehicles in the service area based on the arranged lidars, and uploading the collected data to the server side in real time. The beneficial effects of the present application: According to the actual situation of the service area and the coverage range of the lidars, it is possible to intelligently optimize the lidar layout points, minimize the number of lidars to the greatest extent, and ensure that there are no dead corners in the lidar coverage of each key area of the service area while reducing costs.

Description

Technical Field

[0001] This application relates to the field of intelligent transportation technology, and particularly to a method for identifying vehicles in a service area based on three-dimensional lidar data. Background Art

[0002] Compared with service areas, other vehicle identification technology application scenarios such as parking lots and autonomous driving have significant differences in tasks and technical requirements. Vehicle identification in parking lots mainly focuses on access management and parking space guidance, with the emphasis on vehicle identity verification and parking space allocation, requiring relatively low vehicle position accuracy but high accuracy in vehicle identity recognition. Vehicle identification in autonomous driving, on the other hand, focuses on real-time monitoring of driving states, emphasizing high-precision trajectory recognition and instant response to ensure autonomous decision-making and safe driving of vehicles.

[0003] However, vehicle management in service areas not only involves vehicle identity recognition but also requires real-time monitoring and analysis of vehicle behavior in complex environments. The driving states of vehicles in service areas are more complex, including various states such as deceleration, parking, and starting, and the driving routes of vehicles are not fixed, lacking the regularity in autonomous driving scenarios. In addition, the environment in service areas is complex, with numerous people and facilities, which poses higher anti-interference requirements for radar technology, especially when distinguishing vehicles from other objects, making the technical difficulty greater. Traditional vehicle monitoring technologies rely on limited cameras or manual monitoring, unable to comprehensively cover various dynamic situations in service areas and being inefficient in high-traffic and high-density environments. Summary of the Invention

[0004] One of the purposes of this application is to provide a method for identifying vehicles in a service area based on three-dimensional lidar data that can solve at least one of the defects in the above background art.

[0005] To achieve at least one of the above purposes, the technical solution adopted in this application is as follows: A method for identifying vehicles in a service area based on three-dimensional lidar data, including the following steps:

[0006] S100: Collect service area environmental information and construct a colored base map line model including key ground features of the service area;

[0007] S200: Construct a local coordinate system and a world coordinate system; wherein, the local coordinate system is used for summarizing and analyzing vehicle driving behavior data, and the world coordinate system is used for cross-system data integration and information sharing;

[0008] S300: Divide the colored base map line model into regions; based on the priority of vehicle identification positions, arrange lidars at the safe area positions of the colored base map line model, and use the minimum number of lidars under the condition of achieving full coverage of the service area according to the approximation principle;

[0009] S400: Classify the vehicles in the service area and identify their real-time status based on the arranged lidar, and upload the collected data to the server side in real time.

[0010] Preferably, step S100 includes the following specific processes:

[0011] S110: Uniformly arrange multiple cross targets in the service area, and use GNSS equipment to measure the three-dimensional coordinates of the cross targets;

[0012] S120: Use a drone to take large-range and multi-angle photos of the service area, and construct a colored three-dimensional point cloud model of the service area based on the collected image data;

[0013] S130: Denoise the obtained three-dimensional point cloud model, and use the denoised three-dimensional point cloud model to construct a colored base map line model.

[0014] Preferably, in step S130, the denoising of the three-dimensional point cloud model includes the following specific processes:

[0015] S131: Judge the ground of the point cloud through the RANSAC algorithm;

[0016] S132: Judge the height of each point in the point cloud data from the ground. If the height of the point from the ground is within the set first range, it is considered that the point belongs to the low-altitude points and is retained, otherwise the next step is carried out;

[0017] S133: Judge whether the height of the point from the ground is within the second range. If it is, the next step is carried out, otherwise the point is identified as a noise point and removed;

[0018] S134: Conduct a range search with a set radius distance with this point as the center of the sphere. If the number of other points within the spherical search range is greater than the set threshold number, it is considered that the point is a low-altitude point and is retained, otherwise it is a noise point and is removed.

[0019] Preferably, in step S130, the construction process of the colored base map line model is as follows:

[0020] S135: Based on the denoised point cloud data, remove all points that are more than the first range away from the ground;

[0021] S136: Project the remaining point cloud data onto the ground to generate a colored base map model of the ground;

[0022] S137: Extract the spatial layout and lane information of different regions in the service area from the colored base map model generated in step S136 through computer vision technology to obtain the colored base map line model of the service area.

[0023] Preferably, in step S300, the regional division of the colored base map line model includes the following process:

[0024] S310: Divide the service area into three regions according to the colored base map line model, including the region to be recognized, the immovable object region, and the radar-installable region;

[0025] S320: Divide the entire service area into grids and number the grids in the horizontal and vertical directions;

[0026] S330: Obtain the numbered positions of the region to be recognized and the coordinates of the four corner points of each grid according to the divided grids in combination with the colored base map line model.

[0027] Preferably, the vehicle recognition priority at the service area entrance and exit position is higher than that at other positions; then the lidar arrangement at the service area entrance and exit position in step S300 includes the following process:

[0028] S340: Locate the midpoint coordinates (x 1 , y 1 ) of the service area entrance and exit position in the divided grids, and the center point coordinates (x 2 , y 2 ) of the service area;

[0029] S350: Calculate the placement vector v of the lidar according to the obtained coordinates v = (x 2 - x 1 , y 2 - y 1 );

[0030] S360: Determine the candidate area for radar placement according to the direction of the placement vector v;

[0031] S370: Traverse each grid in the candidate area to calculate the recognition range of the lidar, select the locally optimal grid and determine the precise placement position of the lidar through the approximation principle.

[0032] Preferably, the lidar arrangement process for other areas of the service area except the entrance and exit positions is as follows:

[0033] S381: Determine the grids corresponding to the edge of the coverage range of the lidar at the service area entrance and exit position, and then obtain the grid area numbers that are not fully recognized in the service area;

[0034] S382: Traverse each empty grid area in the horizontal or vertical direction and simulate the placement of the lidar;

[0035] S383: Record the empty grid areas and edge positions covered by the lidar at each placement position.

[0036] S384: Select the placement point that covers the most empty grid areas and has the largest number of cross-grid areas with the grid areas occupied by lidars at the service area entrance and exit positions as the optimal placement grid area;

[0037] S385: Determine the precise placement position of the lidar within the optimal placement grid area through the approximation principle.

[0038] Preferably, the process of determining the precise placement position of the lidar based on the approximation principle is as follows:

[0039] S301: Traverse all the placement areas where the lidar can be placed in the corresponding locally optimal grid;

[0040] S302: Divide the placement area into multiple placement points at a set interval distance;

[0041] S303: Hypothetically place the lidar at each placement point and judge the coverage range of the lidar, and select the placement point with the best coverage range as the precise placement position of the lidar.

[0042] Preferably, step S400 includes the following process:

[0043] S410: Perform target annotation on the three-dimensional point cloud collected by the lidar, including three types of targets: sedans, box trucks, and large trucks, and other types are labeled as negative samples;

[0044] S420: Build a vehicle recognition model on the server side and perform model training;

[0045] S430: Calibrate the coordinate system of the lidar to ensure the unity of the coordinates of the point cloud data collected by the lidar;

[0046] S440: Upload the point cloud data collected by the lidar to the server side, and then perform detection in combination with the trained vehicle recognition model, and track and recognize the behavior of the vehicle according to the detection results.

[0047] Preferably, the process of model training through the PointPillars model in step S420 is as follows:

[0048] S421: Divide the three-dimensional point cloud data into multiple cylinders; if the number of points contained in a single cylinder exceeds the threshold, calculate the characteristics of the cylinder; if the number of points in a single cylinder is less than the threshold, fill it with zeros;

[0049] S422: Aggregate the points within each cylinder, calculate the mean characteristics of the cylinder and compress them into a feature vector of a fixed length; splice the feature vectors of all cylinders to form a two-dimensional feature map;

[0050] S423: Process the two-dimensional feature map through the backbone network, gradually capture the regional shape, texture, and spatial relationship between different regions, so as to extract the high-level spatial features in the two-dimensional feature map;

[0051] S424: Fuse the extracted high-level spatial features through the Feature Pyramid Networks network structure and transfer them to the head module, and then the head module generates the final detection results, including the target category, location, and confidence.

[0052] Compared with the prior art, the beneficial effects of this application are as follows:

[0053] (1) This application generates the service area point cloud by using the unmanned aerial vehicle photogrammetry technology. Compared with traditional manual measurement or high-precision terrestrial laser scanning, it not only greatly reduces the equipment investment and labor cost, but also can quickly complete the three-dimensional modeling of a large-scale area, especially suitable for the infrastructure planning and management of large-scale service areas.

[0054] (2) According to the actual situation of the service area and the coverage range of the lidar, this application can intelligently optimize the lidar layout points, minimize the number of lidars to the greatest extent, and ensure that there is no dead angle in the lidar coverage of each key area in the service area while reducing costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 It is a schematic diagram of the overall work flow of this application.

[0056] Figure 2 It is a schematic diagram of the grid division of the colored base map line model of the service area in this application.

[0057] Figure 3 It is a schematic diagram of arranging lidars at the service area entrance and exit positions in this application.

[0058] Figure 4 It is a schematic diagram of the coverage range of the lidars at the service area entrance and exit positions in this application.

[0059] Figure 5 It is a schematic diagram after the lidar arrangement is completed in this application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0060] Next, in combination with specific embodiments, the present application will be further described. It should be noted that in the description of this specification, the descriptions referring to terms such as "one embodiment", "some embodiments", "example", "specific example", or "some examples" mean that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic expressions of the above terms should not be understood as necessarily referring to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in a suitable manner in any one or more embodiments or examples. In addition, those skilled in the art can combine and combine the different embodiments or examples described in this specification.

[0061] In the description of the present application, it should be noted that for orientation terms, if there are terms such as "center", "lateral", "longitudinal", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise", etc., indicating the orientation and position relationship is based on the orientation or position relationship shown in the drawings. This is only for the convenience of describing the present application and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and should not be construed as limiting the specific protection scope of the present application.

[0062] It should be noted that the terms "first", "second", etc. in the description and claims of the present application are used to distinguish similar objects and do not necessarily have to be used to describe a specific order or sequence.

[0063] In the present application, unless otherwise clearly defined and limited, terms such as "install", "connect", "couple", "fix", etc. should be understood in a broad sense. For example, it can be a connection, a detachable connection, or integrated; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium, and it can be the communication inside two elements or the interaction relationship between two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0064] In this application, unless otherwise clearly specified and defined, the first feature being "on" or "under" the second feature may include direct contact between the first and second features, or may include the situation where the first and second features are not in direct contact but in contact through other features therebetween. Moreover, the first feature being "above", "over" and "on top of" the second feature includes the first feature being directly above and obliquely above the second feature, or merely indicating that the horizontal height of the first feature is higher than that of the second feature. The first feature being "under", "beneath" and "underneath" the second feature includes the first feature being directly below and obliquely below the second feature, or merely indicating that the horizontal height of the first feature is lower than that of the second feature.

[0065] The terms "comprising" and "having" in the description and claims of this application, and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device comprising a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these process, method, product or device.

[0066] One preferred embodiment of this application, as Figure 1 shown, a method for identifying vehicles in a service area based on three-dimensional lidar data, comprising the following steps:

[0067] S100: Collect the environmental information of the service area and construct a colored base map line model including the key ground features of the service area.

[0068] S200: Construct a local coordinate system and a world coordinate system; wherein, the local coordinate system is used for summarizing and analyzing the vehicle driving behavior data, and the world coordinate system is used for cross-system data integration and information sharing.

[0069] S300: Divide the colored base map line model into regions; based on the priority of the vehicle identification position, arrange lidars at the safe area positions of the colored base map line model, and use the minimum number of lidars under the condition of achieving full coverage of the service area according to the approximation principle.

[0070] S400: Classify and identify the real-time status of the vehicles in the service area based on the arranged lidars, and upload the collected data to the server side in real time.

[0071] It can be understood that, in order to more accurately identify the vehicles in the service area, it is necessary to fully collect the environmental information in the service area. At the same time, in order to facilitate the subsequent arrangement of lidars, the map information of the service area can be presented in the form of a base map, and then by traversing the lidar arrangement methods on the base map, the arrangement efficiency of the lidars can be effectively improved.

[0072] Specifically, the service area base map designed in this application covers key information such as parking spaces, lanes and no-parking areas in the service area. By combining with the subsequent two sets of coordinate systems, the base map information can be accurately calibrated, thereby providing comprehensive spatial data support for subsequent vehicle management.

[0073] In the construction of the coordinate system, the local coordinate system summarizes and analyzes vehicle driving behavior data, including traffic flow, speed, parking information, etc., which can facilitate the management and operation of the service area. The world coordinate system is mainly used to achieve the integration of lidar vehicle recognition and global positioning system, so as to provide support for cross-system data integration and information sharing.

[0074] As for the layout of LiDAR, since LiDAR is relatively expensive, in order to reduce the cost, this application designs a solution that can cover the entire service area and reduce the number of LiDAR deployments. This solution can achieve full LiDAR coverage of the service area to the maximum extent while reducing the number of LiDAR devices, thereby effectively reducing equipment costs and improving management efficiency.

[0075] For vehicle identification in the service area, this application can identify vehicles and analyze their behavior based on the point cloud data of the LiDAR. That is, through advanced algorithms, vehicles in the service area are classified, including cars, box trucks, and large trucks, and real-time tracking and behavior judgment of each vehicle are achieved. Through automatic detection of vehicle speeding, illegal parking, and reverse driving, the system can upload behavior data to the server in real time for service area managers to respond and make decisions quickly, thereby improving the safety and management efficiency of the service area.

[0076] For ease of understanding, each step will be described in detail below.

[0077] In this embodiment, step S100 includes the following specific processes:

[0078] S110: Multiple cross targets are evenly distributed in the service area, and the three-dimensional coordinates of the cross targets are measured using GNSS equipment.

[0079] Specifically, the number of cross targets can be set according to actual needs, and generally 5-10 cross targets can be set. The cross target can divide the marking area into four sector areas, and the colors of the two adjacent sector parts are set differently, for example, black and white respectively. These cross targets are facing the sky so that the drone can accurately and clearly capture the positions of these cross targets during the measurement process. The selection of cross targets should ensure that the entire service area is covered to ensure calibration accuracy. After completing the arrangement of the cross targets, GNSS equipment, such as Beidou Positioning System, GPS and Galileo, can be used to accurately measure the three-dimensional coordinates of a single cross target.

[0080] S120: Use a drone to take wide - angle and multi - perspective photos of the service area, and construct a colored three - dimensional point cloud model of the service area based on the collected image data.

[0081] Specifically, the image data collection by the drone needs to ensure sufficient image data for subsequent three - dimensional reconstruction. For the construction of the colored three - dimensional point cloud model, the Structure from Motion (SfM) algorithm is preferably used. By using the SfM algorithm, the position and orientation of the camera carried by the drone, as well as the three - dimensional coordinates of the objects in the scene, are deduced. The SfM algorithm not only needs to identify the feature points in the pictures but also can use the relative position relationship between the cameras to perform three - dimensional reconstruction of the scene. The specific steps of this algorithm are as follows:

[0082] Extract key feature points from each image and match the feature points between different images. According to the known feature point matching results, estimate the relative position and pose between the cameras through the fundamental matrix or the essential matrix. This process is essentially to solve a linear system to determine the internal and external parameters of each camera. Using the previously obtained camera positions and the matched two - dimensional feature points, the algorithm solves for the spatial points through the collinearity equation. The collinearity equation is a key mathematical tool in photogrammetry, which describes the geometric relationship between the points in the image space, the points in the camera coordinate system, and the points in the physical coordinate system. The collinearity equation allows the reverse deduction of the position of the object in the three - dimensional space from the two - dimensional image coordinates by establishing the relationship between the image coordinates and the object coordinate system. The form of the collinearity equation is as follows:

[0083] 。

[0084] 。

[0085] Among them, (x, y) represents the image coordinates, (x 0 , y 0 ) represents the principal point coordinates, c represents the focal length of the camera, (X, Y, Z) represents the object coordinates in the three - dimensional space, (X 0 , Y 0 , Z 0 ) represents the coordinates of the camera projection center in the object - side coordinate system, and R ij represents the rotation transformation from the camera coordinate system to the object - side coordinate system.

[0086] After the initial three - dimensional reconstruction is completed, it will be further optimized through Bundle Adjustment. Bundle Adjustment optimizes the parameters of all cameras and the coordinates of the three - dimensional points by minimizing the reprojection error of the image feature points in each perspective. Solved by the least - squares method, the positions of the cameras and the three - dimensional points are adjusted until the error reaches the minimum.

[0087] The 3D point cloud model obtained after the above-mentioned 3D reconstruction and optimization is still a sparse model. On the basis of sparse reconstruction, the Multi-View Stereo (MVS) technology is used to generate a dense point cloud. The dense point cloud represents the geometric structure of the scene more precisely by adding more 3D points. Through the above steps, a colored 3D point cloud model of the dense service area can be generated. Through the crosshair coordinates in the point cloud and the pre-measured crosshair coordinates, the point cloud can be transformed into the world coordinate system through the four-parameter transformation formula; the specific transformation process is well-known to those skilled in the art, so it will not be elaborated in detail here.

[0088] S130: Denoise the obtained 3D point cloud model, and use the denoised 3D point cloud model to construct a colored base map line model.

[0089] It can be understood that there may be a large number of noise points in the colored 3D point cloud obtained through the foregoing steps, so it is necessary to perform a denoising operation on the point cloud. Specifically, the denoising of the 3D point cloud model includes the following specific process:

[0090] S131: Judge the ground of the point cloud through the RANSAC algorithm.

[0091] It should be noted that for the convenience of subsequent steps, the point cloud space can be first divided into blocks, and the point cloud data is divided into multiple grid regions on the horizontal xy plane. The size of a single grid region can be set according to actual needs. For example, the size of a single grid is 3m×3m. Each grid region contains multiple points, and the z value of each grid is the height information of all points in the grid region.

[0092] S132: Judge the height of each point in the point cloud data from the ground. If the height of the point from the ground is within the set first range, it is considered that the point belongs to the low-altitude points and is retained, otherwise the next step is performed.

[0093] It can be understood that the specific value of the first range can be set according to actual needs; generally speaking, considering the slope change of the road in the service area, the first range is preferably set to ±0.5m.

[0094] S133: Judge whether the height of the point from the ground is within the second range. If it is, the next step is performed, otherwise the point is determined to be a noise point and removed.

[0095] It can be understood that the specific value of the second range can be set according to actual needs. Considering that the height of general vehicles is below 3m, that is, objects above 3m will not interfere with the normal driving of vehicles; therefore, the value of the second range is preferably 3m, that is, points with a height exceeding 3m are regarded as noise points and deleted.

[0096] S134: For points whose height exceeds the first range but lies within the second range, perform a range search with a set radius distance centered at this point. If the number of other points within this spherical search range is greater than the set threshold number, consider this point as a low-altitude point and retain it; otherwise, it is a noise point and is removed.

[0097] It can be understood that for points whose height exceeds the first range but lies within the second range, they may belong to points on buildings or street lamps. These points theoretically belong to part of the buildings or street lamps and cannot be simply judged as noise points. That is, by retaining the high points on buildings and street lamps through step S134 and removing the elevation noise points, the noise points in the high-altitude part of the parking area can be effectively removed, thereby obtaining a more accurate point cloud model and ensuring that the open area in the elevation direction is not affected by noise points. The specific value of the radius of the spherical search range can be selected according to actual needs. For example, the radius can be taken as 0.5 m.

[0098] In this embodiment, after denoising the point cloud data, the construction of the colored base map line model includes the following process:

[0099] S135: Based on the denoised point cloud data, remove all points that are more than the first range away from the ground.

[0100] It can be understood that the colored base map line model mainly records the ground information of the service area, and the denoised point cloud data mentioned above also contains the point clouds of some non-ground objects. For example, a semi-enclosed rain shelter, etc. The area under the rain shelter does not affect vehicle passage. If all these points are projected onto the ground in the subsequent steps, they may be regarded as ground structure points during the data extraction process, resulting in inaccurate subsequent base map establishment.

[0101] S136: Project the remaining point cloud data onto the ground to generate a colored base map model of the ground.

[0102] It should be known that the generated colored base map line model represents the ground information of the service area, including key ground features such as parking lines, zebra crossings, and vehicle markings.

[0103] S137: Through computer vision technology, extract the spatial layout and lane information of different areas within the service area from the colored base map model generated in step S136 to obtain the colored base map line model of the service area.

[0104] It can be understood that there are various computer vision technologies adopted in step S137, such as edge detection methods and threshold segmentation methods, etc. The specific working principles are well-known to those skilled in the art, so they will not be elaborated in detail here. Through computer vision technology, line features such as lane lines and parking lines can be extracted from the colored base map line model, and these features represent the spatial layout and lane information of different areas within the service area. By extracting the lane lines and other markings, a JSON file containing the positions, sizes, and geometric forms of each part is generated. This file contains the spatial information of elements such as lane lines, parking spaces, and no-parking areas, and provides data support for future lane line extraction and vehicle position recognition.

[0105] Based on the above information, an accurate colored base map line model of the service area is generated. This model provides precise ground layout information through the fusion of a three-dimensional coordinate system and spatial data such as lane lines, and is used for subsequent vehicle recognition and position analysis. Using this base map line model, combined with future vehicle position recognition technology, intelligent management and monitoring of vehicles within the service area are achieved. This technology can provide precise basic data support for lane line management, parking space utilization, traffic flow monitoring, etc.

[0106] It should be known that in this application, the drone photogrammetry technology is adopted to generate the point cloud of the service area, so that the base map of the service area can be constructed quickly and at low cost. Compared with the traditional manual measurement and high-precision terrestrial laser scanning methods, this application not only greatly reduces the equipment investment and labor costs, but also can quickly complete the three-dimensional modeling of a large-scale area, especially suitable for the infrastructure planning and management of large-scale service areas. Through the technical solution of this application, an accurate base map can be provided for the service area, and it is ensured that various subsequent technical applications based on the three-dimensional model have high-quality data support.

[0107] In this embodiment, in order to achieve precise vehicle monitoring and information management, two coordinate systems are designed in step S200, namely the local coordinate system and the world coordinate system of the service area; the specific construction processes of the local coordinate system and the world coordinate system are well-known technologies to those skilled in the art; for the convenience of understanding, the two coordinate systems will be briefly described below.

[0108] I. World coordinate system.

[0109] The world coordinate system is mainly used to collect and integrate information of various parts within the service area and the identified vehicles globally. Through the world coordinate system, various types of information within the service area can be integrated with the highway traffic flow data. For example, through the world coordinate system, the positions and trajectories of vehicles within the service area can be monitored in real time, and data exchange can be carried out with external systems (such as traffic control centers).

[0110] The construction of the world coordinate system depends on the Global Navigation Satellite System (GNSS) and the cross targets arranged within the service area. Five to ten cross targets are evenly arranged within the service area. These cross targets should have clear global positioning information, face the sky, and can be accurately captured by drones or other measurement devices. By calibrating the positions of these cross targets in the 3D model and combining the coordinate information provided by GNSS, the base map and 3D point cloud model of the service area can be transformed into the world coordinate system to achieve precise global positioning and information fusion.

[0111] II. Local coordinate system.

[0112] The local coordinate system is mainly used for vehicle monitoring, anomaly detection, and behavior analysis within the service area. Compared with the world coordinate system, the local coordinate system is based on the specific scenario of the service area and can more intuitively and simply represent the positions and states of objects within the service area. Through the local coordinate system, vehicle entry and exit identification, parking status judgment, and real-time monitoring of other events can be efficiently carried out. The use of the local coordinate system enables the system to achieve rapid response within a local range, improving the processing speed and accuracy.

[0113] The establishment of the local coordinate system is based on the transformation principle of the world coordinate system. After obtaining the coordinate data of the world coordinate system, the local coordinate system can be obtained by decentralizing the coordinates within the service area. Specifically, by converting the world coordinate system data of all key objects (such as parking spaces, lane lines, traffic signs, etc.) within the service area into relative coordinates, the local coordinate system can be constructed. This process can be simplified by setting a reference point or a fixed reference point to ensure the smooth progress of subsequent data processing and management in the local coordinate system.

[0114] In the subsequent vehicle monitoring and behavior recognition process, the system will use the local coordinate system to process and judge the data. To achieve seamless conversion between different scenarios, an interface will be provided between the local coordinate system and the world coordinate system to support real-time conversion and data update. Specifically, when it is necessary to associate the recognition results in the local coordinate system with the traffic data on a global scale, the system will convert the coordinates of the local coordinate system into the world coordinate system through a preset conversion algorithm for efficient data sharing and information integration. Through the combination of the two coordinate systems, this application can not only provide local management and control but also facilitate linkage with a larger-scale traffic system, Internet of Things platform, and highway management system. By using the world coordinate system, the data in the service area can be seamlessly connected to the surrounding traffic flow.

[0115] In this embodiment, when performing step S300, based on the principle of covering the entire service area with the minimum number of lidars, the installation height of each lidar needs to be confirmed first when arranging the lidars. The installation height of the lidar needs to meet the following conditions: It can identify all types of vehicles and ensure the recognition effectiveness when the vehicle passes obliquely below the lidar. That is, for a vehicle close to the lidar, the lidar should still be able to correctly identify its presence.

[0116] It should be known that the height of a vehicle generally does not exceed 4.2m, and the length of a vehicle generally does not exceed 18m. According to the working principle of the lidar, the scanning angle and effective scanning distance of the lidar are crucial for the recognition effect. To avoid recognition failure due to too small a longitudinal angle, it is usually required that the scanning angle of the lidar is greater than 30 degrees. Thus, according to the trigonometry principle, the minimum installation height of the lidar can be deduced as: 9×sin30° + 4.2 = 8.7m. To leave a redundant space, the installation height of each lidar in this embodiment is preferably 9 meters.

[0117] For easy understanding, the following can describe the local arrangement process of the lidar in detail based on the simplified structure of the service area. As Figure 2 shown, the black shaded areas in the figure represent the positions suitable for installing lidars. The shaded areas include not only the intersections between the lanes and the parking areas, but also specifically mark some areas above the parking areas. It should be noted that these shadows are only schematic markings to show which areas are suitable for arranging the radars. During actual arrangement, the specific positions of the lidar control points should be considered. These areas are usually located at the intersections between the parking spaces and the lane intersections, and the lidars can effectively monitor vehicles at these key positions. Through Figure 2 it can be intuitively seen which parts are suitable for placing the lidars. The shaded areas shown in the figure cover the key areas of the service area, including lanes, parking spaces, and entrance and exit areas. The arrangement of the lidars should be concentrated in these intersection areas to ensure that no important vehicle flow information and status monitoring are missed.

[0118] Specifically, in step S300, the area division of the colored base map line model includes the following process:

[0119] S310: Divide the service area into three areas according to the colored base map line model, including the area to be recognized, the area of immovable objects, and the area where the lidar can be installed.

[0120] It is understandable that the areas to be recognized include the driving lane, parking spaces in the parking area, gas station area, charging area, etc. The lidar should be able to cover these areas completely, and lidar devices shall not be installed in these areas. The immovable object areas include areas such as the buildings of the server, columns, and signboards. These areas are not suitable for placing radars. The areas where radars can be installed include green belts, roadside areas, the middle part between the parking area and the road, and the intersections of multiple parking spaces, etc.

[0121] S320: Divide the entire service area into grids and number the grids in the horizontal and vertical directions.

[0122] It is understandable that, for the convenience of calculating the layout direction of lidars subsequently, the service area can be divided into spatial grids first. The size of each grid can be set according to actual needs. For example, the size of each grid is 15m×15m. To ensure the effective recognition of vehicles and minimize the number of radar devices, the grids in the service area need to be numbered. For the convenience of distinction, different types of numbers can be used in the horizontal and vertical directions. For example, letters can be used for horizontal numbering and Arabic numerals can be used for vertical numbering. For example Figure 2 As shown, assuming the size of the service area is 225m×120m, then the service area can be divided into 15×8 grid areas. The horizontal numbers range from A to O, and the vertical numbers range from 1 to 8.

[0123] S330: Obtain the numbered positions of the areas to be recognized and the coordinates of the four corner points of each grid according to the divided grids in combination with the colored base map line model.

[0124] It is understandable that after the grid division is completed, it is necessary to further determine which grid areas need to be recognized according to the base map and grid positions. For example Figure 2 As shown, the grids in the first row do not need to be recognized, and grid areas such as D2 to M2 do not need to be recognized either. When determining whether a grid area needs to be recognized, it is also necessary to obtain the coordinates of the four corner points of each grid. Through these coordinates, it can be judged whether the points covered by the lidar are within the grid.

[0125] In this embodiment, the vehicle recognition priority at the service area entrance and exit positions is higher than that at other positions, that is, the lidar needs to first ensure the traffic flow information at the service area entrance and exit, as well as dynamic information such as the driving mode of the vehicle after entering from the entrance and the entering direction before exiting the exit, so as to ensure accurate recognition when the vehicle is driving in the service area.

[0126] Specifically, as Figure 2 and Figure 3 shown, the lidar layout at the service area entrance and exit positions in step S300 includes the following process:

[0127] S340: Locate the midpoint coordinates (x 1 , y 1 ) of the service area entrance position and the center point coordinates (x 2 , y 2 ) in the divided grid.

[0128] S350: Calculate the placement vector v = (x 2 - x 1 , y 2 - y 1 ) of the lidar according to the obtained coordinates.

[0129] S360: Determine the candidate area for lidar placement according to the direction of the placement vector v.

[0130] S370: Traverse each grid in the candidate area to calculate the recognition range of the lidar, select the locally optimal grid and determine the precise placement position of the lidar through the approximation principle.

[0131] For easy understanding, the following will describe in detail the lidar placement method at the service area entrance in combination with Figure 2 .

[0132] As Figure 2 shown, extract the lane information of the service area through image processing technology, and judge the widened, bifurcated or obvious inflection point positions of the lanes. As can be seen from Figure 2 , the vehicle diversion area at the service area entrance position is grid B3. Then it is necessary to ensure that the lidar at the entrance can cover this area and perform a certain redundant coverage of the surrounding area. The center point of the service area is located at the intersection of grids H4 and H5, with coordinates (x 2 , y 2 ); the center point coordinates of grid B3 are (x 1 , y 1 ). According to the size of the service area, the specific values of the two coordinates can be obtained, and then the placement vector of the lidar layout can be calculated.

[0133] According to the direction of the calculated placement vector v, determine the candidate area for lidar placement, and ensure that the effective coverage range of the lidar can completely cover grid B3 and the maximum coverage area of columns A and B. To determine the best placement point of the lidar, traverse each candidate area and calculate whether the recognition range of the lidar can cover area B3 and columns A and B. Finally, select a locally optimal placement grid, and further accurately determine the placement position within this grid through the approximation method. Assume that the effective recognition range of the lidar is 70 meters, and the scanning angle of the lidar meets the requirement of 30 degrees. Let the lidar placement point be (x p , y p) Then, the effective coverage area of the lidar is a circular area with this point as the center and a radius of 70 meters, which can be specifically expressed by the following formula.

[0134] .

[0135] It can be understood that using only the center of the grid as the placement point will affect the placement accuracy of the lidar, and the center points of some areas may be on the road or in the parking space, which cannot meet the placement requirements. Therefore, it is necessary to find the best placement area in the placeable area of this grid; that is, to select the placement point through the approximation principle within a single grid. The process of determining the precise layout position of the lidar based on the approximation principle is as follows:

[0136] S301: Traverse all the layout areas where the lidar can be placed in the corresponding locally optimal grid.

[0137] S302: Divide the layout area into multiple placement points at a set interval distance.

[0138] S303: Assume the placement of the lidar at each placement point and judge the coverage range of the lidar, and select the placement point with the best coverage range as the precise layout position of the lidar.

[0139] It should be noted that the setting of the interval distance in step S302 can be selected according to the actual needs of those skilled in the art. For example, the interval distance can be set to 1m.

[0140] In this embodiment, after completing the layout of the lidar at the service area entrance and exit positions, it is necessary to select the control points at other positions in the middle of the service area. As Figures 3 to 5 shown, the process of placing the lidar in other areas of the service area except the entrance and exit positions is as follows:

[0141] S381: Determine the grid corresponding to the edge of the coverage range of the lidar at the service area entrance and exit positions, and then obtain the grid area numbers that are not fully recognized in the service area.

[0142] S382: Traverse each empty grid area in the horizontal or vertical direction and simulate the placement of the lidar.

[0143] S383: Record the empty grid areas and the edge positions covered by the lidar at each placement position.

[0144] S384: Select the placement point that covers the most empty grid areas and has the largest number of cross-grid numbers with the grid area occupied by the lidar at the service area entrance and exit positions as the best placement grid area.

[0145] S385: Determine the precise layout position of the lidar within the optimal placement grid area through the approximation principle.

[0146] It can be understood that for the layout of lidars at the middle position of the service area, it is mainly to approximate towards the middle based on the coverage range of the lidars at the entrance and exit positions. In this way, the number of lidars to be laid can be effectively reduced. For the convenience of understanding, the layout process of the lidars in the middle area of the service area will be described in detail below with reference to the accompanying drawings.

[0147] As Figure 3 shown, the two circles respectively represent the coverage ranges of the two lidars at the entrance and exit positions of the service area. Extract the fully covered areas of the two lidar points at the entrance and exit. For the areas that are not fully identified, demarcate them as empty areas. As Figure 4 shown, the gray shaded part is the range of the grid area that can be fully covered by the lidars at the entrance and exit positions. It can be seen from the figure that the areas that need to be identified but are still not fully identified in the current service area include E2, E7 - E8, G3 - G8, H3 - H8, I3, and I6 - I8.

[0148] After obtaining these areas, first, it is necessary to divide the span of the areas; since each grid is an interval of 15m × 15m, the ratio and range between the horizontal grid span and the vertical grid span can be judged first. Through this ratio and range, the approximate number of additional lidars to be laid can be further estimated. Taking the current service area as an example, the vertical span of the unrecognized area is six grids, which is 90 meters, that is, 45 meters on each side. The horizontal span is four grids, which is 60 meters, that is, 30 meters on each side. Therefore, theoretically, only one more lidar needs to be laid.

[0149] Next, it is necessary to determine the layout position of this additional lidar; sequentially select the middle grids that are not fully occupied along the horizontal or vertical direction, and then judge whether a lidar can be installed in this grid. The specific judgment method can refer to the layout judgment method of the lidars at the entrance and exit. If the grid area can install a lidar, the lidar can be virtually arranged and its coverage range can be recorded. After the virtual layout of the lidars is completed in all grid areas where lidars can be installed, compare the recorded coverage ranges of all, and select the grid area corresponding to the coverage range that can occupy the most empty grid areas and has the largest cross - overlap range with the lidars at the entrance and exit positions as the optimal placement grid area. After carefully analyzing the uncovered areas and selecting the lidar placement area, further precise adjustment is required. The selection principle of the lidar placement point is the same as that of the previous lidars at the entrance and exit. To ensure the maximum coverage range of each lidar, after selecting a suitable placement point, verify the effective coverage range of the lidar again to ensure that it can cover the areas that are not fully identified. AsFigure 5 As shown, based on the placement points of the current three lidars, full coverage of all areas to be recognized within the service area can be achieved.

[0150] In this embodiment, step S400 includes the following process:

[0151] S410: Perform target annotation on the three-dimensional point cloud collected by the lidar, including three types of targets: cars, box trucks, and large trucks, and other types are labeled as negative samples.

[0152] It can be understood that for the convenience of data management, the LabelCloud tool can be used for the annotation of the three-dimensional point cloud, and labels in the format of the KITTI dataset are generated. It should be noted that when annotating vehicles, the head direction must be correctly marked, otherwise it will affect the subsequent target recognition accuracy.

[0153] S420: Build a vehicle recognition model on the server side and perform model training.

[0154] It can be understood that there are various specific methods for vehicle recognition model training. In this embodiment, the PointPillars model is used for model training, which specifically includes the following process:

[0155] S421: Divide the three-dimensional point cloud data into multiple cylinders; if the number of points contained in a single cylinder exceeds the threshold, calculate the features of the cylinder; if the number of points in a single cylinder is less than the threshold, fill it with zeros.

[0156] It can be understood that each cylinder represents the point cloud information within a fixed area. The PointPillars model divides the space into three-dimensional grids of the same size, that is, cylinders; the grid size can be adjusted according to the actual scenario. After calculating the features of the cylinders, each cylinder is represented as a point cloud feature vector, including coordinate information, reflection intensity, etc.

[0157] S422: Aggregate the points within each cylinder, calculate the mean feature of the cylinder and compress it into a feature vector of a fixed length; splice the feature vectors of all cylinders to form a two-dimensional feature map.

[0158] It should be known that the size of the feature map is the number of cylinders multiplied by the feature dimension of each cylinder.

[0159] S423: Process the two-dimensional feature map through the backbone network to gradually capture the regional shape, texture, and spatial relationship between different regions, so as to extract the high-level spatial features in the two-dimensional feature map.

[0160] It is understandable that a 2D convolutional neural network is used as the backbone network, and then the feature map is processed through multiple convolutional layers to obtain corresponding high-level spatial features. The convolutional layer usually combines BatchNorm and the ReLU activation function to enhance the network's non-linear expression ability.

[0161] S424: The extracted high-level spatial features are fused through the Feature Pyramid Networks network structure and transmitted to the head module, and then the head module generates the final detection results, including the target category, location, and confidence.

[0162] It should be known that the generation of the detection results by the head module includes the following process: first, a classification task is performed, that is, predicting whether each column contains a target; then a regression task is performed, that is, predicting the location and size of the target. By comparing with the labeled data, backpropagation is carried out and the network weights are adjusted. After multiple rounds of training, the vehicle recognition accuracy can be effectively improved.

[0163] S430: Calibrate the coordinate system of the lidar to ensure the unity of the coordinates of the point cloud data collected by the lidar.

[0164] It is understandable that the lidar is calibrated through the above two coordinate systems to ensure that all the point cloud data collected by the lidar can be unified into the calibrated coordinate system. After that, the real-time collected point cloud data is transmitted to the server side and combined with the trained model for detection, and the vehicle can be recognized.

[0165] S440: Upload the point cloud data collected by the lidar to the server side, and then combine it with the trained vehicle recognition model for detection, and track and recognize the behavior of the vehicle according to the detection results.

[0166] It is understandable that in the real-time monitoring stage of the lidar, the vehicle recognition results include information such as 3D bounding boxes, vehicle center coordinates, and sizes. For each recognized vehicle, the system assigns a unique label to it and passes this information into the DeepSORT algorithm. DeepSORT combines the vehicle appearance features extracted by the convolutional neural network (CNN), predicts the movement trajectory of the object through the SORT framework and the Kalman filter, and uses the IoU (Intersection over Union) value between objects for target matching, thereby updating the state of the object (such as location, speed).

[0167] Since the entire service area has a full coverage lidar layout and vehicles are recognized in the calibrated coordinate system, the recognition results between two consecutive lidars can be matched through the vehicle position, thus realizing cross-lidar tracking of vehicles. Finally, the prediction results will be uploaded to the management center of the service area in real time for violation detection and service area management. In this way, the management and operation and maintenance efficiency of the service area can be effectively improved.

[0168] It should be noted that in step S400, by combining the PointPillars algorithm and the DeepSORT algorithm, accurate and real-time recognition and tracking of vehicles in the service area can be achieved. The PointPillars algorithm can accurately recognize different types of vehicles (such as cars, trucks, etc.) through efficient feature extraction of point cloud data and supports continuous tracking of vehicles in complex environments. The DeepSORT algorithm ensures high stability and high precision of vehicle tracking through the Kalman filter and appearance feature extraction. On this basis, the recognition data and position information of all vehicles will be uploaded to the service area management center in real time, realizing the intelligence and automation of service area management, significantly improving the efficiency and safety of vehicle management, and providing intelligent operation and maintenance and data analysis support for the service area.

[0169] The above describes the basic principles, main features and advantages of the present application. Those skilled in the art should understand that the present application is not limited by the above embodiments. What is described in the above embodiments and the specification is only the principle of the present application. Without departing from the spirit and scope of the present application, the present application will have various changes and improvements, and these changes and improvements all fall within the scope of the present application claimed. The scope of protection required by the present application is defined by the appended claims and their equivalents.

Claims

1. A method for identifying vehicles in a service area based on three-dimensional laser radar data, characterized in that: The steps include: S100: Collect service area environmental information and construct a colored base map line model including key ground features of the service area; S200: constructing a local coordinate system and a world coordinate system; wherein the local coordinate system is used to aggregate and analyze vehicle driving behavior data, and the world coordinate system is used for cross-system data integration and information sharing; S300: Divide the colored base map line model into regions; arrange laser radars in safe area positions of the colored base map line model based on the priority of the vehicle identification position, and use the minimum number of laser radars while achieving full coverage of the service area according to the approximation principle; S400: Classify and identify the real-time status of vehicles in the service area based on the deployed laser radar, and upload the collected data to the server in real time; In step S300, the region division of the colored base map line model includes the following process: S310: Divide the service area into three areas according to the colored base map line model, including an area that needs to be identified, an area for immovable objects, and an area where radar can be set up; S320: Divide the entire service area into grids and number the divided grids in horizontal and vertical directions; S330: Obtain the numbered positions of the areas to be identified and the coordinates of the four corner points of each grid according to the divided grids combined with the colored base map line model; The vehicle identification priority at the entrance and exit of the service area is higher than that at other locations; then the laser radar arrangement at the entrance and exit of the service area in step S300 includes the following process: S340: Locate the midpoint coordinates (x1, y1) of the entrance and exit of the service area in the divided grid, and the center point coordinates (x2, y2) of the service area; S350: Calculate the placement vector v=(x2-x1, y2-y1) of the laser radar according to the obtained coordinates; S360: Determine a candidate area for radar placement according to the direction of the placement vector v; S370: traverse each grid in the candidate area to calculate the recognition range of the laser radar, select the local optimal grid and determine the precise layout position of the laser radar through the approximation principle; The laser radar layout process for other areas of the service area except the entrance and exit locations is as follows: S381: Determine the grid corresponding to the edge of the coverage range of the laser radar at the entrance and exit of the service area, and then obtain the grid area number that is not fully recognized in the service area; S382: traverse each empty grid area in the horizontal or vertical direction and simulate the placement of the laser radar; S383: Recording the empty grid area and edge position covered by the laser radar at each placement position; S384: Selecting a placement point that covers the largest number of empty grid areas and has the largest number of intersecting grids with the grid area occupied by the laser radar at the entrance and exit of the service area as the optimal placement grid area; S385: Determine the precise placement position of the laser radar in the optimal placement grid area by using the approximation principle; The process of determining the precise placement of the LiDAR based on the approximation principle is as follows: S301: traversing all layout areas where laser radars can be placed in the corresponding local optimal grid; S302: Divide the layout area into a plurality of placement points according to a set interval distance; S303: Performing a hypothetical placement of the laser radar for each placement point and determining the coverage of the laser radar, and selecting a placement point with the best coverage as the precise placement position of the laser radar; Step S400 includes the following process: S410: Target annotation is performed on the 3D point cloud collected by the LiDAR, including three types of targets: cars, vans, and trucks. Other types are annotated as negative samples. S420: Building a vehicle recognition model on the server and performing model training; S430: Calibrate the coordinate system of the laser radar to ensure that the coordinates of the point cloud data collected by the laser radar are unified; S440: The point cloud data collected by the laser radar is uploaded to the server, and then detected in combination with the trained vehicle recognition model, and the vehicle is tracked and the behavior is recognized according to the detection results.

2. The method for identifying vehicles in a service area based on three-dimensional laser radar data according to claim 1, characterized in that: Step S100 includes the following specific processes: S110: Multiple cross targets are evenly distributed in the service area, and the 3D coordinates of the cross targets are measured using GNSS equipment; S120: Use drones to take photos of the service area in a wide range and at multiple angles, and construct a colored 3D point cloud model of the service area based on the collected image data; S130: De-noising the obtained three-dimensional point cloud model, and using the de-noised three-dimensional point cloud model to construct a colored base map line model.

3. The method for identifying vehicles in a service area based on three-dimensional laser radar data as claimed in claim 2, characterized in that: In step S130, denoising the three-dimensional point cloud model includes the following specific processes: S131: judging the ground of the point cloud by using the RANSAC algorithm; S132: judging the height of each point from the ground in the point cloud data, if the height of the point from the ground is within a set first range, the point is considered to be a low-range point and is retained, otherwise proceeding to the next step; S133: Determine whether the height of the point from the ground is within the second range, if so, proceed to the next step, otherwise, determine that the point is a noise point and remove it; S134: Use the point as the center of the sphere to search within a range with a set radius. If the number of other points within the spherical search range is greater than the set threshold, the point is considered a low-level point and is retained; otherwise, it is considered a noise point and is removed.

4. The method for identifying vehicles in a service area based on three-dimensional laser radar data as claimed in claim 3, characterized in that: In step S130, the construction process of the colored base map line model is as follows: S135: Based on the denoised point cloud data, remove all points that are above a first range from the ground; S136: Project the remaining point cloud data onto the ground to generate a colored base map model of the ground; S137: Using computer vision technology, the spatial layout and lane information of different areas in the service area are extracted from the colored base map model generated in step S136 to obtain a colored base map line model of the service area.

5. The method for identifying vehicles in a service area based on three-dimensional laser radar data as claimed in claim 1, characterized in that: The process of model training using the PointPillars model in step S420 is as follows: S421: Divide the three-dimensional point cloud data into multiple cylinders; if the number of points contained in a single cylinder exceeds a threshold, calculate the features of the cylinder; if the number of points in a single cylinder is less than the threshold, fill it with zero; S422: Aggregate the points in each column, calculate the mean feature of the column and compress it into a feature vector of fixed length; splice the feature vectors of all columns to form a two-dimensional feature map; S423: Processing the two-dimensional feature map through the backbone network, gradually capturing the regional shape, texture and spatial relationship between different regions, so as to extract high-level spatial features in the two-dimensional feature map; S424: The extracted high-level spatial features are fused through the Feature Pyramid Networks network structure and passed to the head module, and then the head module generates the final detection results, including target category, location and confidence.

Images