Method, medium and device for globally tracking a vehicle within a semi-occluded scene

Abstract

This application discloses a method, medium, and device for full-domain vehicle tracking in a semi-enclosed scene. The method includes the following steps: constructing a reference coordinate system and solving the transformation matrix of the original coordinate system of each lidar relative to the reference coordinate system; determining the effective area of ​​the semi-enclosed scene and generating corresponding envelope regions under the original coordinate systems of each lidar; constructing training datasets for each lidar under different simulated conditions to train the vehicle tracking model; during actual tracking, cropping the real-time point cloud data of each lidar based on the envelope region, stitching and fusing the cropped point cloud data, and inputting it into the vehicle tracking model for full-domain vehicle tracking. The device and medium are used to implement the above method. The beneficial effects of this application are: during the training phase, it systematically constructs various data sample sets such as single-radar, multi-radar partial fusion, and full-radar fusion, enabling the model to learn the point cloud distribution characteristics under different radar online states.

Description

Technical Field

[0001] This application relates to the field of intelligent transportation technology, and in particular to a method, medium and device for full-area vehicle tracking in a semi-enclosed scenario. Background Technology

[0002] In recent years, fixed-deployment LiDAR has been widely used for vehicle perception and monitoring tasks in traffic scenarios such as service areas, parking lots, urban roads, and highways due to its all-weather operation capability, high spatial measurement accuracy, and strong resistance to light interference. Compared with traditional vision-based perception solutions, LiDAR can directly acquire high-precision 3D point cloud information and maintain relatively stable perception performance under low light, backlight, or complex lighting conditions.

[0003] In existing application scenarios, multiple LiDARs are typically deployed to achieve full coverage, with some overlap in coverage areas between adjacent LiDARs. However, existing vehicle tracking solutions often prioritize computational efficiency, using a single LiDAR to independently identify vehicles and then binding and tracking the identification results from different LiDARs. However, in LiDAR point cloud-based vehicle identification and tracking, the deployment location, coverage area, and viewing angle of the LiDARs significantly impact the quality of the vehicle point cloud. In practical applications, a single LiDAR is susceptible to factors such as large vehicles obstructing the view, small vehicles being located at the radar edge, and short-term radar outages or malfunctions, leading to sparse, partially missing, or discontinuous vehicle point clouds, thus reducing the stability and accuracy of vehicle identification. These point cloud quality issues further cause unstable vehicle identification results, target loss, and tracking interruptions, making continuous vehicle tracking across the entire area difficult and severely limiting the effectiveness of refined vehicle management and intelligent scheduling in semi-closed scenarios.

[0004] Current research on improving the performance of LiDAR vehicle recognition and tracking mainly focuses on improving the target detection or tracking algorithms themselves. However, most of these methods assume that the input point cloud data is of stable quality, and they are difficult to fundamentally solve recognition and tracking failures caused by data-level issues such as radar occlusion, distance attenuation, and missing point cloud data. Summary of the Invention

[0005] One objective of this application is to provide a method for full-domain vehicle tracking in a semi-enclosed environment that can overcome at least one of the deficiencies in the aforementioned background art.

[0006] Another objective of this application is to provide an electronic device capable of implementing a method for full-domain vehicle tracking in a semi-enclosed environment that addresses at least one of the deficiencies in the aforementioned background technology.

[0007] Another object of this application is to provide a computer-readable storage medium capable of implementing a method for full-domain vehicle tracking in a semi-enclosed scenario that addresses at least one of the deficiencies in the aforementioned background art.

[0008] To achieve at least one of the above objectives, one aspect of this application provides a method for full-domain vehicle tracking in a semi-enclosed environment, comprising the following steps: S100: Construct a unified reference coordinate system and solve the transformation parameters of the original coordinate systems of multiple lidars in a semi-enclosed scene relative to the reference coordinate system. S200: Determine the effective area for vehicle driving in a semi-enclosed scene, and generate the envelope region of the corresponding effective area in the original coordinate system of each lidar. S300: Construct training datasets for each LiDAR under different simulated working conditions, and use the obtained training datasets to train the vehicle tracking model; wherein, the training datasets include at least a first type of sample set consisting of a single LiDAR point cloud, a second type of sample set consisting of the fusion of multiple LiDAR point clouds that are not all of them, and a third type of sample set consisting of the fusion of all LiDAR point clouds. S400: During the actual vehicle tracking process, the real-time point cloud data of each LiDAR is cropped based on the corresponding envelope region; according to the obtained transformation parameters, all cropped point cloud data are transformed to the reference coordinate system for stitching and fusion, and the resulting real-time point cloud fusion data is input into the trained vehicle tracking model for full-domain vehicle tracking.

[0009] Preferably, the transformation parameters include rotation matrix and translation vector. In step S100, the solution for the transformation parameters between the original coordinate system and the reference coordinate system of each lidar includes the following process: a lidar in the center of the semi-enclosed scene is selected as the reference lidar, and its original coordinate system is used as the reference coordinate system; for other lidars with known installation parameters, the rotation matrix and translation vector relative to the reference coordinate system are directly solved by the point cloud fine registration method based on the main direction features of the point cloud of each lidar; for other lidars with unknown installation parameters, coarse registration is first performed by the overlapping area with the lidar with known installation parameters or the reference lidar, and then the rotation matrix and translation vector relative to the reference coordinate system are solved by the point cloud fine registration method.

[0010] Preferably, the coarse registration for lidar with unknown installation parameters includes the following process: if the overlap area between the lidar and the reference lidar with known installation parameters is greater than a set threshold, a coarse registration process is performed to estimate the coarse rotation relationship between the two lidars using a fast point feature histogram method; if the overlap area between the lidar and the reference lidar with known installation parameters is less than or equal to the set threshold, multiple common point pairs corresponding to the positions of the two lidars in the overlap area are manually selected, and a coarse registration process for aligning the original coordinate system is performed based on the obtained common point pairs.

[0011] Preferably, in step S200, the acquisition of the envelope region for each lidar includes the following process: transforming the point cloud data of each lidar to a reference coordinate system and fusing them according to the obtained transformation parameters; extracting the envelope point set covering the entire effective area of ​​the semi-closed scene from the fused point cloud data; inversely transforming the point cloud in the envelope point set back to the original coordinate system of each lidar, and then each lidar merges the point cloud corresponding to the envelope point set in its own original coordinate system to obtain the corresponding required envelope region.

[0012] Preferably, in step S300, the construction of the second type of sample set includes the following process: calculating the projection coverage of each lidar in the reference coordinate system; determining the set of fusionable lidars that have effective spatial overlap with each lidar based on the overlap rate between the projection coverage of each lidar; generating multiple different lidar combinations based on the obtained set of fusionable lidars; after deduplicating the lidar combinations generated from different set of fusionable lidars according to the overlap difference of the projection coverage, summarizing the fusion point cloud samples corresponding to the remaining lidar combinations to obtain the second type of sample set.

[0013] Preferably, in step S300, the construction of the second type of sample set includes the following process: obtaining dimensional parameters to determine whether any two lidars have complementarity, including spatial coverage overlap, viewing angle complementarity coefficient, effective detection range matching degree, and installation height difference; weighted fusion of the obtained dimensional parameters to construct a comprehensive complementarity score; fusion of lidar pairs with comprehensive complementarity scores greater than a set score threshold to construct a set of fusionable lidars that have complementary relationships with each lidar; construction of a lidar complementarity graph based on the obtained set of fusionable lidars, and generation of all candidate lidar combinations using the connected subgraphs of the lidar complementarity graph; construction of the comprehensive value of each candidate lidar combination through a value function, and selection of the top m candidate lidar combinations that maximize the comprehensive value using a greedy algorithm or heuristic search as the initial lidar combination set for the second type of sample set; after deduplication of the initial lidar combination set based on spatial coverage equivalence, summarizing the fused point cloud samples corresponding to the remaining lidar combinations in the initial lidar combination set to obtain the second type of sample set.

[0014] Preferably, the comprehensive complementarity score S corresponding to any two lidars i and j ij The expression is: ; In the formula, ω1, ω2, ω3, and ω4 represent the weight coefficients corresponding to each item, and η ij θ represents the overlap ratio between lidar i and lidar j. ij ρ represents the view complementarity coefficient between lidar i and lidar j. ij This indicates the effective detection range matching degree between lidar i and lidar j. h ij H represents the difference in installation height between lidar i and lidar j. max This indicates the preset maximum installation height difference.

[0015] Preferably, the expression for the comprehensive value F(C) of any candidate radar combination C is: ; In the formula, α, β, and γ represent the weight coefficients of each item, This indicates the number of lidar units in candidate radar combination C. S represents the overall complementarity score of all radar pairs in candidate radar combination C. C The sum of these values, Redundancy(C), represents the combination redundancy, used to penalize combinations with highly overlapping spatial coverage.

[0016] Another aspect of this application provides an electronic device including a processor and a memory; the memory is used to store a computer program, and the processor is used to execute the computer program to implement the above-described method for full-domain vehicle tracking in a semi-enclosed scenario.

[0017] Another aspect of this application provides a computer-readable storage medium storing a computer program; when the computer program is executed by a processor, it implements the above-described method for full-domain vehicle tracking in a semi-closed scenario.

[0018] Compared with the prior art, the beneficial effects of this application are as follows: (1) This application does not rely on the theoretical coordinates or ideal deployment parameters of the lidar during installation. Instead, it constructs a unified reference coordinate system for the field area through point cloud sampling and rotation matrix solving, achieving spatial alignment of multiple lidars. Even when there are positional deviations, angle errors, or incomplete calibration information during lidar installation, reliable coordinate unification and point cloud fusion processing can still be completed. This method reduces the dependence on the accuracy of manual calibration and the consistency of construction, significantly improving the applicability of the system in actual engineering deployment environments. It is especially suitable for scenarios with complex installation conditions and differences in lidar deployment, such as service areas and parking lots.

[0019] (2) This application systematically constructs various data sample sets during the training phase, including single-radar, multi-radar partial fusion, and full-radar fusion, enabling the model to learn the point cloud distribution characteristics under different radar online states. When one or more lidars go offline, are blocked, or malfunction during actual operation, the model can still complete stable vehicle recognition based on existing training experience. This method effectively avoids the performance degradation problem caused by traditional training based solely on "full radar online" conditions, fundamentally improving the robustness and reliability of the system under complex working conditions.

[0020] (3) This application constructs an envelope of the effective region and performs unified cropping on the multi-radar point cloud, effectively eliminating point cloud data from irrelevant areas such as buildings, green belts, and areas outside the site. This cropping mechanism significantly reduces the size of the input point cloud without losing effective vehicle information, thereby significantly reducing the computational complexity of subsequent recognition algorithms. At the same time, reducing invalid point cloud interference helps improve the stability and accuracy of vehicle detection and classification. This method is particularly suitable for engineering application scenarios that are sensitive to real-time performance and computing power. Attached Figure Description

[0021] Figure 1 This is a schematic diagram of the overall working steps of this application. Detailed Implementation

[0022] The present application will now be further described in conjunction with specific embodiments. It should be noted that, in the description of this specification, the use of terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicates that the specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms should not be construed as necessarily referring to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. In addition, those skilled in the art can combine and integrate the different embodiments or examples described in this specification.

[0023] In the description of this application, it should be noted that the terms "center", "lateral", "longitudinal", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise", etc., which indicate the orientation and positional relationship based on the orientation or positional relationship shown in the accompanying drawings, are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and should not be construed as limiting the specific protection scope of this application.

[0024] It should be noted that the terms "first," "second," etc., in the specification and claims of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0025] In this application, unless otherwise expressly specified and limited, the terms "installation," "connection," "joining," and "fixing," etc., should be interpreted broadly. For example, they can refer to a connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0026] In this application, unless otherwise expressly specified and limited, "above" or "below" the second feature can include direct contact between the first and second features, or contact between the first and second features through another feature between them. Furthermore, "above," "over," and "on top" of the second feature includes the first feature being directly above or diagonally above the second feature, or simply indicates that the first feature is at a higher horizontal level than the second feature. "Below," "below," and "under" the second feature includes the first feature being directly below or diagonally below the second feature, or simply indicates that the first feature is at a lower horizontal level than the second feature.

[0027] The terms “comprising” and “having”, and any variations thereof, in the specification and claims of this application are intended to cover non-exclusive inclusion, for example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units that are explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to such process, method, product, or device.

[0028] One aspect of this application provides a method for full-domain vehicle tracking in a semi-enclosed environment, such as... Figure 1As shown, one preferred embodiment includes the following steps: S100: Construct a unified reference coordinate system and solve for the transformation parameters of the original coordinate systems of multiple LiDARs in a semi-enclosed scene relative to the reference coordinate system.

[0029] It is understandable that, during actual installation, the physical installation position of a LiDAR may deviate from its pre-set position. Since the technical solution of this application achieves full-domain tracking of vehicles in a semi-enclosed environment, multiple LiDARs may need to work collaboratively. Therefore, a unified data benchmark is required among these multiple LiDARs to ensure the accuracy of their point cloud data calculations. Consequently, the coordinate system and center point position given in the equipment installation parameters cannot be directly used as the reference coordinate system for point cloud fusion. Therefore, a reference coordinate system needs to be constructed. By calculating the transformation relationship between all the multiple LiDARs in the semi-enclosed scene and the reference coordinate system, subsequent data conversion calculations can be facilitated.

[0030] Compared to traditional methods, this application does not rely on theoretical coordinates or ideal deployment parameters during lidar installation. Instead, it constructs a unified reference coordinate system for the field area through point cloud sampling and rotation matrix solving, achieving spatial alignment of multiple lidars. Even when there are positional deviations, angular errors, or incomplete calibration information during lidar installation, reliable coordinate unification and point cloud fusion processing can still be achieved. This method reduces reliance on the accuracy of manual calibration and construction consistency, significantly improving the system's applicability in real-world engineering deployment environments. It is particularly suitable for scenarios with complex installation conditions and varying lidar deployments, such as service areas and parking lots.

[0031] S200: Determine the effective area for vehicle movement in a semi-enclosed scene, and generate the envelope region of the corresponding effective area in the original coordinate system of each lidar.

[0032] Understandably, in a semi-enclosed scene, the internal area includes both drivable areas for vehicle movement and non-drivable areas such as buildings and green belts. For full-domain vehicle tracking, only the drivable area within the semi-enclosed scene needs to be considered, i.e., the effective area. However, LiDAR point cloud data typically contains a large number of non-interesting point clouds from buildings, green belts, and areas outside the site. These point clouds not only consume significant computing resources but may also interfere with subsequent vehicle recognition and tracking processes, thereby reducing recognition accuracy. Therefore, in the technical solution of this application, the effective area of ​​the semi-enclosed scene can be determined first, and the envelope region of each LiDAR within the effective area can be generated. This allows the model training to focus only on the envelope region of the LiDAR corresponding to the effective area, thereby reducing the resource consumption of non-interesting point clouds and improving the accuracy of vehicle tracking and recognition.

[0033] Compared to traditional methods, this application constructs an envelope of the effective region and performs unified cropping of multi-radar point clouds, effectively eliminating point cloud data from irrelevant areas such as buildings, green belts, and areas outside the site. This cropping mechanism significantly reduces the size of the input point cloud without losing effective vehicle information, thereby significantly reducing the computational complexity of subsequent recognition algorithms. At the same time, reducing invalid point cloud interference helps improve the stability and accuracy of vehicle detection and classification. This method is particularly suitable for engineering applications sensitive to real-time performance and computational power.

[0034] S300: Construct training datasets for each LiDAR under different simulated operating conditions, and use the obtained training datasets to train the vehicle tracking model; wherein, the training datasets include at least a first type of sample set consisting of a single LiDAR point cloud, a second type of sample set consisting of the fusion of multiple LiDAR point clouds that are not all of them, and a third type of sample set consisting of the fusion of all LiDAR point clouds.

[0035] Understandably, in the actual use of semi-enclosed scenarios, depending on the installation location and working status of the LiDAR, there may be situations where only one LiDAR can work normally and identify the vehicle, multiple LiDARs may not all work normally and identify the vehicle, or all LiDARs may work normally and identify the vehicle. Therefore, it is necessary to construct corresponding sample sets for these three situations for model training, thereby improving the tracking accuracy of the model in actual use.

[0036] Compared to traditional methods, this application systematically constructs various data sample sets during the training phase, including single-radar, multi-radar partial fusion, and full-radar fusion, enabling the model to learn the point cloud distribution characteristics under different radar online conditions. When one or more LiDARs go offline, are obstructed, or malfunction during actual operation, the model can still achieve stable vehicle recognition based on existing training experience. This method effectively avoids the performance degradation problem caused by traditional training relying solely on "full radar online" conditions, fundamentally improving the robustness and reliability of the system under complex operating conditions.

[0037] S400: During the actual vehicle tracking process, the real-time point cloud data of each LiDAR is cropped based on the corresponding envelope region; according to the obtained transformation parameters, all cropped point cloud data are transformed to the reference coordinate system for stitching and fusion, and the resulting real-time point cloud fusion data is input into the trained vehicle tracking model for full-domain vehicle tracking.

[0038] Understandably, this application employs the same point cloud pruning, coordinate transformation, and fusion processing flow as the training data construction phase during real-time operation, ensuring a high degree of consistency in data distribution between training and actual deployment. This design effectively avoids performance degradation caused by inconsistencies between training and deployment, improving the model's generalization performance in real-world scenarios. Furthermore, the overall algorithm flow is clearly modularized, facilitating direct integration and deployment in existing multi-radar systems. This method can be seamlessly adapted to multi-radar perception systems such as service areas and parking lots, demonstrating significant engineering implementation value.

[0039] In this embodiment, there are multiple ways to select the reference coordinate system constructed in step S100. For example, the original coordinate system of one of the lidars in the semi-enclosed scene can be selected as the reference coordinate system, or the world coordinate system can be selected as the reference coordinate system. The specific selection can be made according to the actual needs of those skilled in the art. In this embodiment, the original coordinate system of one of the lidars is preferred as the reference coordinate system. After obtaining the reference coordinate system, aligning the original coordinate systems of other lidars with the reference coordinate system requires rotation and translation of the coordinates. That is, the transformation parameters between the original coordinate system and the reference coordinate system include the rotation matrix and the translation vector. For ease of understanding, the process of solving the transformation parameters between the original coordinate system and the reference coordinate system of each lidar will be described in detail below.

[0040] Specifically, when constructing the reference coordinate system, to avoid significant offsets in the overall point cloud during the fusion process, a specific LiDAR located at the center of the semi-enclosed scene can be selected as the reference LiDAR, and its original coordinate system can be used as the reference coordinate system. After obtaining the reference coordinate system, it is necessary to solve for the rotation matrix and translation vector of each LiDAR's original coordinate system relative to the reference coordinate system. Several cases exist: The first scenario involves the lidar installation locations being relatively precise, and the coordinates of the installation center points of each lidar in a unified space being obtainable; that is, the installation parameters of the lidars are known. In this case, based on the principal direction features of the point cloud of each lidar, the rotation matrix and translation vector of each lidar reaching the reference coordinate system can be directly solved using point cloud fine registration methods, such as the ICP (Iterative Closest Point) method.

[0041] The second scenario involves the lidar's installation location being inaccurate or its installation parameters being unknown. In this case, coarse registration is first performed using the overlapping area between the lidar and a reference lidar with known installation parameters. Then, a point cloud fine registration method is used to solve for its rotation matrix and translation vector relative to the reference coordinate system. It's important to note that this second scenario can be further divided into two sub-scenarios; these will be described in more detail below.

[0042] The first scenario is: the overlap area between the lidar to be registered and the adjacent lidar with known installation parameters or the reference lidar is greater than a set threshold, for example, the overlap area accounts for more than 10% of the coverage area of ​​any lidar; then a coarse registration process can be performed to estimate the coarse rotation relationship between the two lidars based on the Fast Point Feature Histogram (FPFH) method, and then the rotation matrix and translation vector of each lidar to the reference coordinate system can be solved by the point cloud fine registration method.

[0043] The second scenario is: the overlap area between the lidar to be registered and the adjacent lidar with known installation parameters or the reference lidar is less than or equal to a set threshold, for example, the overlap area is less than 10% of the coverage area of ​​any lidar. Since the overlap area is small, the coarse rotation matrix cannot be stably solved by the FPFH method. In this case, multiple common point pairs corresponding to the positions of the two lidars in the overlap area can be selected manually, and a coarse registration process of aligning the original coordinate system can be performed based on the obtained common point pairs. Then, the rotation matrix and translation vector of each lidar to the reference coordinate system can be solved by the point cloud fine registration method.

[0044] In this embodiment, when performing step S200, after obtaining the rotation matrix and translation vector corresponding to all lidars, it is necessary to determine the region of interest (ROI) of all lidars, i.e., the envelope region. This specifically includes the following process: First, using the obtained rotation matrix R and translation vector t, the point cloud data collected by each lidar is uniformly transformed into the reference coordinate system; taking the nth lidar in a semi-enclosed scene as an example, the specific transformation expression is as follows: .

[0045] In the formula, R represents the coordinates of the point cloud data of the nth lidar in the reference coordinate system. n and t n Let p represent the rotation matrix and translation vector corresponding to the nth lidar, respectively. n This represents the coordinates of the point cloud data of the nth lidar in the original coordinate system.

[0046] Then, in the reference coordinate system, point selection is performed manually, using tools such as CloudCompare, to select regions from the fused point cloud and extract the envelope point set covering the entire effective area of ​​the semi-closed scene. After obtaining the envelope point set, to ensure that subsequent processing can still be performed in the original coordinate system of each LiDAR, an inverse transformation of the selected point set is required. Specifically, for the point cloud belonging to each LiDAR in the envelope point set, the corresponding inverse rotation matrix and inverse translation vector are used to transform it back to the original coordinate system of the corresponding LiDAR; taking the nth LiDAR in the semi-closed scene as an example, the specific transformation expression is: .

[0047] In the formula, This represents the inverse rotation matrix corresponding to the nth lidar.

[0048] Finally, each lidar merges the point clouds corresponding to the envelope point set in its own original coordinate system to obtain the required envelope region, which contains the original point cloud data from all lidars and located within a unified effective area.

[0049] In this embodiment, when constructing the training dataset in step S300, the core idea of ​​this application is to generate a large-scale training sample set covering multiple radar operating states, so that the trained model can maintain stable recognition and tracking performance under different radar online or offline conditions. For the production of the training dataset, the training samples designed in this application must contain at least the following three types of data: (1) sample data from a single LiDAR, i.e., the first type of sample set; (2) sample data fused from multiple LiDARs (not all radars), i.e., the second type of sample set; (3) sample data fused from all LiDARs, i.e., the third type of sample set. These three sample types complement each other during training and are used to cover the vehicle point cloud feature distribution under different radar operating states.

[0050] It should be understood that the first type of sample set is used in the extreme case where all other radars around a certain radar are offline, enabling the model to still learn vehicle features based on a single radar point cloud. The second sample set is used when some adjacent radars are online and there is limited overlap in observations, enabling the model to adapt to vehicle point cloud features from different radar perspectives. The third type of sample set is used to provide the model with vehicle samples that have the highest point cloud density and the most complete features when all radars are in normal working condition.

[0051] It is important to know that the main difference between the three types of sample sets mentioned above lies in the different observation angle distribution, point cloud density, and spatial coverage integrity of the point cloud when the vehicle receives scans from different lidars.

[0052] For the first type of sample set, the vehicle is scanned by only a single LiDAR, and its point cloud mainly comes from a single observation direction. Furthermore, the point cloud density typically decreases gradually with increasing distance between the vehicle and the radar center point, and at long distances, the vehicle point cloud may even become extremely sparse. In practical applications, when some radars go offline, creating "radar islands," this type of sample set can provide the model with effective single-radar vehicle features, thereby reducing the problem of vehicle misidentification or decreased identification accuracy.

[0053] For the second type of sample set, the vehicle is scanned by multiple adjacent lidars from different angles, and its point cloud features are significantly different from those of the first type of sample set. The purpose of this type of sample set is to provide the model with vehicle point cloud features with multi-view characteristics under the condition that a limited number of lidars are online.

[0054] For the third type of sample set, since all LiDARs scan the vehicle simultaneously, the vehicle point cloud density is the highest and the number of sample points is the largest. Therefore, this type of sample can characterize the ideal point cloud state of the LiDAR system under complete operating conditions, and its internal features are also the richest.

[0055] It's also important to know that for both the first and third sample sets, during model training, all samples corresponding to each LiDAR need to be transformed to a unified reference coordinate system to ensure consistency in the spatial representation of the training samples. In the first sample set, a separate training data subset needs to be constructed for each LiDAR; therefore, the number of training subsets in the first sample set equals the total number of LiDARs in the semi-enclosed scene. In the third sample set, since the point cloud of each monostation LiDAR is rotated to a unified coordinate system using the previously solved rotation matrix, all points in space are merged together; therefore, the training subset of the third sample set is itself. For the second sample set, due to differences in the overlap range between different LiDARs, it may correspond to multiple LiDAR combinations. Therefore, it is necessary to comprehensively consider the spatial location, height information, and actual receptive field range of the LiDARs to determine how many different training data subsets should be constructed. There are several specific methods for constructing the second sample set; for ease of understanding, two specific examples will be provided below for detailed explanation.

[0056] Example 1: The construction of the second type of sample set includes the following process: For each lidar, calculate its projected coverage area in the reference coordinate system based on its actual collected point cloud data. Based on the overlap rate between the projected coverage areas of each lidar, determine the set of lidars that have effective spatial overlap with each lidar; generate various different lidar combinations based on the obtained set of fusionable lidars, and summarize the fused point cloud samples corresponding to each lidar combination to obtain the second type of sample set.

[0057] In this example, the overlap rate is calculated as follows: the ratio of the overlapping area of ​​the projected coverage of the two lidars to the projected coverage area of ​​one of the lidars is calculated; when the calculated ratio is greater than a preset overlap rate threshold, it is determined that there is effective spatial overlap between the two lidars. The specific value of the overlap rate threshold can be determined by those skilled in the art according to their actual needs, for example, it can be 10%. For ease of understanding, the overlap rate η between lidars i and j is... ij The calculation expression is as follows: .

[0058] In the formula, Poly i and Poly j These represent the projected coverage areas of lidar i and lidar j, respectively, with Area(·) indicating area calculation.

[0059] Example 2 addresses the issue that Example 1 only considered the overlap rate of projected coverage on a two-dimensional plane, neglecting key factors such as overlap in the height direction, effective radar detection range, and field of view. This could lead to the omission of some truly complementary radars or the misclassification of radars with poor complementarity as fusionable. Therefore, this example constructs a second type of sample set based on a multi-dimensional radar complementarity evaluation method, which will be described in detail below.

[0060] In this example, the dimensional parameters used to evaluate the complementarity of any two LiDARs include spatial coverage overlap, viewing angle complementarity coefficient, effective detection range matching degree, and installation height difference. These dimensional parameters are weighted and fused to construct a comprehensive complementarity score as the criterion for judging whether any two LiDARs are complementary. Specifically, spatial coverage overlap refers to the overlap rate of the projected coverage area in Example 1; the viewing angle complementarity coefficient refers to the angle between the observation directions of the two LiDARs relative to the scene center, used to determine whether the two LiDARs have good viewing angle complementarity, i.e., they can observe the same vehicle from different angles, reducing the impact of occlusion; the effective detection range matching degree refers to the difference in the effective detection range of the two LiDARs, used to determine whether the two LiDARs have good matching in detection range, ensuring a relatively uniform point cloud density after fusion; and the installation height difference is used to determine whether the two LiDARs have good horizontal viewing angle consistency, ensuring a more harmonious top-down projection of the fused point cloud. For ease of understanding, the following will use any two LiDARs i and j as an example to construct the comprehensive complementarity score S. ij The specific expression is as follows: .

[0061] .

[0062] .

[0063] In the formula, ω1, ω2, ω3, and ω4 represent the weight coefficients corresponding to each item; θ ij This represents the view complementarity coefficient between lidar i and lidar j; and ρ represents the observation direction vectors of lidar i and lidar j, respectively, that is, the direction vectors from the center point of lidar i to the center of the overlapping position of their envelope regions; ij r represents the effective detection range matching degree between lidar i and lidar j. i and r j These represent the effective detection radii of lidar i and lidar j, respectively. h ij H represents the difference in installation height between lidar i and lidar j. max This indicates the preset maximum installation height difference.

[0064] It's important to understand that all values ​​in the comprehensive complementarity score are normalized. Therefore, after calculating the comprehensive complementarity score, the complementarity relationship between the two LiDARs can be determined by setting a scoring threshold. The specific value of the scoring threshold can be chosen according to the actual needs of those skilled in the art; for example, a threshold of 0.6 can be set. If the calculated comprehensive complementarity score is greater than 0.6, it can be determined that there is an effective complementary relationship between the two LiDARs, and radar fusion can be performed.

[0065] It should be understood that after determining the number and combination relationships of each lidar with which it has effective spatial overlap or complementarity, different fusionable radar sets can be obtained, generating multiple radar combinations. For the obtained radar combinations, deduplication and rearrangement are required to avoid generating duplicate training datasets with equivalent spatial coverage.

[0066] For example, N lidars in a semi-enclosed scene can be labeled L1, L2, ..., L... N When the fusionable radar set of lidar L1 is [L2, L3, L4] and the fusionable radar set of lidar L2 is [L1, L3, L4], the fusionable radar set of one lidar can be deleted, while retaining the fusionable radar set of the other lidar. For example, if the fusionable radar set based on lidar L1 is retained as [L2, L3, L4], then the radar combinations that can be constructed are: {L1, L2, L3, L4}, {L1, L2, L3}, {L1, L2, L4}, {L1, L3, L4}, {L1, L2}, {L1, L3}, {L1, L4}.

[0067] It is important to know that each of the radar combinations mentioned above can construct a separate subset of training data. For each separate subset of training data, the point cloud data synchronously acquired by the corresponding lidar is uniformly transformed to the reference coordinate system through a pre-calibrated rotation matrix and translation vector, and the transformed point cloud is then stitched together.

[0068] In this embodiment, when deduplicating radar combinations generated from different fusionable radar sets, multiple radar combinations with the same overlapping range can be deduplicated based on the difference in the overlap of their projected coverage areas. Taking the aforementioned fusionable radar sets for lidar L1 as [L2, L3, L4] and lidar L2 as [L1, L3, L4] as an example, when these two fusionable radar sets generate their respective radar combinations, the overlapping area of ​​the projected coverage areas of all radar combinations generated by these two fusionable radar sets can be calculated. If the overlapping area of ​​the projected coverage areas of all radar combinations is located within the maximum overlapping area of ​​the publicly disclosed radar combinations {L1, L2, L3, L4}, these two fusionable radar sets can be considered equivalent, and therefore one of the fusionable radar sets can be deleted.

[0069] It is important to note that when deduplicating radar combinations using the above method, since it requires exhaustively listing all possible radar combinations, the number of LiDARs in the fusionable radar set is large (e.g., more than eight). Exhaustive listing of all possible radar combinations will lead to an exponential increase in the number of combinations, resulting in an excessively large training dataset and increasing the cost of data acquisition, annotation, and model training. Therefore, in this embodiment, a value function for each radar combination can be constructed to evaluate the comprehensive value of each combination. This comprehensive value is then used to generate radar combinations for the second type of sample set and to deduplicatize them. For ease of understanding, a detailed description will follow.

[0070] Specifically, when obtaining the fusionable radar set through Example 2 above, a radar complementarity graph G=(V, E) can be constructed; where node V represents each lidar, and edge E represents an effective complementarity relationship between two lidars, i.e., the calculated comprehensive complementarity score is greater than the score threshold. All possible candidate radar combinations correspond to a connected subgraph of the radar complementarity graph G. Then, the comprehensive value F of each candidate radar combination can be constructed using a value function, and a greedy algorithm or heuristic search is used to select the top m candidate radar combinations that maximize the comprehensive value F as the initial radar combination set for the second type of sample set. The expression for calculating the comprehensive value F(C) of any candidate radar combination C is: .

[0071] In the formula, α, β, and γ represent the weight coefficients of each item, This indicates the number of lidar units in candidate radar combination C. S represents the overall complementarity score of all radar pairs in candidate radar combination C. C The sum of these values, Redundancy(C), represents the combination redundancy, used to penalize combinations with highly overlapping spatial coverage.

[0072] After obtaining the initial radar combination set of the second type of sample set, it is necessary to deduplicate combinations with equivalent spatial coverage. Specifically, we can define any two radar combinations C in the initial radar combination set. a and C b Spatial coverage difference D(C) a C b The specific calculation expression is as follows: D(C a C b =1-Area(C a ∩C b ) / Area(C a ∪C b ).

[0073] When calculating the spatial coverage difference D(C) a C b When the difference is less than the set difference threshold, the two radar combinations C are determined to be... a and C b For spatially equivalent combinations, one of the radar combinations needs to be removed. Finally, the fused point cloud samples corresponding to the remaining radar combinations in the initial radar combination set are summarized to obtain the second type of sample set. The specific value of the difference threshold can be selected according to the actual needs of those skilled in the art, for example, it can be 10%.

[0074] In this embodiment, when training the vehicle tracking model using the constructed training dataset, since the point clouds of all LiDARs ultimately need to be stitched together into a unified reference coordinate system, the annotation information corresponding to the training samples also needs to be transformed into this unified reference coordinate system. In actual annotation, LiDARs typically annotate vehicles based on the original coordinate system of a single LiDAR; therefore, the original annotation information needs to undergo a unified coordinate transformation. For any i-th LiDAR, its vehicle annotation information Label in the original coordinate system... i It can be represented as: .

[0075] In the formula, (x i y i , z i ) represents the coordinates of the vehicle's center point, (w i h i, l i ) represents the width, height, and length dimensions of the vehicle's surround frame, r i This represents the rotation angle of the vehicle around the original coordinate system's z-axis.

[0076] Based on the above annotation information, the coordinates of the eight corner points of the vehicle bounding box can be calculated. T represents the corner point sequence; and the rotation matrix R from the lidar to the reference coordinate system obtained previously is used. i and translation vector t i Transform it to the reference coordinate system to obtain the coordinates of the eight corner points of the vehicle bounding box in the reference coordinate system. The specific transformation expression is as follows: .

[0077] In this embodiment, after the training dataset is constructed, when training the vehicle tracking model, the point cloud data needs to be cropped based on the envelope region obtained in the aforementioned steps. The main purpose of this cropping operation is to reduce the number of input point clouds to the model, thereby reducing computational complexity, and to improve the effectiveness and accuracy of model training by pre-filtering irrelevant point cloud data. In the aforementioned steps, polygonal or polyhedral envelope regions for filtering have been obtained in the reference coordinate system. Since all LiDAR point clouds have been uniformly transformed to the reference coordinate system, for any set of LiDAR point clouds, it is possible to determine whether a point is located within the valid area based on this envelope region. By performing the above cropping operation on the point cloud data of each sample in the sample set, the size of the LiDAR point cloud data can be significantly reduced, thereby improving the overall computational efficiency. After cropping all samples, a multi-radar fusion point cloud training dataset covering different online LiDAR states in the current semi-closed scene can be obtained. This dataset can fully reflect the point cloud distribution characteristics under different radar visibility conditions, providing a stable and reliable data foundation for subsequent vehicle recognition tasks.

[0078] In this embodiment, after training the model using the constructed training dataset and obtaining the corresponding model weight file, the vehicle detection and tracking task in step S400 can be executed through the real-time LiDAR data stream of the semi-closed scene. For the real-time collected point cloud data, in order to meet the real-time requirements of online recognition, a series of preprocessing operations need to be performed before inputting it into the model to ensure that the data format is consistent with the model input requirements.

[0079] Specifically, assuming N LiDAR base stations are deployed in a semi-closed scenario, and each base station transmits point cloud data through independent data pipelines, after multi-radar data synchronization, the point cloud data collected by each base station needs to be preprocessed. The overall preprocessing process includes three steps: point cloud trimming, coordinate transformation, and point cloud stitching. Point cloud trimming is performed based on the envelope region corresponding to each LiDAR in its original coordinate system. After trimming, the point cloud data from each radar station is transformed to a reference coordinate system using a pre-calibrated rotation matrix and translation vector. The transformed point clouds are then stitched together to generate a complete real-time LiDAR fused point cloud frame. Finally, this fused point cloud frame is input into the vehicle tracking model to achieve real-time vehicle detection and continuous tracking.

[0080] Another aspect of this application provides an electronic device, in one preferred embodiment of which includes a processor and a memory; the memory is used to store a computer program, and the processor is used to execute the computer program to implement the above-described method for full-domain vehicle tracking in a semi-enclosed scenario.

[0081] Another aspect of this application provides a computer-readable storage medium, in a preferred embodiment of which a computer program is stored on the storage medium; when the computer program is executed by a processor, it implements the method for full-domain tracking of vehicles in a semi-closed scenario described above.

[0082] The basic principles, main features, and advantages of this application have been described above. Those skilled in the art should understand that this application is not limited to the above embodiments. The embodiments and descriptions in the specification are merely the principles of this application. Various changes and modifications can be made to this application without departing from its spirit and scope, and all such changes and modifications fall within the scope of the claims. The scope of protection claimed by this application is defined by the appended claims and their equivalents.

Claims

1. A method for full-domain vehicle tracking in a semi-enclosed environment, characterized in that, Includes the following steps: S100: Construct a unified reference coordinate system and solve the transformation parameters of the original coordinate systems of multiple lidars in a semi-enclosed scene relative to the reference coordinate system. S200: Determine the effective area for vehicle driving in a semi-enclosed scene, and generate the envelope region of the corresponding effective area in the original coordinate system of each lidar. S300: Construct training datasets for each LiDAR under different simulated working conditions, and use the obtained training datasets to train the vehicle tracking model; wherein, the training datasets include at least a first type of sample set consisting of a single LiDAR point cloud, a second type of sample set consisting of the fusion of multiple LiDAR point clouds that are not all of them, and a third type of sample set consisting of the fusion of all LiDAR point clouds. S400: During the actual vehicle tracking process, the real-time point cloud data of each LiDAR is cropped based on the corresponding envelope region; according to the obtained transformation parameters, all cropped point cloud data are transformed to the reference coordinate system for stitching and fusion, and the resulting real-time point cloud fusion data is input into the trained vehicle tracking model for full-domain vehicle tracking.

2. The method for full-domain vehicle tracking in a semi-enclosed scenario as described in claim 1, characterized in that, The transformation parameters include the rotation matrix and translation vector; in step S100, the solution for the transformation parameters between the original coordinate system and the reference coordinate system of each lidar includes the following process: Select a lidar in the center of a semi-enclosed scene as the reference lidar, and use its original coordinate system as the reference coordinate system. For other lidars with known installation parameters, the rotation matrix and translation vector relative to the reference coordinate system are solved directly by the point cloud fine registration method based on the main direction characteristics of each lidar's point cloud. For other lidars with unknown installation parameters, coarse registration is first performed using the overlapping area with lidars whose installation parameters are known or reference lidars. Then, the rotation matrix and translation vector relative to the reference coordinate system are solved using the point cloud fine registration method.

3. The method for full-domain vehicle tracking in a semi-enclosed scenario as described in claim 2, characterized in that, For coarse registration of lidar with unknown installation parameters, the process includes the following steps: If the overlap area between the lidar and the reference lidar, whose installation parameters are known, is greater than a set threshold, a coarse registration process is performed to estimate the coarse rotational relationship between the two lidars using the fast point feature histogram method. If the overlap area between the lidar and the reference lidar, whose installation parameters are known, is less than or equal to a set threshold, multiple pairs of common points corresponding to the positions of the two lidars in the overlap area are manually selected, and a coarse registration process for aligning the original coordinate system is performed based on the obtained pairs of common points.

4. The method for full-domain vehicle tracking in a semi-enclosed scenario as described in claim 1, characterized in that, In step S200, the acquisition of the envelope region for each lidar includes the following process: Based on the obtained transformation parameters, the point cloud data of each lidar is transformed to the reference coordinate system and then fused. Extract the envelope point set covering the entire effective area in the semi-closed scene from the fused point cloud data; The point cloud in the envelope point set is inversely transformed back to the original coordinate system of each lidar. Then, each lidar merges the point cloud corresponding to the envelope point set in its own original coordinate system to obtain the corresponding required envelope region.

5. The method for full-domain vehicle tracking in a semi-enclosed scenario as described in any one of claims 1-4, characterized in that, In step S400, the construction of the second type of sample set includes the following process: Calculate the projected coverage area of ​​each lidar in the reference coordinate system; Based on the overlap rate between the projection coverage areas of each lidar, determine the set of lidars that can be fused with each lidar having effective spatial overlap. Based on the obtained fusionable radar set, various different radar combinations are generated. After deduplication of the radar combinations generated from different fusionable radar sets according to the overlap difference of the projection coverage, the fusion point cloud samples corresponding to the remaining radar combinations are summarized to obtain the second type of sample set.

6. The method for full-domain vehicle tracking in a semi-enclosed scenario as described in any one of claims 1-4, characterized in that, In step S300, the construction of the second type of sample set includes the following process: Obtain the dimensional parameters to determine whether any two lidars are complementary. These parameters include spatial coverage overlap, view complementarity coefficient, effective detection range matching degree, and installation height difference. The obtained dimensional parameters are weighted and fused to construct a comprehensive complementary score; Radar pairs with a comprehensive complementarity score greater than a set score threshold are fused to construct a fusionable radar set that has a complementary relationship with each lidar. Construct a radar complementarity graph based on the obtained fusionable radar set, and generate all candidate radar combinations using the connected subgraphs of the radar complementarity graph; The comprehensive value of each candidate radar combination is constructed by the value function, and the top m candidate radar combinations that maximize the comprehensive value are selected by greedy algorithm or heuristic search as the initial radar combination set of the second type of sample set. After deduplication based on spatial coverage equivalence, the fused point cloud samples corresponding to the remaining radar combinations in the initial radar combination set are summarized to obtain the second type of sample set.

7. The method for full-domain vehicle tracking in a semi-enclosed scenario as described in claim 6, characterized in that, The comprehensive complementarity score S corresponding to any two lidars i and j ij The expression is: ； In the formula, ω1, ω2, ω3, and ω4 represent the weight coefficients corresponding to each item, and η ij θ represents the overlap ratio between lidar i and lidar j. ij ρ represents the view complementarity coefficient between lidar i and lidar j. ij This indicates the effective detection range matching degree between lidar i and lidar j. h ij H represents the difference in installation height between lidar i and lidar j. max This indicates the preset maximum installation height difference.

8. The method for full-domain vehicle tracking in a semi-enclosed scenario as described in claim 6, characterized in that, The expression for the comprehensive value F(C) of any candidate radar combination C is: ； In the formula, α, β, and γ represent the weight coefficients of each item, This indicates the number of lidar units in candidate radar combination C. S represents the overall complementarity score of all radar pairs in candidate radar combination C. C The sum of these values, Redundancy(C), represents the combination redundancy, used to penalize combinations with highly overlapping spatial coverage.

9. An electronic device, characterized in that, It includes a processor and a memory; the memory is used to store a computer program, and the processor is used to execute the computer program to implement the method for full-domain tracking of vehicles in a semi-enclosed scene as described in any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, The storage medium stores a computer program; when the computer program is executed by a processor, it implements the method for full-domain tracking of vehicles in a semi-enclosed scene as described in any one of claims 1-8.

Images