A target tracking method, system, medium, and device

Abstract

The application discloses a target tracking method, system, medium and equipment, and belongs to the technical field of automatic driving. The method comprises the following steps: acquiring original point cloud data around a vehicle, and converting to the same coordinate system to obtain preprocessed point cloud data; performing target detection on the preprocessed point cloud data, determining the position of a candidate target, then extracting target point cloud data; performing multi-view depth image projection on the target point cloud data to obtain a depth pseudo image; classifying the depth pseudo image to obtain the target category, confidence and feature vector of the corresponding candidate target; and performing target tracking on the candidate target. The point cloud data acquired by the laser radar is processed, the point cloud data of a target region is extracted and projected to obtain a corresponding depth pseudo image, and the detector is used to detect the depth pseudo image, so that the processing capacity and efficiency of the point cloud data are improved.

Description

Technical Field

[0001] This application relates to the field of autonomous driving technology, and in particular to a target tracking method, system, medium, and device. Background Technology

[0002] Autonomous driving technology has developed rapidly in recent years, especially in scenarios with limited conditions, such as highways and ports, which poses many challenges to the reliability of autonomous driving technology. To ensure the redundancy and robustness of the perception capabilities of autonomous driving systems, multiple sensors are typically involved, mainly including cameras, LiDAR, millimeter-wave radar, and ultrasonic radar. Multi-target tracking is a core algorithm in autonomous driving, and stable and accurate detection of the actual positions of other vehicles in three-dimensional space is crucial. Due to its inherent characteristics, LiDAR has the ability to provide accurate spatial position information, making it highly suitable for multi-target tracking.

[0003] Online multi-object tracking algorithms can be broadly categorized into two types: traditional methods and methods incorporating deep learning. Traditional methods primarily involve single-frame detection and multi-frame tracking. Deep learning-integrated methods fall into two main paradigms: Tracking by Detection, which replaces the single-frame detection module in traditional methods with a deep learning detection method, and Joint Detection & Tracking, which encodes historical frame information into the detection of the next frame. Traditional methods lack semantic understanding and cannot effectively generate candidate boxes in complex scenes. In deep learning-integrated tracking methods, most detectors are developed with a visual detection background and directly applied to the processing of LiDAR point cloud data, resulting in poor performance and low efficiency in processing LiDAR point clouds. Summary of the Invention

[0004] This application proposes a target tracking method, system, medium, and device to solve the problem of poor point cloud processing effect and low processing efficiency when identifying and tracking targets.

[0005] One technical solution of this application provides a target tracking method, comprising: acquiring raw point cloud data around a vehicle using multiple lidar sensors and transforming it to the same coordinate system to obtain preprocessed point cloud data; performing target detection on the preprocessed point cloud data to determine the positions of candidate targets in the preprocessed point cloud data; extracting target point cloud data corresponding to the candidate targets from the preprocessed point cloud data based on the positions of the candidate targets; performing multi-view depth image projection on the target point cloud data to obtain a depth pseudo-image; classifying the depth pseudo-image to obtain the target category, confidence level, and feature vector of the corresponding candidate targets; and updating the tracking target trajectory or creating a new tracking trajectory based on the target category, confidence level, and feature vector of the candidate targets to perform target tracking.

[0006] Optionally, the original point cloud data around the vehicle is acquired and transformed to the same coordinate system to obtain preprocessed point cloud data, including: performing extrinsic parameter transformation on the acquired original point cloud data to transform the acquired original point cloud data to the same coordinate system; and stitching the original point cloud data in the same coordinate system to obtain preprocessed point cloud data.

[0007] Optionally, target detection is performed on the preprocessed point cloud data to determine the location of candidate targets in the preprocessed point cloud data, including: classifying the preprocessed point cloud data according to the distance between the point cloud points and the autonomous vehicle to obtain near-range point cloud data and long-range point cloud data; performing target detection on the near-range point cloud data to obtain the first candidate target and its corresponding location; and performing target clustering on the long-range point cloud data to obtain the second candidate target and its corresponding location.

[0008] Optionally, the preprocessed point cloud data is classified according to the distance between the point cloud points and the autonomous vehicle to obtain near-range point cloud data and long-range point cloud data, including: identifying preprocessed point cloud data with a distance less than a first preset distance threshold as near-range point cloud data; and identifying preprocessed point cloud data with a distance greater than a second preset distance threshold as long-range point cloud data, wherein the second preset distance threshold is less than the first preset distance threshold.

[0009] Optionally, target detection is performed on the near-range point cloud data to obtain a first candidate target and its corresponding location, including: projecting the near-range point cloud data to obtain a pseudo image; and detecting the pseudo image to obtain the first candidate target and its corresponding location.

[0010] Optionally, before tracking a target, the method further includes: fusing the first and second candidate targets in the overlapping parts to obtain a fused target; and updating the tracking target trajectory or creating a new tracking trajectory for the first candidate target, the second candidate target, and the fused target in the non-overlapping parts using a tracker to track the target trajectory or create a new tracking trajectory.

[0011] Optionally, the first candidate target and the second candidate target in the overlapping part are fused to obtain a fused target, including: calculating the cross-union ratio of the depth pseudo-images corresponding to the first candidate target and the second candidate target in the overlapping part; if the cross-union ratio is greater than a preset cross-union ratio threshold, the target with higher confidence among the first candidate target and the second candidate target is taken as the real target, and the target is fused to obtain a fused target.

[0012] One technical solution of this application provides a target tracking system, comprising: a point cloud acquisition and preprocessing module, which acquires raw point cloud data around a vehicle and transforms it to the same coordinate system to obtain preprocessed point cloud data; a target detection module, which performs target detection on the preprocessed point cloud data, determines the position of candidate targets in the preprocessed point cloud data, and extracts the target point cloud data corresponding to the candidate targets from the preprocessed point cloud data based on the position of the candidate targets; an image projection module, which performs multi-view depth image projection on the target point cloud data to obtain a depth pseudo-image; a target classification module, which classifies the depth pseudo-image to obtain the target category, confidence level, and feature vector of the corresponding candidate targets; and a tracker, which updates the tracking target trajectory or creates a new tracking trajectory based on the target category, confidence level, and feature vector of the candidate targets to perform target tracking.

[0013] In one technical solution of this application, a computer-readable storage medium is provided, which stores computer instructions, wherein the computer instructions are operated to perform the target tracking method in Solution 1.

[0014] In one technical solution of this application, a computer device is provided, which includes a processor and a memory, wherein the memory stores computer instructions, and the processor operates the computer instructions to execute the target tracking method in Solution 1.

[0015] The beneficial effects of this application are as follows: This application improves the processing effect of point cloud data by converting point cloud data acquired by multiple sensing devices to the same coordinate system for processing, extracts point cloud data of the target area and projects it to obtain the corresponding depth pseudo-image, improves the detection effect of point cloud data and improves the detection accuracy by visualizing the point cloud data and then detecting the depth pseudo-image, thereby accelerating the processing capability and efficiency of point cloud data, so that the target tracking process has both the accuracy characteristics of point cloud data and the high efficiency characteristics of image detection. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of one implementation of the target tracking method of this application;

[0017] Figure 2 This is a schematic diagram of one implementation of the target tracking system of this application;

[0018] Figure 3 This is a schematic diagram of an example of the target tracking system of this application. Detailed Implementation

[0019] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0020] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented, for example, in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a product or device comprising a series of steps or units is not necessarily limited to those units explicitly listed, but may include other units not explicitly listed or inherent to such products or devices.

[0021] The rapid development of autonomous driving technology in recent years, especially in scenarios with limited conditions such as highways and ports, has brought many challenges to the reliability of autonomous driving systems. To ensure the redundancy and robustness of the perception capabilities of autonomous driving systems, multiple sensors are typically involved, mainly including cameras, LiDAR, millimeter-wave radar, and ultrasonic radar. Multi-target tracking is a core algorithm in autonomous driving, and the stable and accurate detection of the actual positions of other vehicles in three-dimensional space is crucial. Due to its inherent characteristics, LiDAR has the ability to provide accurate spatial position information, making it highly suitable for multi-target tracking.

[0022] Online multi-object tracking algorithms can be broadly categorized into two types: traditional methods and methods combined with deep learning. Traditional methods mainly consist of two steps: single-frame detection and multi-frame tracking, with the SORT method as a representative example. The main algorithms for single-frame detection are segmentation or clustering algorithms, while multi-frame tracking primarily includes three parts: generating trajectories from single-frame detection, updating existing trajectories using motion models (such as the Kalman filter algorithm), and matching existing trajectories with the single-frame detection results (such as the Hungarian algorithm). The deep learning-integrated approach falls into two main paradigms: Tracking by Detection, which replaces the single-frame detection module in traditional methods with deep learning detection methods; and Joint Detection & Tracking, which encodes historical frame information into the detection of the next frame. Traditional methods lack semantic understanding capabilities and cannot effectively provide candidate boxes in complex scenes; Joint Detection & Tracking requires additional computing power to process historical frame information, making it difficult to apply in online systems with limited computing power; Tracking by Detection is a relatively mature paradigm, but most detectors are developed in a vision-oriented context, and their direct application to LiDAR results in poor performance or low efficiency.

[0023] To address the aforementioned problems, this application proposes a target tracking method, system, medium, and device. The method includes: acquiring multiple sets of raw point cloud data around a vehicle and transforming them to the same coordinate system to obtain preprocessed point cloud data; performing target detection on the preprocessed point cloud data to determine the positions of candidate targets within the preprocessed point cloud data; extracting target point cloud data corresponding to the candidate targets from the preprocessed point cloud data based on their positions; projecting multi-view depth images onto the target point cloud data to obtain a depth pseudo-image; classifying the depth pseudo-image to obtain the target category, confidence level, and feature vector of the corresponding candidate targets; and updating the tracking target trajectory or creating a new tracking trajectory based on the target category, confidence level, and feature vector of the candidate targets to perform target tracking.

[0024] This application improves the processing effect of point cloud data by converting point cloud data acquired by multiple sensing devices to the same coordinate system, extracting point cloud data of the target area and projecting it to obtain the corresponding depth pseudo-image. By visualizing the point cloud data and then detecting the depth pseudo-image, the detection effect of point cloud data is improved, and the detection accuracy is increased. This accelerates the processing capability and efficiency of point cloud data, so that the target tracking process maintains both the accuracy of point cloud data and the high efficiency of image detection.

[0025] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0026] Figure 1 An embodiment of the target tracking method of this application is shown.

[0027] exist Figure 1 In the embodiment shown, the target tracking method of this application includes process S101, which involves acquiring raw point cloud data around the vehicle and converting it to the same coordinate system to obtain preprocessed point cloud data.

[0028] In this implementation, when tracking a target based on point cloud data, raw point cloud data of the area surrounding the vehicle is first acquired using multiple acquisition devices mounted on the vehicle, such as multiple LiDARs. Then, the acquired raw point cloud data of the current frame is preprocessed. Since the point cloud data acquired by different LiDARs are all centered on their respective LiDARs, coordinate transformation is required to convert them to the same coordinate system, ultimately obtaining preprocessed point cloud data for subsequent processing.

[0029] Optionally, acquire raw point cloud data around the vehicle and transform it to the same coordinate system to obtain preprocessed point cloud data. This includes: performing extrinsic parameter transformation on the acquired raw point cloud data to transform it to the same coordinate system; and stitching the raw point cloud data in the same coordinate system to obtain preprocessed point cloud data.

[0030] In this optional embodiment, different acquisition devices have their own corresponding device parameters. For example, different lidars correspond to different external parameters, such as installation height and orientation angle. For the raw point cloud data acquired by different lidars, the external parameters corresponding to the lidar point clouds are transformed to convert the point cloud data acquired by multiple lidars to the same coordinate system. Within the same coordinate system, there will be overlap in the point cloud data acquired by different lidars. Therefore, the point cloud data under the same coordinate system is stitched together, and related operations such as filtering and removing point cloud noise are also included to finally obtain preprocessed point cloud data.

[0031] exist Figure 1 In the embodiment shown, the target tracking method of this application includes process S102, which performs target detection on preprocessed point cloud data to determine the position of candidate targets in the preprocessed point cloud data.

[0032] In this embodiment, target detection is performed on preprocessed point cloud data in the same coordinate system. The location of candidate targets is determined in the preprocessed point cloud data by using the density and other characteristics of the point cloud data.

[0033] Optionally, target detection is performed on the preprocessed point cloud data to determine the location of candidate targets in the preprocessed point cloud data, including: classifying the preprocessed point cloud data according to the distance between the point cloud and the autonomous vehicle to obtain near-range point cloud data and long-range point cloud data; performing target detection on the near-range point cloud data to obtain the first candidate target and its corresponding location; and performing target clustering on the long-range point cloud data to obtain the second candidate target and its corresponding location.

[0034] In this optional embodiment, due to the inherent properties of LiDAR, nearby objects correspond to a larger number of point cloud points, while distant objects correspond to fewer point cloud points. Therefore, there are a large number of dense point cloud points nearby, while distant objects have relatively sparse point cloud points. To enhance target detection performance and reduce the time spent on target detection, the preprocessed point cloud data is divided according to its distance from the autonomous vehicle, resulting in near-range point cloud data (closer to the vehicle) and long-range point cloud data (farther from the vehicle). Then, a target detection method, such as using a BEV detector and the BEV (Bird's Eye View) detection method, is used to detect targets in the near-range point cloud data, obtaining the corresponding first candidate target and its location. A target clustering method, such as using a target clustering algorithm on the long-range point cloud data, is then used to cluster targets, determining the second candidate target and its location.

[0035] Optionally, the preprocessed point cloud data is classified according to the distance between the point cloud points and the autonomous vehicle to obtain near-range point cloud data and long-range point cloud data, including: identifying preprocessed point cloud data with a distance less than a first preset distance threshold as near-range point cloud data; and identifying preprocessed point cloud data with a distance greater than a second preset distance threshold as long-range point cloud data, wherein the second preset distance threshold is less than the first preset distance threshold.

[0036] In this optional embodiment, the preprocessed point cloud data is classified according to a first preset distance threshold and a second preset distance threshold. Preprocessed point cloud data with a distance less than the first preset distance threshold is identified as near-range point cloud data; preprocessed point cloud data with a distance greater than the second preset distance threshold is identified as far-range point cloud data. To avoid omissions or incomplete classification when dividing near-range and far-range point cloud data, the first preset distance threshold is greater than the second preset distance threshold.

[0037] Specifically, the first preset distance threshold can be 100 meters, and the second preset distance threshold can be 80 meters. It should be noted that the first and second preset distance thresholds can be reasonably set according to the model of the LiDAR, the requirements for point cloud data processing, etc. The above are preferred examples and do not limit the scope of protection of this application.

[0038] Optionally, target detection is performed on the near-range point cloud data to obtain a first candidate target and its corresponding location, including: projecting the near-range point cloud data to obtain a pseudo image; and detecting the pseudo image to obtain the first candidate target and its corresponding location.

[0039] In this optional embodiment, for example, when performing target detection on a near-range point cloud using a BEV detector, the near-range point cloud data is first projected using a BEV target detection method to obtain a pseudo-image. Then, the obtained pseudo-image is detected using a BEV detector to obtain the first candidate target in the near-range point cloud and its corresponding position.

[0040] exist Figure 1 In the embodiment shown, the target tracking method of this application includes process S103, which extracts the target point cloud data corresponding to the candidate target from the preprocessed point cloud data according to the position of the candidate target.

[0041] In this implementation, after determining the location of the candidate target, the target point cloud data corresponding to the candidate target is extracted from the preprocessed point cloud data for subsequent processing. By extracting the target point cloud data and then processing it, the amount of point cloud data to be processed is reduced, and the processing speed is accelerated.

[0042] exist Figure 1 In the embodiment shown, the target tracking method of this application includes process S104, which involves multi-view depth image projection of the target point cloud data to obtain a depth pseudo-image.

[0043] In this embodiment, the segmented target point cloud data is subjected to multi-view RangeImage projection to convert the 3D point cloud data into an image of pixel values, thereby obtaining a depth pseudo-image corresponding to each candidate target. The process of performing multi-view RangeImage projection is a common processing procedure and will not be described in detail here.

[0044] exist Figure 1 In the embodiment shown, the target tracking method of this application includes process S105, which classifies the depth pseudo-image to obtain the target category, confidence level and feature vector of the corresponding candidate target.

[0045] In this embodiment, the depth pseudo-images corresponding to each candidate target are input into a multi-task classification network to predict the type and confidence level of the target corresponding to each depth pseudo-image, and output the corresponding results.

[0046] Specifically, the target categories can be set to seven: cars, trucks, buses, trailers, motorcycles, pedestrians, and background noise. The multi-task classification neural network processes each depth pseudo-image, outputting a vector with 7 + 64 = 71 dimensions. The first seven dimensions correspond to the confidence scores of the neural network's predictions for the seven categories, with each prediction ranging from 0 to 1, and the sum of these seven values ​​being 1. The category with the highest confidence score among these seven values ​​is taken as the actual category corresponding to the current depth pseudo-image. The remaining 64-dimensional vector output by the neural network represents the target's feature vector. This feature vector expresses the neural network's encoding of the vehicle's position and shape information. The same vehicle operating at different times will produce similar feature vectors; while different vehicles will produce significantly different feature vectors. Therefore, the similarity of these 64-dimensional feature vectors can be used to determine whether they represent the same target, thus enabling target tracking. The specific method for calculating the similarity of feature vectors can be to calculate the mean squared error of two 64-dimensional features. When the mean squared error is less than a first preset threshold 'a', the two objects are marked as the same object; when the mean squared error is greater than a second preset threshold 'b', the two objects are marked as different objects; when the mean squared error is between the first preset threshold 'a' and the second preset threshold 'b', the two objects are marked as undetermined. Preferably, the first preset threshold 'a' can be 0.2 and the second preset threshold 'b' can be 0.9. When two objects are marked as undetermined, similarity information is not used; only the object in the current frame is compared with the predicted position of the historical trajectory for Hungarian matching; that is, it degenerates into a classic tracking algorithm. It should be noted that the above-mentioned first preset threshold 'a' and second preset threshold 'b' can be reasonably valued based on judgment experience and can be reasonably adjusted according to actual judgment requirements. The above values ​​are only preferred choices and do not specifically limit the scope of protection of this application.

[0047] exist Figure 1 In the embodiment shown, the target tracking method of this application includes process S106, which updates the tracking target trajectory or creates a new tracking trajectory based on the target category, confidence level and feature vector of the candidate target, and performs target tracking.

[0048] In this embodiment, the tracker can process the target category, confidence level, and feature vector of the depth pseudo-image obtained after the classification network processing. Based on the similarity between the feature vector corresponding to the current frame and the feature vector corresponding to the previous frame, it can be determined whether the two belong to the same target. If they do, the tracking target trajectory is updated; if not, a new tracking trajectory is created to complete the target tracking.

[0049] Specifically, when determining whether two targets are the same target, the mean square error of the two 64-dimensional vectors is calculated for the 64-dimensional feature vector of the target in the current frame and the 64-dimensional feature vector of the target in the previous frame. If the mean square error is less than a first preset threshold, then the two are the same target; if the mean square error is greater than a second preset threshold, then the two are different targets.

[0050] Optionally, the target category, confidence level, and feature vector of the candidate target are updated to track the target trajectory or a new tracking trajectory is created. Before target tracking, the following steps are also included: fusing the first candidate target and the second candidate target in the overlapping part to obtain the fused target; and updating the target trajectory or creating a new tracking trajectory for the first candidate target, the second candidate target, and the fused target in the non-overlapping part through the tracker to perform target tracking.

[0051] In this optional embodiment, the above process involves classifying the preprocessed point cloud to obtain near-range point cloud data and long-range point cloud data. Subsequent target detection and classification network processing then yields targets in the near-range point cloud data and the long-range point cloud data. Since there is overlap between the near-range and long-range point cloud data, the overlapping targets need to be fused before proceeding with the subsequent target tracking process.

[0052] Optionally, the first candidate target and the second candidate target in the overlapping part are fused to obtain a fused target, including: calculating the cross-union ratio of the depth pseudo-images corresponding to the first candidate target and the second candidate target in the overlapping part; if the cross-union ratio is greater than a preset cross-union ratio threshold, the target with higher confidence among the first candidate target and the second candidate target is taken as the real target, and the target is fused to obtain a fused target.

[0053] In this optional embodiment, it is first determined whether two targets can be fused. The intersection-union ratio (IUR) of the first candidate target and the second candidate target is calculated. If the IUR is greater than a preset IUR threshold, they are considered the same target, and subsequent target fusion is performed. During target fusion, the confidence levels of the two targets are compared, and the target category of the target with the higher confidence level is taken as the fused target category. Then, fusion is performed to obtain the fused target.

[0054] Specifically, let A represent the first candidate target and B represent the second candidate target. The crossover ratio (IoU) of the first candidate target is denoted as S_A, and the crossover ratio of the second candidate target is denoted as S_B. Then, the crossover ratio IoU(AB) = S_AB / (S_A + S_B). The IoU threshold can be set to 0.3. That is, when IoU(AB) is greater than 0.3, target A and target B are considered the same target. In this case, the confidence levels of target A and target B are compared, and the target category with the higher confidence level is selected as the fused target category. If IoU(AB) is not greater than 0.3, target A and target B are considered different objects, and no fusion process is performed. It should be noted that the IoU threshold can be reasonably set according to the actual judgment requirements. The above is only a preferred example and does not limit the scope of protection of this application.

[0055] The target tracking method of this application improves the processing effect of point cloud data by converting point cloud data acquired by multiple sensing devices to the same coordinate system, extracting point cloud data of the target area and projecting it to obtain the corresponding depth pseudo-image, and then detecting the depth pseudo-image by image-izing the point cloud data, thereby improving the detection effect and accuracy of the point cloud data, thus accelerating the processing capability and efficiency of point cloud data, so that the target tracking process maintains both the accuracy of point cloud data and the high efficiency of image detection.

[0056] Figure 2 An embodiment of the target tracking system of this application is shown.

[0057] exist Figure 2 In the embodiment shown, the target tracking system of this application includes: a point cloud acquisition and preprocessing module 201, which acquires the original point cloud data around the vehicle and transforms it to the same coordinate system to obtain preprocessed point cloud data; a target detection module 202, which performs target detection on the preprocessed point cloud data, determines the position of candidate targets in the preprocessed point cloud data, and extracts the target point cloud data corresponding to the candidate targets from the preprocessed point cloud data based on the position of the candidate targets; an image projection module 203, which performs multi-view depth image projection on the target point cloud data to obtain a depth pseudo-image; a target classification module 204, which classifies the depth pseudo-image to obtain the target category, confidence level, and feature vector of the corresponding candidate targets; and a tracker 205, which updates the tracking target trajectory or creates a new tracking trajectory based on the target category, confidence level, and feature vector of the candidate targets to perform target tracking.

[0058] Optionally, in the point cloud acquisition and preprocessing module 201, the acquired raw point cloud data is converted to an external parameter to transform the acquired raw point cloud data to the same coordinate system; and the acquired raw point cloud data in the same coordinate system are stitched together to obtain preprocessed point cloud data.

[0059] Optionally, in the target detection module 202, the preprocessed point cloud data is classified according to the distance between the point cloud points and the autonomous vehicle to obtain near-range point cloud data and long-range point cloud data; target detection is performed on the near-range point cloud data to obtain the first candidate target and its corresponding location; and target clustering is performed on the long-range point cloud data to obtain the second candidate target and its corresponding location.

[0060] Optionally, in the target detection module 202, preprocessed point cloud data with a distance less than a first preset distance threshold is identified as near-range point cloud data; and preprocessed point cloud data with a distance greater than a second preset distance threshold is identified as far-range point cloud data, wherein the second preset distance threshold is less than the first preset distance threshold.

[0061] Optionally, in the target detection module 202, the near-range point cloud data is projected to obtain a pseudo image; the pseudo image is detected to obtain the first candidate target and its corresponding position.

[0062] Optionally, it also includes a target fusion module, which fuses the first candidate target and the second candidate target in the overlapping part to obtain the fused target; and updates the tracking target trajectory or creates a new tracking trajectory for the first candidate target, the second candidate target and the fused target in the non-overlapping part to perform target tracking.

[0063] Optionally, in the target fusion module, the cross-union ratio (CUI) of the depth pseudo-images corresponding to the first candidate target and the second candidate target in the overlapping part of the calculation is calculated. If the CUI is greater than the preset CUI threshold, the target with higher confidence among the first candidate target and the second candidate target is taken as the real target, and target fusion is performed to obtain the fused target.

[0064] The target tracking system of this application improves the processing effect of point cloud data by converting point cloud data acquired by multiple sensing devices into the same coordinate system, extracting point cloud data of the target area and projecting it to obtain the corresponding depth pseudo-image. By visualizing the point cloud data and then detecting the depth pseudo-image, the detection effect of point cloud data is improved, and the detection accuracy is increased. This accelerates the processing capability and efficiency of point cloud data, so that the target tracking process maintains both the accuracy of point cloud data and the high efficiency of image detection.

[0065] Figure 3 A schematic diagram of an example of the target tracking system of this application is shown.

[0066] exist Figure 3In the example shown, the target tracking system of this application includes a memory and a processor. The processor includes: a point cloud acquisition and preprocessing module, a target detection module, an image projection module, a target classification module, and a tracker. The acquired point cloud data is processed by each module in the processor, and intermediate and final data are stored in the memory, working together to complete the entire processing. The specific working principles of each processing module in the processor are the same as those in the target tracking system and method described above, and will not be repeated here.

[0067] In one specific embodiment of this application, a computer-readable storage medium stores computer instructions, wherein the computer instructions are operated to perform the target tracking method described in any embodiment. The storage medium may be located directly in hardware, in a software module executed by a processor, or in a combination of both.

[0068] Software modules may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disks, removable disks, CD-ROMs, or any other form of storage medium known in this art. An exemplary storage medium is coupled to the processor, enabling the processor to read information from and write information to the storage medium.

[0069] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic, discrete hardware components, or any combination thereof. A general-purpose processor can be a microprocessor, but alternatively, it can be any conventional processor, controller, microcontroller, or state machine. The processor can also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors incorporating a DSP core, or any other such configuration. Alternatively, the storage medium can be integrated with the processor. The processor and storage medium can reside in an ASIC. The ASIC can reside in the user terminal. Alternatively, the processor and storage medium can reside as discrete components in the user terminal.

[0070] In one specific embodiment of this application, a computer device includes a processor and a memory, the memory storing computer instructions, wherein the processor operates the computer instructions to perform the target tracking method described in any embodiment.

[0071] In the embodiments provided in this application, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0072] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0073] The above are merely embodiments of this application and do not limit the scope of this patent application. Any equivalent structural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of this application.

Claims

1. A target tracking method, characterized in that, include: Acquire the raw point cloud data around the vehicle and transform it to the same coordinate system to obtain preprocessed point cloud data; Target detection is performed on the preprocessed point cloud data to determine the location of candidate targets in the preprocessed point cloud data, including classifying the preprocessed point cloud data according to the distance between the point cloud and the vehicle to obtain near-range point cloud data and long-range point cloud data; Target detection is performed on the near-field point cloud data to obtain the first candidate target and its corresponding location; Target clustering is performed on the remote point cloud data to obtain the second candidate target and its corresponding location; Based on the location of the candidate target, extract the target point cloud data corresponding to the candidate target from the preprocessed point cloud data; Multi-view depth image projection is performed on the target point cloud data to obtain a depth pseudo-image; The depth pseudo-image is classified to obtain the target category, confidence level, and feature vector of the corresponding candidate target; The first candidate target and the second candidate target in the overlapping area are fused to obtain the fused target; For the first candidate target, the second candidate target, and the fused target in the non-overlapping parts of their positions, update the target trajectory or create a new tracking trajectory to perform target tracking; The target tracking trajectory is updated or a new tracking trajectory is created based on the target category, confidence level, and feature vector of the candidate target to perform target tracking.

2. The target tracking method according to claim 1, characterized in that, The process of acquiring raw point cloud data around the vehicle and transforming it to the same coordinate system to obtain preprocessed point cloud data includes: The acquired raw point cloud data is transformed by extrinsic parameter conversion to convert the acquired raw point cloud data to the same coordinate system; The original point cloud data is then stitched together within the same coordinate system to obtain the preprocessed point cloud data.

3. The target tracking method according to claim 1, characterized in that, The preprocessed point cloud data is classified according to the distance between the point cloud and the vehicle to obtain near-range point cloud data and long-range point cloud data, including: The preprocessed point cloud data whose distance is less than a first preset distance threshold is determined as the near-range point cloud data; The preprocessed point cloud data whose distance is greater than a second preset distance threshold is determined as the remote point cloud data, wherein the second preset distance threshold is less than the first preset distance threshold.

4. The target tracking method according to claim 1, characterized in that, The step of performing target detection on the near-range point cloud data to obtain a first candidate target and its corresponding location includes: Projecting the near-field point cloud data yields a pseudo-image; The pseudo-image is detected to obtain the first candidate target and its corresponding location.

5. The target tracking method according to claim 1, characterized in that, The process of fusing the first candidate target and the second candidate target in the overlapping area to obtain the fused target includes: Calculate the intersection-union ratio of the depth pseudo-images corresponding to the first candidate target and the second candidate target in the overlapping part of the location; If the cross-union ratio is greater than the preset cross-union ratio threshold, then the target with higher confidence among the first candidate target and the second candidate target is used as the real target, and target fusion is performed to obtain the fused target.

6. A target tracking system, characterized in that, include: The point cloud acquisition and preprocessing module acquires the raw point cloud data around the vehicle and transforms it to the same coordinate system to obtain preprocessed point cloud data. The target detection module performs target detection on the preprocessed point cloud data, determines the location of candidate targets in the preprocessed point cloud data, and extracts target point cloud data corresponding to the candidate targets from the preprocessed point cloud data based on the location of the candidate targets. Specifically, the step of performing target detection on the preprocessed point cloud data to determine the location of candidate targets includes classifying the preprocessed point cloud data according to the distance between the point cloud and the vehicle to obtain near-range point cloud data and long-range point cloud data; performing target detection on the near-range point cloud data to obtain a first candidate target and its corresponding location; and performing target clustering on the long-range point cloud data to obtain a second candidate target and its corresponding location. The image projection module performs multi-view depth image projection on the target point cloud data to obtain a depth pseudo-image. The target classification module classifies the depth pseudo-image to obtain the target category, confidence level, and feature vector of the corresponding candidate target; The fusion module fuses the first candidate target and the second candidate target in the overlapping area to obtain a fused target; The target tracking module updates or creates new tracking trajectories for the first candidate target, the second candidate target, and the fused target in non-overlapping locations, thereby performing target tracking. The tracker updates the target trajectory or creates a new tracking trajectory based on the target category, confidence level, and feature vector of the candidate target to perform target tracking.

7. A computer-readable storage medium storing computer instructions, wherein the computer instructions are operated to perform the target tracking method according to any one of claims 1-5.

8. A computer device comprising a processor and a memory, the memory storing computer instructions, wherein: The processor operates the computer instructions to execute the target tracking method according to any one of claims 1-5.

Images