Target vehicle 3D real-time tracking method and system
By determining the search area and target template in the target vehicle 3D real-time tracking method, acquiring and fusion characteristics, and generating candidate boxes using the perceptron network and the PointNet++ network, the problem of low tracking accuracy in the prior art is solved, and the problem of low tracking accuracy of target vehicle in the existing technology is realized. Efficient and accurate tracking in complex environments.
Patent Information
- Application Number
- CN202111347923.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-11-15
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2041-11-15
AI Technical Summary
In the prior art, the tracking accuracy of target vehicles is low, especially in complex environments. The performance of the 2D target tracking algorithm is greatly affected by lighting transformation and object occlusion, while the 3D target tracking algorithm is processed due to the sparseness and disorder of point cloud data. The problem is large, resulting in insufficient tracking accuracy.
A target vehicle 3D real-time tracking method is adopted to determine the search area and target template, obtain point features and local features, generate global features and similarity features, and generate guidance points based on these features, and use the perceptron network and the PointNet++ network for feature fusion. and candidate box prediction, determine the candidate box with the highest prediction score as the target vehicle.
It realizes the real-time tracking of target vehicles efficiently and accurately in complex environments, overcomes the problems of lighting interference and point cloud processing, and improves the tracking accuracy.
Smart Images

Figure CN114140497B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of vehicle tracking, and in particular to a 3D real-time tracking method and system for a target vehicle. Background Art
[0002] Vehicle target tracking plays an important role in the field of autonomous driving. Its main steps are to determine the vehicle to be tracked at a certain moment and then continuously lock the position of the target vehicle over a period of time. The current tracking methods mainly include 2D target tracking based on image data and 3D target tracking based on LIDAR.
[0003] 2D target tracking mainly relies on image data captured by the camera, and can work well in environments with good conditions such as laboratories. However, in actual application scenarios, due to its complex lighting changes and object occlusion, the performance of the 2D target tracking algorithm relying on the camera will be seriously lost. The 3D target tracking algorithm based on LiDAR can overcome the problem of lighting very well. At the same time, in a three-dimensional scene, LiDAR can effectively calculate the position relationship of the surrounding environment relative to the radar and obtain the position distance in real time. Therefore, 3D target tracking has better robustness than 2D tracking and is closer to practical applications. LiDAR mainly reflects the surrounding environment through point cloud data, and feeds back each object in the real world as points. Due to the sparsity and disorder of point clouds, point cloud processing has always been a difficult problem for 3D computer vision.
[0004] Therefore, how to improve the real-time tracking accuracy of the target vehicle has become a technical problem that technical personnel in this field need to solve urgently. Summary of the invention
[0005] The present invention provides a 3D real-time tracking method and system for a target vehicle, which are used to solve the defect of low target vehicle tracking accuracy in the prior art and achieve efficient and accurate real-time tracking of the target vehicle.
[0006] The present invention provides a target vehicle 3D real-time tracking method, comprising:
[0007] Determine the search area and target template;
[0008] Acquire point features of the search area, local features of the search area, point features of the target template, and local features of the target template;
[0009] Generate global features of the target template according to the local features of the target template;
[0010] Determining similarity features between the search area and the target template according to local features of the search area and local features of the target template;
[0011] Based on the similarity feature, the point feature of the template, the local feature of the target template and the global feature of the template are integrated into the search area to generate a guide point;
[0012] A candidate box and a prediction score of the candidate box are generated according to the guidance point, and the candidate box with the highest prediction score is determined to be the target vehicle.
[0013] According to a target vehicle 3D real-time tracking method provided by the present invention, the determining of the search area includes:
[0014] Collect the three-dimensional space point cloud data at the current moment;
[0015] Determining the target vehicle position predicted at the last moment in the three-dimensional space point cloud data;
[0016] A circular area with the target vehicle position predicted at the last moment as the center and a preset distance as the radius is used as the search area.
[0017] According to a target vehicle 3D real-time tracking method provided by the present invention, the step of acquiring the point features of the search area, the local features of the search area, the point features of the target template, and the local features of the target template includes:
[0018] Determine the point features of the search area and the point features of the target template through the three-dimensional coordinate data collected by the laser radar;
[0019] The point cloud data in the search area and the target template are passed through the PointNet++ network to obtain the local features of the search area and the local features of the target template respectively.
[0020] According to a target vehicle 3D real-time tracking method provided by the present invention, the point cloud data in the search area and the target template are respectively obtained through the PointNet++ network to obtain the local features of the search area and the local features of the target template, including:
[0021] Determine by random copying or deletion that the amount of point cloud data in the target template is the first amount of data, and the amount of point cloud data in the search area is the second amount of data;
[0022] Based on preset weights and preset local feature dimensions, the target template point cloud data of the first data volume and the search area point cloud data of the second data volume are input into the PointNet++ network to obtain local features of the search area and local features of the target template.
[0023] According to a target vehicle 3D real-time tracking method provided by the present invention, generating global features of the target template according to local features of the target template includes:
[0024] The local features of all point cloud data in the target template are passed through the Max-Pool network in the feature dimension to generate the global features of the target template.
[0025] According to a target vehicle 3D real-time tracking method provided by the present invention, the determining of similarity features between the search area and the target template includes:
[0026] Determine a similarity function;
[0027] Based on the similarity function, the local features of the search area and the local features of the target template, the cosine similarity of the search area and the target template is constructed to generate a similarity feature.
[0028] According to a target vehicle 3D real-time tracking method provided by the present invention, based on the similarity feature, the point feature of the template, the local feature of the target template and the global feature of the template are integrated into the search area to generate a guidance point, including:
[0029] Splicing the similarity feature, the point feature of the target template and the local feature of the target template to obtain a first splicing feature;
[0030] Using a perceptron network with a preset number of layers to fuse the first concatenated features to obtain a first fused feature;
[0031] Performing a Max-Pool operation on the first fusion feature in a preset dimension;
[0032] Splicing the feature after the Max-Pool operation with the global feature of the target template to obtain a second splicing feature;
[0033] Using a perceptron network with a preset number of layers to fuse the second concatenated features, to obtain a second fused feature;
[0034] The point features of the search area are concatenated with the second fused features to generate guidance points.
[0035] According to a target vehicle 3D real-time tracking method provided by the present invention, generating a candidate frame and a prediction score of the candidate frame according to the guidance point, and determining the candidate frame with the highest prediction score as the target vehicle, comprises:
[0036] Obtaining voting features, predicted target center positions and predicted scores of each of the guiding points respectively;
[0037] Based on the proposal network, determining a candidate box and a prediction score of the candidate box according to the voting characteristics, the predicted target center position and the prediction score of each of the guide points;
[0038] Determine the candidate box with the highest prediction score among the candidate boxes as the target vehicle.
[0039] According to a target vehicle 3D real-time tracking method provided by the present invention, the step of respectively obtaining the voting features, the predicted target center position and the predicted score of each of the guiding points includes:
[0040] Each of the guiding points is input into a perceptron network with a preset number of layers and shared weights to obtain the voting features, predicted target center position and predicted score of each of the guiding points.
[0041] The present invention also provides a target vehicle 3D real-time tracking system, comprising:
[0042] A 3D twin network module, used to determine a search area and a target template; obtain point features of the search area, local features of the search area, point features of the target template, and local features of the target template;
[0043] A feature fusion network module, used to generate global features of the target template according to the local features of the target template; determine similarity features between the search area and the target template according to the local features of the search area and the local features of the target template; based on the similarity features, fuse the point features of the template, the local features of the target template and the global features of the template to the search area to generate guidance points;
[0044] The voting proposal network module is used to generate a candidate box and a prediction score of the candidate box according to the guidance point, and determine that the candidate box with the highest prediction score is the target vehicle.
[0045] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of any of the above-mentioned methods for real-time 3D tracking of a target vehicle are implemented.
[0046] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of any of the above-mentioned methods for real-time 3D tracking of a target vehicle are implemented.
[0047] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the steps of any of the above-mentioned methods for real-time 3D tracking of a target vehicle are implemented.
[0048] The present invention provides a method and system for real-time 3D tracking of a target vehicle. The method determines a search area and a target template; obtains point features of the search area, local features of the search area, point features of the target template, and local features of the target template; generates global features of the target template according to the local features of the target template; determines similarity features between the search area and the target template according to the local features of the search area and the local features of the target template; based on the similarity features, integrates point features of the template, local features of the target template, and global features of the template into the search area to generate guidance points; generates candidate frames and prediction scores of the candidate frames according to the guidance points, and determines the candidate frame with the highest prediction score as the target vehicle. The method can not only overcome complex environmental interference, but also efficiently and accurately complete real-time tracking of the target vehicle. BRIEF DESCRIPTION OF THE DRAWINGS
[0049] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0050] Figure 1 is a flow chart of a target vehicle 3D real-time tracking method provided by an embodiment of the present invention;
[0051] Figure 2 It is a schematic diagram of the principle of multi-level feature fusion provided by an embodiment of the present invention;
[0052] Figure 3 It is a schematic diagram of the principle of the voting proposal process provided by an embodiment of the present invention;
[0053] Figure 4 is a schematic structural diagram of a target vehicle 3D real-time tracking system provided by an embodiment of the present invention;
[0054] Figure 5 3D real-time tracking system for a target vehicle provided by an embodiment of the present invention;
[0055] Figure 6 It is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0056] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0057] Combine the following Figure 1-Figure 6 The present invention describes a method and system for real-time 3D tracking of a target vehicle. The vehicle tracking process is to select a target vehicle to be tracked at a certain moment in a real scene, determine the position of the target, and then continuously determine the position and direction of the target within a subsequent period of time after the target position is determined. It is mainly used in unmanned tracking fields such as autonomous mobile robots and automatic driving. The sensors used include lidar sensors, and optionally, auxiliary positioning sensors can be used. In a preferred embodiment of the present invention, the lidar sensor used is LADIR. It should be noted that the present invention is also applicable to other laser sensors, such as lidar. The present invention can also add auxiliary positioning sensors, including code discs and inertial measurement units (IMU).
[0058] Figure 1 is a flow chart of a target vehicle 3D real-time tracking method provided by an embodiment of the present invention; Figure 2 It is a schematic diagram of the principle of multi-level feature fusion provided by an embodiment of the present invention; Figure 3 It is a schematic diagram of the principle of the voting proposal process provided by an embodiment of the present invention.
[0059] like Figure 1 As shown, a target vehicle 3D real-time tracking method provided by an embodiment of the present invention includes the following steps:
[0060] 101. Determine the search area and target template.
[0061] Specifically, the search area can be determined by using a sensor to collect three-dimensional space point cloud data at the current moment, and determining the target vehicle position predicted at the previous moment in the three-dimensional space point cloud data; using the target vehicle position predicted at the previous moment as the center and a circular area with a preset distance as the radius as the search area, for example, a circular area with a radius of 3 meters as the search area.
[0062] The target template can be determined by using the target vehicle selected at the first moment (earliest moment) or all previous prediction results. In this embodiment, the target vehicle selected at the earliest moment and the sum of the target box point set predicted at the previous moment are preferably used as the target template. The target template is updated at each update moment. In the current frame point cloud data collected in real time, the target vehicle to be tracked is selected, and the 3D tracking box of the target vehicle is determined as the target template.
[0063] 102. Obtain point features of the search area, local features of the search area, point features of the target template, and local features of the target template.
[0064] Specifically, before obtaining the point features of the search area, the local features of the search area, the point features of the target template, and the local features of the target template, the search area and the target template need to be standardized. When inputting the back-end local feature extraction network, the different number of points each time will reduce the efficiency of the algorithm. Therefore, in this embodiment, the number of points in the target template and the search area can be fixed to 1024 by randomly copying or deleting.
[0065] After standardization is completed, point features and local features are extracted. Point feature extraction can be three-dimensional coordinate data collected by laser radar LIDAR. The feature dimensions are three, representing the x, y, and z coordinates of this point in space. The coordinate system is established on the laser radar to determine the point features of the search area and the point features of the target template. In other words, the point features of the target template and the point features of the search area are directly derived from the three-dimensional spatial coordinate position information of the point cloud. The point features of the target template and the point features of the search area are composed of various points in the area, and the features of each point are composed of three-dimensional coordinates with the observer as the coordinate origin.
[0066] The extraction of local features can be done by passing the point cloud data in the search area and the target template through the PointNet++ network to obtain the local features of the search area and the local features of the target template respectively. Specifically, it includes unifying the point cloud data in the target template and the search area, and determining the amount of point cloud data in the target template as the first data amount N1 and the amount of point cloud data in the search area as the second data amount N2 by random copying or deletion; based on the preset weights and the preset local feature dimension d, input the first data amount of target template point cloud data and the second data amount of search area point cloud data into the PointNet++ network to obtain the local features of the search area and the local features of the target template. That is, the backbone network used is the three-layer set-abstraction (SA) layer in the PointNet++ algorithm with shared weights, with local radii of 0.3, 0.5, and 0.7 meters respectively, and the farthest point sampling algorithm (Farthest Point Sampling) is used, and the number of output key points is 128 and the dimension is 128.
[0067] 103. Generate global features of the target template based on the local features of the target template.
[0068] After obtaining the local features of the target template, the global features of the target template are obtained by applying the local features (N1*d) of all point cloud data in the target template to the Max-Pool network algorithm in the feature dimension to generate a (1*d)-dimensional global feature of the target template.
[0069] 104. Determine similarity features of the search area and the target template based on local features of the search area and local features of the target template.
[0070] Specifically, according to the local features of the search area and the local features of the target template, the similarity features of the search area and the target template are determined, and a similarity function can be determined, as shown in formula (1):
[0071]
[0072] Where T and S represent the target template and the point set in the search area respectively; t i is the i-th point in T; s j is the jth point in S; is the local feature of the template midpoint; is the local feature of the points in the search area.
[0073] Then, based on the similarity function, the local features of the search area and the local features of the target template, the cosine similarity of the search area and the target template is constructed to generate a similarity feature, that is, the output similarity feature Sim is (N1*N2*1) dimensional.
[0074] 105. Based on the similarity feature, the point features of the template, the local features of the target template and the global features of the template are fused to the search area to generate guidance points.
[0075] Specifically, based on the similarity feature, the point features of the template, the local features of the target template and the global features of the template are fused to the search area to generate guidance points, that is, to perform multi-level feature fusion. There are many fusion methods. In this embodiment, feature splicing and multi-layer perceptron fusion are used to achieve feature fusion, which may include: splicing the similarity feature, the point features of the target template and the local features of the target template to obtain a first splicing feature, and using a perceptron network with a preset number of layers (3 layers) to fuse the first splicing feature to obtain a first fused feature; performing a Max-Pool operation on the first fused feature in a preset dimension (N1 dimension); splicing the features after the Max-Pool operation with the global features of the target template to obtain a second splicing feature; using a perceptron network with a preset number of layers (3 layers) to fuse the second splicing feature to obtain a second fused feature; splicing the point features of the search area with the second fused features to generate guidance points, that is, finally generating 128 guidance points. Figure 2 As shown, it is a flowchart of a specific guidance point generation process. The generation process of 128 guidance points can be more clearly understood in conjunction with the attached figure.
[0076] 106. Generate candidate boxes and prediction scores of the candidate boxes according to the guidance points, and determine that the candidate box with the highest prediction score is the target vehicle.
[0077] Specifically, candidate boxes and prediction scores of candidate boxes are generated according to the guidance points, and the candidate box with the highest prediction score is determined as the target vehicle, including: obtaining the voting features, predicted target center position and prediction score of each guidance point through the voting module, that is, inputting each guidance point into the perceptron network with a preset number of shared weights, and obtaining the voting features, predicted target center position and prediction score of each guidance point. Among them, the voting module can be a 3-layer perceptron network with shared weights. The network training loss function is formula (2):
[0078]
[0079] Among them, G obj is the total number of guidance points on the target surface; Δx gi represents the distance from the guidance point to the prediction center (prediction); The actual distance from the guidance point to the target center (label), [g i On target] means only calculate points on the target surface.
[0080] Then, based on the proposal network, the candidate boxes and their predicted scores are determined according to the voting features, predicted target center position and predicted scores of each guiding point; that is, the voting features, predicted target center position and predicted scores of each guiding point are input into the proposal module, and the candidate boxes and their predicted scores are output. The proposal module is the SA layer and the 3-layer perceptron layer in PonintNet++, which outputs K candidate boxes and their corresponding predicted scores, and finally determines the candidate box with the highest predicted score as the target vehicle.
[0081] The proposal module includes clustering and prediction of the final target box. The clustering uses the SA layer in PointNet++ with a clustering radius of 0.3 meters. The 128 prediction centers in the voting module are clustered into 64. The target prediction is a three-layer perceptron network, which outputs the (3+1+128)-dimensional data of the voting output as 4-dimensional data (x, y, z, rot_y, score), a total of 64. Among them, the x, y, z dimensions predict the position of the target, rot_y is the direction of the target vehicle, and score is the prediction score. The highest prediction score is taken as the final prediction result. Figure 3 As shown in FIG. 1 , it is a specific principle process diagram of obtaining a 3D target frame, i.e., a target vehicle, through a voting module and a proposal module.
[0082] Since this embodiment uses an end-to-end training method before use, which is a first-order target tracking algorithm, the overall training loss can be expressed as formula (3):
[0083] L total =L reg +λ1L cls +λ2L obj +λ3L box (3)
[0084] Among them, L reg represents voting loss; L cls Indicates the classification loss of the target position in the proposal network, which is 1 within the threshold and 0 outside the threshold; L obj and L box Represents the final predicted position and target box size error loss.
[0085] The present embodiment provides a method for real-time 3D tracking of a target vehicle, which determines a search area and a target template; obtains point features of the search area, local features of the search area, point features of the target template, and local features of the target template; generates global features of the target template based on the local features of the target template; determines similarity features between the search area and the target template based on the local features of the search area and the local features of the target template; based on the similarity features, fuses the point features of the template, the local features of the target template, and the global features of the template to the search area to generate guidance points; generates candidate boxes and prediction scores of the candidate boxes based on the guidance points, and determines the candidate box with the highest prediction score as the target vehicle. The method can not only overcome complex environmental interference, but also efficiently and accurately complete real-time tracking of the target vehicle.
[0086] Based on the same general inventive concept, the present application also protects a target vehicle 3D real-time tracking system. The target vehicle 3D real-time tracking system provided by the present invention is described below. The target vehicle 3D real-time tracking system described below and the target vehicle 3D real-time tracking method described above can be referenced to each other.
[0087] Figure 4 is a schematic diagram of the structure of a target vehicle 3D real-time tracking system provided by an embodiment of the present invention, Figure 5 It is a schematic diagram of the principle of a target vehicle 3D real-time tracking system provided by an embodiment of the present invention.
[0088] like Figure 4 As shown, a target vehicle 3D real-time tracking system provided by an embodiment of the present invention includes:
[0089] The 3D twin network module 41 is used to determine the search area and the target template; obtain the point features of the search area, the local features of the search area, the point features of the target template, and the local features of the target template;
[0090] The feature fusion network module 42 is used to generate the global features of the target template according to the local features of the target template; determine the similarity features of the search area and the target template according to the local features of the search area and the local features of the target template; based on the similarity features, fuse the point features of the template, the local features of the target template and the global features of the template to the search area to generate guidance points;
[0091] The voting proposal network module 43 is used to generate a candidate box and a prediction score of the candidate box according to the guidance point, and determine that the candidate box with the highest prediction score is the target vehicle.
[0092] like Figure 5As shown, a, b and c in this embodiment correspond to the 3D twin network module 41, the feature fusion network module 42 and the voting proposal network module 43 respectively. Through the principle diagram, the whole process of tracking the target vehicle can be more clearly understood, which is the process of selecting the target vehicle to be tracked in the current frame point cloud data collected in the real-time scene, determining the 3D tracking frame of the target vehicle, and then obtaining the final 3D frame through the action of the three network modules.
[0093] The present invention provides a target vehicle 3D real-time tracking system, which determines a search area and a target template; obtains point features of the search area, local features of the search area, point features of the target template and local features of the target template; generates global features of the target template according to the local features of the target template; determines similarity features between the search area and the target template according to the local features of the search area and the local features of the target template; based on the similarity features, integrates the point features of the template, the local features of the target template and the global features of the template into the search area to generate guidance points; generates candidate frames and prediction scores of the candidate frames according to the guidance points, and determines the candidate frame with the highest prediction score as the target vehicle. The system can not only overcome complex environmental interference, but also efficiently and accurately complete real-time tracking of the target vehicle.
[0094] Furthermore, the 3D twin network module 41 in this embodiment is specifically used for:
[0095] Collect the three-dimensional space point cloud data at the current moment;
[0096] Determining the target vehicle position predicted at the last moment in the three-dimensional space point cloud data;
[0097] A circular area with the target vehicle position predicted at the last moment as the center and a preset distance as the radius is used as the search area.
[0098] Furthermore, the 3D twin network module 41 in this embodiment is also specifically used for:
[0099] Determine the point features of the search area and the point features of the target template through the three-dimensional coordinate data collected by the laser radar;
[0100] The point cloud data in the search area and the target template are passed through the PointNet++ network to obtain the local features of the search area and the local features of the target template respectively.
[0101] Furthermore, the 3D twin network module 41 in this embodiment is also specifically used for:
[0102] Determine by random copying or deletion that the amount of point cloud data in the target template is the first amount of data, and the amount of point cloud data in the search area is the second amount of data;
[0103] Based on preset weights and preset local feature dimensions, the target template point cloud data of the first data volume and the search area point cloud data of the second data volume are input into the PointNet++ network to obtain local features of the search area and local features of the target template.
[0104] Furthermore, the feature fusion network module 42 in this embodiment is specifically used for:
[0105] The local features of all point cloud data in the target template are passed through the Max-Pool network in the feature dimension to generate the global features of the target template.
[0106] Furthermore, the feature fusion network module 42 in this embodiment is also specifically used for:
[0107] Determine a similarity function;
[0108] Based on the similarity function, the local features of the search area and the local features of the target template, the cosine similarity of the search area and the target template is constructed to generate a similarity feature.
[0109] Furthermore, the feature fusion network module 42 in this embodiment is also specifically used for:
[0110] Splicing the similarity feature, the point feature of the target template and the local feature of the target template to obtain a first splicing feature;
[0111] Using a perceptron network with a preset number of layers to fuse the first concatenated features to obtain a first fused feature;
[0112] Performing a Max-Pool operation on the first fusion feature in a preset dimension;
[0113] Splicing the feature after the Max-Pool operation with the global feature of the target template to obtain a second splicing feature;
[0114] Using a perceptron network with a preset number of layers to fuse the second concatenated features, to obtain a second fused feature;
[0115] The point features of the search area are concatenated with the second fused features to generate guidance points.
[0116] Furthermore, the voting proposal network module 43 in this embodiment is specifically used for:
[0117] Obtaining voting features, predicted target center positions and predicted scores of each of the guiding points respectively;
[0118] Based on the proposal network, determining a candidate box and a prediction score of the candidate box according to the voting characteristics, the predicted target center position and the prediction score of each of the guide points;
[0119] Determine the candidate box with the highest prediction score among the candidate boxes as the target vehicle.
[0120] Furthermore, the voting proposal network module 43 in this embodiment is also specifically used for:
[0121] Each of the guiding points is input into a perceptron network with a preset number of layers and shared weights to obtain the voting features, predicted target center position and predicted score of each of the guiding points.
[0122] Figure 6 It is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention.
[0123] like Figure 6 As shown, the electronic device may include: a processor 610, a communication interface 620, a memory 630 and a communication bus 640, wherein the processor 610, the communication interface 620 and the memory 630 communicate with each other through the communication bus 640. The processor 610 may call the logic instructions in the memory 630 to execute the target vehicle 3D real-time tracking method, which includes: determining the search area and the target template; obtaining the point features of the search area, the local features of the search area, the point features of the target template and the local features of the target template; generating the global features of the target template according to the local features of the target template; determining the similarity features of the search area and the target template according to the local features of the search area and the local features of the target template; based on the similarity features, fusing the point features of the template, the local features of the target template and the global features of the template to the search area to generate a guide point; generating a candidate box and a prediction score of the candidate box according to the guide point, and determining the candidate box with the highest prediction score as the target vehicle.
[0124] In addition, the logic instructions in the above-mentioned memory 630 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.
[0125] On the other hand, the present invention also provides a computer program product, which includes a computer program, which can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the target vehicle 3D real-time tracking method provided by the above methods, and the method includes: determining a search area and a target template; obtaining point features of the search area, local features of the search area, point features of the target template, and local features of the target template; generating global features of the target template based on the local features of the target template; determining similarity features between the search area and the target template based on the local features of the search area and the local features of the target template; based on the similarity features, fusing the point features of the template, the local features of the target template, and the global features of the template to the search area to generate guidance points; generating candidate boxes and prediction scores of the candidate boxes based on the guidance points, and determining that the candidate box with the highest prediction score is the target vehicle.
[0126] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the target vehicle 3D real-time tracking method provided by the above-mentioned methods, the method comprising: determining a search area and a target template; obtaining point features of the search area, local features of the search area, point features of the target template, and local features of the target template; generating global features of the target template based on the local features of the target template; determining similarity features between the search area and the target template based on the local features of the search area and the local features of the target template; based on the similarity features, fusing the point features of the template, the local features of the target template, and the global features of the template to the search area to generate guidance points; generating candidate boxes and prediction scores of the candidate boxes based on the guidance points, and determining the candidate box with the highest prediction score as the target vehicle.
[0127] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.
[0128] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.
[0129] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A target vehicle 3D real-time tracking method, characterized in that: include: Determine the search area and target template; Acquire point features of the search area and point features of the target template, and pass point cloud data in the search area and the target template through a PointNet++ network to obtain local features of the search area and local features of the target template respectively; The local features of all point cloud data in the target template are passed through the Max-Pool network in the feature dimension to generate the global features of the target template; Determining similarity features between the search area and the target template according to local features of the search area and local features of the target template; Splicing the similarity feature, the point feature of the target template and the local feature of the target template to obtain a first splicing feature; fusing the first splicing feature using a perceptron network with a preset number of layers to obtain a first fused feature; Performing a Max-Pool operation on the first fused feature in a preset dimension; splicing the feature after the Max-Pool operation with the global feature of the target template to obtain a second spliced feature; fusing the second spliced feature using a perceptron network with a preset number of layers to obtain a second fused feature; splicing the point feature of the search area with the second fused feature to generate a guide point; The voting features, predicted target center position and predicted score of each of the guiding points are obtained respectively; based on the proposal network, the candidate boxes and the predicted scores of the candidate boxes are determined according to the voting features, predicted target center position and predicted scores of each of the guiding points; the candidate box with the highest predicted score is determined as the target vehicle.
2. The target vehicle 3D real-time tracking method according to claim 1, characterized in that: The determining of the search area comprises: Collect the three-dimensional space point cloud data at the current moment; Determining the target vehicle position predicted at the last moment in the three-dimensional space point cloud data; A circular area with the target vehicle position predicted at the last moment as the center and a preset distance as the radius is used as the search area.
3. The target vehicle 3D real-time tracking method according to claim 1, characterized in that: The acquiring the point features of the search area and the point features of the target template includes: The point features of the search area and the point features of the target template are determined through the three-dimensional coordinate data collected by the laser radar.
4. The target vehicle 3D real-time tracking method according to claim 1, characterized in that: The step of passing the point cloud data in the search area and the target template through a PointNet++ network to obtain local features of the search area and local features of the target template respectively includes: Determine by random copying or deletion that the amount of point cloud data in the target template is the first amount of data, and the amount of point cloud data in the search area is the second amount of data; Based on preset weights and preset local feature dimensions, the target template point cloud data of the first data volume and the search area point cloud data of the second data volume are input into the PointNet++ network to obtain local features of the search area and local features of the target template.
5. The target vehicle 3D real-time tracking method according to claim 1, characterized in that: The determining of the similarity feature between the search area and the target template includes: Determine a similarity function; Based on the similarity function, the local features of the search area and the local features of the target template, the cosine similarity of the search area and the target template is constructed to generate a similarity feature.
6. The target vehicle 3D real-time tracking method according to claim 1, characterized in that: The step of respectively obtaining the voting features, the predicted target center position and the predicted score of each of the guiding points includes: Each of the guiding points is input into a perceptron network with a preset number of layers and shared weights to obtain the voting features, predicted target center position and predicted score of each of the guiding points.
7. A target vehicle 3D real-time tracking system, characterized in that: include: 3D twin network module, used to determine the search area and target template; Acquire point features of the search area and point features of the target template; Passing the point cloud data in the search area and the target template through the PointNet++ network to obtain the local features of the search area and the local features of the target template respectively; A feature fusion network module is used to generate global features of the target template by passing the local features of all point cloud data in the target template through the Max-Pool network in the feature dimension; determine the similarity features of the search area and the target template according to the local features of the search area and the local features of the target template; splice the similarity features, the point features of the target template and the local features of the target template to obtain a first spliced feature; and fuse the first spliced feature using a perceptron network with a preset number of layers to obtain a first fused feature; Performing a Max-Pool operation on the first fused feature in a preset dimension; splicing the feature after the Max-Pool operation with the global feature of the target template to obtain a second spliced feature; fusing the second spliced feature using a perceptron network with a preset number of layers to obtain a second fused feature; splicing the point feature of the search area with the second fused feature to generate a guide point; The voting proposal network module is used to respectively obtain the voting characteristics, predicted target center position and predicted score of each of the guiding points; based on the proposal network, determine the candidate box and the predicted score of the candidate box according to the voting characteristics, predicted target center position and predicted score of each of the guiding points; determine the candidate box with the highest prediction score as the target vehicle.
