Vehicle tracking method, device, equipment and medium based on BEV characteristics

Abstract

The present application provides a vehicle tracking method, apparatus, device, and medium based on BEV features. The method comprises: determining target point cloud data based on an acquisition device, and determining a pillar feature map corresponding to the target point cloud data based on a pointpillars network; determining a target trajectory corresponding to all vehicles in the pillar feature map based on a target detection algorithm; determining a target detection frame and a target feature region corresponding to the target detection frame in the pillar feature map, and determining, based on the target feature region, trajectory features corresponding to the target trajectory and detection frame features corresponding to the target detection frame; determining a target feature matching value based on the detection frame features, all trajectory features, and a preset weight; and determining a target vehicle in the target detection frame based on the target feature matching value and a preset matching threshold. In this way, the flexibility and accuracy of vehicle tracking in complex environments are improved.

Description

Technical Field

[0001] The present application relates to the field of vehicle tracking technology, and in particular to a vehicle tracking method, device, equipment and medium based on BEV characteristics. Background Art

[0002] The port environment is a highly complex and dynamic scene, which contains vehicles of various shapes and sizes, such as container trucks, tractors, trailers, forklifts, etc. In the port autonomous driving system, vehicle tracking is a crucial link, which provides the necessary dynamic scene understanding for obstacle avoidance, path planning and decision-making.

[0003] Existing vehicle tracking methods generally use the output bounding box information and adopt a two-stage framework of detection and tracking. First, an object detector is used to obtain the target bounding box in each frame, and then various association algorithms are used to associate the detection results between different frames to form the target's motion trajectory.

[0004] However, existing vehicle tracking methods suffer from low accuracy and poor adaptability. Summary of the Invention

[0005] The present application provides a vehicle tracking method, apparatus, device and medium based on BEV features, which are used to solve the problems of low accuracy and poor adaptability of existing vehicle tracking methods.

[0006] In a first aspect, the present application provides a vehicle tracking method based on BEV features, the method comprising:

[0007] Determine the target point cloud data based on the acquisition device, and determine the pillar feature map corresponding to the target point cloud data based on the pointpillars network;

[0008] According to the target detection algorithm, the target trajectories corresponding to all vehicles in the column feature map are determined;

[0009] Determine the target detection frame and the target feature area corresponding to the target detection frame in the cylindrical feature map, and determine the trajectory features corresponding to the target trajectory and the detection frame features corresponding to the target detection frame based on the target feature area;

[0010] Determine the target feature matching value based on the detection frame features, all trajectory features and preset weights;

[0011] The target vehicle in the target detection frame is determined based on the target feature matching value and the preset matching threshold.

[0012] In some embodiments of the present application, target point cloud data is determined based on an acquisition device, and a pillar feature map corresponding to the target point cloud data is determined based on a pointpillars network, including:

[0013] The target point cloud data is divided into cylinders of preset cylinder sizes, and the data in the cylinders are feature encoded according to the pointpillars network to obtain the corresponding cylinder data features;

[0014] The column feature map is determined according to the spatial position of the column corresponding to the column data feature.

[0015] In some embodiments of the present application, determining a cylinder feature map based on the spatial position of the cylinder corresponding to the cylinder data feature includes:

[0016] Determine the preset convolutional neural network;

[0017] According to the preset convolutional neural network and the spatial position of the cylinder, a high-resolution cylinder feature map, a medium-resolution cylinder feature map, and a low-resolution cylinder feature map are determined;

[0018] A column feature map is determined according to the high-resolution column feature map, the medium-resolution column feature map, and the low-resolution column feature map.

[0019] In some embodiments of the present application, determining the target trajectories corresponding to all vehicles in the column feature map according to the target detection algorithm includes:

[0020] Determine the convolution area of ​​a preset size in the cylindrical feature map;

[0021] According to the target detection algorithm, the vehicles in the convolution area and the target trajectories corresponding to the vehicles are determined.

[0022] In some embodiments of the present application, determining a target detection frame and a target feature area corresponding to the target detection frame in a cylindrical feature map, and determining, based on the target feature area, a trajectory feature corresponding to the target trajectory and a detection frame feature corresponding to the target detection frame, includes:

[0023] Determine a high-resolution feature map, a medium-resolution feature map, and a low-resolution feature map in the column feature map, and their corresponding feature areas;

[0024] Perform feature map fusion on the feature area corresponding to the high-resolution feature map, the feature area corresponding to the medium-resolution feature map, and the feature area corresponding to the low-resolution feature map to obtain the target feature area;

[0025] According to the target feature area, the trajectory features corresponding to the target trajectory and the detection frame features corresponding to the target detection frame are determined.

[0026] In some embodiments of the present application, determining a target feature matching value based on the detection frame features, all trajectory features, and preset weights includes:

[0027] Determine the cosine similarity, intersection-over-union (IoU) and distance metrics between the detection box features and each trajectory feature, as well as preset weights corresponding to the cosine similarity, the IoU and the distance metrics;

[0028] According to the preset weights, the cosine similarity, intersection-over-union ratio and distance metrics are weighted and combined to obtain the target feature matching value.

[0029] In some embodiments of the present application, determining a target vehicle in a target detection frame according to a target feature matching value and a preset matching threshold includes:

[0030] Compare the target feature matching value with the preset matching threshold to obtain the comparison result;

[0031] If the comparison result shows that the target feature matching value is greater than the preset matching threshold, the vehicle in the target detection frame is determined to be the target vehicle corresponding to the target trajectory;

[0032] If the comparison result is that the target feature matching value is not greater than the preset matching threshold, the vehicle in the target detection frame is determined to be an unmatched vehicle.

[0033] In a second aspect, the present application provides a vehicle tracking device based on BEV features, the device comprising:

[0034] The feature map determination module is used to determine the target point cloud data according to the acquisition device, and determine the pillar feature map corresponding to the target point cloud data according to the pointpillars network;

[0035] A trajectory determination module is used to determine the target trajectories corresponding to all vehicles in the column feature map based on the target detection algorithm;

[0036] A feature determination module is used to determine the target detection frame and the target feature area corresponding to the target detection frame in the cylindrical feature map, and determine the trajectory features corresponding to the target trajectory and the detection frame features corresponding to the target detection frame based on the target feature area;

[0037] A matching value determination module is used to determine the target feature matching value based on the detection frame features, all trajectory features and preset weights;

[0038] The vehicle determination module is used to determine the target vehicle in the target detection frame based on the target feature matching value and the preset matching threshold.

[0039] In a third aspect, the present application provides a computer device, comprising: a processor, and a memory communicatively connected to the processor;

[0040] Memory stores computer-executable instructions;

[0041] The processor executes the computer-executable instructions stored in the memory to implement the method of the present application.

[0042] In a fourth aspect, the present application provides a computer-readable storage medium, in which program code is stored. When the program code is executed by a processor, it is used to implement the method of the present application.

[0043] The vehicle tracking method, apparatus, device and medium based on BEV features provided in the present application determine target point cloud data according to an acquisition device, and determine a pillar feature map corresponding to the target point cloud data according to a pointpillars network; determine the target trajectory corresponding to all vehicles in the pillar feature map according to a target detection algorithm; determine a target detection frame and a target feature area corresponding to the target detection frame in the pillar feature map, and determine the trajectory features corresponding to the target trajectory and the detection frame features corresponding to the target detection frame according to the target feature area; determine the target feature matching value according to the detection frame features, all trajectory features and preset weights; and determine the target vehicle in the target detection frame according to the target feature matching value and the preset matching threshold.

[0044] In this way, by determining the bounding box information output by the target detection module and introducing the BEV feature map into the tracking process in a tightly coupled manner, especially in the target matching and filter observation stages, accurate and stable tracking of various vehicles in the port environment can be achieved. Unlike the traditional detection and tracking separation architecture, the detection module is no longer merely used as a front-end input provider for the tracking module, but deep information sharing and fusion between the detection and tracking modules is achieved, that is, the internal intermediate feature representation of the detection module (BEV feature map) is directly involved in the core calculation process of the tracking module, forming a close collaboration at the "feature level" rather than just a simple concatenation at the "result level", which improves the flexibility and accuracy of vehicle tracking in complex environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0046] Figure 1 A flow chart of a vehicle tracking method based on BEV features provided in an embodiment of the present application;

[0047] Figure 2 A system architecture diagram of a vehicle tracking method based on BEV features provided in an embodiment of the present application;

[0048] Figure 3 A schematic diagram of a vehicle tracking method based on BEV features provided in an embodiment of the present application;

[0049] Figure 4A schematic structural diagram of a vehicle tracking device based on BEV features provided in an embodiment of the present application;

[0050] Figure 5 4 is a structural block diagram of a device for executing a vehicle tracking method based on BEV features according to an embodiment of the present application. DETAILED DESCRIPTION

[0051] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.

[0052] The following specific embodiments describe in detail the technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0053] Figure 1 The flowchart of a vehicle tracking method based on BEV features provided in an embodiment of the present application is as follows. Figure 1 As shown, the vehicle tracking method based on BEV features may include the following steps:

[0054] S110 , determining target point cloud data according to the acquisition device, and determining a pillar feature map corresponding to the target point cloud data according to the pointpillars network.

[0055] Among them, the acquisition device can be a vehicle-mounted lidar as the main sensor, and a camera can be optionally used for auxiliary perception. Point cloud data is a discrete data set used to represent objects or scenes in three-dimensional space. It consists of a large number of three-dimensional coordinate points (x, y, z). Each point usually also contains additional information such as color, reflection intensity, and normal direction. The geometric and attribute characteristics of the object surface or space are recorded through dense sampling.

[0056] The pointpillars network is a deep learning network for three-dimensional object detection. It can be used to process point cloud data with high efficiency and accuracy.

[0057] The pillar feature map is a key feature representation in the pointpillars network, which can convert point cloud data into a format suitable for two-dimensional convolution processing, so that the subsequent two-dimensional convolution backbone network can effectively extract high-level features for target detection and three-dimensional bounding box regression.

[0058] Based on this, the point cloud data is determined by the vehicle-mounted lidar, and the three-dimensional point cloud data is converted into a column feature map that can be processed in two dimensions according to the pointpillars network, so that the column feature map can be used to determine whether the vehicle to be detected in the detection frame is the target vehicle to be tracked.

[0059] S120: Determine target trajectories corresponding to all vehicles in the column feature map according to a target detection algorithm.

[0060] Among them, the target detection algorithm is an algorithm for detecting vehicles, which can be a detection head algorithm. In the pointpillars network, the detection head is the key component responsible for predicting the position, size, direction and category of three-dimensional objects from the feature map. It can convert the high-level features extracted by the feature encoder and backbone network into the final detection results.

[0061] The vehicles are all the vehicles included in the point cloud data, and the target trajectory is the driving trajectory corresponding to each vehicle.

[0062] Based on this, the target detection algorithm is used to determine the driving trajectories of all vehicles in the collected point cloud data in the cylindrical feature map, so that the detection frame trajectory corresponding to the vehicle to be detected can be determined based on the vehicle trajectory to determine whether it matches the trajectory of any vehicle, thereby determining whether the vehicle to be detected is the target vehicle.

[0063] S130 , determining the target detection frame and the target feature area corresponding to the target detection frame in the cylinder feature map, and determining the trajectory features corresponding to the target trajectory and the detection frame features corresponding to the target detection frame based on the target feature area.

[0064] Among them, the target detection box is the detection box corresponding to the target detection algorithm, which is used to determine the current vehicle to be detected.

[0065] The target feature area is the area including the target detection frame and the target trajectory; the trajectory feature is the characteristic expression of the vehicle's target trajectory in the target feature area, and the detection frame feature is the characteristic expression of the target detection frame in the target feature area. For example, the system is tracking a truck in the port, and the target trajectory is the trajectory of the truck in the past 10 frames. The trajectory features may include position, speed, size and BEV features. A bounding box is detected in the new frame, namely the target detection frame, which includes the target vehicle to be detected, which may be the same truck (or offset due to occlusion). The detection frame feature is the feature corresponding to the target vehicle.

[0066] Furthermore, for each detection box di and existing trajectory tj, a region of interest (ROI) is defined on the BEV feature map according to its position and size, and the ROI Align or ROI Pooling method is used to extract feature patches from the multi-scale feature maps F1, F2 and F3, and the feature patches of different scales are fused to obtain the feature representation of each target, which is recorded as fi (detection box feature) and fj (trajectory feature).

[0067] Based on this, by determining the target feature area in the cylindrical feature map, and determining the corresponding features of the target detection frame and the target vehicle trajectory in the target feature area, feature matching can be performed subsequently to determine whether the vehicle to be detected in the detection frame is the vehicle corresponding to the target trajectory.

[0068] S140: Determine a target feature matching value based on the detection frame features, all trajectory features, and preset weights.

[0069] The preset weight is a pre-set weight value used to weight the target feature matching value; the target feature matching value is used to indicate whether the detection frame feature and the trajectory feature match.

[0070] Based on this, by performing feature processing on the detection frame features and trajectory features, for example, the cosine similarity, intersection-over-union ratio, and distance metric of the detection frame features and the trajectory features can be determined, thereby determining the target feature matching value, so as to further determine whether the vehicle to be detected in the detection frame is the vehicle corresponding to the target trajectory.

[0071] S150: Determine the target vehicle in the target detection frame according to the target feature matching value and the preset matching threshold.

[0072] The preset matching threshold is a pre-set matching threshold used to determine whether the target feature matching value meets the matching requirements determined by the user.

[0073] Based on this, by determining the preset matching threshold, it is determined whether the vehicle to be detected is the vehicle corresponding to the target trajectory according to the target feature matching value and the preset matching threshold, thereby improving the tracking accuracy and achieving real-time tracking.

[0074] Based on the feasible implementation of the above S110, the present application further provides a method for determining target point cloud data according to an acquisition device, and determining a pillar feature map corresponding to the target point cloud data according to a pointpillars network, including the following steps:

[0075] The target point cloud data is divided into cylinders of preset cylinder sizes, and the data in the cylinders are feature encoded according to the pointpillars network to obtain the corresponding cylinder data features;

[0076] The column feature map is determined according to the spatial position of the column corresponding to the column data feature.

[0077] The preset cylinder size is the size of a pre-set data cylinder, thereby dividing the point cloud data. For example, the point cloud data is divided into several vertical cylinders, and the bottom surface size of each cylinder is 0.16m×0.16m.

[0078] Feature encoding refers to the process of converting raw data (such as image pixels, text characters, sensor measurements, etc.) into an abstract feature representation that is more suitable for algorithm processing. The goal is to extract key information from the data and remove redundancy or noise. In the pointpillars network, feature encoding of the points in each pillar is a key step. This step converts the raw point cloud data into a more expressive feature representation. The feature encoding of the points in each pillar includes features such as the three-dimensional coordinates of the points, density, and the offset of the points relative to the center of the pillar.

[0079] The spatial position is the spatial position arrangement of the column features. By processing the point features in each column, a column-level feature representation is generated, and the column features are arranged according to their spatial positions to form a BEV feature map with dimensions of C×H×W, where C is the number of feature channels, and H and W are the height and width of the BEV feature map, respectively.

[0080] Based on this, by determining the data cylinder of a preset size corresponding to the point cloud data and encoding the features of the data cylinder, the cylinder data features are obtained, and the cylinder feature map is determined according to the spatial position arrangement order of the cylinder.

[0081] Based on the feasible implementation of the above S110, the present application further provides a method for determining a column feature map according to the spatial position of the column corresponding to the column data feature, including the steps of:

[0082] Determine the preset convolutional neural network;

[0083] According to the preset convolutional neural network and the spatial position of the cylinder, a high-resolution cylinder feature map, a medium-resolution cylinder feature map, and a low-resolution cylinder feature map are determined;

[0084] A column feature map is determined according to the high-resolution column feature map, the medium-resolution column feature map, and the low-resolution column feature map.

[0085] The preset convolutional neural network is a pre-set convolutional neural network, for example, it can be a 2D convolutional neural network.

[0086] Based on this, three feature maps of different scales are extracted from the BEV backbone network through convolutional neural networks, which are respectively recorded as F1 (high resolution, low semantics), F2 (medium resolution, medium semantics) and F3 (low resolution, high semantics) feature maps, so that these feature maps can be directly introduced into the tracking process in the future, realizing the tight coupling of detection and tracking modules.

[0087] Based on the feasible implementation of the above S120, the present application further provides a method for determining target trajectories corresponding to all vehicles in the column feature map according to a target detection algorithm, including the following steps:

[0088] Determine the convolution area of ​​a preset size in the cylindrical feature map;

[0089] According to the target detection algorithm, the vehicles in the convolution area and the target trajectories corresponding to the vehicles are determined.

[0090] The preset-sized convolutional region can be understood as applying a preset-sized convolutional layer on the multi-scale BEV feature map. Furthermore, a 3×3 convolutional layer is applied on the multi-scale BEV feature map to generate classification and regression branches. The classification branch predicts the probability that each location belongs to different categories, including cars, trucks, trailers, and other common vehicle types in ports. The regression branch predicts the target geometric attributes of each location, including center coordinates, size, and orientation angle.

[0091] Based on this, by determining the convolution area in the cylindrical feature map, the vehicles in the convolution layer and their corresponding trajectories are determined to obtain the target trajectory.

[0092] Based on the feasible implementation of the above S130, the present application further provides a method for determining a target detection frame and a target feature area corresponding to the target detection frame in the cylindrical feature map, and determining, based on the target feature area, a trajectory feature corresponding to the target trajectory and a detection frame feature corresponding to the target detection frame, including the following steps:

[0093] Determine a high-resolution feature map, a medium-resolution feature map, and a low-resolution feature map in the column feature map, and their corresponding feature areas;

[0094] Perform feature map fusion on the feature area corresponding to the high-resolution feature map, the feature area corresponding to the medium-resolution feature map, and the feature area corresponding to the low-resolution feature map to obtain the target feature area;

[0095] According to the target feature area, the trajectory features corresponding to the target trajectory and the detection frame features corresponding to the target detection frame are determined.

[0096] Among them, the high-resolution feature map, medium-resolution feature map and low-resolution feature map are feature maps of three different scales extracted from the BEV backbone network through convolutional neural networks; the corresponding feature areas are the feature areas corresponding to the feature maps of three different scales, thereby introducing the multi-scale feature map in the detection network BEV backbone network into the target tracking process to enhance the accuracy of target matching and state estimation.

[0097] Based on this, feature blocks of different scales are fused to determine the detection frame features corresponding to the detection frame and the trajectory features corresponding to the target trajectory in the cylindrical feature map, thereby improving the accuracy of feature determination.

[0098] Based on the feasible implementation of the above S140, the present application further provides a method for determining a target feature matching value based on the detection frame features, all trajectory features, and preset weights, including the following steps:

[0099] Determine the cosine similarity, intersection-over-union (IoU) and distance metrics between the detection box features and each trajectory feature, as well as preset weights corresponding to the cosine similarity, the IoU and the distance metrics;

[0100] According to the preset weights, the cosine similarity, intersection-over-union ratio and distance metrics are weighted and combined to obtain the target feature matching value.

[0101] Among them, the detection box feature can be represented by fi, the trajectory feature can be represented by fj, and the cosine similarity can be represented by Indicates that the intersection and union ratio can be used Indicates that the distance metric can be used Represents that the cosine similarity , intersection and union ratio , distance metric .

[0102] The weight corresponding to the cosine similarity can be used Indicates that the weight corresponding to the intersection ratio can be used Indicates that the weight corresponding to the distance metric can be used Indicates that the target feature matching value can be used Indicates that ,The preset weights w1, w2 and w3 are weight coefficients, which can be determined through experimental tuning.

[0103] Based on this, the target feature matching value is obtained by weightedly combining the cosine similarity, intersection-over-union ratio and distance metrics between the detection box features and the trajectory features.

[0104] Based on the feasible implementation of the above S150, the present application further provides a method for determining a target vehicle in a target detection frame according to a target feature matching value and a preset matching threshold, including the following steps:

[0105] Compare the target feature matching value with the preset matching threshold to obtain the comparison result;

[0106] If the comparison result shows that the target feature matching value is greater than the preset matching threshold, the vehicle in the target detection frame is determined to be the target vehicle corresponding to the target trajectory;

[0107] If the comparison result is that the target feature matching value is not greater than the preset matching threshold, the vehicle in the target detection frame is determined to be an unmatched vehicle.

[0108] Based on this, by comparing the target feature matching value with the preset matching threshold, it is determined whether the current target feature matching value exceeds the matching threshold, so as to determine whether the detection frame feature and the trajectory feature match, that is, whether the vehicle corresponding to the target detection frame is the vehicle corresponding to the target trajectory, thereby realizing vehicle identification and tracking.

[0109] Please refer to Figure 2 , Figure 2 A system architecture diagram of a vehicle tracking method based on BEV features provided in an embodiment of the present application; Figure 2 As shown in the figure, the overall architecture includes the data flow relationship between modules such as sensor data acquisition, BEV feature extraction, target detection, feature enhancement matching, feature-assisted filtering and trajectory management; in particular, the figure emphasizes how BEV feature information is tightly coupled from the feature extraction module to the feature enhancement matching and feature-assisted filtering modules, forming a feature-level deep fusion architecture for detection and tracking.

[0110] Among them, the BEV feature information is tightly coupled to the observation stage of the filter, forming a feature-state collaborative estimation mechanism to improve the accuracy of state estimation.

[0111] Please refer to Figure 3 , Figure 3 A schematic diagram of a vehicle tracking method based on BEV features provided in an embodiment of the present application; Figure 3 The figure shows how to generate and retain feature maps (F1, F2, F3) at different scales. These feature maps are not only used for detection but also directly participate in the subsequent tracking process, forming the basis for a tightly coupled design.

[0112] In some embodiments of the present application, the multi-scale feature map in the detection network BEV backbone network is introduced into the target tracking process in a tightly coupled manner, point cloud data in a complex environment is collected, and the point cloud data is divided into data pillars according to the pointpillars network, and feature encoding is performed to generate a pillar feature map corresponding to the point cloud data. At the same time, feature maps of different scales are generated through a 2D convolutional neural network, and feature information is retained at multiple scales and tightly coupled with the tracking module; the feature representation of the target is extracted from the BEV feature map through the ROI Align or ROI Pooling method, and the feature similarity is calculated for data association, thereby realizing the tight coupling of the feature and matching process, and the Hungarian algorithm can be used for optimal matching based on the comprehensive matching score of feature similarity, IoU and position distance to realize the deep fusion of feature information and geometric information; in addition, feature similarity information is introduced in the observation stage of the Kalman filter, and the observation noise covariance matrix is ​​dynamically adjusted to realize a tightly coupled adaptive mechanism for feature and state estimation.

[0113] By tightly coupling BEV feature information from the detection network into the matching and filtering process of vehicle tracking, the loosely coupled serial model of traditional detection and tracking is broken, and a tracking framework with deep feature-level fusion is constructed. This tightly coupled design enables the tracking module to directly access and utilize the rich intermediate representations of the detection process, rather than relying solely on the final detection results. A target matching algorithm based on BEV feature similarity is developed, which closely combines feature similarity with traditional position and size information to improve matching accuracy. This is particularly true in occluded and dense scenes. Feature similarity is not only considered in terms of the geometric properties of the target, but also fully utilizes the semantic information contained in deep features, significantly improving matching performance in difficult scenarios. Feature similarity is introduced into the observation model of the Kalman filter, and by dynamically adjusting the observation noise, an adaptive assessment of observation reliability is achieved, improving the accuracy of state estimation. This tight coupling mechanism between features and filters enables the system to intelligently cope with various complex scenarios, especially when the target is partially or completely occluded. In the event of temporary occlusion or detection failure, previously saved BEV feature information is used to assist in tracking recovery, reducing the problems of ID switching and target loss. Through the temporal transmission and update mechanism of features, the system can accurately identify and associate targets after long-term occlusion and maintain tracking continuity.

[0114] Figure 4 Schematic diagram of a vehicle tracking device 400 based on BEV features provided in an embodiment of the present application. Figure 4 As shown, the vehicle tracking device 400 based on BEV features includes: a feature map determination module 410, a trajectory determination module 420, a feature determination module 430, a matching value determination module 440, and a vehicle determination module 450; wherein:

[0115] The feature map determination module 410 is used to determine the target point cloud data according to the acquisition device, and determine the pillar feature map corresponding to the target point cloud data according to the pointpillars network;

[0116] A trajectory determination module 420 is used to determine the target trajectories corresponding to all vehicles in the pillar feature map according to the target detection algorithm;

[0117] A feature determination module 430 is configured to determine a target detection frame and a target feature region corresponding to the target detection frame in the cylindrical feature map, and determine, based on the target feature region, a trajectory feature corresponding to the target trajectory and a detection frame feature corresponding to the target detection frame;

[0118] A matching value determination module 440 is used to determine a target feature matching value based on the detection frame features, all trajectory features, and preset weights;

[0119] The vehicle determination module 450 is used to determine the target vehicle in the target detection frame according to the target feature matching value and the preset matching threshold.

[0120] In the embodiment of the present application, the feature map determination module 410 may also be specifically configured to:

[0121] The target point cloud data is divided into cylinders of preset cylinder sizes, and the data in the cylinders are feature encoded according to the pointpillars network to obtain the corresponding cylinder data features;

[0122] The column feature map is determined according to the spatial position of the column corresponding to the column data feature.

[0123] In the embodiment of the present application, the feature map determination module 410 may also be specifically configured to:

[0124] Determine the preset convolutional neural network;

[0125] According to the preset convolutional neural network and the spatial position of the cylinder, a high-resolution cylinder feature map, a medium-resolution cylinder feature map, and a low-resolution cylinder feature map are determined;

[0126] A column feature map is determined according to the high-resolution column feature map, the medium-resolution column feature map, and the low-resolution column feature map.

[0127] In the embodiment of the present application, the trajectory determination module 420 may also be specifically configured to:

[0128] Determine the convolution area of ​​a preset size in the cylindrical feature map;

[0129] According to the target detection algorithm, the vehicles in the convolution area and the target trajectories corresponding to the vehicles are determined.

[0130] In the embodiment of the present application, the feature determination module 430 may also be specifically configured to:

[0131] Determine a high-resolution feature map, a medium-resolution feature map, and a low-resolution feature map in the column feature map, and their corresponding feature areas;

[0132] Perform feature map fusion on the feature area corresponding to the high-resolution feature map, the feature area corresponding to the medium-resolution feature map, and the feature area corresponding to the low-resolution feature map to obtain the target feature area;

[0133] According to the target feature area, the trajectory features corresponding to the target trajectory and the detection frame features corresponding to the target detection frame are determined.

[0134] In the embodiment of the present application, the matching value determination module 440 may also be specifically configured to:

[0135] Determine the cosine similarity, intersection-over-union (IoU) and distance metrics between the detection box features and each trajectory feature, as well as preset weights corresponding to the cosine similarity, the IoU and the distance metrics;

[0136] According to the preset weights, the cosine similarity, intersection-over-union ratio and distance metrics are weighted and combined to obtain the target feature matching value.

[0137] In the embodiment of the present application, the vehicle determination module 450 may also be specifically configured to:

[0138] Compare the target feature matching value with the preset matching threshold to obtain the comparison result;

[0139] If the comparison result shows that the target feature matching value is greater than the preset matching threshold, the vehicle in the target detection frame is determined to be the target vehicle corresponding to the target trajectory;

[0140] If the comparison result is that the target feature matching value is not greater than the preset matching threshold, the vehicle in the target detection frame is determined to be an unmatched vehicle.

[0141] Figure 5 This is a schematic diagram of the structure of the device provided in the embodiment of this application. Figure 5 As shown, the device 500 includes:

[0142] The device 500 may include one or more processors 501 , one or more computer-readable storage media memories 502 , a communication component 503 , and other components. The processor 501 , the memory 502 , and the communication component 503 are connected via a bus 504 .

[0143] In a specific implementation process, at least one processor 501 executes the computer-executable instructions stored in the memory 502 , so that the at least one processor 501 executes the above-mentioned vehicle tracking method based on BEV features.

[0144] The specific implementation process of the processor 501 can be found in the above method embodiment. Its implementation principle and technical effects are similar and will not be repeated here in this embodiment.

[0145] Furthermore, the processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), or application-specific integrated circuits (ASICs). A general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in this application may be directly executed by a hardware processor or by a combination of hardware and software modules within the processor.

[0146] The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage.

[0147] A bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus. Buses can be categorized as address buses, data buses, and control buses. For ease of illustration, the buses in the drawings of this application are not limited to just one bus or just one type of bus.

[0148] In some embodiments, a computer program product is further provided, comprising a computer program or instructions, which implement the steps of any of the above-mentioned vehicle tracking methods based on BEV features when executed by a processor.

[0149] The specific implementation of the above operations can be found in the previous embodiments and will not be repeated here.

[0150] Those skilled in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be accomplished by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0151] To this end, an embodiment of the present application provides a computer-readable storage medium storing a plurality of program codes, which can be loaded by a processor to execute the steps of any one of the vehicle tracking methods based on BEV features provided in the embodiments of the present application.

[0152] The storage medium may include a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0153] According to one aspect of the present application, a computer program product or a computer program is provided. The computer program product or the computer program comprises computer instructions stored in a computer-readable storage medium.

[0154] Since the instructions stored in the storage medium can execute the steps of any one of the vehicle tracking methods based on BEV features provided in the embodiments of the present application, the beneficial effects that can be achieved by any one of the vehicle tracking methods based on BEV features provided in the embodiments of the present application can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0155] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present application that follow the general principles of the present application and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered merely as exemplary, and the true scope and spirit of the present application are indicated by the claims set forth above.

[0156] It should be understood that the present application is not limited to the exact structure described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A vehicle tracking method based on BEV characteristic information, characterized in that: The method comprises: Divide the target point cloud data into cylinders of preset cylinder sizes, and perform feature encoding on the data in the cylinders according to the pointpillars network to obtain corresponding cylinder data features; Determine the preset convolutional neural network; Determining a high-resolution column feature map, a medium-resolution column feature map, and a low-resolution column feature map according to the preset convolutional neural network and the spatial position of the column; Determining the column feature map according to the high-resolution column feature map, the medium-resolution column feature map, and the low-resolution column feature map; Determine a convolution area of ​​a preset size in the cylindrical feature map; Determining, according to a target detection algorithm, vehicles in the convolution region and target trajectories corresponding to the vehicles; Determining a target detection frame and a target feature region corresponding to the target detection frame in the cylindrical feature map, and determining, based on the target feature region, a trajectory feature corresponding to the target trajectory and a detection frame feature corresponding to the target detection frame; Determining a target feature matching value based on the detection frame features, all the trajectory features, and a preset weight; The target vehicle in the target detection frame is determined according to the target feature matching value and a preset matching threshold.

2. The method according to claim 1, characterized in that The determining of the target detection frame and the target feature region corresponding to the target detection frame in the cylindrical feature map, and determining, based on the target feature region, the trajectory features corresponding to the target trajectory and the detection frame features corresponding to the target detection frame, includes: Determine a high-resolution feature map, a medium-resolution feature map, and a low-resolution feature map in the column feature map, and their corresponding feature regions; Performing feature map fusion on the feature region corresponding to the high-resolution feature map, the feature region corresponding to the medium-resolution feature map, and the feature region corresponding to the low-resolution feature map to obtain the target feature region; According to the target feature area, a trajectory feature corresponding to the target trajectory and a detection frame feature corresponding to the target detection frame are determined.

3. The method according to claim 1, characterized in that The determining of the target feature matching value according to the detection frame feature, all the trajectory features and the preset weight includes: Determining the cosine similarity, intersection-over-union ratio, and distance metric between the detection frame feature and each of the trajectory features, as well as the preset weight corresponding to the cosine similarity, the preset weight corresponding to the intersection-over-union ratio, and the preset weight corresponding to the distance metric; According to the preset weights, the cosine similarity, intersection-over-union ratio and distance metric are weightedly combined to obtain the target feature matching value.

4. The method according to claim 1, wherein The determining the target vehicle in the target detection frame according to the target feature matching value and the preset matching threshold comprises: Comparing the target feature matching value with the preset matching threshold to obtain a comparison result; If the comparison result is that the target feature matching value is greater than the preset matching threshold, determining that the vehicle in the target detection frame is the target vehicle corresponding to the target track; If the comparison result is that the target feature matching value is not greater than the preset matching threshold, the vehicle in the target detection frame is determined to be an unmatched vehicle.

5. A vehicle tracking device based on BEV characteristic information, characterized in that: The device comprises: The encoding module is used to divide the target point cloud data into cylinders of preset cylinder sizes, and perform feature encoding on the data in the cylinders according to the pointpillars network to obtain corresponding cylinder data features; A network determination module, used to determine a preset convolutional neural network; a feature map determination module, configured to determine a high-resolution column feature map, a medium-resolution column feature map, and a low-resolution column feature map based on the preset convolutional neural network and the spatial position of the column; a pillar determining module, configured to determine the pillar feature map according to the high-resolution pillar feature map, the medium-resolution pillar feature map, and the low-resolution pillar feature map; A region determination module, configured to determine a convolution region of a preset size in the cylindrical feature map; a trajectory determination module, configured to determine the vehicles in the convolution area and the target trajectories corresponding to the vehicles based on a target detection algorithm; a feature determination module, configured to determine a target detection frame and a target feature region corresponding to the target detection frame in the cylindrical feature map, and determine, based on the target feature region, a trajectory feature corresponding to the target trajectory and a detection frame feature corresponding to the target detection frame; a matching value determination module, configured to determine a target feature matching value based on the detection frame features, all the trajectory features, and a preset weight; The vehicle determination module is used to determine the target vehicle in the target detection frame according to the target feature matching value and a preset matching threshold.

6. A device, characterized in that include: one or more processors; Memory; One or more programs, wherein the one or more programs are stored in a memory and configured to be executed by one or more processors, the one or more programs being configured to perform the method according to any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that The computer-readable storage medium stores program codes, which can be called by a processor to execute the method according to any one of claims 1 to 4.

Images