Target tracking method and device, computer equipment, storage medium and program product

Abstract

The embodiment of the invention relates to a target tracking method and device, computer equipment, a storage medium and a program product. Acquiring a multi-view image at the current moment; inputting the multi-view image into a prediction model, and using the prediction model to call a target guide feature to predict the multi-view image to obtain a prediction result of each target object in the multi-view image; the target guiding feature comprises a target guiding aerial view BEV feature; the target guiding BEV feature is obtained by performing reverse optimization on a historical BEV feature according to a historical prediction result. By adopting the method, the accuracy of determining the position information of the target object in the current frame can be improved.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the field of autonomous driving technology, and in particular to a target tracking method, apparatus, computer equipment, storage medium, and program product. Background Art

[0002] With the development of artificial driving technology, it is necessary to detect and track target objects in the driving environment during driving in order to formulate one's own driving route. Therefore, the detection and tracking of target objects plays an important role in artificial driving.

[0003] In traditional technology, when detecting and tracking a target object, the Bird's Eye View (BEV) features of the current frame are learned based on the BEV features in the historical frames, so as to obtain the position information of the target object in the current frame.

[0004] However, the traditional method has the problem that the accuracy of the position information of the target object in the current frame is low. Summary of the invention

[0005] The embodiments of the present disclosure provide a target tracking method, apparatus, computer device, storage medium and program product, which can be used to improve the accuracy of the position information of the target object in the current frame.

[0006] In a first aspect, an embodiment of the present disclosure provides a target tracking method, the method comprising:

[0007] Get the multi-view image at the current moment;

[0008] The multi-view image is input into a prediction model, and the prediction model is used to call the target-oriented feature to predict the multi-view image, so as to obtain the prediction result of each target object in the multi-view image; the target-oriented feature includes the target-oriented bird's-eye view BEV feature; the target-oriented BEV feature is obtained by reversely optimizing the historical BEV feature according to the historical prediction results.

[0009] In a second aspect, an embodiment of the present disclosure provides a target tracking device, the device comprising:

[0010] A first acquisition module is used to acquire a multi-view image at the current moment;

[0011] The second acquisition module is used to input the multi-view image into the prediction model, use the prediction model to call the target-oriented feature to predict the multi-view image, and obtain the prediction results of each target object in the multi-view image; the target-oriented feature includes the target-oriented bird's-eye view BEV feature; the target-oriented BEV feature is obtained by reversely optimizing the historical BEV feature according to the historical prediction results.

[0012] In a third aspect, an embodiment of the present disclosure provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in the first aspect when executing the computer program.

[0013] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the method described in the first aspect above.

[0014] In a fifth aspect, an embodiment of the present disclosure provides a computer program product, including a computer program, which implements the method described in the first aspect when executed by a processor.

[0015] The target tracking method, apparatus, computer equipment, storage medium and program product provided by the embodiments of the present disclosure obtain a multi-view image at the current moment, input the multi-view image into a prediction model, and use the prediction model to call a target-oriented feature to predict the multi-view image. The target-oriented feature may include a target-oriented bird's-eye view BEV feature obtained by reversely optimizing the historical BEV feature according to the historical prediction result, that is, the historical BEV feature has been reversely corrected according to the historical prediction result, and the background feature information in the historical BEV feature that is not related to the target object has been filtered out from the obtained target-oriented BEV feature, thereby avoiding the interference of the background feature information in the target-oriented BEV feature with the BEV feature learning of the multi-view image at the current moment, so that the multi-view image at the current moment can be accurately predicted according to the target-oriented BEV feature, and the prediction result of each target object in the multi-view image at the current moment can be accurately obtained, thereby ensuring the accuracy of the prediction result of each target object in the multi-view image at the current moment. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 A diagram showing an application environment of a target tracking method in an embodiment;

[0017] Figure 2 A schematic diagram of a target tracking method according to an embodiment of the present invention;

[0018] Figure 3 is a flow chart of a target tracking method in another embodiment;

[0019] Figure 4 is a flow chart of a target tracking method in another embodiment;

[0020] Figure 5 A schematic diagram of obtaining prediction results of each target object in a multi-view image at the current moment in an embodiment;

[0021] Figure 6 is a flow chart of a target tracking method in another embodiment;

[0022] Figure 7 is a flow chart of a target tracking method in another embodiment;

[0023] Figure 8 is a flow chart of a target tracking method in another embodiment;

[0024] Fig. 9 is a flow chart of a target tracking method in another embodiment;

[0025] Fig.10 is a flow chart of a target tracking method in another embodiment;

[0026] Fig.11 is a flow chart of a target tracking method in another embodiment;

[0027] Fig.12 A schematic diagram of obtaining a target-oriented BEV feature and obtaining a target-oriented image feature in one embodiment;

[0028] Fig.13 is a flow chart of a target tracking method in another embodiment;

[0029] Fig.14 is a flow chart of a target tracking method in another embodiment;

[0030] Fig.15 is a flow chart of a target tracking method in another embodiment;

[0031] Fig.16 is a flow chart of a target tracking method in another embodiment;

[0032] Fig.17 is a schematic diagram of tracking target objects in one embodiment;

[0033] Fig.18 is a structural block diagram of a target tracking device in one embodiment;

[0034] Fig.19 is a structural block diagram of a target tracking device in one embodiment;

[0035] Fig. 20 is a structural block diagram of a target tracking device in one embodiment;

[0036] Fig.21 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0037] In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure more clear, the embodiments of the present disclosure are further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the embodiments of the present disclosure and are not used to limit the embodiments of the present disclosure.

[0038] First, before specifically introducing the technical solution of the embodiment of the present disclosure, the technical background or technical evolution context on which the embodiment of the present disclosure is based is introduced. Generally, when detecting and tracking a target object, the current technical background is: constructing a three-dimensional detection and tracking model based on BEV visual perception technology, supplementing the BEV feature in the current frame with the timing information according to the timing information of the BEV in the historical frame, thereby enhancing the feature information of each target object, obtaining the position information of the target object in the current frame, and detecting and tracking the target object. However, in the autonomous driving scenario, the historical frame contains a large amount of background information that is irrelevant to the target object, and the background information will affect the learning accuracy of the BEV feature in the current frame, thereby resulting in the problem of low accuracy of the position information of the target object determined in the current frame. Based on this background, the applicant has found through long-term model simulation research and development and the collection, demonstration and verification of experimental data that when the historical frame includes a lot of background information that is irrelevant to the target object, it will greatly affect the prediction accuracy of the position information of the target object in the current frame. Therefore, how to improve the accuracy of the position information of the determined target object has become a difficult problem that needs to be solved urgently. In addition, it should be noted that the background information of determining the BEV features in the historical frame will affect the learning accuracy of the BEV features in the current frame, resulting in lower accuracy of the predicted position information of the target object, as well as the technical solutions introduced in the following embodiments, and the applicants have put in a lot of creative work.

[0039] The technical solutions involved in the embodiments of the present disclosure are introduced below in combination with the scenarios to which the embodiments of the present disclosure are applied.

[0040] The target tracking method provided by the embodiment of the present disclosure can be applied to Figure 1 In the application environment shown, the computer device 102 communicates with the vehicle 104 through a network. The computer device 102 may be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, and portable wearable devices, and the vehicle 104 may be various motor vehicles such as cars and trucks. The vehicle 104 is provided with an image acquisition device and a communication component. The image acquisition device may acquire multi-view images of the environment in which the vehicle 104 is located at the current moment, and the acquired multi-view images may be sent to the computer device through the communication component.

[0041] In one embodiment, Figure 2 As shown, a target tracking method is provided, which is applied to Figure 1 The computer device in the example is used to illustrate, including the following steps:

[0042] S201, obtaining a multi-view image at the current moment.

[0043] It should be noted that in an autonomous driving scenario, in order to detect and track a target object, it is necessary to obtain images output by multiple image acquisition devices with different positions and orientations to estimate the position, size, direction and other information of the target object in three-dimensional (3D) space. In this embodiment, the multi-perspective image at the current moment is an image obtained based on the images output by multiple image acquisition devices with different positions and orientations at the current moment.

[0044] In this embodiment, multiple image acquisition devices may be set at multiple positions of the vehicle in different directions to acquire image information in different directions, and then the multiple image information in different directions acquired at the current moment may be transmitted to a computer device for fusion to obtain a multi-view image; or, the multiple image information in different directions at the current moment may be transmitted to a processor of the vehicle, and the multiple image information in different directions acquired at the current moment may be fused by the processor of the vehicle, and the fused image may be transmitted to a computer device to obtain a multi-view image at the current moment. Optionally, when acquiring a multi-view image at the current moment, the multiple image information in different directions at the current moment may be fused at the pixel level to obtain a multi-view image; or, the multiple image information in different directions at the current moment may be fused at the feature level to obtain a multi-view image.

[0045] S202, input the multi-view image into the prediction model, use the prediction model to call the target-oriented features to predict the multi-view image, and obtain the prediction results of each target object in the multi-view image; the target-oriented features include target-oriented bird's-eye view BEV features; the target-oriented BEV features are obtained by reversely optimizing the historical BEV features according to the historical prediction results.

[0046] Optionally, the prediction model in this embodiment may be a convolutional neural network, or a deep neural network, etc., which is not limited in this embodiment.

[0047] The historical prediction result refers to the prediction result obtained by predicting the position information of each target object in the multi-view image at the historical moment according to the BEV features of the multi-view image at the historical moment. For example, the historical prediction result can be the position information of pedestrians, the position information of trees, the information of road signs, and the driving information of other vehicles in the multi-view image at the historical moment.

[0048] It should be noted that in the BEV visual perception technology, in order to accurately perceive the 3D spatial position of each target object in different time frames, the BEV features at the current moment can be supplemented with time information based on the BEV features at the historical moment to enhance the feature information of each target object at the current moment. However, in the autonomous driving scenario, the captured image may contain a large amount of irrelevant background feature information, so that the acquired BEV features also contain a large amount of background feature information irrelevant to the target object, which will affect the accuracy of the acquired BEV features at the current moment. Based on this, the target objects of the multi-view images at the historical moments are predicted according to the historical BEV features. After obtaining the historical prediction results, the historical BEV features can be reversely optimized according to the historical prediction results, and the irrelevant background feature information in the historical BEV features can be filtered out to obtain the target-oriented BEV features. Then, the target-oriented BEV features are used to enhance the time information of each target object in the multi-view images at the current moment, and the target objects in the multi-view images at the current moment are predicted to obtain the prediction results.

[0049] In this embodiment, the multi-view image acquired at the current moment can be input into the prediction model. In the prediction process, the prediction model can call the target-oriented features at the historical moment to provide time series information for the features of each target object in the multi-view image at the current moment, so as to enhance the features of each target object in the multi-view image at the current moment, thereby analyzing the enhanced features, and determining the prediction results of each target object in the multi-view image at the current moment according to the analysis results. As an optional implementation, since the BEV feature can reflect the feature information of the object, the target-oriented feature can include the target-oriented bird's-eye view BEV feature obtained by reversely optimizing the historical BEV feature according to the historical prediction results, that is, the target-oriented BEV feature at the historical moment can be called by the prediction model, and the target-oriented BEV feature at the historical moment can be used to provide time series information for the BEV feature in the multi-view image at the current moment, so as to enhance the BEV features of each target object in the multi-view image at the current moment, thereby analyzing the enhanced BEV features, and determining the prediction results of each target object in the multi-view image at the current moment according to the analysis results.

[0050] In the above target tracking method, by acquiring a multi-view image at the current moment and inputting the multi-view image into a prediction model, the prediction model is used to call a target-oriented feature to predict the multi-view image. The target-oriented feature may include a target-oriented bird's-eye view BEV feature obtained by reversely optimizing the historical BEV feature according to the historical prediction result. That is to say, the historical BEV feature has been reversely corrected according to the historical prediction result, and the background feature information irrelevant to the target object in the historical BEV feature has been filtered out in the obtained target-oriented BEV feature, thereby avoiding the interference of the background feature information in the target-oriented BEV feature with the BEV feature learning of the multi-view image at the current moment, so that the multi-view image at the current moment can be accurately predicted according to the target-oriented BEV feature, and the prediction result of each target object in the multi-view image at the current moment can be accurately obtained, thereby ensuring the accuracy of the prediction result of each target object in the multi-view image at the current moment.

[0051] In a scenario where the above-mentioned prediction model calls the target-oriented feature to predict the multi-view image and obtains the prediction results of each target object in the multi-view image, in one embodiment, the above-mentioned target-oriented feature also includes a target-oriented image feature, which is obtained by reversely optimizing the historical image feature according to the historical prediction results.

[0052] It should be noted that since the BEV feature is a 3D spatial feature from a bird's-eye view, and the multi-view images captured by the image acquisition device are 2D images, it is necessary to first extract the image features of the multi-view images, and then map and transform the image features to obtain the BEV features corresponding to the multi-view images.

[0053] In this embodiment, the multi-view images of historical moments can be input into the prediction model, and the historical image features of the multi-view images of historical moments can be obtained through the prediction model. The historical image features are then mapped to obtain historical BEV features, and then the corresponding historical prediction results are obtained based on the historical BEV features. Furthermore, after obtaining the historical prediction results of the multi-view images of historical moments, the historical image features can be reversely optimized based on the historical prediction results, and background features in the historical image features that are not related to the target object can be filtered out to obtain target-oriented image features. Furthermore, the target-oriented image features can be mapped and transformed to obtain target-oriented BEV features.

[0054] In this embodiment, after the multi-view image at the current moment is input into the prediction model, after the prediction model performs image feature extraction processing on the input multi-view image at the current moment, the prediction model can be used to call the above-mentioned target-oriented image features to perform feature fusion processing on the extracted image features to obtain fused image features, and then the fused feature map is converted into BEV features through the prediction model, and the prediction model is used to call the above-mentioned target-oriented BEV features to perform feature fusion processing on the obtained BEV features to obtain fused BEV features, and then the prediction model is used to use the fused BEV features to classify the multi-view image at the current moment, determine the target objects in the multi-view image according to the classification results, and obtain the prediction results of each target object in the multi-view image at the current moment.

[0055] In the present embodiment, since the target-oriented features also include target-oriented image features obtained by reversely optimizing historical image features according to historical prediction results, and the target-oriented image features and target-oriented BEV features are obtained by reversely correcting historical image features and historical BEV features according to historical prediction results and filtering out background feature information irrelevant to the target object in the historical image features and historical BEV features, the target-oriented image features and target-oriented BEV features contain less irrelevant background feature information, and thus by calling the target-oriented image features and target-oriented BEV features through the prediction model, the target object in the multi-view image at the current moment can be accurately predicted through the prediction model and the called target-oriented image features and target-oriented BEV features, thereby improving the accuracy of the prediction results of the target object in the multi-view image at the current moment.

[0056] In one embodiment, the prediction model may include a first processing network, a second processing network, and a classification network. The following will describe the detailed process of obtaining the prediction results of each target object in the multi-view image at the current moment through the prediction model. Figure 3 As shown, the above S202 includes:

[0057] S301, obtaining initial BEV features of the multi-view image through a first processing network, the multi-view image and the called target-oriented image features.

[0058] The first processing network is used to perform feature extraction processing on the multi-view image at the current moment, and obtain the initial BEV features of the multi-view image at the current moment according to the extracted features and the called target-oriented image features. In this embodiment, the multi-view image at the current moment can be input into the first processing network. During the process of the first processing network processing the multi-view image at the current moment, the image features of the multi-view image at the current moment can be supplemented with time sequence information by calling the target-oriented image features of the historical moments, so as to obtain the initial BEV features of the multi-view image by performing perspective conversion on the image features after the supplementation of the time sequence information.

[0059] As an optional implementation, the first processing network may include a backbone network, a fusion network and a mapping network. Figure 4 As shown, the above S301 may include:

[0060] S401, input the multi-view image into the backbone network to obtain the initial image features of the multi-view image.

[0061] Among them, the backbone network is a neural network used to extract feature information in the multi-view image at the current moment. Optionally, the backbone network can be a visual geometry group (Visual Geometry Group, VGG) network, a residual network (Residual Network, ResNet), or a vision transformer (Vision Transformer, ViT) network, etc., which is not limited in this embodiment.

[0062] In this embodiment, the multi-view image acquired at the current moment can be input into the backbone network, and the backbone network can extract feature information in the multi-view image to obtain corresponding initial image features, wherein the initial image features may include shape features, texture features, spatial relationship features and other features of the area of ​​interest in the image.

[0063] S402, inputting the initial image features and the called target-oriented image features into a fusion network to obtain fused image features.

[0064] Among them, the fusion network fuses different input features into a neural network with more discriminative features than the input features.

[0065] In this embodiment, the initial image features obtained above can be input into the fusion network, and the fusion network will fuse the called target-oriented image features and the initial image features to obtain the fused image features. It can be understood that the target-oriented image features contain the features of each object in the multi-view image at the historical moment and the interference of background information has been filtered out, and the initial image features contain the features of each object in the multi-view image at the current moment. After the fusion network, the target-oriented image features can be used to fuse and correct the features of each object in the initial image features, so as to obtain the fused image features containing more comprehensive and accurate feature information.

[0066] S403, inputting the fused image features into a mapping network to obtain initial BEV features.

[0067] In this embodiment, the mapping network is used to map the fused image features to the BEV perspective to obtain BEV features corresponding to the fused image features. Optionally, the mapping network can be an auto-encoding neural network or a self-organizing mapping network, which is not limited in this embodiment.

[0068] In the above implementation, since the multi-view images are input into the backbone network of the prediction model, the initial image features can be quickly obtained, and the process of using the fusion network to fuse the called target-oriented image features with the initial image features is relatively simple, and the fused image features can be quickly obtained, so that the mapping network can be used to map the fused image features to the BEV view, and the initial BEV features can be quickly obtained, thereby improving the efficiency of obtaining the initial BEV features.

[0069] S302, inputting the initial BEV features and the called target-oriented BEV features into a second processing network to obtain target BEV features of the multi-view image.

[0070] The second processing network is used to process the above-mentioned initial BEV features and the called target-oriented BEV features to obtain the target BEV features of the multi-view image at the current moment. In this embodiment, the second processing network can use the called target-oriented BEV features to fuse the initial BEV features. Since the called target-oriented BEV features filter out background features that are not related to the target object, a target BEV feature containing more comprehensive and accurate BEV features can be obtained.

[0071] S303, inputting the target BEV features into the classification network to obtain the prediction results of each target object in the multi-view image.

[0072] Among them, the classification network refers to a neural network that classifies the target BEV features. Optionally, the classification network can be a convolutional neural network or a residual network, which is not limited in this embodiment. The classification method used in the classification network can be a support vector machine method, a Bayesian method, or a K-nearest neighbor method, which is not limited in this embodiment.

[0073] In this embodiment, the target BEV features can be input into the classification network, and the classification network classifies and identifies the target BEV features to obtain prediction results for each target object in the multi-view image. For example, the support vector machine method can be used to calculate the decision surface corresponding to the target BEV features, and the category of unknown samples in the target BEV features is determined according to the calculated decision surface, so that the target BEV features are classified and identified according to the obtained category, and the classification and identification results are used as the prediction results for each target object in the multi-view image.

[0074] The following will be combined Figure 5 The detailed process of obtaining the prediction results of the target objects in the multi-view images at the current moment is described. Figure 5 As shown, first, the multi-view images of historical moments are input into the prediction model, and the historical image features of the multi-view images of historical moments are obtained through the backbone network in the prediction model. The historical image features are then mapped through the perspective mapping network to obtain historical BEV features, and then the corresponding historical prediction results are obtained according to the historical BEV features. Furthermore, after the historical prediction results of the multi-view images of historical moments are obtained, the historical image features can be reversely optimized according to the historical prediction results, and the background features irrelevant to the target object in the historical image features are filtered out to obtain target-oriented image features. Furthermore, the historical prediction results can be used to reversely optimize the historical BEV features, and the background features irrelevant to the target object in the historical BEV features are filtered out to obtain target-oriented BEV features. Then, the multi-view image at the current moment is input into the prediction model. After the prediction model performs image feature extraction on the input multi-view image at the current moment, the target-oriented image feature is called to perform feature fusion processing on the extracted image features to obtain fused image features. The fused features are then mapped and transformed by the prediction model to obtain BEV features. The prediction model is used to call the target-oriented BEV features to perform feature fusion processing on the obtained BEV features to obtain fused BEV features. The fused BEV features are then classified by the classification network in the prediction model. The target objects in the multi-view image are determined according to the classification results to obtain the prediction results of each target object in the multi-view image at the current moment.

[0075] In this embodiment, since background features irrelevant to the target object are filtered out from the target-oriented image features, the initial BEV features of the multi-view image obtained through the first processing network, the multi-view image and the called target-oriented image features will not include background features. Therefore, the initial BEV features and the called target-oriented BEV features are input into the second processing network, so that the target BEV features of the multi-view image with higher accuracy can be obtained. Furthermore, the target BEV features are input into the classification network, so that the prediction results of each target object in the multi-view image with higher accuracy can be obtained through the classification network, thereby improving the accuracy of the prediction results of each target object in the determined multi-view image.

[0076] The detailed process of obtaining the goal-oriented BEV features in the above scenario is described below. Figure 6 As shown, the method for obtaining the target-oriented BEV feature includes:

[0077] S501, performing center mapping on the historical prediction results to obtain the historical prediction vectors corresponding to the historical prediction results.

[0078] Among them, center mapping refers to the mapping process of obtaining the projection of each object in the historical prediction results on the two-dimensional plane according to the central coordinates of the object in the historical prediction results, and determining the historical prediction vector corresponding to the historical prediction result according to the projection result of the two-dimensional plane.

[0079] Optionally, in this embodiment, the center of each target object in the historical prediction results in the 3D space position can be determined based on the historical prediction results, so that each target object can be center mapped according to the obtained center of each target object to obtain the historical prediction vector corresponding to each target object in the historical prediction results.

[0080] S502 : Acquire goal-oriented BEV features according to historical BEV features and historical prediction vectors.

[0081] Among them, the historical BEV features refer to the BEV features corresponding to the multi-view images at the historical moments. In this embodiment, the historical BEV features can be feature screened according to the historical prediction vectors, and the feature information of the target object in the historical BEV features can be determined according to the screening results, thereby filtering out the background features in the historical BEV features that are not related to the target object, and obtaining the target-oriented BEV features.

[0082] In this embodiment, since the center mapping process is relatively simple, the historical prediction vector can be obtained quickly and accurately, and then the target-oriented BEV features can be quickly and accurately obtained based on the historical BEV features and the historical prediction vector, thereby improving the efficiency and accuracy of obtaining the target-oriented BEV features.

[0083] The following describes the detailed process of obtaining the target-oriented BEV features based on the historical prediction vector and the historical BEV features. Figure 7 As shown, the above S502 includes:

[0084] S601, perform central mapping on historical BEV features to obtain historical BEV feature vectors corresponding to the historical BEV features.

[0085] In this embodiment, the center of the BEV features corresponding to each target object in the historical prediction results can be determined based on the historical BEV features, and the historical BEV features can be subjected to center mapping transformation processing based on the determined centers of the BEV features corresponding to each target object to obtain the historical BEV feature vectors corresponding to each target object in each prediction result.

[0086] S602, obtaining a target-oriented BEV feature according to a historical BEV feature vector, a historical BEV feature and a historical prediction vector.

[0087] In this embodiment, background features irrelevant to the target object in the historical BEV feature vector can be filtered out according to the historical prediction vector, so that each historical BEV feature is screened according to the filtering result, and the screened features are determined as the target-oriented BEV features.

[0088] Optional, such as Figure 8 As shown, the above S602 includes:

[0089] S701, mapping targets of different sizes to historical BEV feature vectors and historical prediction vectors to obtain distribution weights of targets of different sizes in the historical BEV feature vectors.

[0090] Among them, the target mapping of different sizes refers to the screening of historical BEV feature vectors of different sizes. In this embodiment, the historical BEV feature vectors and historical prediction vectors can be input into the target mapping network, and the historical BEV feature vectors can be size screened according to the mapping network. The probability of each target object belonging to a small, medium, or large target is determined according to the screening results, as well as the fine-grained information of different parts, that is, the distribution weights of targets of different sizes in the historical BEV feature vector. It should be noted that the division of small, medium, and large in this embodiment can be determined by the target mapping network based on the self-learning results.

[0091] S702 , obtaining first target-oriented screening grids of different sizes according to the distribution weights of targets of different sizes in the historical BEV feature vector.

[0092] Among them, the first target-oriented screening grid is used to perform feature screening on the historical BEV feature vector. According to the different sizes corresponding to each target object, first target-oriented screening grids of different sizes can be obtained. It should be noted that the weight of the background area of ​​the non-target object in the first target-oriented screening grid is 0.

[0093] In this embodiment, the weight distribution of targets of different sizes can be mapped to different areas of the first target-oriented screening grid according to the target center of each target object in the prediction results, forming corresponding first target-oriented screening grids of small, medium and large sizes.

[0094] S703: Filter out the historical BEV features using the first target-oriented screening grid to obtain filtered BEV features.

[0095] In this embodiment, the historical BEV features can be multiplied with the first target-oriented screening grids of different sizes, and the filtered BEV features can be obtained according to the product. It can be understood that since the first target screening grid of each size only contains target object features and does not contain background features unrelated to the target object, after the historical BEV features are multiplied with the first target-oriented screening grid, the values ​​of the background features in the historical BEV feature matrix can be changed to 0, thereby filtering out the historical BEV features and obtaining the filtered BEV features.

[0096] S704, processing the screened BEV features according to dynamic convolution kernels of different kernel sizes to obtain target-oriented BEV features.

[0097] Among them, the dynamic convolution kernel is a window for convolution operation of the convolution network. In this embodiment, the selected BEV features can be convolved according to dynamic convolution kernels of different kernel sizes to obtain target-oriented BEV features. It should be noted that in order to avoid the target being too large to fully extract feature information, or the target being too small to affect the convolution result, in this embodiment, dynamic convolution kernels of different kernel sizes can be selected to extract BEV feature information of different sizes after screening.

[0098] In this embodiment, a convolution network with dynamic convolution kernels of different kernel sizes can be used to perform convolution processing on the screened BEV features of different sizes to obtain BEV features after convolution processing of different sizes. The BEV features after convolution processing are spliced ​​and fused, and the dimension of the BEV features after splicing and kernel fusion is reduced to obtain target-oriented BEV features.

[0099] In this embodiment, the historical BEV features are centrally mapped to obtain the historical BEV feature vectors corresponding to the historical BEV features. Since the process of central mapping is relatively simple, the efficiency of obtaining the historical BEV feature vectors can be improved. Then, based on the historical BEV feature vectors, historical BEV features and historical prediction vectors, the target-oriented BEV features can be obtained, which can improve the efficiency of obtaining the target-oriented BEV features.

[0100] The detailed process of obtaining target-oriented image features in the above scenario is described below. Fig. 9 As shown, the method for obtaining target-oriented image features includes:

[0101] S801, performing center mapping on the historical prediction results to obtain the historical prediction vectors corresponding to the historical prediction results.

[0102] Among them, center mapping refers to the mapping process of obtaining the projection of each object in the historical prediction results on the two-dimensional plane according to the central coordinates of the object in the historical prediction results, and determining the historical prediction vector corresponding to the historical prediction result according to the projection result of the two-dimensional plane.

[0103] Optionally, in this embodiment, the center of the 3D spatial position of each target object in the historical prediction results can be determined based on the historical prediction results, and then a center mapping transformation is performed on each target object according to the obtained center of each target object to obtain a historical prediction vector corresponding to each target object in the historical prediction results.

[0104] S802, obtaining target-oriented image features according to historical image features and historical prediction vectors.

[0105] Among them, historical image features refer to image features corresponding to multi-perspective images of historical moments. In this embodiment, the historical image features can be feature screened according to the historical prediction vector, and the feature information of the target object in the historical image features can be determined according to the screening results, thereby filtering out background features in the historical image features that are not related to the target object to obtain target-oriented image features.

[0106] In this embodiment, by performing center mapping on historical prediction results, the historical prediction vector corresponding to the historical prediction result can be obtained, so that the target-oriented image feature can be obtained based on the historical image feature and the historical prediction vector. Since the center mapping process is relatively simple, the historical prediction vector can be obtained quickly and accurately, and the target-oriented image feature can be obtained quickly and accurately based on the historical image feature and the historical prediction vector, thereby improving the efficiency and accuracy of obtaining the target-oriented image feature.

[0107] The following describes the detailed process of obtaining target-oriented image features based on historical prediction vectors and historical image features. Fig.10 As shown, the above S802 includes:

[0108] S901, performing center mapping on historical image features to obtain historical image feature vectors corresponding to the historical image features.

[0109] In this embodiment, the center of the historical image features corresponding to each target object in the historical prediction results can be determined based on the historical image features, and the center mapping transformation processing is performed on each historical image feature based on the determined center of the historical image features corresponding to each target object to obtain the historical image feature vector corresponding to each target object in each prediction result.

[0110] S902, obtaining target-oriented image features according to the historical image feature vector, the historical image features and the historical prediction vector.

[0111] In this embodiment, background features irrelevant to the target object in the historical image feature vector can be filtered out according to the historical prediction vector, and then the historical image features can be screened according to the filtering result, and the screened features can be determined as the target-oriented image features.

[0112] Optional, such as Fig.11 As shown, the above S902 includes:

[0113] S1001, mapping targets of different sizes to the historical image feature vector and the historical prediction vector to obtain distribution weights of targets of different sizes in the historical image feature vector.

[0114] Among them, the target mapping of different sizes refers to the screening processing of different sizes of historical image feature vectors. In this embodiment, the historical image feature vectors and historical prediction vectors can be input into the target mapping network, and the historical image feature vectors can be size screened according to the mapping network. According to the screening results, the probability of each target object belonging to a small, medium, or large target is determined, as well as the fine-grained information of different parts, that is, the distribution weights of targets of different sizes in the historical image feature vector. It should be noted that the division of small, medium, and large in this embodiment can be determined by the target mapping network based on the self-learning results.

[0115] S1002, obtaining second target-oriented screening grids of different sizes according to the distribution weights of targets of different sizes in the historical image feature vector.

[0116] Among them, the second target-oriented screening grid is used to perform feature screening on the feature vectors of historical images. According to the different sizes corresponding to each target object, second target-oriented screening grids of different sizes can be obtained. It should be noted that the weight of the background area of ​​non-target objects in the second target-oriented screening grid is 0.

[0117] In this embodiment, the weight distribution of targets of different sizes can be mapped to different areas of the second target-oriented screening grid according to the target center of each target object in the prediction results, forming corresponding second target-oriented screening grids of small, medium and large sizes.

[0118] S1003, using the second target-oriented screening grid to filter out the historical image features to obtain screened image features.

[0119] In this embodiment, the historical image features can be multiplied with the second target-oriented screening grids of different sizes, and the filtered image features can be obtained according to the product. It can be understood that since the second target screening grid of each size only contains target object features and does not contain background features unrelated to the target object, after the historical image features are multiplied with the second target-oriented screening grid, the values ​​of the background features in the historical image feature matrix can be changed to 0, thereby achieving the filtering of the historical image features and obtaining the filtered image features.

[0120] S1004, processing the filtered image features according to dynamic convolution kernels of different kernel sizes to obtain target-oriented image features.

[0121] The dynamic convolution kernel is a window for performing convolution operations in the convolution network. In this embodiment, the filtered image features can be convolved according to dynamic convolution kernels of different kernel sizes to obtain target-oriented image features. It should be noted that in order to avoid the target being too large to fully extract feature information, or the target being too small to affect the convolution result, in this embodiment, dynamic convolution kernels of different kernel sizes can be selected to extract filtered image feature information of different sizes.

[0122] In this embodiment, a convolution network with dynamic convolution kernels of different kernel sizes can be used to perform convolution processing on filtered image features of different sizes to obtain image features after convolution processing of different sizes. The image features after convolution processing are spliced ​​and fused, and the dimensionality of the image features after splicing and kernel fusion is reduced to obtain target-oriented image features.

[0123] Exemplarily, the schematic diagram of obtaining the target-oriented BEV feature and obtaining the target-oriented image feature can be as follows: Fig.12 As shown, according to the above description and Fig.12It can be seen that the process of acquiring the goal-oriented BEV feature can be as follows: inputting the historical prediction vector and the historical BEV feature vector into the target mapping network, classifying the size of each target object in the historical prediction result, thereby obtaining the distribution weight corresponding to each target object of different sizes, further, obtaining the first goal-oriented screening grid including different sizes according to the distribution weight, multiplying the historical BEV feature with the first goal-oriented screening grid of different sizes to obtain the screened feature, and then using dynamic convolution kernels of different kernel sizes to convolve the screened BEV feature to obtain the historical BEV feature after convolution, and then splicing and fusing the historical BEV feature after convolution to obtain the goal-oriented BEV. EV features; the process of acquiring target-oriented image features can be as follows: inputting historical prediction vectors and historical image feature vectors into the target mapping network, classifying the sizes of each target object in the historical prediction results, thereby obtaining the distribution weights corresponding to different sizes of each target object, and further obtaining a second target-oriented screening grid including different sizes according to the distribution weights, multiplying the historical image features with the second target-oriented screening grids of different sizes to obtain the screened features, and then using dynamic convolution kernels of different kernel sizes to convolve the screened image features to obtain the historical image features after convolution, and then splicing and fusing the historical image features after convolution to obtain the target-oriented image features.

[0124] In this embodiment, the historical image features are centrally mapped to obtain the historical image feature vectors corresponding to the historical image features. Since the process of central mapping is relatively simple, the efficiency of obtaining the historical image feature vectors can be improved. Then, based on the historical image feature vectors, historical image features and historical prediction vectors, the target-oriented image features can be obtained, which can improve the efficiency of obtaining the target-oriented BEV features.

[0125] After the prediction results of each target object in the multi-view image are determined, each target object in the multi-view image at the current moment can also be tracked according to the tracking trajectory at the historical moment. In one embodiment, the method further includes: tracking each target object according to the prediction results of each target object in the multi-view image and a preset target trajectory; wherein the target trajectory is a trajectory formed by tracking each object in multiple historical prediction results.

[0126] Among them, the preset target trajectory is a trajectory determined based on the position information of each object in the historical prediction results of the multi-view images at historical moments. It should be noted that each historical prediction result corresponds to a target trajectory, and each target trajectory may contain feature information of multiple objects, wherein the feature information of the object may include information such as the size and direction of the object.

[0127] In this embodiment, the feature information of each object in each target trajectory can be compared with the feature information of each object in the prediction result. If any feature information of the size and direction of each object in the target trajectory is consistent with the feature information of a target object in the prediction result, it can be determined that the timing information of the target object exists in the target trajectory. Furthermore, the corresponding trajectory information of the target object in the target trajectory can be obtained, so that the target object can be tracked according to the feature information in the prediction result of the target object and the trajectory information in the target trajectory.

[0128] In this embodiment, since the preset target trajectory is a trajectory formed by tracking each object in a plurality of historical prediction results, the obtained target trajectory can accurately obtain the trajectory information of the target object at a historical moment, so that each target object can be tracked more accurately according to the prediction results of each target object in the multi-view image and the preset target trajectory, thereby improving the accuracy of tracking each target object.

[0129] In this embodiment, the detailed process of tracking the trajectory of each target object in the multi-view image at the current moment is described. In one embodiment, Fig.13 As shown, the above “tracking each target object according to the prediction results of each target object in the multi-view image and the preset target trajectory” includes:

[0130] S1101, determining a first object in each prediction result that matches the target trajectory according to the prediction results of each target object and the target trajectory.

[0131] Among them, the prediction result of each target object refers to the prediction result of each target object in the multi-view image at the current moment, and the target trajectory refers to the trajectory formed by the position information of each object in the multi-view image at multiple historical moments. Exemplarily, if the prediction result of each target object is the prediction result at moment t, the historical moments may include t-3, t-2, and t-1. For object N in the target trajectory, the target trajectory may contain feature information corresponding to object N at the three moments t-3, t-2, and t-1, respectively.

[0132] In this embodiment, the corresponding feature information in the prediction results of each target object and the feature information of the object corresponding to each trajectory in the target trajectory can be matched, and whether the prediction result matches the target trajectory is determined based on the matching result, wherein the feature information can be a size feature, a direction feature, etc., which is not limited in this embodiment.

[0133] Optionally, the directional features in the prediction results of each target object can be matched with the directional features corresponding to different moments in each target trajectory. If the matching result is greater than a preset matching threshold, it can be determined that the object corresponding to the target trajectory matches the target object in the prediction result, and the target object in the prediction result is determined to be the first object, wherein the preset matching threshold can be set according to the accuracy requirement of the actual usage scenario. For example, the preset matching threshold can be 80% or 70%, and this embodiment does not impose any restrictions on this. The size features in the prediction results of each target object can also be matched with the size features corresponding to different moments in each target trajectory. If the matching result is greater than the preset matching threshold, it can be determined that the object corresponding to the target trajectory matches the target object in the prediction result, and the target object in the prediction result is the first object.

[0134] S1102: Track the first object according to the target trajectory.

[0135] It is understandable that if an object in each prediction result matches an object in the target trajectory, the position information corresponding to the object in multiple historical moments can be obtained according to the target trajectory. In this embodiment, multiple position information corresponding to the first object in historical moments can be obtained according to the target trajectory, so that the motion trajectory of the first object is formed according to the obtained multiple position information and the position information of the first object in the prediction result, and the first object is tracked according to the formed motion trajectory.

[0136] In this embodiment, based on the prediction results and target trajectory of each target object, the first object matching the target trajectory in each prediction result can be accurately determined, thereby improving the accuracy of the determined first object, so that the first object can be accurately tracked, thereby improving the accuracy of tracking the first object.

[0137] The following describes the detailed process of determining the first object in the prediction result that matches the target trajectory based on the prediction results and the target trajectory of each target object. Fig.14 As shown, the above S1101 includes:

[0138] S1201, determining a first image feature vector, a first BEV feature vector, and a first prediction result feature vector corresponding to a prediction result.

[0139] Among them, the first image feature vector is the feature vector of the initial image feature corresponding to the multi-view image at the current moment, the first BEV feature vector is the feature vector of the BEV feature corresponding to the multi-view image at the current moment, and the first prediction result feature vector is the feature vector of the prediction result of each target object corresponding to the multi-view image at the current moment.

[0140] In this embodiment, the initial image features obtained above can be centrally mapped to obtain a first image feature vector, the initial BEV features obtained above can be centrally mapped to obtain a first BEV feature vector, and the prediction results can be centrally mapped to obtain a first prediction result feature vector.

[0141] S1202, determining a second image feature vector, a second BEV feature vector, and a second prediction result feature vector corresponding to the target trajectory.

[0142] Among them, the second image feature vector is the feature vector of the historical image features corresponding to the multi-view images at the historical moment, the second BEV feature vector is the feature vector of the BEV features corresponding to the multi-view images at the historical moment, and the second prediction result feature vector is the feature vector of the historical prediction results of each target object corresponding to the multi-view images at the historical moment.

[0143] In this embodiment, the historical image features obtained above can be centrally mapped to obtain a second image feature vector, the historical BEV features obtained above can be centrally mapped to obtain a second BEV feature vector, and the historical prediction results can be centrally mapped to obtain a second prediction result feature vector.

[0144] S1203: Obtain a first similarity matrix according to the first image feature vector and the second image feature vector.

[0145] The first similarity matrix is ​​used to characterize the similarity between the first image feature vector of each target object in the prediction result and the second image feature vector corresponding to each object in each target track.

[0146] In this embodiment, the first image feature vector of each target object in the prediction result and the second image feature vector corresponding to each object in each target trajectory can be multiplied, and the product value is used as the matrix value in the first similarity matrix to obtain the first similarity matrix.

[0147] S1204: Obtain a second similarity matrix according to the first BEV feature vector and the second BEV feature vector.

[0148] The second similarity matrix is ​​used to characterize the similarity between the first BEV feature vector of each target object in the prediction result and the second BEV feature vector corresponding to each object in each target trajectory.

[0149] In this embodiment, the first BEV feature vector of each target object in the prediction result and the second BEV feature vector corresponding to each object in each target trajectory can be multiplied, and the product value is used as the matrix value in the second similarity matrix to obtain the second similarity matrix.

[0150] S1205: Obtain a third similarity matrix according to the first prediction result feature vector and the second prediction result feature vector.

[0151] The third similarity matrix is ​​used to characterize the similarity between the first prediction result feature vector of each target object in the prediction result and the second prediction result feature vector corresponding to each object in each target trajectory.

[0152] In this embodiment, the first prediction result feature vector of each target object in the prediction result and the second prediction result feature vector corresponding to each object in each target trajectory can be multiplied, and the product value is used as the matrix value in the third similarity matrix to obtain the first similarity matrix.

[0153] S1206: Determine the first object according to the first similarity matrix, the second similarity matrix, the third similarity matrix and a preset matching algorithm.

[0154] Among them, the first object refers to the object in each target trajectory that matches the prediction result, and the preset matching algorithm refers to an algorithm for matching each target object in the prediction result with each object in each target trajectory. Optionally, the matching algorithm can be a Hungarian matching algorithm; or, the preset matching algorithm can also be a Kalman filtering algorithm, which is not limited in this embodiment.

[0155] In this embodiment, different weights can be assigned to the first similarity matrix, the second similarity matrix, and the third similarity matrix, respectively, and each element in each similarity matrix is ​​multiplied by the corresponding weight to obtain three weight matrices corresponding to the three similarity matrices. The three weight matrices are added together to obtain a sum matrix corresponding to the three similarity matrices, and the sum matrix is ​​used as a cost matrix of the matching algorithm. The matching result is calculated based on the cost matrix. If the matching result is greater than the matching threshold, it can be determined that the two are successfully matched, and the target object in the prediction result that successfully matches the object in the target trajectory is determined as the first object.

[0156] In this embodiment, since the first image feature vector, the first BEV feature vector and the first prediction result feature vector corresponding to the prediction result can be simply and accurately determined, and the second image feature vector, the second BEV feature vector and the second prediction result feature vector corresponding to the target trajectory can be determined, it is possible to quickly and accurately obtain the first similarity matrix based on the first image feature vector and the second image feature vector, obtain the second similarity matrix based on the first BEV feature vector and the second BEV feature vector, and obtain the third similarity matrix based on the first prediction result feature vector and the second prediction result feature vector, and then the first object can be quickly and accurately determined based on the first similarity matrix, the second similarity matrix, the third similarity matrix and the preset matching algorithm, thereby improving the efficiency and accuracy of determining the first object.

[0157] In some scenarios, the above prediction results may contain a second object that does not match the target trajectory. In this embodiment, the tracking method of the second object will be described in detail. In one embodiment, Fig.15 As shown, the above “tracking each target object according to the prediction results of each target object in the multi-view image and the preset target trajectory” also includes:

[0158] S1301, according to the prediction result and the target trajectory, determining a second object in the prediction result that does not match the target trajectory.

[0159] In this embodiment, the corresponding feature information in the prediction results of each target object and the feature information of the object corresponding to each trajectory in the target trajectory can be matched, and whether the prediction result matches the target trajectory can be determined based on the matching result. If the matching result is less than or equal to a preset matching threshold, the target object in the prediction result can be determined as the second object.

[0160] S1302, determining a second object bounding box and bounding boxes of objects in the remaining target tracks; the remaining target tracks are target tracks that do not match any prediction results.

[0161] The bounding box of the second object refers to a closed curve formed by the boundary lines of the outer contour of the second object. It is understandable that if the target trajectory does not successfully match each prediction result, the unmatched target trajectory can be determined as the remaining target trajectory, and the boundary lines of the outer contours of the objects at each historical moment in each remaining target trajectory are determined as the bounding boxes of the objects in the remaining target trajectory.

[0162] In this embodiment, the boundary line of the outer contour of the second object can be determined based on the position, size, direction and other characteristics of each second object, and the boundary line is determined as the bounding box of the second object. The boundary line of the outer contour of the object in the remaining target trajectory is determined based on the position, size, direction and other characteristics of the object at each corresponding historical moment in each remaining target trajectory, and the boundary line is determined as the bounding box of the object in the remaining target trajectory.

[0163] S1303: Track the second object according to the bounding box of the second object and the bounding boxes of the objects in the remaining target track.

[0164] In this embodiment, the shape of the bounding box of the second object can be compared with the shape of the bounding box of the object in the remaining target trajectory. If the comparison result is greater than a preset comparison threshold, it can be determined that the bounding box of the second object is similar to the bounding box of the object in the remaining target trajectory, and the object in the remaining target trajectory is determined as the object corresponding to the second object at the historical moment, and the trajectory information corresponding to the object is determined as the trajectory information corresponding to the second object at the historical moment, so that the second object can be tracked according to the determined trajectory information and the position information of the second object.

[0165] In this embodiment, based on the prediction results and the target trajectory, the second object in the prediction results that does not match the target trajectory is determined, so that the second object boundary box and the boundary boxes of the objects in the remaining target trajectories that do not match the prediction results can be determined relatively simply and quickly, and then the second object can be tracked based on the boundary box of the second object and the boundary boxes of the objects in the remaining target trajectories. Since the similarity between the boundary box of the second object and the boundary box of the objects in the remaining target trajectories can be accurately calculated, the second object can be accurately tracked based on the calculation result, thereby improving the accuracy of tracking the second object.

[0166] The following describes the detailed process of tracking the second object based on the bounding box of the second object and the bounding boxes of the objects in the remaining target track. Fig.16 As shown, the above S1303 includes:

[0167] S1401, expanding the bounding box of the second object and the bounding boxes of the objects in the remaining target track according to a preset classification size to obtain an expanded bounding box of the second object and an expanded bounding box of the objects in the remaining target track.

[0168] Among them, the preset classification size is determined according to the maximum size range of each target object in the multi-view image. It should be noted that since the bounding box of the object in the autonomous driving scenario is small, if the bounding box of the second object is directly matched with the bounding box of the object in the remaining target trajectory, the matching degree will be reduced. Therefore, the bounding box of the second object and the bounding box of the object in the remaining target trajectory can be expanded in length and width.

[0169] In this embodiment, the classification size of the second object and the objects in the remaining target track can be determined, and the extension size of the bounding box of the second object and the bounding box of the objects in the remaining target track can be determined according to the classification size, and the bounding box of the second object and the bounding box of the objects in the remaining target track can be extended according to the extension size. Exemplarily, if the classification size of the second object is a small target size, the size of the bounding box of the second object after expansion cannot exceed the maximum size range of a small target, and if the classification size of the object in the remaining target track is a medium target size, the size of the bounding box of the object in the remaining target track after expansion cannot exceed the maximum size range of a medium target.

[0170] S1402: Determine an intersection-over-union ratio between the expanded bounding box of the second object and the expanded bounding boxes of the objects in the remaining target tracks.

[0171] The intersection-and-union ratio refers to the ratio of the intersection and the union of two bounding boxes. In this embodiment, for each second object, the expanded bounding box of the second object can be respectively calculated with the expanded bounding boxes of the objects in the remaining target tracks, and the intersection-and-union ratio of the expanded bounding box of the second object and the expanded bounding boxes of the objects in the remaining target tracks is determined according to the calculation result.

[0172] S1403: Track the second object according to the IoU ratio and a preset IoU ratio threshold.

[0173] The IoU threshold is used to determine the similarity between two bounding boxes. A larger IoU value indicates a higher similarity between the two bounding boxes. Optionally, the IoU threshold may be 0.6 or 0.5, which is not limited in this embodiment.

[0174] In this embodiment, the above-mentioned IoU can be compared with a preset IoU threshold, and the similarity between the expanded bounding box of the second object and the expanded bounding box of the object in the remaining target trajectory is determined based on the comparison result, and whether the second object matches the object in the remaining target trajectory is determined based on the similarity, and the second object is tracked based on the matching result.

[0175] As an optional implementation, if the above-mentioned intersection-and-union ratio is greater than the intersection-and-union ratio threshold, it means that the position information of the target object in the prediction result exists in the historical moment, and the second object can be further tracked according to the remaining target trajectory corresponding to the target object.

[0176] It can be understood that if the above-mentioned intersection-and-union ratio is greater than the intersection-and-union ratio threshold, it means that the similarity between the expanded bounding box of the second object and the expanded bounding box of the object in the remaining target trajectory reaches a preset threshold, and it can be determined that the object in the remaining target trajectory is the object corresponding to the second object at the historical moment, and the trajectory information of the object in the remaining target trajectory is determined to be the trajectory information of the second object at the historical moment. Therefore, the second object can be tracked according to the trajectory information of the object in the target trajectory and the position information of the second object.

[0177] As another optional implementation, if the above-mentioned intersection-and-union ratio is less than or equal to the intersection-and-union ratio threshold, it means that the position information of the target object in the prediction result does not exist in the historical moment. Then, the second object can be tracked according to the position information corresponding to the second object and the position information corresponding to the second object at subsequent moments to obtain the tracking trajectory of the second object.

[0178] It can be understood that if the above-mentioned intersection-and-union ratio is less than or equal to the intersection-and-union ratio threshold, it means that the similarity between the expanded bounding box of the second object and the expanded bounding box of the object in the remaining target trajectory does not reach the preset threshold, then it can be determined that there is no trajectory information corresponding to the second object in the remaining target trajectory. In this embodiment, the position information and timing information of the second object can be used as new trajectory information to form a tracking trajectory corresponding to the second object.

[0179] It can be understood that if the remaining target track fails to match the second object within three time moments, it means that the object in the remaining target track disappears and the target track is deleted. For example, if the current time is time t, the remaining target track object A is matched with the second object at time t, time t+1, and time t+2 respectively, and the matches all fail, it can be determined that object A disappears after time t, and the target track corresponding to object A will be deleted.

[0180] Exemplarily, the schematic diagram of tracking each target object according to the prediction results of each target object in the multi-view image and the preset target trajectory can be as follows: Fig.17As shown in the figure, firstly, the first prediction result feature vector, the first BEV feature vector, and the first image feature vector corresponding to the multi-view image at the current moment are obtained through the center mapping, and the second prediction result feature vector, the second BEV feature vector, and the second image feature vector corresponding to the multi-view image at the historical moment are obtained through the center mapping. The similarity matrix between the prediction result at the current moment and the target trajectory at the historical moment is calculated according to the above feature vectors, and different weights are assigned to each similarity matrix. The weights are summed up as the cost matrix of the matching algorithm, and the target object that successfully matches the target trajectory in the prediction result is determined as the first object, and the target object that does not match the target trajectory in the prediction result is determined as the first object. The successfully matched target object is determined as the second object, and the target trajectory in the target trajectory that does not successfully match the prediction result is determined as the remaining target trajectory. Further, the bounding boxes of the second object and the objects in the remaining target trajectory are determined, and the bounding boxes are expanded according to a preset classification size, so as to calculate the intersection-and-union ratio between the expanded bounding box of the second object and the expanded bounding box of the remaining target trajectory. If the intersection-and-union ratio is greater than a preset intersection-and-union ratio threshold, the second object is tracked according to the remaining target trajectory corresponding to the target object. If the intersection-and-union ratio is less than or equal to the intersection-and-union ratio threshold, the position information corresponding to the second object is determined as the tracking trajectory corresponding to the second object.

[0181] In the present embodiment, since the bounding box of the second object and the bounding box of the object in the remaining target trajectory are expanded according to the preset classification size, the expanded bounding box of the second object and the expanded bounding box of the object in the remaining target trajectory are obtained, the probability of the bounding box generating an intersection-and-union ratio is improved, so that the intersection-and-union ratio of the expanded bounding box of the second object and the expanded bounding box of the object in the remaining target trajectory can be quickly and accurately determined, and then the second object can be tracked according to the intersection-and-union ratio and the preset intersection-and-union ratio threshold, thereby improving the efficiency and accuracy of tracking the second object.

[0182] To facilitate understanding by those skilled in the art, the target tracking method provided by the present application is described in detail below. The method may include the following steps:

[0183] S1, input the multi-view images of historical moments into the backbone network of the prediction model to obtain historical image features.

[0184] S2, input the historical image features into the mapping network in the prediction model to obtain the historical BEV features.

[0185] S3, input the historical BEV features into the classification network in the prediction model to obtain the historical prediction results.

[0186] S4, reversely optimize the historical image features according to the historical prediction results to obtain the target-oriented image features.

[0187] S5, reversely optimize the historical BEV characteristics according to the historical prediction results to obtain the target-oriented BEV characteristics.

[0188] S6, inputting the multi-view image at the current moment into the backbone network of the prediction model to obtain the initial image features of the multi-view image.

[0189] S7, calling the target-oriented image features through the prediction model, inputting the initial image features and the called target-oriented image features into the fusion network of the prediction model to obtain the fused image features.

[0190] S8, input the fused image features into the mapping network of the prediction model to obtain the initial BEV features.

[0191] S9, calling the target-oriented BEV feature through the prediction model, inputting the initial BEV feature and the called target-oriented BEV feature into the second processing network of the prediction model, and obtaining the target BEV feature of the multi-view image.

[0192] S10, inputting the target BEV features into the classification network of the prediction model to obtain the prediction results of each target object in the multi-view image.

[0193] S11, determining a first object in each prediction result that matches the target trajectory according to the prediction results of each target object and the target trajectory.

[0194] S12: Track the first object according to the target trajectory.

[0195] S13, determining a second object in the prediction result that does not match the target trajectory according to the prediction result and the target trajectory.

[0196] S14, determining a second object bounding box and bounding boxes of objects in the target tracks that are not matched with the prediction results in the remaining target tracks.

[0197] S15: Track the second object according to the bounding box of the second object and the bounding boxes of the objects in the remaining target track.

[0198] It should be understood that, although the various steps in the above flowchart are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the above flowchart may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0199] In one embodiment, Fig.18 As shown, a target tracking device is provided, comprising: a first acquisition module 11 and a second acquisition module 12, wherein:

[0200] The first acquisition module 11 is used to acquire the multi-view images at the current moment.

[0201] The second acquisition module 12 is used to input the multi-view image into the prediction model, use the prediction model to call the target-oriented feature to predict the multi-view image, and obtain the prediction results of each target object in the multi-view image; the target-oriented feature includes the target-oriented bird's-eye view BEV feature; the target-oriented BEV feature is obtained by reversely optimizing the historical BEV feature according to the historical prediction results.

[0202] The target tracking device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, which will not be repeated here.

[0203] In one embodiment, the above-mentioned goal-oriented features also include goal-oriented image features; the goal-oriented image features are obtained by reversely optimizing historical image features according to historical prediction results.

[0204] The target tracking device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, which will not be repeated here.

[0205] In one embodiment, the prediction model includes a first processing network, a second processing network, and a classification network, such as Fig.19 As shown, the second acquisition module 12 includes: a first acquisition unit 121, a second acquisition unit 122 and a third acquisition unit 123, wherein:

[0206] The first acquisition unit 121 is used to obtain the initial BEV features of the multi-view image through the first processing network, the multi-view image and the called target-oriented image features.

[0207] The second acquisition unit 122 is used to input the initial BEV feature and the called target-oriented BEV feature into the second processing network to obtain the target BEV feature of the multi-view image.

[0208] The third acquisition unit 123 inputs the target BEV feature into the classification network to obtain the prediction results of each target object in the multi-view image.

[0209] The target tracking device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, which will not be repeated here.

[0210] In one embodiment, the first processing network includes a backbone network, a fusion network and a mapping network. Fig.19 The above-mentioned first acquisition unit 121 is specifically used to: input the multi-view image into the backbone network to obtain the initial image features of the multi-view image; input the initial image features and the called target-oriented image features into the fusion network to obtain the fused image features; input the fused image features into the mapping network to obtain the initial BEV features.

[0211] The target tracking device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, which will not be repeated here.

[0212] In one embodiment, please continue to refer to Fig.19 The above device further includes: a third acquisition module 13 and a fourth acquisition module 14, wherein:

[0213] The third acquisition module 13 is used to perform center mapping on the historical prediction results to obtain the historical prediction vectors corresponding to the historical prediction results.

[0214] The fourth acquisition module 14 is used to acquire the goal-oriented BEV characteristics according to the historical BEV characteristics and the historical prediction vector.

[0215] The target tracking device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, which will not be repeated here.

[0216] In one embodiment, please continue to refer to Fig.19 The fourth acquisition module 14 includes a fourth acquisition unit 141 and a fifth acquisition unit 142, wherein:

[0217] The fourth acquisition unit 141 is used to perform center mapping on the historical BEV features to acquire the historical BEV feature vector corresponding to the historical BEV features.

[0218] The fifth acquisition unit 142 is used to acquire the goal-oriented BEV feature according to the historical BEV feature vector, the historical BEV feature and the historical prediction vector.

[0219] The target tracking device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, which will not be repeated here.

[0220] In one embodiment, please continue to refer to Fig.19The fifth acquisition unit 142 is specifically used to: perform target mapping of different sizes on the historical BEV feature vector and the historical prediction vector to obtain the distribution weights of targets of different sizes in the historical BEV feature vector; obtain first target-oriented screening grids of different sizes according to the distribution weights of targets of different sizes in the historical BEV feature vector; use the first target-oriented screening grid to filter out the historical BEV features to obtain filtered BEV features; process the filtered BEV features according to dynamic convolution kernels of different kernel sizes to obtain target-oriented BEV features.

[0221] The target tracking device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, which will not be repeated here.

[0222] In one embodiment, please continue to refer to Fig.19 The above device further includes: a fifth acquisition module 15 and a sixth acquisition module 16, wherein:

[0223] The fifth acquisition module 15 is used to perform center mapping on the historical prediction results to obtain the historical prediction vectors corresponding to the historical prediction results.

[0224] The sixth acquisition module 16 is used to acquire target-oriented image features according to historical image features and historical prediction vectors.

[0225] The target tracking device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, which will not be repeated here.

[0226] In one embodiment, please continue to refer to Fig.19 The sixth acquisition module 16 includes a sixth acquisition unit 161 and a seventh acquisition unit 162, wherein:

[0227] The sixth acquisition unit 161 is used to perform center mapping on the historical image features to acquire the historical image feature vector corresponding to the historical image features.

[0228] The seventh acquisition unit 162 is used to acquire the target-oriented image feature according to the historical image feature vector, the historical image feature and the historical prediction vector.

[0229] The target tracking device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, which will not be repeated here.

[0230] In one embodiment, please continue to refer to Fig.19The seventh acquisition unit 162 is specifically used for: performing target mapping of different sizes on the historical image feature vector and the historical prediction vector to obtain the distribution weights of targets of different sizes in the historical image feature vector; obtaining second target-oriented screening grids of different sizes according to the distribution weights of targets of different sizes in the historical image feature vector; filtering the historical image features using the second target-oriented screening grid to obtain filtered image features; processing the filtered image features according to dynamic convolution kernels of different kernel sizes to obtain target-oriented image features.

[0231] The target tracking device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, which will not be repeated here.

[0232] In one embodiment, please continue to refer to Fig.19 The above device further comprises: a tracking module 17, wherein:

[0233] The tracking module 17 is used to track each target object according to the prediction results of each target object in the multi-view image and a preset target trajectory; wherein the target trajectory is a trajectory formed by tracking each object in multiple historical prediction results.

[0234] The target tracking device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, which will not be repeated here.

[0235] In one embodiment, please continue to refer to Fig.19 The tracking module 17 includes: a first determining unit 171 and a first tracking unit 172, wherein:

[0236] The first determining unit 171 is configured to determine, according to the prediction results of each target object and the target trajectory, a first object in each prediction result that matches the target trajectory.

[0237] The first tracking unit 172 is used to track the first object according to the target trajectory.

[0238] The target tracking device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, which will not be repeated here.

[0239] In one embodiment, please continue to refer to Fig.19The above-mentioned first determination unit 171 is specifically used to: determine the first image feature vector, the first BEV feature vector and the first prediction result feature vector corresponding to the prediction result; determine the second image feature vector, the second BEV feature vector and the second prediction result feature vector corresponding to the target trajectory; obtain a first similarity matrix according to the first image feature vector and the second image feature vector; obtain a second similarity matrix according to the first BEV feature vector and the second BEV feature vector; obtain a third similarity matrix according to the first prediction result feature vector and the second prediction result feature vector; determine the first object according to the first similarity matrix, the second similarity matrix, the third similarity matrix and a preset matching algorithm.

[0240] The target tracking device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, which will not be repeated here.

[0241] In one embodiment, please continue to refer to Fig.19 The first determination unit 171 is further specifically used to: obtain a sum matrix of the first similarity matrix, the second similarity matrix and the third similarity matrix; use the sum matrix as a cost matrix of a preset matching algorithm, match each target object in the prediction result with the target trajectory, and determine the first object.

[0242] The target tracking device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, which will not be repeated here.

[0243] In one embodiment, Fig. 20 As shown, the tracking module 17 further includes: a second determining unit 173, a third determining unit 174 and a second tracking unit 175, wherein:

[0244] The second determining unit 173 is configured to determine, according to the prediction result and the target trajectory, a second object in the prediction result that does not match the target trajectory.

[0245] The third determining unit 174 is used to determine the second object bounding box and the bounding box of the object in the remaining target track; the remaining target track is the target track that does not match each prediction result.

[0246] The second tracking unit 175 is configured to track the second object according to the bounding box of the second object and the bounding boxes of the objects in the remaining target track.

[0247] The target tracking device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, which will not be repeated here.

[0248] In one embodiment, please continue to refer to Fig. 20The second tracking unit 175 is specifically used to: expand the bounding box of the second object and the bounding box of the object in the remaining target track according to the preset classification size, so as to obtain the expanded bounding box of the second object and the expanded bounding box of the object in the remaining target track; determine the intersection-and-union ratio of the expanded bounding box of the second object and the expanded bounding box of the object in the remaining target track; and track the second object according to the intersection-and-union ratio and the preset intersection-and-union ratio threshold.

[0249] The target tracking device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, which will not be repeated here.

[0250] In one embodiment, please continue to refer to Fig. 20 The above-mentioned second tracking unit 175 is also specifically used to: if the intersection-and-union ratio is greater than the intersection-and-union ratio threshold, track the second object according to the remaining target trajectory; if the intersection-and-union ratio is less than or equal to the intersection-and-union ratio threshold, use the position information corresponding to the second object to form a tracking trajectory corresponding to the second object.

[0251] The target tracking device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, which will not be repeated here.

[0252] For the specific definition of the target tracking device, please refer to the definition of the target tracking method above, which will not be repeated here. Each module in the above-mentioned target tracking device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0253] Fig.21 1 is a block diagram of a computer device 1300 according to an exemplary embodiment. Fig.21 , the computer device 1300 may include one or more of the following components: a processing component 1302, a memory 1304, a power component 1306, a multimedia component 1308, an audio component 1310, an input / output (I / O) interface 1312, a sensor component 1314, and a communication component 1316. The memory stores computer programs or instructions that run on the processor.

[0254] The processing component 1302 generally controls the overall operation of the computer device 1300, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 1302 may include one or more processors 1320 to execute instructions to complete all or part of the steps of the above method. In addition, the processing component 1302 may include one or more modules to facilitate the interaction between the processing component 1302 and other components. For example, the processing component 1302 may include a multimedia module to facilitate the interaction between the multimedia component 1308 and the processing component 1302.

[0255] The memory 1304 is configured to store various types of data to support the operation of the computer device 1300. Examples of such data include instructions for any application or method operating on the computer device 1300, contact data, phone book data, messages, pictures, videos, etc. The memory 1304 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0256] The power supply component 1306 provides power to the various components of the computer device 1300. The power supply component 1306 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the computer device 1300.

[0257] The multimedia component 1308 includes a touch screen that provides an output interface between the computer device 1300 and the user. In some embodiments, the touch screen may include a liquid crystal display (LCD) and a touch panel (TP). The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor can not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 1308 includes a front camera and / or a rear camera. When the computer device 1300 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera can receive external multimedia data. Each front camera and rear camera can be a fixed optical lens system or have a focal length and optical zoom capability.

[0258] The audio component 1310 is configured to output and / or input audio signals. For example, the audio component 1310 includes a microphone (MIC), and when the computer device 1300 is in an operation mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 1304 or sent via the communication component 1316. In some embodiments, the audio component 1310 also includes a speaker for outputting audio signals.

[0259] I / O interface 1312 provides an interface between processing component 1302 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.

[0260] The sensor assembly 1314 includes one or more sensors for providing various aspects of status assessment for the computer device 1300. For example, the sensor assembly 1314 can detect the open / closed state of the computer device 1300, the relative positioning of components, such as the display and keypad of the computer device 1300, and the sensor assembly 1314 can also detect the position change of the computer device 1300 or a component of the computer device 1300, the presence or absence of user contact with the computer device 1300, the orientation or acceleration / deceleration of the computer device 1300, and the temperature change of the computer device 1300. The sensor assembly 1314 may include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor assembly 1314 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 1314 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0261] The communication component 1316 is configured to facilitate wired or wireless communication between the computer device 1300 and other devices. The computer device 1300 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 1316 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 1316 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0262] In an exemplary embodiment, the computer device 1300 can be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above-mentioned target tracking method.

[0263] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 1304 including instructions, and the instructions can be executed by a processor 1320 of a computer device 1300 to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0264] In an exemplary embodiment, a storage medium including instructions is also provided, such as a memory 1304 including instructions, and the instructions can be executed by a processor 1320 of a computer device to perform the above method. The storage medium can be a non-transitory computer-readable storage medium, for example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0265] In an exemplary embodiment, a computer program product is also provided, and when the computer program is executed by a processor, the above method can be implemented. The computer program product includes one or more computer instructions. When these computer instructions are loaded and executed on a computer, part or all of the above method can be implemented in whole or in part according to the process or function described in the embodiment of the present disclosure.

[0266] It should be noted that the solutions described in this specification and the embodiments, if they involve the processing of personal information, will be processed on the premise of having a legal basis (such as obtaining the consent of the subject of personal information, or being necessary for the performance of a contract, etc.), and will only be processed within the scope of regulations or agreements. If a user refuses to process personal information other than the necessary information for basic functions, it will not affect the user's use of basic functions.

[0267] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in the embodiments of the present disclosure can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0268] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0269] The above-described embodiments only express several implementation methods of the embodiments of the present disclosure, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for those of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the embodiments of the present disclosure, and these all belong to the protection scope of the embodiments of the present disclosure. Therefore, the protection scope of the patent of the embodiments of the present disclosure shall be subject to the attached claims.

Claims

1. A target tracking method, characterized in that: The method comprises: Get the multi-view image at the current moment; The multi-view image is input into a prediction model, and the prediction model is used to call the target-oriented feature to predict the multi-view image, so as to obtain the prediction results of each target object in the multi-view image; the target-oriented feature includes a target-oriented bird's-eye view BEV feature; the target-oriented BEV feature is obtained by reversely optimizing the historical BEV feature according to the historical prediction results.

2. The method according to claim 1, characterized in that The target-oriented features also include target-oriented image features; the target-oriented image features are obtained by reversely optimizing historical image features according to the historical prediction results.

3. The method according to claim 2, characterized in that The prediction model includes a first processing network, a second processing network and a classification network; the multi-view image is input into the prediction model, and the prediction result of each target object in the multi-view image is obtained by calling the target-oriented feature of the prediction model, including: Obtaining initial BEV features of the multi-view image through the first processing network, the multi-view image and the called target-oriented image features; Inputting the initial BEV feature and the called target-oriented BEV feature into the second processing network to obtain the target BEV feature of the multi-view image; The target BEV features are input into the classification network to obtain prediction results of each target object in the multi-view image.

4. The method according to claim 3, characterized in that The first processing network includes a backbone network, a fusion network and a mapping network. The initial BEV features of the multi-view image are obtained through the first processing network, the multi-view image and the called target-oriented image features, including: Inputting the multi-view image into the backbone network to obtain initial image features of the multi-view image; Inputting the initial image features and the called target-oriented image features into the fusion network to obtain fused image features; The fused image features are input into the mapping network to obtain the initial BEV features.

5. The method according to claim 1, characterized in that The method for acquiring the target-oriented BEV characteristics includes: Performing center mapping on the historical prediction results to obtain historical prediction vectors corresponding to the historical prediction results; The goal-oriented BEV feature is obtained according to the historical BEV feature and the historical prediction vector.

6. The method according to claim 5, characterized in that The step of acquiring the target-oriented BEV feature according to the historical BEV feature and the historical prediction vector includes: Performing center mapping on the historical BEV feature to obtain a historical BEV feature vector corresponding to the historical BEV feature; The goal-oriented BEV feature is obtained according to the historical BEV feature vector, the historical BEV feature and the historical prediction vector.

7. The method according to claim 6, characterized in that The step of acquiring the target-oriented BEV feature according to the historical BEV feature vector, the historical BEV feature and the historical prediction vector includes: Performing target mapping of different sizes on the historical BEV feature vector and the historical prediction vector to obtain distribution weights of targets of different sizes in the historical BEV feature vector; acquiring first target-oriented screening grids of different sizes according to distribution weights of targets of different sizes in the historical BEV feature vector; Using the first target-oriented screening grid to filter out the historical BEV features to obtain screened BEV features; The screened BEV features are processed according to dynamic convolution kernels with different kernel sizes to obtain the target-oriented BEV features.

8. The method according to claim 2, characterized in that: The method for acquiring the target-oriented image features comprises: Performing center mapping on the historical prediction results to obtain historical prediction vectors corresponding to the historical prediction results; The target-oriented image feature is obtained according to the historical image feature and the historical prediction vector.

9. The method according to claim 8, characterized in that The step of acquiring the target-oriented image feature according to the historical image feature and the historical prediction vector includes: Performing center mapping on the historical image feature to obtain a historical image feature vector corresponding to the historical image feature; The target-oriented image feature is obtained according to the historical image feature vector, the historical image feature and the historical prediction vector.

10. The method according to claim 9, characterized in that The step of acquiring the target-oriented image feature according to the historical image feature vector, the historical image feature and the historical prediction vector comprises: Performing object mapping of different sizes on the historical image feature vector and the historical prediction vector to obtain distribution weights of objects of different sizes in the historical image feature vector; acquiring second target-oriented screening grids of different sizes according to distribution weights of targets of different sizes in the historical image feature vector; Using the second target-oriented screening grid to filter out the historical image features to obtain screened image features; The filtered image features are processed according to dynamic convolution kernels with different kernel sizes to obtain the target-oriented image features.

11. The method according to any one of claims 1 to 10, characterized in that: The method further comprises: According to the prediction results of each target object in the multi-view image and the preset target trajectory, each target object is tracked; wherein the target trajectory is a trajectory formed by tracking each object in a plurality of historical prediction results.

12. The method according to claim 11, characterized in that The step of tracking each target object according to the prediction result of each target object in the multi-view image and the preset target trajectory includes: Determining, according to the prediction results of each of the target objects and the target trajectory, a first object in each of the prediction results that matches the target trajectory; The first object is tracked according to the target trajectory.

13. The method according to claim 12, characterized in that The step of determining, based on the prediction results of each of the target objects and the target trajectory, a first object in the prediction results that matches the target trajectory comprises: Determine a first image feature vector, a first BEV feature vector, and a first prediction result feature vector corresponding to the prediction result; Determine a second image feature vector, a second BEV feature vector, and a second prediction result feature vector corresponding to the target trajectory; Obtaining a first similarity matrix according to the first image feature vector and the second image feature vector; Obtaining a second similarity matrix according to the first BEV feature vector and the second BEV feature vector; Obtaining a third similarity matrix according to the first prediction result feature vector and the second prediction result feature vector; The first object is determined according to the first similarity matrix, the second similarity matrix, the third similarity matrix and a preset matching algorithm.

14. The method according to claim 13, characterized in that The determining the first object according to the first similarity matrix, the second similarity matrix, the third similarity matrix and a preset matching algorithm includes: Obtaining a sum matrix of the first similarity matrix, the second similarity matrix, and the third similarity matrix; The sum matrix is ​​used as a cost matrix of a preset matching algorithm, and each target object in the prediction result is matched with the target trajectory to determine the first object.

15. The method according to claim 11, characterized in that The step of tracking each target object according to the prediction result of each target object in the multi-view image and the preset target trajectory includes: Determining, according to the prediction result and the target trajectory, a second object in the prediction result that does not match the target trajectory; Determine the second object bounding box and the bounding box of the object in the remaining target track; the remaining target track is the target track that does not match each of the prediction results; The second object is tracked according to the bounding box of the second object and the bounding boxes of the objects in the remaining target track.

16. The method according to claim 15, characterized in that Tracking the second object according to the bounding box of the second object and the bounding boxes of the objects in the remaining target track includes: Expanding the bounding box of the second object and the bounding box of the object in the remaining target track according to a preset classification size to obtain an expanded bounding box of the second object and an expanded bounding box of the object in the remaining target track; Determining an intersection-over-union ratio of the expanded bounding box of the second object and the expanded bounding boxes of the objects in the remaining target tracks; The second object is tracked according to the IoU ratio and a preset IoU ratio threshold.

17. The method according to claim 16, characterized in that Tracking the second object according to the IoU ratio and a preset IoU ratio threshold includes: If the intersection-over-union ratio is greater than the intersection-over-union ratio threshold, tracking the second object according to the remaining target trajectory; If the IoU is less than or equal to the IoU threshold, the tracking trajectory corresponding to the second object is formed by using the position information corresponding to the second object.

18. A target tracking device, characterized in that: The device comprises: A first acquisition module is used to acquire a multi-view image at the current moment; The second acquisition module is used to input the multi-view image into the prediction model, use the prediction model to call the target-oriented feature to predict the multi-view image, and obtain the prediction results of each target object in the multi-view image; the target-oriented feature includes the target-oriented bird's-eye view BEV feature; the target-oriented BEV feature is obtained by reversely optimizing the historical BEV feature according to the historical prediction results.

19. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 17 are implemented.

20. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 17 are implemented.

21. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 17 are implemented.

Images