Training method of target tracking model, target tracking method and device

Abstract

The application discloses a target tracking model training method, a target tracking method and a device. The target tracking model comprises a target feature vector extraction submodel and a target object classification submodel. The training method comprises the following steps: acquiring a training sample set comprising multiple groups of training samples, each group of training samples comprising two sample images, and the same object in the two sample images having a mapping label; inputting the training sample set into an initial feature vector extraction submodel to extract initial feature vectors of objects contained in each sample image; adjusting model parameters of the initial feature vector extraction submodel according to the mapping label until a first convergence condition is met, thereby obtaining the target feature vector extraction submodel and target feature vectors of the objects contained in each sample image output by the target feature vector extraction submodel; and inputting the target feature vectors corresponding to each group of training samples in the training sample set into an initial object classification submodel for classification training, thereby obtaining the target object classification submodel.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and more specifically, to a training method for a target tracking model, a target tracking method, and an apparatus. Background Technology

[0002] Visual object tracking is an important research area in computer vision, with wide applications such as video surveillance, human-computer interaction, and autonomous driving. In autonomous driving, visual object tracking can detect and track obstacles (such as pedestrians and other vehicles) ahead, determine their behavioral intentions, and thus plan appropriate paths.

[0003] Current methods for visual object tracking mainly include using the IoU (Intersection over Union) algorithm to calculate the overlap between objects in the previous frame and objects in the current frame. If the overlap is greater than or equal to a target overlap threshold, the objects are identified as the same; otherwise, they are identified as different objects. However, in practical applications, the pose and field of view of the same object may differ in images captured at different times. Therefore, when the differences are significant, the IoU algorithm may mistakenly classify them as different objects. Furthermore, since the IoU algorithm is a rule-based tracking algorithm, and the target detection model involved in object tracking is a neural network model, the coupling between the IoU algorithm and the target detection model is relatively poor.

[0004] Therefore, it is clear that improving the accuracy of target tracking while enhancing coupling is an urgent problem to be solved. Summary of the Invention

[0005] This application provides a training method for a target tracking model, a target tracking method, and an apparatus that can solve the problems of poor coupling between the IoU algorithm and the target detection model, and low target tracking accuracy.

[0006] The specific technical solution is as follows:

[0007] In a first aspect, embodiments of this application provide a method for training a target tracking model, the target tracking model including a target feature vector extraction sub-model and a target object classification sub-model, the method comprising:

[0008] Obtain a training sample set, which includes multiple sets of training samples. Each set of training samples includes two frames of sample images, and the same objects in the two frames of sample images have mapping labels.

[0009] The training sample set is input into the initial feature vector extraction sub-model to extract the initial feature vector of the object contained in each frame of the sample image;

[0010] The model parameters of the initial feature vector extraction sub-model are adjusted according to the mapping annotation until the first convergence condition is met, and the target feature vector extraction sub-model and the target feature vector of the object contained in each frame of the sample image output by the target feature vector extraction sub-model are obtained. The first convergence condition includes that the similarity of the feature vectors of the same object in each group of training samples is greater than or equal to the target similarity threshold.

[0011] The target feature vector corresponding to each group of training samples in the training sample set is input into the initial object classification sub-model for classification training to obtain the target object classification sub-model. The target object classification sub-model is used to calculate the probability that two objects located in different sample images in each group of training samples are the same.

[0012] As can be seen from the above scheme, the embodiments of this application can first train a target feature vector extraction sub-model and the target feature vector of the object contained in each frame sample image output by the target feature vector extraction sub-model based on multiple sets of training samples including the same object mapping annotation and the first convergence condition. Then, classification training is performed based on the target feature vector to obtain the target object classification sub-model. This allows subsequent target tracking to be performed based on the target tracking model including the target feature vector extraction sub-model and the target object classification sub-model. This makes both target tracking and target detection use neural network models, thereby improving the coupling between the target tracking algorithm and the target detection algorithm. Furthermore, since the target tracking model is not affected by the pose and field of view of the object in the image, the target tracking model can improve the accuracy of target tracking.

[0013] In a first possible implementation of the first aspect, obtaining the training sample set includes:

[0014] Acquire consecutive multi-frame sample images;

[0015] Based on the object detection model, object detection is performed on each frame of the sample image to determine the objects contained in each frame of the sample image;

[0016] Based on the overlap algorithm and the bipartite graph algorithm, the objects contained in each two adjacent sample images in the continuous multi-frame sample images are matched, and the objects with the same matching result are mapped and labeled.

[0017] The training sample set is obtained by taking each pair of adjacent sample images with added mapping annotations as a set of training samples.

[0018] As can be seen from the above scheme, the embodiments of this application can first automatically detect the objects contained in each frame of sample image based on the target detection model, and then match the objects contained in each two adjacent frames of sample images in a series of consecutive sample images through the overlap algorithm and the bipartite graph algorithm, so as to realize the automatic mapping and labeling of adjacent two frames of sample images without manual labeling, thereby improving the efficiency of obtaining the training sample set.

[0019] In a second possible implementation of the first aspect, before obtaining the training sample set after taking each of the adjacent two-frame sample images with added mapping annotations as a set of training samples, the method further includes:

[0020] If it is determined that the target type exists among the object types contained in the two adjacent sample images, a manual review prompt message is output. The manual review prompt message is used to prompt whether the mapping label of the object of the target type is correct, or to prompt whether the mapping label of the two adjacent sample images is correct.

[0021] If there is an error in the review, receive the mapping annotation correction operation; if there is no error in the review, receive the confirmation instruction.

[0022] As can be seen from the above scheme, the embodiments of this application can, on the basis of automatic annotation, output manual review prompts to allow manual review of whether the mapping annotation of the target type of object is correct, or to review whether the mapping annotation in two adjacent sample images containing the target type of object is correct. In this way, while improving the efficiency of mapping annotation, the accuracy of mapping annotation can be further ensured, especially the accuracy of the mapping annotation of the target type of focus.

[0023] In a third possible implementation of the first aspect, the step of performing target detection on each frame of the sample image based on the target detection model to determine the objects contained in each frame of the sample image includes:

[0024] For each frame of the sample image, multiple convolution results of the sample image are obtained from multiple convolution channels in the target detection model. Different convolution channels are used to calculate different attribute information of each object. The convolution result is a value that transforms the true value of the attribute information into a value within a target numerical range. The multiple convolution results are located within the same target numerical range. The target numerical range is matched with the shared stride of the multiple convolution results.

[0025] The multiple convolution results are merged based on the shared stride to obtain an initial target detection result;

[0026] Based on a preset mapping function that satisfies the target error range, the final target detection result corresponding to the initial target detection result is calculated, and the final target detection result includes the true value of each attribute information of each object;

[0027] The objects contained in the sample image are determined based on the final target detection results.

[0028] As can be seen from the above scheme, compared with related technologies where the output ranges of the quantized attribute values ​​differ through different convolution channels, and they share the same stride, resulting in unreasonable strides used by different convolution channels and thus inaccurate target detection results, the embodiments of this application can first convert the true values ​​of different attribute information output by different convolution channels to the same target numerical range, and this target numerical range matches the shared stride of multiple convolution results. Then, based on the shared stride, the values ​​of different attributes in multiple target numerical ranges are merged to obtain an initial target detection result. Finally, the final target detection result, including the true values ​​of each attribute information of each object, is determined according to a preset mapping function that satisfies the target error range. Based on the final target detection result, the objects contained in the sample image are determined. This not only enables different convolution channels to share the same stride to overcome the unreasonable problem, but also ensures that the true values ​​of each attribute information obtained in the end are within the target error range, thereby improving the accuracy of the target detection result.

[0029] In a fourth possible implementation of the first aspect, when the target numerical range is [-10, 10] and the target error range is dy ≤ max(0.1, 0.01y), the preset mapping function includes:

[0030]

[0031] Wherein, y represents the true value of each attribute information, x represents the convolution result, and a, b, c, and d represent different fixed values.

[0032] In the fifth possible implementation of the first aspect, adjusting the model parameters of the initial feature vector extraction sub-model according to the mapping annotation includes:

[0033] The model parameters of the initial feature vector extraction sub-model are adjusted based on the push-pull loss function and the mapping label to increase the similarity of feature vectors of the same objects in each group of training samples and decrease the similarity of feature vectors of different objects in each group of training samples.

[0034] Secondly, embodiments of this application provide a target tracking method, the method comprising:

[0035] The first target image and the second target image are respectively input into the target feature vector extraction sub-model in the target tracking model to obtain the first feature vector of the first object contained in the first target image and the second feature vector of the second object contained in the second target image. The target tracking model is trained according to the method described in any possible implementation of the first aspect.

[0036] Input the first feature vector and the second feature vector into the target object classification sub-model in the target tracking model to obtain the target probability that the first object and the second object are the same object;

[0037] Based on the target probability and a preset probability threshold, determine whether the first object and the second object are the same object to obtain the target tracking result.

[0038] As can be seen from the above scheme, after obtaining the target tracking model, the first target image and the second target image can be input into the target feature vector extraction sub-model of the target tracking model to obtain the first feature vector of the first object contained in the first target image and the second feature vector of the second object contained in the second target image. Then, the first feature vector and the second feature vector are input into the target object classification sub-model of the target tracking model to obtain the target probability that the first object and the second object are the same object. Finally, the target probability and the preset probability threshold are used to determine whether the first object and the second object are the same object. It can be seen that the tracking algorithm provided by this application is based on a neural network model, which has a high coupling with the target detection model. Since the target tracking model is not affected by the pose and field of view of the object in the image, the target tracking model can improve the accuracy of target tracking.

[0039] In a first possible implementation of the second aspect, when the target tracking result shows that the same first object is identical to multiple second objects, and / or the same second object is identical to multiple first objects, after determining whether the first object and the second object are the same object based on the target probability and a preset probability threshold to obtain the target tracking result, the method further includes:

[0040] The first object and the second object in the target tracking result are matched using a bipartite graph algorithm to obtain the final target tracking result. The final target tracking result includes that each first object is identical to at most one second object, and each second object is identical to at most one first object.

[0041] As can be seen from the above solution, in the embodiments of the present application, when there are the same first object and multiple second objects in the target tracking result, and / or when there are the same second object and multiple first objects, the bipartite graph algorithm is used to match the first object and the second object in the target tracking result, so that each first object is the same as at most one second object, and each second object is the same as at most one first object, thereby avoiding unrealistic target tracking results and improving the accuracy of the target tracking result.

[0042] In the second possible implementation manner of the second aspect, determining whether the first object and the second object are the same object according to the target probability and the preset probability threshold includes:

[0043] When the preset probability threshold is one and the target probability is greater than or equal to the preset probability threshold, determining that the first object and the second object are the same object; or,

[0044] When the preset probability threshold is multiple and the target probability satisfies a preset formula, determining that the first object and the second object are the same object,

[0045] The preset formula includes:

[0046] (bd_score>a1&&track_time<b1)||(bd_score>a2&&b1≤track_time<b2)||(bd_score>a3&&track_time≥b3)

[0047] where a1 > a2 > a3 and b1 < b2 < b3, the bd_score represents the target probability, the track_time represents the number of frames for continuously tracking the same object, a1, a2, and a3 respectively represent different preset probability thresholds, and b1, b2, and b3 respectively represent different frame number thresholds.

[0048] In a third aspect, an embodiment of the present application provides a training device for a target tracking model. The target tracking model includes a target feature vector extraction sub-model and a target object classification sub-model. The device includes:

[0049] An acquisition unit, configured to acquire a training sample set, where the training sample set includes multiple groups of training samples, each group of training samples includes two-frame sample images, and the same object in the two-frame sample images has a mapping annotation;

[0050] An extraction unit, configured to input the training sample set into an initial feature vector extraction sub-model, and extract the initial feature vectors of the objects included in each frame of the sample images;

[0051] The adjustment unit is used to adjust the model parameters of the initial feature vector extraction sub-model according to the mapping label until the first convergence condition is met, so as to obtain the target feature vector extraction sub-model and the target feature vector of the object contained in each frame of the sample image output by the target feature vector extraction sub-model. The first convergence condition includes the feature vector similarity of the same object in each group of training samples being greater than or equal to the target similarity threshold.

[0052] The classification training unit is used to input the target feature vector corresponding to each group of training samples in the training sample set into the initial object classification sub-model for classification training to obtain the target object classification sub-model. The target object classification sub-model is used to calculate the probability that two objects located in different sample images in each group of training samples are the same.

[0053] In a first possible implementation of the third aspect, the acquiring unit includes:

[0054] The acquisition module is used to acquire multiple consecutive frames of sample images;

[0055] The object detection module is used to perform object detection on each frame of the sample image based on the object detection model to determine the objects contained in each frame of the sample image;

[0056] The matching and labeling module is used to match objects contained in every two adjacent sample images in the continuous multi-frame sample images based on the overlap algorithm and the bipartite graph algorithm, and to map and label objects that have the same matching result.

[0057] The determination module is used to obtain the training sample set by taking each pair of adjacent sample images with added mapping annotations as a set of training samples.

[0058] In a second possible implementation of the third aspect, the acquisition unit further includes:

[0059] The output module is used to output manual review prompt information when it is determined that there is a target type among the object types contained in the adjacent two-frame sample images after the respective adjacent two-frame sample images with added mapping labels are used as a set of training samples and before obtaining the training sample set. The manual review prompt information is used to prompt whether the mapping label of the target type of object is correct, or to prompt whether the mapping label of the adjacent two-frame sample images is correct.

[0060] The receiving module is used to receive mapping annotation correction operations if there are errors in the review, and to receive confirmation instructions if there are no errors in the review.

[0061] In a third possible implementation of the third aspect, the target detection module is configured to, for each frame of the sample image, acquire multiple convolution results of multiple convolution channels in the target detection model on the sample image, wherein different convolution channels are used to calculate different attribute information of each object, the convolution result is a value that transforms the true value of the attribute information to a target numerical range, the multiple convolution results are located within the same target numerical range, and the target numerical range matches the shared stride of the multiple convolution results; merge the multiple convolution results based on the shared stride to obtain an initial target detection result; calculate the final target detection result corresponding to the initial target detection result according to a preset mapping function that satisfies the target error range, the final target detection result including the true value of each attribute information of each object; and determine the objects contained in the sample image based on the final target detection result.

[0062] In a fourth possible implementation of the third aspect, when the target numerical range is [-10, 10] and the target error range is dy ≤ max(0.1, 0.01y), the preset mapping function includes:

[0063]

[0064] Wherein, y represents the true value of each attribute information, x represents the convolution result, and a, b, c, and d represent different fixed values.

[0065] In a fifth possible implementation of the third aspect, the adjustment unit is configured to adjust the model parameters of the initial feature vector extraction sub-model according to the push-pull loss function and the mapping label, so as to increase the similarity of the feature vectors of the same objects in each group of training samples and decrease the similarity of the feature vectors of different objects in each group of training samples.

[0066] The target tracking model training device provided in this application embodiment can first train a target feature vector extraction sub-model and the target feature vector of the object contained in each frame sample image output by the target feature vector extraction sub-model based on multiple sets of training samples including the same object mapping labels and a first convergence condition. Then, it performs classification training based on the target feature vector to obtain a target object classification sub-model. This allows subsequent target tracking to be performed based on the target tracking model including the target feature vector extraction sub-model and the target object classification sub-model. This enables both target tracking and target detection to use neural network models, thereby improving the coupling between the target tracking algorithm and the target detection algorithm. Furthermore, since the target tracking model is not affected by the pose and field of view of the object in the image, this target tracking model can improve the accuracy of target tracking.

[0067] Fourthly, embodiments of this application provide a target tracking device, the device comprising:

[0068] An extraction unit is used to input the first target image and the second target image into the target feature vector extraction sub-model of the target tracking model, respectively, to obtain the first feature vector of the first object contained in the first target image and the second feature vector of the second object contained in the second target image. The target tracking model is trained according to the method described in any possible implementation of the first aspect.

[0069] A classification unit is used to input the first feature vector and the second feature vector into the target object classification sub-model in the target tracking model to obtain the target probability that the first object and the second object are the same object.

[0070] The determining unit is used to determine whether the first object and the second object are the same object based on the target probability and a preset probability threshold, so as to obtain the target tracking result.

[0071] In a first possible implementation of the fourth aspect, the device further includes:

[0072] A matching unit is configured to, when the target tracking result contains the same first object and multiple second objects, and / or the same second object and multiple first objects, after determining whether the first object and the second object are the same object based on the target probability and a preset probability threshold to obtain the target tracking result, use a bipartite graph algorithm to match the first object and the second object in the target tracking result to obtain the final target tracking result. The final target tracking result includes that each first object is at most the same as one second object, and each second object is at most the same as one first object.

[0073] In a second possible implementation of the fourth aspect, the determining unit includes: a first determining module or a second determining module;

[0074] The first determining module is configured to determine that the first object and the second object are the same object when the preset probability threshold is one and the target probability is greater than or equal to the preset probability threshold; or,

[0075] The second determining module is used to determine that the first object and the second object are the same object when there are multiple preset probability thresholds and the target probability satisfies a preset formula.

[0076] The preset formula includes:

[0077] (bd_score > a1 && track_time < b1) || (bd_score > a2 && b1 ≤ track_time < b2) || (bd_score > a3 && track_time ≥ b3)

[0078] Among them, a1 > a2 > a3 and b1 < b2 < b3, where the bd_score represents the target probability, the track_time represents the number of frames for continuously tracking the same object, a1, a2, and a3 respectively represent different preset probability thresholds, and b1, b2, and b3 respectively represent different frame number thresholds.

[0079] After obtaining the target tracking model, the target tracking device provided by the embodiments of the present application can first input the first target image and the second target image into the target feature vector extraction sub-model in the target tracking model respectively to obtain the first feature vector of the first object included in the first target image and the second feature vector of the second object included in the second target image, and then input the first feature vector and the second feature vector into the target object classification sub-model in the target tracking model to obtain the target probability that the first object and the second object are the same object. Finally, according to the target probability and the preset probability threshold, it is determined whether the first object and the second object are the same object. It can be seen that the tracking algorithm provided by the embodiments of the present application is an algorithm based on a neural network model, which has a high coupling with the target detection model. And because the target tracking model is not affected by the pose and field of view of the objects in the image, the target tracking model can improve the accuracy of target tracking.

[0080] In a fifth aspect, the embodiments of the present application provide a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the method described in any possible implementation manner of the first aspect or the second aspect is implemented.

[0081] In a sixth aspect, the embodiments of the present application provide an electronic device, which includes:

[0082] One or more processors;

[0083] The processor is coupled to the storage device, and the storage device is used to store one or more programs;

[0084] When one or more programs are executed by one or more processors, the electronic device implements the method described in any possible implementation manner of the first aspect or the second aspect.

[0085] In a seventh aspect, the embodiments of the present application provide a vehicle, which includes the device described in any possible implementation manner of the third aspect or the fourth aspect, or includes the electronic device described in the sixth aspect.

[0086] Eighthly, embodiments of this application provide a computer program product containing instructions that, when executed on a computer or processor, cause the computer or processor to perform the method described in any possible implementation of the first aspect. Attached Figure Description

[0087] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0088] Figure 1 A flowchart illustrating a training method for a target tracking model provided in an embodiment of this application;

[0089] Figure 2 An example curve of a preset mapping function provided in this application embodiment;

[0090] Figure 3 A flowchart illustrating a target tracking method provided in an embodiment of this application;

[0091] Figure 4 A block diagram illustrating the composition of a training apparatus for a target tracking model provided in this application embodiment;

[0092] Figure 5 A block diagram of a target tracking device provided in an embodiment of this application;

[0093] Figure 6 This is a schematic diagram of the structure of an electronic device or computer device provided in an embodiment of this application. Detailed Implementation

[0094] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0095] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. The terms "comprising" and "having," and any variations thereof, in the embodiments and drawings of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices.

[0096] Figure 1 This is a flowchart illustrating a training method for a target tracking model provided in an embodiment of this application. This method can be applied to electronic devices or computer equipment, specifically to vehicles or servers, and may include the following steps:

[0097] S110: Obtain the training sample set.

[0098] The training sample set includes multiple sets of training samples, each set of training samples includes two frames of sample images, and the same objects in the two frames of sample images have mapping labels. This application embodiment does not limit the presentation method of the mapping labels, as long as it can represent the mapping relationship between the same objects in the two frames of sample images. For example, a line can be used to connect the same objects located in the two frames of sample images, such as connecting pedestrian 1 in sample image 1 and pedestrian 1 in sample image 2 with a line.

[0099] This application embodiment allows for manual or automatic mapping and annotation of multiple sample images. The automatic annotation method can be as described in steps A1-A4:

[0100] A1. Obtain multiple consecutive frames of sample images.

[0101] The sample image may come from one vehicle or multiple vehicles. It may come from the same vehicle as the first target image and the second target image in the following embodiments, or it may come from different vehicles.

[0102] A2. Based on the object detection model, perform object detection on each frame of sample image to determine the objects contained in each frame of sample image.

[0103] In related technologies, in the process of identifying depth information (i.e., the depth of the object from the camera) and object size and other attribute information in an image based on an object detection model, in order to improve computational efficiency, int8 quantization is performed, for example, quantizing float32 to int8, so that the output attribute values ​​are in the range of 0-256 or -128-127. However, since the output range of different convolution channels is different in reality and they share the same stride, some convolution channels may use an unreasonable stride, which will lead to a large error in the output attribute values, resulting in a large error in the final object detection result.

[0104] To address this technical problem, in this embodiment, when performing target detection on each frame of sample images based on the target detection model, multiple convolution results of the sample images from multiple convolution channels in the target detection model can be obtained for each frame of sample images. Different convolution channels are used to calculate different attribute information for each object, and the convolution result is a value that transforms the true value of the attribute information to a value within a target numerical range. Multiple convolution results lie within the same target numerical range, and the target numerical range matches the shared stride of the multiple convolution results. The multiple convolution results are merged based on the shared stride to obtain an initial target detection result. A final target detection result corresponding to the initial target detection result is calculated according to a preset mapping function that satisfies the target error range. The final target detection result includes the true values ​​of each attribute information of each object. The objects contained in the sample images are determined based on the final target detection result.

[0105] Attribute information includes depth information, object size, etc. The preset mapping function is a continuously differentiable mapping function. When the target value range is [-10, 10], and the target error range is dy ≤ max(0.1, 0.01y), the preset mapping function includes:

[0106]

[0107] Where y represents the true value of each attribute, x represents the convolution result, and a, b, c, and d represent different fixed values.

[0108] Different attribute information can correspond to different preset mapping functions. For example, the preset mapping function for depth information can have values ​​of a = 10, b = 23.02585, c = 6.3, and d = 18.12377, respectively. The curve of this preset mapping function can be shown as follows: Figure 2 As shown in the figure, the horizontal axis represents x in the preset mapping function, and the vertical axis represents y in the preset mapping function.

[0109] The target error range of dy≤max(0.1,0.01y) refers to an error of 0.1m or 0.01y. When very close to the target object, the first segment of the preset mapping function can be used to ensure an error range of 0.1m. When relatively close to the target object, the second segment of the preset mapping function can be used to ensure an error range of 0.01y. When relatively far from the target object, the error requirement can be relaxed, and the third segment of the preset mapping function can be used to satisfy the error range of 0.1y.

[0110] As can be seen from the above scheme, compared with related technologies where the output ranges of the quantized attribute values ​​differ through different convolution channels, and they share the same stride, resulting in unreasonable strides used by different convolution channels and thus inaccurate target detection results, the embodiments of this application can first convert the true values ​​of different attribute information output by different convolution channels to the same target numerical range, and this target numerical range matches the shared stride of multiple convolution results. Then, based on the shared stride, the values ​​of different attributes in multiple target numerical ranges are merged to obtain an initial target detection result. Finally, the final target detection result, including the true values ​​of each attribute information of each object, is determined according to a preset mapping function that satisfies the target error range. Based on the final target detection result, the objects contained in the sample image are determined. This not only enables different convolution channels to share the same stride to overcome the unreasonable problem, but also ensures that the true values ​​of each attribute information obtained in the end are within the target error range, thereby improving the accuracy of the target detection result.

[0111] A3. Based on the overlap algorithm and the bipartite graph algorithm, the objects contained in each two adjacent sample images in a series of consecutive sample images are matched, and the objects with the same matching result are mapped and labeled.

[0112] First, the IoU algorithm is used to calculate the overlap between the first object in the first sample image and the second object in the second sample image, respectively. When the overlap is greater than or equal to the target overlap threshold, it is preliminarily determined that the first object and the second object match, that is, it is preliminarily determined that they are the same. When there is one first object and multiple second objects in two adjacent sample images, or one second object and multiple first objects, a bipartite graph algorithm can be used to further match the first object and the second object. At most one second object is selected from the multiple second objects that match the first object to determine that it is the same as the first object, or at most one first object is selected from the multiple first objects that match the second object to determine that it is the same as the second object. The same objects are then mapped and labeled. This bipartite graph algorithm includes, but is not limited to, the Hungarian algorithm.

[0113] A4. After taking each pair of adjacent sample images with added mapping annotations as a set of training samples, the training sample set is obtained.

[0114] To improve mapping and annotation efficiency, and to further ensure the accuracy of mapping and annotation, especially for key target categories, after using each pair of adjacent sample images with added mapping and annotation as a training set, before obtaining the training sample set, a manual review prompt can be output if the target category is found among the object categories contained in the adjacent sample images. This prompt is used to check whether the mapping and annotation of the target category is correct, or to check whether the mapping and annotation of the adjacent sample images is correct. If an error is found, a mapping and annotation correction operation is received; if no error is found, a confirmation instruction is received. Target categories include vehicles, pedestrians, and other categories that drivers pay close attention to during driving.

[0115] It should be added that when the embodiments of this application are applied to vehicles, manual review prompts can be output on the vehicle's central control screen, and the mapping labels can be corrected through human-computer interaction. Alternatively, the vehicle can send the manual review prompts to a mobile terminal (such as a user's mobile phone) and output the manual review prompts on the mobile terminal, allowing the user to perform mapping label correction operations on the mobile terminal.

[0116] S120: Input the training sample set into the initial feature vector extraction sub-model to extract the initial feature vector of the object contained in each frame of sample image.

[0117] The initial feature vector and the target feature vector can specifically be embedding features.

[0118] S130: Adjust the model parameters of the initial feature vector extraction sub-model according to the mapping annotation until the first convergence condition is met, and obtain the target feature vector extraction sub-model and the target feature vector of the object contained in each frame sample image output by the target feature vector extraction sub-model.

[0119] The first convergence criterion is that the similarity of feature vectors of the same objects in each training sample group is greater than or equal to the target similarity threshold. The target similarity threshold can be determined based on practical experience.

[0120] The specific implementation method for adjusting the model parameters of the initial feature vector extraction sub-model based on the mapping annotation includes: adjusting the model parameters of the initial feature vector extraction sub-model based on the push-pull loss function and the mapping annotation to increase the similarity of feature vectors of the same object in each training sample and reduce the similarity of feature vectors of different objects in each training sample, so that the feature vectors of the same object output by the target feature vector extraction sub-model have higher similarity and the feature vectors of different objects have lower similarity.

[0121] S140: Input the target feature vector corresponding to each group of training samples in the training sample set into the initial object classification sub-model for classification training to obtain the target object classification sub-model.

[0122] The target object classification sub-model is used to calculate the probability that two objects in different sample images in each training sample are the same, and the neural network used by the target object classification sub-model can be a fully connected network.

[0123] When inputting the target feature vectors corresponding to each group of training samples in the training sample set into the initial object classification sub-model for classification training, the target feature vectors of the same object can be given a probability label of 1, and the target feature vectors of different objects can be given a probability label of 0. Training continues until the second convergence condition is met, at which point the target object classification sub-model can be obtained. The second convergence condition can be determined by whether the loss function, determined by the difference between the probability that the currently trained object classification sub-model identifies two objects as the same and the true value, is less than or equal to a preset loss threshold.

[0124] The target tracking model training method provided in this application embodiment can first train a target feature vector extraction sub-model and the target feature vector of the object contained in each frame sample image output by the target feature vector extraction sub-model based on multiple sets of training samples including the same object mapping label and a first convergence condition. Then, classification training is performed based on the target feature vector to obtain a target object classification sub-model. This allows subsequent target tracking to be performed based on the target tracking model including the target feature vector extraction sub-model and the target object classification sub-model. This makes both target tracking and target detection use neural network models, thereby improving the coupling between the target tracking algorithm and the target detection algorithm. Furthermore, since the target tracking model is not affected by the pose and field of view of the object in the image, this target tracking model can improve the accuracy of target tracking.

[0125] Based on the above method embodiments, another embodiment of this application provides a target tracking method, such as... Figure 3 As shown, the method includes:

[0126] S210: Input the first target image and the second target image into the target feature vector extraction sub-model in the target tracking model respectively, to obtain the first feature vector of the first object included in the first target image and the second feature vector of the second object included in the second target image.

[0127] The target tracking model is trained according to the method of the training method embodiment of the above target tracking model. The first target image includes at least one first object, and the second target image includes at least one second object. The first object and the second object are only used to distinguish the objects located in different target images and do not represent the order. The first feature vector and the second feature vector can be embedding feature or other vectors.

[0128] S220: Input the first feature vector and the second feature vector into the target object classification sub-model in the target tracking model, to obtain the target probability that the first object and the second object are the same object.

[0129] The neural network used by the target object classification sub-model can be a fully connected network.

[0130] S230: Determine whether the first object and the second object are the same object according to the target probability and the preset probability threshold, to obtain the target tracking result.

[0131] When the preset probability threshold is one and the target probability is greater than or equal to the preset probability threshold, determine that the first object and the second object are the same object; or,

[0132] When the preset probability threshold is multiple and the target probability satisfies the preset formula, determine that the first object and the second object are the same object.

[0133] The preset formula includes:

[0134] (bd_score > a1 && track_time < b1) || (bd_score > a2 && b1 ≤ track_time < b2) || (bd_score > a3 && track_time ≥ b3)

[0135] Where, a1 > a2 > a3 and b1 < b2 < b3, bd_score represents the target probability, track_time represents the number of frames of continuously tracking the same object, a1, a2 and a3 respectively represent different preset probability thresholds, and b1, b2 and b3 respectively represent different frame number thresholds. For example, a1 = 0.9, b1 = 3, a2 = 0.5, b2 = 8, a3 = 0.3, b3 = 8.

[0136] It should be added that when there is only one preset probability threshold and the target probability is less than the preset probability threshold, the first object and the second object are determined to be different objects; when there are multiple preset probability thresholds and the target probability does not satisfy the preset formula, the first object and the second object are determined to be different objects.

[0137] The target tracking method provided in this application, after obtaining the target tracking model, firstly inputs the first target image and the second target image into the target feature vector extraction sub-model of the target tracking model to obtain the first feature vector of the first object contained in the first target image and the second feature vector of the second object contained in the second target image. Then, the first feature vector and the second feature vector are input into the target object classification sub-model of the target tracking model to obtain the target probability that the first object and the second object are the same object. Finally, the target probability and a preset probability threshold are used to determine whether the first object and the second object are the same object. It can be seen that the tracking algorithm provided in this application is based on a neural network model, which has a high coupling with the target detection model. Furthermore, since the target tracking model is not affected by the pose and field of view of the object in the image, the target tracking model can improve the accuracy of target tracking.

[0138] In one implementation, when the target tracking result contains the same first object that is the same as multiple second objects, and / or the same second object that is the same as multiple first objects, after determining whether the first object and the second object are the same object based on the target probability and a preset probability threshold to obtain the target tracking result, the first object and the second object in the target tracking result can be matched using a bipartite graph algorithm to obtain the final target tracking result. The final target tracking result includes that each first object is the same as at most one second object, and each second object is the same as at most one first object.

[0139] The bipartite graph algorithm is used to match the first and second objects in the target tracking result to obtain the final target tracking result. This includes: matching the first and second objects in the target tracking result using the bipartite graph algorithm; selecting at most one second object from multiple second objects that match the first object to determine that it is the same as the first object; or selecting at most one first object from multiple first objects that match the second object to determine that it is the same as the second object; and mapping and labeling the same objects.

[0140] For example, when the preliminary target tracking results include that the first object 1 in the first target image is the same as the second object 1 and the second object 2 in the second target image, the bipartite graph algorithm can be used to match the first object 1 with the second object 1 and the second object 2 again, and finally determine that the second object 1 or the second object 2 is truly the same as the first object 1, or that the second object 1 and the second object 2 are different from the first object 1.

[0141] The embodiments of this application can match the first object and the second object in the target tracking result when the same first object is the same as multiple second objects and / or the same second object is the same as multiple first objects. This is achieved by using a bipartite graph algorithm to match the first object and the second object in the target tracking result, so that each first object is the same as at most one second object and each second object is the same as at most one first object. This avoids target tracking results that do not conform to reality and improves the accuracy of the target tracking result.

[0142] Based on the above method embodiments, another embodiment of this application provides a training device for a target tracking model. The target tracking model includes a target feature vector extraction sub-model and a target object classification sub-model. This device can be applied to electronic devices or computer devices, specifically to vehicles or servers, such as… Figure 4 As shown, the device includes:

[0143] The acquisition unit 310 is used to acquire a training sample set, which includes multiple sets of training samples. Each set of training samples includes two frames of sample images, and the same objects in the two frames of sample images have mapping labels.

[0144] Extraction unit 320 is used to input the training sample set into the initial feature vector extraction sub-model and extract the initial feature vector of the object contained in each frame of the sample image;

[0145] The adjustment unit 330 is used to adjust the model parameters of the initial feature vector extraction sub-model according to the mapping label until the first convergence condition is met, so as to obtain the target feature vector extraction sub-model and the target feature vector of the object contained in each frame of the sample image output by the target feature vector extraction sub-model. The first convergence condition includes the feature vector similarity of the same object in each group of training samples being greater than or equal to the target similarity threshold.

[0146] The classification training unit 340 is used to input the target feature vector corresponding to each group of training samples in the training sample set into the initial object classification sub-model for classification training to obtain the target object classification sub-model. The target object classification sub-model is used to calculate the probability that two objects located in different sample images in each group of training samples are the same.

[0147] In one possible implementation, the acquisition unit 310 includes:

[0148] The acquisition module is used to acquire multiple consecutive frames of sample images;

[0149] The object detection module is used to perform object detection on each frame of the sample image based on the object detection model to determine the objects contained in each frame of the sample image;

[0150] The matching and labeling module is used to match objects contained in every two adjacent sample images in the continuous multi-frame sample images based on the overlap algorithm and the bipartite graph algorithm, and to map and label objects that have the same matching result.

[0151] The determination module is used to obtain the training sample set by taking each pair of adjacent sample images with added mapping annotations as a set of training samples.

[0152] In one possible implementation, the acquisition unit 310 further includes:

[0153] The output module is used to output manual review prompt information when it is determined that there is a target type among the object types contained in the adjacent two-frame sample images after the respective adjacent two-frame sample images with added mapping labels are used as a set of training samples and before obtaining the training sample set. The manual review prompt information is used to prompt whether the mapping label of the target type of object is correct, or to prompt whether the mapping label of the adjacent two-frame sample images is correct.

[0154] The receiving module is used to receive mapping annotation correction operations if there are errors in the review, and to receive confirmation instructions if there are no errors in the review.

[0155] In one possible implementation, the object detection module is configured to, for each frame of the sample image, acquire multiple convolution results of multiple convolution channels in the object detection model on the sample image, wherein different convolution channels are used to calculate different attribute information of each object, and the convolution result is a value that transforms the true value of the attribute information to a target numerical range, the multiple convolution results are located within the same target numerical range, and the target numerical range matches the shared stride of the multiple convolution results; merge the multiple convolution results based on the shared stride to obtain an initial object detection result; calculate the final object detection result corresponding to the initial object detection result according to a preset mapping function that satisfies the target error range, the final object detection result including the true value of each attribute information of each object; and determine the objects contained in the sample image based on the final object detection result.

[0156] In one possible implementation, when the target numerical range is [-10, 10] and the target error range is dy ≤ max(0.1, 0.01y), the preset mapping function includes:

[0157]

[0158] Wherein, y represents the true value of each attribute information, x represents the convolution result, and a, b, c, and d represent different fixed values.

[0159] In one possible implementation, the adjustment unit 330 is used to adjust the model parameters of the initial feature vector extraction sub-model according to the push-pull loss function and the mapping label, so as to improve the similarity of the feature vectors of the same objects in each group of training samples and reduce the similarity of the feature vectors of different objects in each group of training samples.

[0160] The target tracking model training device provided in this application embodiment can first train a target feature vector extraction sub-model and the target feature vector of the object contained in each frame sample image output by the target feature vector extraction sub-model based on multiple sets of training samples including the same object mapping labels and a first convergence condition. Then, it performs classification training based on the target feature vector to obtain a target object classification sub-model. This allows subsequent target tracking to be performed based on the target tracking model including the target feature vector extraction sub-model and the target object classification sub-model. This enables both target tracking and target detection to use neural network models, thereby improving the coupling between the target tracking algorithm and the target detection algorithm. Furthermore, since the target tracking model is not affected by the pose and field of view of the object in the image, this target tracking model can improve the accuracy of target tracking.

[0161] Based on the above method embodiments, another embodiment of this application provides a target tracking device that can be applied to electronic devices or computer devices, specifically to vehicles or servers, such as... Figure 5 As shown, the device includes:

[0162] Extraction unit 410 is used to input the first target image and the second target image into the target feature vector extraction sub-model in the target tracking model, respectively, to obtain the first feature vector of the first object contained in the first target image and the second feature vector of the second object contained in the second target image. The target tracking model is trained according to the method described in any possible implementation of the first aspect.

[0163] Classification unit 420 is used to input the first feature vector and the second feature vector into the target object classification sub-model in the target tracking model to obtain the target probability that the first object and the second object are the same object;

[0164] The determining unit 430 is used to determine whether the first object and the second object are the same object based on the target probability and a preset probability threshold, so as to obtain the target tracking result.

[0165] In a possible implementation, the device further includes:

[0166] A matching unit, configured to, when there are multiple second objects identical to the same first object and / or multiple first objects identical to the same second object in the target tracking result, after determining whether the first object and the second object are the same object according to the target probability and a preset probability threshold to obtain a target tracking result, use the bipartite graph algorithm to match the first object and the second object in the target tracking result to obtain a final target tracking result, where the final target tracking result includes that each first object is identical to at most one second object, and each second object is identical to at most one first object.

[0167] In a possible implementation, the determining unit 430 includes: a first determining module or a second determining module;

[0168] The first determining module is configured to determine that the first object and the second object are the same object when the preset probability threshold is one and the target probability is greater than or equal to the preset probability threshold; or

[0169] The second determining module is configured to determine that the first object and the second object are the same object when the preset probability threshold is multiple and the target probability satisfies a preset formula.

[0170] The preset formula includes:

[0171] (bd_score > a1 && track_time < b1) || (bd_score > a2 && b1 ≤ track_time < b2) || (bd_score > a3 && track_time ≥ b3)

[0172] Where a1 > a2 > a3 and b1 < b2 < b3, the bd_score represents the target probability, the track_time represents the number of frames of continuously tracking the same object, a1, a2, and a3 respectively represent different preset probability thresholds, and b1, b2, and b3 respectively represent different frame number thresholds.

[0173] The target tracking device provided in this application, after obtaining the target tracking model, can first input the first target image and the second target image into the target feature vector extraction sub-model of the target tracking model to obtain the first feature vector of the first object contained in the first target image and the second feature vector of the second object contained in the second target image. Then, the first feature vector and the second feature vector are input into the target object classification sub-model of the target tracking model to obtain the target probability that the first object and the second object are the same object. Finally, the target probability and a preset probability threshold are used to determine whether the first object and the second object are the same object. It can be seen that the tracking algorithm provided in this application is based on a neural network model, which has a high coupling with the target detection model. Since the target tracking model is not affected by the pose and field of view of the object in the image, the target tracking model can improve the accuracy of target tracking.

[0174] Based on the above method embodiments, another embodiment of this application provides a computer-readable storage medium storing executable instructions thereon, which, when executed by a processor, cause the processor to implement the training method of the target tracking model as described in any of the above embodiments, or to implement the target tracking method as described in any of the above embodiments.

[0175] Based on the above method embodiments, another embodiment of this application provides an electronic device or computer device, such as... Figure 6 As shown, it includes:

[0176] One or more processors 510;

[0177] The processor 510 is coupled to a storage device 520, the storage device 520 being used to store one or more programs;

[0178] When the one or more programs are executed by the one or more processors 510, the electronic device or computer device implements the training method of the target tracking model as described in any of the above embodiments, or implements the target tracking method as described in any of the above embodiments.

[0179] Based on the above method embodiments, another embodiment of this application provides a vehicle that includes a training device for a target tracking model as described in any of the above embodiments, or includes a target tracking device as described in any of the above embodiments, or includes an electronic device as described above.

[0180] The vehicle includes an image acquisition unit, a CPU (Central Processing Unit), and a T-Box (Telematics Box). The image acquisition unit is used to acquire sample images; the T-Box can act as a gateway to communicate with the server; the CPU can execute the target tracking model training method described in any of the above embodiments, or execute the target tracking method described in any of the above embodiments. The CPU can also acquire multiple frames of sample images acquired by the image acquisition unit and report them to the server via the T-Box, so that the server can execute the target tracking model training method described in any of the above embodiments. Furthermore, the CPU can acquire a first target image and a second target image acquired by the image acquisition unit and report them to the server via the T-Box, so that the server can execute the target tracking method described in any of the above embodiments.

[0181] Based on the above embodiments, another embodiment of this application provides a computer program product, which includes instructions that, when executed on a computer or processor, cause the computer or processor to perform the method described in any of the above embodiments.

[0182] The above-described apparatus embodiments correspond to the method embodiments and have the same technical effects. For detailed descriptions, please refer to the method embodiments. The apparatus embodiments are derived from the method embodiments; detailed descriptions can be found in the method embodiments section, and will not be repeated here. Those skilled in the art will understand that the accompanying drawings are merely schematic diagrams of one embodiment, and the modules or processes shown in the drawings are not necessarily essential for implementing this application.

[0183] Those skilled in the art will understand that the modules in the apparatus of the embodiments can be distributed in the apparatus of the embodiments as described in the embodiments, or they can be located in one or more devices different from this embodiment with corresponding changes. The modules of the above embodiments can be combined into one module, or they can be further divided into multiple sub-modules.

[0184] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A training method for a target tracking model, characterized in that, The target tracking model includes a target feature vector extraction sub-model and a target object classification sub-model, and the method includes: Obtain a training sample set, which includes multiple sets of training samples. Each set of training samples includes two frames of sample images, and the same objects in the two frames of sample images have mapping labels. The training sample set is input into the initial feature vector extraction sub-model to extract the initial feature vector of the object contained in each frame of the sample image; The model parameters of the initial feature vector extraction sub-model are adjusted according to the mapping annotation until the first convergence condition is met, and the target feature vector extraction sub-model and the target feature vector of the object contained in each frame of the sample image output by the target feature vector extraction sub-model are obtained. The first convergence condition includes that the similarity of the feature vectors of the same object in each group of training samples is greater than or equal to the target similarity threshold. The target feature vector corresponding to each group of training samples in the training sample set is input into the initial object classification sub-model for classification training to obtain the target object classification sub-model. The target object classification sub-model is used to calculate the probability that two objects located in different sample images in each group of training samples are the same.

2. The method according to claim 1, characterized in that, The acquisition of the training sample set includes: Acquire consecutive multi-frame sample images; Based on the object detection model, object detection is performed on each frame of the sample image to determine the objects contained in each frame of the sample image; Based on the overlap algorithm and the bipartite graph algorithm, the objects contained in each two adjacent sample images in the continuous multi-frame sample images are matched, and the objects with the same matching result are mapped and labeled. The training sample set is obtained by taking each pair of adjacent sample images with added mapping annotations as a set of training samples.

3. The method according to claim 2, characterized in that, Before obtaining the training sample set after taking each of the adjacent two frames of sample images with added mapping annotations as a set of training samples, the method further includes: If it is determined that the target type exists among the object types contained in the two adjacent sample images, a manual review prompt message is output. The manual review prompt message is used to prompt whether the mapping label of the object of the target type is correct, or to prompt whether the mapping label of the two adjacent sample images is correct. If there is an error in the review, receive the mapping annotation correction operation; if there is no error in the review, receive the confirmation instruction.

4. The method according to claim 2, characterized in that, The step of performing target detection on each frame of the sample image based on the target detection model to determine the objects contained in each frame of the sample image includes: For each frame of the sample image, multiple convolution results of the sample image are obtained from multiple convolution channels in the target detection model. Different convolution channels are used to calculate different attribute information of each object. The convolution result is a value that transforms the true value of the attribute information into a value within a target numerical range. The multiple convolution results are located within the same target numerical range. The target numerical range is matched with the shared stride of the multiple convolution results. The multiple convolution results are merged based on the shared stride to obtain an initial target detection result; Based on a preset mapping function that satisfies the target error range, the final target detection result corresponding to the initial target detection result is calculated, and the final target detection result includes the true value of each attribute information of each object; The objects contained in the sample image are determined based on the final target detection results.

5. The method according to claim 4, characterized in that, When the target numerical range is [-10, 10] and the target error range is dy ≤ max(0.1, 0.01y), the preset mapping function includes: Wherein, y represents the true value of each attribute information, x represents the convolution result, and a, b, c, and d represent different fixed values.

6. The method according to any one of claims 1-5, characterized in that, The step of adjusting the model parameters of the initial feature vector extraction sub-model based on the mapping annotation includes: The model parameters of the initial feature vector extraction sub-model are adjusted based on the push-pull loss function and the mapping label to increase the similarity of feature vectors of the same objects in each group of training samples and decrease the similarity of feature vectors of different objects in each group of training samples.

7. A target tracking method, characterized in that, The method includes: The first target image and the second target image are respectively input into the target feature vector extraction sub-model in the target tracking model to obtain the first feature vector of the first object contained in the first target image and the second feature vector of the second object contained in the second target image. The target tracking model is trained by the method according to any one of claims 1-6. Input the first feature vector and the second feature vector into the target object classification sub-model in the target tracking model to obtain the target probability that the first object and the second object are the same object; Based on the target probability and a preset probability threshold, determine whether the first object and the second object are the same object, so as to obtain the target tracking result.

8. The method according to claim 7, characterized in that, When the target tracking result shows that the same first object is the same as multiple second objects, and / or the same second object is the same as multiple first objects, after determining whether the first object and the second object are the same object based on the target probability and a preset probability threshold to obtain the target tracking result, the method further includes: The first object and the second object in the target tracking result are matched using a bipartite graph algorithm to obtain the final target tracking result. The final target tracking result includes that each first object is identical to at most one second object, and each second object is identical to at most one first object.

9. The method according to claim 7 or 8, characterized in that, Determining whether the first object and the second object are the same object based on the target probability and a preset probability threshold includes: When the preset probability threshold is one, and the target probability is greater than or equal to the preset probability threshold, the first object and the second object are determined to be the same object; or, When there are multiple preset probability thresholds and the target probability satisfies a preset formula, the first object and the second object are determined to be the same object. The preset formula includes: (bd_score > a1 && track_time < b1) || (bd_score > a2 && b1 <= track_time < b2) || (bd_score > a3 && track_time >= b3) Where, a1 > a2 > a3 and b1 < b2 < b3, the bd_score represents the target probability, the track_time represents the number of frames for continuously tracking the same object, a1, a2, and a3 respectively represent different preset probability thresholds, and b1, b2, and b3 respectively represent different frame number thresholds.

10. A training device for a target tracking model, characterized in that, The target tracking model includes a target feature vector extraction sub-model and a target object classification sub-model, and the device includes: An acquisition unit, configured to acquire a training sample set, the training sample set includes multiple groups of training samples, each group of the training samples includes two frame sample images, and the same object in the two frame sample images has a mapping annotation; An extraction unit, configured to input the training sample set into an initial feature vector extraction sub-model, and extract the initial feature vectors of the objects included in each frame of the sample images; An adjustment unit, configured to adjust the model parameters of the initial feature vector extraction sub-model according to the mapping annotation until the first convergence condition is satisfied, and obtain the target feature vector extraction sub-model and the target feature vectors of the objects included in each frame of the sample images output by the target feature vector extraction sub-model, where the first convergence condition includes that the feature vector similarity of the same object in each group of the training samples is greater than or equal to a target similarity threshold; A classification training unit, configured to input the target feature vectors corresponding to each group of the training samples in the training sample set into an initial object classification sub-model for classification training, and obtain the target object classification sub-model, where the target object classification sub-model is used to calculate the probability that two objects located in different sample images in each group of the training samples are the same.

11. The apparatus according to claim 10, characterized in that, The acquisition unit includes: An acquisition module, configured to acquire multiple consecutive frame sample images; A target detection module, configured to perform target detection on each frame of the sample images based on a target detection model, and determine the objects included in each frame of the sample images; A matching annotation module, configured to match the objects included in every two adjacent frame sample images in the multiple consecutive frame sample images based on an overlap degree algorithm and a bipartite graph algorithm, and perform mapping annotation on the objects with the matching result being the same; A determination module, configured to obtain the training sample set after respectively taking each of the adjacent two frame sample images with mapping annotation as a group of training samples.

12. The apparatus according to claim 11, characterized in that, The acquisition unit further includes: An output module, configured to output an artificial review prompt message before obtaining the training sample set after respectively taking each of the adjacent two frame sample images with mapping annotation as a group of training samples, where the artificial review prompt message is used to prompt to review whether the mapping annotation of the object of the target type is correct, or is used to prompt to review whether the mapping annotation of the adjacent two frame sample images is correct; The receiving module is used to receive mapping annotation correction operations if there are errors in the review, and to receive confirmation instructions if there are no errors in the review.

13. The apparatus according to claim 11, characterized in that, The target detection module is used to acquire multiple convolution results of the target detection model on each frame of the sample image, wherein different convolution channels are used to calculate different attribute information of each object, and the convolution result is a value that transforms the true value of the attribute information to a target numerical range. The multiple convolution results are located within the same target numerical range, and the target numerical range matches the shared stride of the multiple convolution results. The multiple convolution results are merged based on the shared stride to obtain an initial target detection result. The final target detection result is calculated according to a preset mapping function that satisfies the target error range. The final target detection result includes the true value of each attribute information of each object. The objects contained in the sample image are determined based on the final target detection result.

14. The apparatus according to claim 13, characterized in that, When the target numerical range is [-10, 10] and the target error range is dy ≤ max(0.1, 0.01y), the preset mapping function includes: Wherein, y represents the true value of each attribute information, x represents the convolution result, and a, b, c, and d represent different fixed values.

15. A target tracking device, characterized in that, The device includes: An extraction unit is used to input the first target image and the second target image into the target feature vector extraction sub-model in the target tracking model, respectively, to obtain the first feature vector of the first object contained in the first target image and the second feature vector of the second object contained in the second target image, wherein the target tracking model is trained by the method according to any one of claims 1-6; A classification unit is used to input the first feature vector and the second feature vector into the target object classification sub-model in the target tracking model to obtain the target probability that the first object and the second object are the same object. The determining unit is used to determine whether the first object and the second object are the same object based on the target probability and a preset probability threshold, so as to obtain the target tracking result.

16. The apparatus according to claim 15, characterized in that, The device further includes: A matching unit is configured to, when the target tracking result contains the same first object and multiple second objects, and / or the same second object and multiple first objects, after determining whether the first object and the second object are the same object based on the target probability and a preset probability threshold to obtain the target tracking result, use a bipartite graph algorithm to match the first object and the second object in the target tracking result to obtain the final target tracking result. The final target tracking result includes that each first object is at most the same as one second object, and each second object is at most the same as one first object.

17. The apparatus according to claim 15 or 16, characterized in that, The determining unit includes: a first determining module or a second determining module; The first determination module is configured to determine that the first object and the second object are the same object when the preset probability threshold is one and the target probability is greater than or equal to the preset probability threshold; or, The second determination module is configured to determine that the first object and the second object are the same object when the preset probability threshold is multiple and the target probability satisfies a preset formula, The preset formula includes: (bd_score > a1 && track_time < b1) || (bd_score > a2 && b1 ≤ track_time < b2) || (bd_score > a3 && track_time ≥ b3) where a1 > a2 > a3 and b1 < b2 < b3, the bd_score represents the target probability, the track_time represents the number of frames for continuously tracking the same object, a1, a2, and a3 respectively represent different preset probability thresholds, and b1, b2, and b3 respectively represent different frame number thresholds.

18. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method according to any one of claims 1-6 or any one of claims 7-9.

19. An electronic device, characterized in that, The electronic device includes: One or more processors; The processor is coupled to the storage device, and the storage device is configured to store one or more programs; When the one or more programs are executed by the one or more processors, the electronic device implements the method according to any one of claims 1-6 or any one of claims 7-9.

20. A vehicle, characterized in that, The vehicle includes the device according to any one of claims 10-14 or any one of claims 15-17, or includes the electronic device according to claim 19.

Images