Target tracking method and device, electronic equipment and storage medium
By obtaining and matching the cross-type similarity of the target feature combination in the autonomous driving module, the tracking loss problem caused by type jump is solved, and stable tracking of cross-type targets is achieved.
Patent Information
- Application Number
- CN202510054414.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-01-14
- Publication Date
- 2025-04-18
AI Technical Summary
Tracking loss problems caused by type jumps in the area around the bicycle in the autonomous driving module, especially the loss of targets tracked across cameras.
By acquiring a combination of multiple target features, the cross-type similarity between the first target feature and the second target feature is determined, and an optimal match is performed based on this to obtain the target optimal match pair, and finally the tracking attribute of the second target feature is determined based on the reference attribute of the first target feature in the target matching pair.
It is achieved that when the target type confidence is not high and the target changes across types in the front and back frames, it can still accurately track targets to ensure stable tracking of different types of targets.
Smart Images
Figure CN119850681A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of intelligent driving, and in particular to a target tracking method, device, electronic device and storage medium. Background Art
[0002] Benefiting from the vigorous development of automobile intelligence and AI technology, autonomous driving technology has been greatly developed, making it possible to use autonomous driving to replace humans, which will greatly reduce labor costs and transportation costs. However, autonomous driving technology still faces many challenges. Among them, the accuracy and stability of perception of other traffic participants are extremely high, especially the perception and tracking of targets around the vehicle, which has important academic and application value.
[0003] The BEV technology of bird's-eye view spatial mapping achieves 360° spatial feature coverage around the vehicle by fusing features in the BEV space. Compared with the traditional method, it can avoid the interference of single sensor truncation target and the loss of useful information caused by post-fusion, and can also greatly enhance the spatial feature expression ability and effectively improve the accuracy of detection. This solution has gradually attracted the attention of academia and industry and has become the mainstream solution.
[0004] Currently, multi-target tracking of different types of targets is mainly performed independently. The premise of this method is that the confidence of the target type is high, and the accuracy of cross-target type tracking is low when the types are inconsistent. Summary of the invention
[0005] In order to solve the problem of tracking loss in the area around the vehicle due to type jumping in the current autonomous driving module, especially the technical problem of target loss in cross-camera tracking, the present application provides a target tracking method, device, electronic device and storage medium.
[0006] In a first aspect, the present application provides a target tracking method, comprising:
[0007] Acquire multiple target feature combinations, each of which includes: a first target feature whose tracking attribute has been determined and a second target feature whose tracking attribute is to be determined, wherein the first target feature and the second target feature are determined according to a previously acquired panoramic image frame and a subsequently acquired panoramic image frame in two adjacent panoramic image frames, respectively;
[0008] For each target feature combination, determine the cross-type similarity between the first target feature and the second target feature, and perform optimal matching based on the cross-type similarity to obtain a target optimal matching pair;
[0009] The tracking attribute of the second target feature is determined based on the reference attribute of the first target feature in the target best matching pair.
[0010] Optionally, multiple target feature combinations are obtained, including:
[0011] Acquire a tracking target list extracted from a previously acquired panoramic image frame of two adjacent panoramic image frames and a list of to-be-preprocessed targets extracted from a subsequently acquired panoramic image frame, wherein the tracking target list includes a plurality of first target features;
[0012] Preprocessing the third target feature in the to-be-preprocessed target list to obtain a preprocessed target list, wherein the preprocessed target list includes a plurality of second target features;
[0013] For each first target feature in the tracking target list, a second target feature corresponding to a position is determined in the preprocessing target list according to the position of the first target feature, and a determined target feature combination including the first target feature and the second target feature is established.
[0014] Optionally, preprocessing the third target feature in the to-be-preprocessed target list to obtain a preprocessed target list includes:
[0015] For each of the third target features in the to-be-preprocessed target list, determining a feature type of the third target feature;
[0016] Performing filtering processing on the third target feature in the preprocessing target list based on the feature type;
[0017] A pre-processing target list is determined based on the remaining third target features.
[0018] Optionally, filtering the third target feature in the preprocessing target list based on the feature type includes:
[0019] Performing position filtering on the third target feature in the preprocessing target list to obtain a first remaining target feature set;
[0020] Performing confidence filtering on each of the third target features in the first remaining target feature set to obtain a second remaining target feature set;
[0021] For each third target feature in the second remaining target feature set, if the feature type is the first feature type, the third target feature is subjected to non-maximum suppression processing in the image domain and angle-based corner detection algorithm processing; if the feature type is the second feature type, the third target feature is subjected to non-maximum suppression processing in the image domain and VCS coordinate system.
[0022] Optionally, for each target feature combination, determining a cross-type similarity between the first target feature and the second target feature, and performing optimal matching based on the cross-type similarity to obtain a target optimal matching pair includes:
[0023] Obtaining a cross-type similarity probability matrix, wherein parameters of the cross-type similarity probability matrix are determined based on prior information of misrecognition situations caused by occlusion recorded in historical records;
[0024] For each of the target feature combinations, based on the cross-type similarity probability matrix, a first cross-type similarity matrix of the first target feature and the second target feature is determined, and optimal matching is performed based on the first cross-type similarity matrix to obtain a first optimal matching pair;
[0025] If the similarity of the first optimal matching pair is greater than a preset similarity threshold, the first optimal matching pair is determined as a target optimal matching pair.
[0026] Optionally, determining a first cross-type similarity matrix of the first target feature and the second target feature based on the cross-type similarity probability matrix, and performing optimal matching based on the first cross-type similarity matrix to obtain a first optimal matching pair includes:
[0027] Calculating the Iou similarity matrix of the first target feature and the second target feature;
[0028] An optimal match is performed based on the cross-type similarity probability matrix and the Iou similarity matrix to obtain the first optimal matching pair.
[0029] Optionally, for each target feature combination, determining a cross-type similarity between the first target feature and the second target feature, and performing optimal matching based on the cross-type similarity to obtain a target optimal matching pair, further comprising:
[0030] If the similarity of the best matching pair is less than or equal to the similarity threshold, based on the cross-type similarity probability matrix, cascade matching is performed on the second target feature and the first target feature in any other target feature combination to obtain a second best matching pair;
[0031] If the similarity of the second best matching pair is greater than a preset similarity threshold, the second best matching pair is determined as a target best matching pair.
[0032] Optionally, based on the cross-type similarity probability matrix, cascade matching is performed on the second target feature with the first target feature in any other target feature combination to obtain a second optimal matching pair, including:
[0033] Calculating a Real3d similarity matrix between the second target feature and the first target feature in any other target feature combination;
[0034] An optimal match is performed based on the cross-type similarity probability matrix and the Real3d similarity matrix to obtain the second optimal matching pair.
[0035] Optionally, before obtaining the cross-type similarity probability matrix, the method further includes:
[0036] Determine whether the position of the first target feature and / or the second target feature is located in the intersection area of the panoramic field of view angle;
[0037] If the position of the first target feature and / or the second target feature is located in the intersection area of the panoramic field of view angle, the step of obtaining a cross-type similarity probability matrix is performed.
[0038] In a second aspect, the present application provides a target tracking device, comprising:
[0039] an acquisition module, configured to acquire a plurality of target feature combinations, each of which comprises: a first target feature whose tracking attribute has been determined and a second target feature whose tracking attribute is to be determined, wherein the first target feature and the second target feature are respectively determined based on a previously acquired panoramic image frame and a subsequently acquired panoramic image frame in two adjacent panoramic image frames;
[0040] A first determination module, configured to determine, for each target feature combination, a cross-type similarity between the first target feature and the second target feature, and perform optimal matching based on the cross-type similarity to obtain a target optimal matching pair;
[0041] The second determination module is used to determine the tracking attribute of the second target feature according to the reference attribute of the first target feature in the target optimal matching pair.
[0042] In a third aspect, the present application provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other via the communication bus;
[0043] Memory, used to store computer programs;
[0044] The processor is used to implement any target tracking method described in the first aspect when executing a program stored in the memory.
[0045] In a fourth aspect, the present application provides a computer-readable storage medium, on which a program of a target tracking method is stored. When the program of the target tracking method is executed by a processor, the steps of the target tracking method described in any one of the first aspects are implemented.
[0046] Beneficial effects of the present invention: The embodiment of the present application determines the cross-category similarity for the first target feature and the second target feature in each target feature combination, and performs optimal matching based on the cross-type similarity to obtain the optimal target matching pair. Finally, the tracking attribute of the second target feature can be determined according to the reference attribute of the first target feature in the target matching pair. By considering the cross-type factor in the process of determining the target similarity, it is possible to accurately track the target when the confidence level of the target type is not high, that is, when the target in the previous and next frames changes across types, thereby achieving stable tracking of targets of different types. BRIEF DESCRIPTION OF THE DRAWINGS
[0047] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.
[0048] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.
[0049] Figure 1 A flow chart of a target tracking method provided in an embodiment of the present application;
[0050] Figure 2 A schematic diagram of a surrounding target feature stitching area provided in an embodiment of the present application;
[0051] Figure 3 for Figure 1 Flow chart of step S101;
[0052] Figure 4 A preprocessing flow chart provided for an embodiment of the present application;
[0053] Figure 5 for Figure 1 Flow chart of step S102;
[0054] Figure 6 A schematic diagram of a KM cascade tracking process based on a type similarity matrix provided in an embodiment of the present application;
[0055] Figure 7 A structural diagram of a target tracking device provided in an embodiment of the present application;
[0056] Figure 8 A structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0057] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.
[0058] Since multi-target tracking of different types of targets is mainly performed independently, the premise of this method is that the confidence of the target type is high, and the cross-target type tracking accuracy is low when the types are inconsistent. To this end, the embodiments of the present application provide a target tracking method, device, electronic device and storage medium.
[0059] The present application embodiment provides a target tracking method, such as Figure 1 As shown, including:
[0060] Step S101, obtaining multiple target feature combinations;
[0061] In the embodiment of the present application, each target feature combination includes: a first target feature whose tracking attribute has been determined and a second target feature whose tracking attribute is to be determined, wherein the first target feature and the second target feature are respectively determined according to a panoramic image frame acquired earlier and a panoramic image frame acquired later in two adjacent panoramic image frames;
[0062] In this step, two adjacent panoramic image frames may be first acquired, a first target feature may be determined based on the previously acquired panoramic image frame, a second target feature may be determined based on the subsequently acquired panoramic image frame, and the first target feature and the second target feature may be determined as a target feature combination.
[0063] Step S102, for each target feature combination, determining the cross-type similarity between the first target feature and the second target feature, and performing optimal matching based on the cross-type similarity to obtain a target optimal matching pair;
[0064] Due to the existence of splicing features in the truncated area in the current BEV space, when the panoramic camera is used as input, and the target is in Figure 2 In the different camera angle areas shown, the features extracted in the BEV space at this time are often difficult to accurately express the target features, and the possibility of target type jumps increases. Therefore, in an embodiment of the present application, the cross-type similarity between the first target feature and the second target feature is determined. The cross-type similarity refers to the similarity between different types of targets. When the cross-type similarity exceeds a certain threshold, it indicates that the two targets are the same target, otherwise they are different targets.
[0065] In this step, the cross-type similarity between the first target feature and the second target feature in each target feature combination can be determined, and the cross-type similarity can be used as the input of the optimal match to obtain the optimal match pair, and then the target optimal match pair can be determined based on the optimal match pair.
[0066] Step S103: determining the tracking attribute of the second target feature according to the reference attribute of the first target feature in the target optimal matching pair.
[0067] In this step, the tracking attribute of the second target feature may be determined according to the reference attribute of the first target feature in the target optimal matching pair, that is, the tracking attribute of the second target feature is determined as the reference attribute.
[0068] In an embodiment of the present application, the cross-category similarity is determined for the first target feature and the second target feature in each target feature combination, and the optimal match is performed based on the cross-type similarity to obtain an optimal target matching pair. Finally, the tracking attribute of the second target feature can be determined according to the reference attribute of the first target feature in the target matching pair. By considering the cross-type factor in the process of determining the target similarity, it is possible to accurately track the target even when the confidence level of the target type is not high, that is, when the target in the previous and next frames changes across types, thereby achieving stable tracking of targets of different types.
[0069] In another embodiment of the present application, step S101 obtains multiple target feature combinations, such as Figure 3 As shown, including:
[0070] Step S201, obtaining a list of tracking targets extracted from a previously acquired panoramic image frame of two adjacent panoramic image frames and a list of targets to be preprocessed extracted from a subsequently acquired panoramic image frame, wherein the list of tracking targets includes a plurality of first target features;
[0071] Step S202, preprocessing the third target feature in the to-be-preprocessed target list to obtain a preprocessed target list, wherein the preprocessed target list includes a plurality of second target features;
[0072] In order to avoid the redundant detection frame in the subsequent tracking link affecting the normal tracking of the real target, the third target feature in the preprocessed target list can be treated as follows: Figure 4 The preprocessing shown in the figure includes confidence filtering, FOV filtering and NMS deduplication, which are mainly divided into NMS3D and NMS2D for vehicles, NMS2D for cyclists and pedestrians, and angle-based target filtering.
[0073] In one embodiment of the present application, the third target features in the target list to be preprocessed are preprocessed to obtain a preprocessed target list, including: for each of the third target features in the target list to be preprocessed, determining the feature type of the third target feature; filtering the third target features in the preprocessed target list based on the feature type; and determining the preprocessed target list based on the remaining third target features.
[0074] Among them, filtering the third target feature in the pre-processing target list based on the feature type includes: performing position filtering on the third target feature in the pre-processing target list to obtain a first set of remaining target features; performing confidence filtering on each of the third target features in the first set of remaining target features to obtain a second set of remaining target features; for each third target feature in the second set of remaining target features, if the feature type is the first feature type, performing non-maximum suppression processing in the image domain and angle-based corner detection algorithm processing on the third target feature; if the feature type is the second feature type, performing non-maximum suppression processing in the image domain and VCS coordinate system on the third target feature.
[0075] Step S203: for each first target feature in the tracking target list, determine the second target feature corresponding to the position in the pre-processing target list according to the position of the first target feature, and establish a determined target feature combination including the first target feature and the second target feature.
[0076] When acquiring a target feature combination, the embodiment of the present application can preprocess the third target feature in the acquired list of targets to be preprocessed to eliminate redundant detection frames, and then form a target feature combination with each second target feature obtained by preprocessing and the first target feature corresponding to the position, thereby improving the efficiency of target tracking.
[0077] In another embodiment of the present application, step S102 determines the cross-type similarity between the first target feature and the second target feature for each target feature combination, and performs optimal matching based on the cross-type similarity to obtain a target optimal matching pair, such as Figure 5 As shown, including:
[0078] Step S301, obtaining a cross-type similarity probability matrix, wherein parameters of the cross-type similarity probability matrix are determined based on prior information of misrecognition situations caused by occlusion recorded in historical records;
[0079] Before obtaining the cross-type similarity probability matrix in step S301, the method also includes: determining whether the position of the first target feature and / or the second target feature is located in the intersection area of the surrounding field of view angle; if the position of the first target feature and / or the second target feature is located in the intersection area of the surrounding field of view angle, executing the step of obtaining the cross-type similarity probability matrix.
[0080] Because the panoramic input needs to splice the target features in the BEV space when crossing cameras, and the target poses in the image domain are rich at this time, it is found in practical applications that the target type is usually more prone to mutation. Therefore, if the target is in the FOV, a cross-type similarity probability matrix should be introduced. Furthermore, the value of the cross-type similarity probability matrix can be improved.
[0081] Step S302: for each target feature combination, based on the cross-type similarity probability matrix, determine a first cross-type similarity matrix of the first target feature and the second target feature, and perform optimal matching based on the first cross-type similarity matrix to obtain a first optimal matching pair;
[0082] In this step, the cross-type similarity calculation can be performed on all targets in the previous and next frames, and IouSimFunction and Real3dSimFunction can be calculated respectively, which are used as the similarity matrix of the target; then, when crossing types, a cross-type similarity probability matrix needs to be introduced, and its initial value is zero, and the assignment of the matrix is guided by the cross-type tracking examples that often appear in practical applications. For example, cyclists are misidentified as pedestrians due to occlusion in the early stage of recognition, trucks and buses are misidentified, and motorcycles, bicycles and tricycles are misidentified. Introducing this prior information is of great value in practical applications.
[0083] like Figure 6 As shown in the figure, after the target similarity matrix of all target types between the previous and next frames is obtained in the first matching, the problem can be solved by the optimal matching algorithm to achieve tracking with the minimum global tracking error. The KM algorithm is an improvement on the Hungarian algorithm, which is used to find the optimal match of the weighted maximum complete match of the bipartite graph; finally, the tracking threshold is set based on the optimal matching result. If the similarity between the optimal matching targets of the previous and next frames is higher than the threshold, the target tracking in the BEV space is considered to be completed.
[0084] In one embodiment of the present application, based on the cross-type similarity probability matrix, a first cross-type similarity matrix of the first target feature and the second target feature is determined, and optimal matching is performed based on the first cross-type similarity matrix to obtain a first optimal matching pair, including: calculating the Iou similarity matrix of the first target feature and the second target feature; performing optimal matching based on the cross-type similarity probability matrix and the Iou similarity matrix to obtain the first optimal matching pair.
[0085] Exemplarily, the Iou similarity matrix of the first target feature and the second target feature may be calculated first, and the first similarity matrix may be obtained by multiplying the Iou similarity matrix by the cross-type similarity probability matrix, and optimal matching may be performed based on the first similarity matrix.
[0086] Step S303: if the similarity of the first optimal matching pair is greater than a preset similarity threshold, the first optimal matching pair is determined as a target optimal matching pair.
[0087] Step S304: if the similarity of the best matching pair is less than or equal to the similarity threshold, based on the cross-type similarity probability matrix, cascade matching is performed on the second target feature and the first target feature in any other target feature combination to obtain a second best matching pair;
[0088] Based on the cross-type similarity probability matrix, the second target feature is cascade matched with the first target feature in any other target feature combination to obtain a second optimal matching pair, including: calculating the Real3d similarity matrix of the second target feature and the first target feature in any other target feature combination; performing optimal matching based on the cross-type similarity probability matrix and the Real3d similarity matrix to obtain the second optimal matching pair.
[0089] Exemplarily, the Real3d similarity matrix of the first target feature in the second target feature and any other target feature combination can be calculated first, and the Real3d similarity matrix can be multiplied by the cross-type similarity probability matrix to obtain the second similarity matrix, and the optimal matching is performed based on the first similarity matrix.
[0090] Step S305: if the similarity of the second best matching pair is greater than a preset similarity threshold, the second best matching pair is determined as a target best matching pair.
[0091] In the embodiment of the present application, in the process of determining the similarity matrix of the first target feature and the second target feature, a cross-type similarity probability matrix is introduced to obtain a first cross-type similarity matrix, and optimal matching is performed based on the first cross-type similarity matrix to obtain a first optimal matching pair. When the similarity of the optimal matching pair is greater than the similarity threshold, it is considered that the matching is successful, and the first optimal matching pair is determined as the target optimal matching pair; otherwise, it is considered that the matching fails, and cascade matching is performed, that is, cascade matching is performed with the second target feature and the first target feature in any other target feature combination. In the process of calculating the similarity by cascade matching, the cross-type similarity probability matrix is also introduced to obtain the second optimal matching pair. If the similarity of the second optimal matching pair is greater than the preset similarity threshold, it is considered that the matching is successful, and the second optimal matching pair is determined as the target optimal matching pair. That is to say, in the present application, by introducing the cross-type similarity probability matrix, cross-type factors are taken into account in the process of determining the target similarity, so as to achieve accurate target tracking when the confidence of the target type is not high, that is, when the target in the previous and next frames changes across types, stable tracking of targets of different types is achieved.
[0092] The present invention proposes a method for stable target tracking in the vehicle BEV space, so as to introduce the prior information of the target characteristics in the BEV space as much as possible to achieve stable target tracking, thereby achieving stable tracking of the target with 360° visual coverage of the vehicle, and maximizing the performance of the BEV model output target. Its characteristics are that a cross-type similarity probability matrix is introduced with an initial value of zero, and the matrix is initialized using a cross-type tracking instance during the algorithm iteration process. In addition, the scheme also considers whether the target is in the surrounding target feature splicing area of the vehicle to improve the cross-type tracking performance of the target in the area. For ease of understanding, the present application provides a target tracking method in practical applications. The feature area around the vehicle is such as Figure 2 As shown, the present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments.
[0093] S1. Preprocess the input target on the model side. The preprocessing module designed in this paper is as follows: Figure 3 As shown, based on the confidence statistics of the target output on the model side, confidence thresholds of different categories and positions are set in combination with the position of the target, and the input targets are filtered to obtain the target input set before the NMS stage.
[0094] S2. Vehicle targets are easily overlapped in the image domain, but are usually separated from each other in the VCS coordinate system, while pedestrian targets are just the opposite. Therefore, it is necessary to filter targets by category and coordinate system during NMS. Specifically, vehicles need to perform NMS in both the image domain and the VCS coordinate system, while pedestrians only need to perform NMS in the image domain. However, as a supplement, an angle-based method is introduced to filter pedestrian targets. The NMS methods in the image domain and the VCS coordinate system are both implemented using the IOU method. The specific calculation method is as follows:
[0095]
[0096] Where r represents the area of the intersection, r1 and r2 represent the area of the two targets respectively.
[0097] S3, taking the preprocessed target and the list of tracked targets maintained in the previous frame as input, calculate the IouSimFunction similarity matrix of all targets in the previous and next frames and realize the first matching, and then calculate the Real3dSimFunction matrix to realize the cascade matching of untracked targets. The IouSimFunction similarity matrix is calculated by intersection over Union (IOU). The specific calculation method refers to the calculation formula of NMS. Real3dSimFunction uses horizontal and vertical diff to calculate the horizontal and vertical cost and the overall cost, thereby completing the preliminary calculation of the similarity matrix. However, the pose difference of the input target on the BEV model side is large, and it is closely related to the target's surrounding field of view. It is necessary to introduce such prior information to improve the performance of target tracking.
[0098] S4. Furthermore, the coefficients of the cross-type similarity probability matrix are introduced. Specifically, the matrix can be initialized as a unit matrix, that is, only targets of the same type can be associated with each other. Based on the cross-type association characteristics and cross-type association instances, other positions of the matrix are assigned values.
[0099] S5. Furthermore, based on the fact that target feature splicing easily leads to type jumps, whether the target is located in the intersection area of the panoramic field of view is introduced to further correct the cross-type similarity probability matrix.
[0100] S6. Based on the target similarity matrix obtained in the above steps, the KM algorithm is used to achieve the optimal match between the tracking list and the input target. The optimal matching result obtained at this time may force the two targets with zero similarity to match, so it is necessary to filter the matching results in combination with the similarity threshold, and finally update the attributes of the corresponding target in the tracking list based on the matching results using the attributes of the input target, and finally complete the stable tracking of the bev target.
[0101] Through the above example description, this method introduces a cross-type similarity probability matrix, initializes the matrix based on the cross-type tracking instance, and introduces the panoramic feature stitching area to further improve the target tracking performance in the intersection area of the panoramic field of view, so that the entire tracking method can better adapt to the input target characteristics of the BEV model side and achieve stable tracking of the panoramic BEV target.
[0102] In another embodiment of the present application, a target tracking device is also provided. Figure 7 As shown, including:
[0103] An acquisition module 11 is used to acquire multiple target feature combinations, each of which includes: a first target feature whose tracking attribute has been determined and a second target feature whose tracking attribute is to be determined, wherein the first target feature and the second target feature are determined according to a previously acquired panoramic image frame and a subsequently acquired panoramic image frame in two adjacent panoramic image frames, respectively;
[0104] A first determination module 12 is used to determine, for each target feature combination, a cross-type similarity between the first target feature and the second target feature, and perform optimal matching based on the cross-type similarity to obtain a target optimal matching pair;
[0105] The second determination module 13 is used to determine the tracking attribute of the second target feature according to the reference attribute of the first target feature in the target optimal matching pair.
[0106] In another embodiment of the present application, an electronic device is provided, including a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus;
[0107] Memory, used to store computer programs;
[0108] The processor is used to implement the target tracking method described in any of the above method embodiments when executing the program stored in the memory.
[0109] In the electronic device provided by an embodiment of the present invention, the processor determines the cross-category similarity for the first target feature and the second target feature in each target feature combination by executing a program stored in the memory, and performs optimal matching based on the cross-type similarity to obtain an optimal target matching pair. Finally, the tracking attribute of the second target feature can be determined according to the reference attribute of the first target feature in the target matching pair. By considering the cross-type factor in the process of determining the target similarity, it is possible to accurately track the target even when the confidence level of the target type is not high, that is, when the target in the previous and next frames changes across types, thereby achieving stable tracking of targets of different types.
[0110] The communication bus 1140 mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The communication bus 1140 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 8 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.
[0111] The communication interface 1120 is used for communication between the above electronic device and other devices.
[0112] The memory 1130 may include a random access memory (RAM) or a non-volatile memory, such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.
[0113] The above-mentioned processor 1110 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.
[0114] In another embodiment of the present application, a computer-readable storage medium is also provided, on which a program of a target tracking method is stored, and when the program of the target tracking method is executed by a processor, the steps of the target tracking method described in any of the aforementioned method embodiments are implemented. It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements that are not explicitly listed, or also includes elements inherent to such a process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "including one..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.
[0115] The foregoing is merely a specific embodiment of the present invention, which enables those skilled in the art to understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features claimed herein.
Claims
1. A target tracking method, characterized in that: include: Acquire multiple target feature combinations, each of which includes: a first target feature whose tracking attribute has been determined and a second target feature whose tracking attribute is to be determined, wherein the first target feature and the second target feature are determined according to a previously acquired panoramic image frame and a subsequently acquired panoramic image frame in two adjacent panoramic image frames, respectively; For each target feature combination, determine the cross-type similarity between the first target feature and the second target feature, and perform optimal matching based on the cross-type similarity to obtain a target optimal matching pair; The tracking attribute of the second target feature is determined based on the reference attribute of the first target feature in the target best matching pair.
2. The target tracking method according to claim 1, characterized in that: Get multiple target feature combinations, including: Acquire a tracking target list extracted from a previously acquired panoramic image frame of two adjacent panoramic image frames and a list of to-be-preprocessed targets extracted from a subsequently acquired panoramic image frame, wherein the tracking target list includes a plurality of first target features; Preprocessing the third target feature in the to-be-preprocessed target list to obtain a preprocessed target list, wherein the preprocessed target list includes a plurality of second target features; For each first target feature in the tracking target list, a second target feature corresponding to a position is determined in the preprocessing target list according to the position of the first target feature, and a determined target feature combination including the first target feature and the second target feature is established.
3. The target tracking method according to claim 2, characterized in that: Preprocessing the third target feature in the target list to be preprocessed to obtain a preprocessed target list, including: For each of the third target features in the to-be-preprocessed target list, determining a feature type of the third target feature; Performing filtering processing on the third target feature in the preprocessing target list based on the feature type; A pre-processing target list is determined based on the remaining third target features.
4. The target tracking method according to claim 3, characterized in that: Filtering the third target feature in the preprocessing target list based on the feature type includes: Performing position filtering on the third target feature in the preprocessing target list to obtain a first remaining target feature set; Performing confidence filtering on each of the third target features in the first remaining target feature set to obtain a second remaining target feature set; For each third target feature in the second remaining target feature set, if the feature type is the first feature type, the third target feature is subjected to non-maximum suppression processing in the image domain and angle-based corner detection algorithm processing; if the feature type is the second feature type, the third target feature is subjected to non-maximum suppression processing in the image domain and VCS coordinate system.
5. The target tracking method according to claim 1, characterized in that: For each target feature combination, determining the cross-type similarity between the first target feature and the second target feature, and performing optimal matching based on the cross-type similarity to obtain a target optimal matching pair, including: Obtaining a cross-type similarity probability matrix, wherein parameters of the cross-type similarity probability matrix are determined based on prior information of misrecognition situations caused by occlusion recorded in historical records; For each of the target feature combinations, based on the cross-type similarity probability matrix, a first cross-type similarity matrix of the first target feature and the second target feature is determined, and optimal matching is performed based on the first cross-type similarity matrix to obtain a first optimal matching pair; If the similarity of the first optimal matching pair is greater than a preset similarity threshold, the first optimal matching pair is determined as a target optimal matching pair.
6. The target tracking method according to claim 5, characterized in that: Determining a first cross-type similarity matrix of the first target feature and the second target feature based on the cross-type similarity probability matrix, and performing optimal matching based on the first cross-type similarity matrix to obtain a first optimal matching pair, including: Calculating the Iou similarity matrix of the first target feature and the second target feature; An optimal match is performed based on the cross-type similarity probability matrix and the Iou similarity matrix to obtain the first optimal matching pair.
7. The target tracking method according to claim 5, characterized in that: For each target feature combination, determining the cross-type similarity between the first target feature and the second target feature, and performing optimal matching based on the cross-type similarity to obtain a target optimal matching pair, further comprising: If the similarity of the best matching pair is less than or equal to the similarity threshold, based on the cross-type similarity probability matrix, cascade matching is performed on the second target feature and the first target feature in any other target feature combination to obtain a second best matching pair; If the similarity of the second best matching pair is greater than a preset similarity threshold, the second best matching pair is determined as a target best matching pair.
8. The target tracking method according to claim 7, characterized in that: Based on the cross-type similarity probability matrix, cascade matching is performed on the second target feature with the first target feature in any other target feature combination to obtain a second optimal matching pair, including: Calculating a Real3d similarity matrix between the second target feature and the first target feature in any other target feature combination; An optimal match is performed based on the cross-type similarity probability matrix and the Real3d similarity matrix to obtain the second optimal matching pair.
9. The target tracking method according to claim 5, characterized in that: Before obtaining the cross-type similarity probability matrix, the method further includes: Determine whether the position of the first target feature and / or the second target feature is located in the intersection area of the panoramic field of view angle; If the position of the first target feature and / or the second target feature is located in the intersection area of the panoramic field of view angle, the step of obtaining a cross-type similarity probability matrix is performed.
10. A target tracking device, characterized in that: include: an acquisition module, configured to acquire a plurality of target feature combinations, each of which comprises: a first target feature whose tracking attribute has been determined and a second target feature whose tracking attribute is to be determined, wherein the first target feature and the second target feature are respectively determined based on a previously acquired panoramic image frame and a subsequently acquired panoramic image frame in two adjacent panoramic image frames; A first determination module, configured to determine, for each target feature combination, a cross-type similarity between the first target feature and the second target feature, and perform optimal matching based on the cross-type similarity to obtain a target optimal matching pair; The second determination module is used to determine the tracking attribute of the second target feature according to the reference attribute of the first target feature in the target optimal matching pair.
11. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; Memory, used to store computer programs; The processor is used to implement the target tracking method described in any one of claims 1 to 9 when executing the program stored in the memory.
12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a program of the target tracking method, and when the program of the target tracking method is executed by the processor, the steps of the target tracking method described in any one of claims 1 to 9 are implemented.