A single-target tracking method

Abstract

The present disclosure relates to a single target tracking method, device, electronic equipment and storage medium. The method comprises the following steps: predicting the target score map of the current frame and the previous frame in a video, thereby obtaining two groups of feature vectors of all target candidate objects in the two frames; inputting the two groups of feature vectors of the target candidate objects of the current frame and the previous frame in the video into a target matching module, thereby obtaining the pairing relationship between the target candidate objects and the interference objects between the adjacent two frames of the video. After the target candidate objects are paired between the adjacent two frames of the video, the tracking target in the current frame can be inferred and used as the initial prediction result; feature extraction is performed through a shared weight twin network, and feature level fusion operation is performed on the extracted features; corner position estimation is performed on the fused features to generate a final tracking bounding box, thereby completing the target tracking task.

Description

Technical Field

[0001] This disclosure relates to the field of target tracking based on two-dimensional vision, and more specifically, to a method, apparatus, electronic device, and computer-readable storage medium for single target tracking. Background Technology

[0002] The framework of a 2D vision-based target tracking system consists of four parts: a target dynamic model, visual feature extraction, an observation state model, and an optimization strategy. It aims to estimate the target's state between video frames and generate tracking bounding boxes. With the rapid development of computer vision, target tracking technology has made significant progress. Currently, there are three main types of target tracking methods: those based on traditional models, those based on correlation filtering, and those based on deep network models.

[0003] Traditional modeling methods such as particle filtering, mean shift, and optical flow are simple in structure, but their limited feature representation capabilities make them susceptible to changes in illumination and target occlusion, resulting in low tracking accuracy and limiting their practical applications. Correlation filtering methods achieve a good balance between accuracy and computational speed, but they still have shortcomings when dealing with target rotation and non-rigid deformation. In recent years, deep network modeling methods have shown excellent performance in terms of speed, accuracy, and robustness, but they rely on large amounts of labeled data and computational resources, their performance is easily affected by network structure and parameter configuration, and they risk tracking failure when target features are insufficient.

[0004] In existing tracking technologies, the ability to effectively resist interference from similar targets and improve the accuracy of target bounding box scale estimation directly affects tracking accuracy and robustness. Therefore, this invention aims to propose a stable single-target tracking method to address the aforementioned technical challenges and improve the accuracy and reliability of the tracking process.

[0005] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0006] The purpose of this disclosure is to provide a method, apparatus, electronic device, and computer-readable storage medium for single-target tracking, thereby overcoming, at least to some extent, one or more problems caused by limitations and defects in related technologies.

[0007] The above-mentioned technical objective of the present invention is achieved through the following technical solution:

[0008] Using the PrDiMP tracking algorithm as the basic tracking algorithm, when predicting the target score map of the current frame and the previous frame in the video, the algorithm finds the position of the target with higher score through preset constraints to complete the selection of candidate targets and generates two sets of feature vectors of all target candidate objects in the two frames.

[0009] The feature vectors of the current and previous frame target candidates in the video are input into the target matching module. Preset constraints are used to determine the pairing relationship between candidate targets and interference objects in adjacent frames. After pairing, the target in the current frame can be inferred and used as the initial prediction result.

[0010] The search region is truncated to twice the size of the initial prediction result region and input together with the template frame set into a Siamese network architecture with shared weights. The features of the two branches are extracted and fused hierarchically.

[0011] Estimate the corner positions of the fused features and generate the final tracking bounding box to complete the target tracking task.

[0012] In one exemplary embodiment of this disclosure, the method further includes:

[0013] The target matching algorithm takes the set of targets detected in the previous frame, the set of candidate targets, and the selected tracking target in the previous frame as input;

[0014] Verify whether a candidate target in the current frame matches any target detected in the previous frame, and check whether its assigned probability exceeds a preset threshold. If so, directly pair the candidate target with the corresponding target in the previous frame.

[0015] The system searches for a target to track within the target set detected in the current frame, prioritizing the determination of whether an object selected as a target for tracking existed in the previous frame. If such an object exists and its number matches the number of a candidate target in the target set of the current frame, then the candidate target with the same number is selected as the tracking target for the current frame.

[0016] In one exemplary embodiment of this disclosure, the method further includes:

[0017] The two sets of feature vectors of the candidate target in the previous frame and the current frame of the video sequence are input into the target matching module. Based on the preset constraints, the matching relationship between the interference object and the candidate target between two adjacent frames is obtained.

[0018] In one exemplary embodiment of this disclosure, the method further includes:

[0019] If there is another candidate target in the current frame of the video with a score higher than the historical score of the currently selected target, then the other candidate target is selected as the target to be tracked in the current frame.

[0020] When the selected tracking target in the previous frame of the video is no longer visible, the detection step needs to be re-executed, and it needs to be determined whether the target score with the highest target classifier score in the current frame exceeds a preset threshold. If so, the candidate target is temporarily selected as the tracking target for the current frame.

[0021] In one exemplary embodiment of this disclosure, the feature-level fusion of the method further includes:

[0022] The search regions in the current video frame and the template frame are input into a parameter-shared Siamese network for feature extraction. The extracted features are denoted as C and F, and F is decomposed into a first preset number of 3×3 filters F. m The second preset number of filters F (5×5) n ;

[0023] Using the first preset number of filters F respectively m The second preset number of filters F n The similarity between the template feature F and the feature C of the search region in the current frame is calculated, and the corresponding similarity maps are obtained in sequence.

[0024] Based on preset weights, the similarity maps are concatenated and merged to obtain target features optimized by the reference branch.

[0025] In one exemplary embodiment of this disclosure, the method further includes:

[0026] The corner positions are estimated on the fused features. Here, a prediction head based on the heatmap output is used for corner localization. The positions of the two corners of the bounding box are determined by the generated heatmap, and finally the tracking bounding box is generated to complete the target tracking task.

[0027] In one aspect of this disclosure, a single-target tracking device is provided, comprising:

[0028] The target feature extraction and modeling module is used to predict the target score map of the current frame and the previous frame in the video, and generate the feature vector of all candidate targets in the two frames.

[0029] The target matching module takes the target candidate feature vectors of the current frame and the previous frame as input, and determines the pairing relationship between candidate targets and interference objects in adjacent frames based on preset constraints. After completing the pairing, the target of the current frame can be inferred and an initial prediction result can be generated.

[0030] The feature-level fusion module is used to enlarge the initial prediction result region by two times and then crop it into the search region. This region is then input into a Siamese network with shared weights along with the template frame set to extract features from the two branches and perform hierarchical fusion.

[0031] The tracking result generation module is used to estimate the corner positions of the fused features and generate the final tracking bounding box.

[0032] In one aspect of this disclosure, an electronic device is provided, comprising:

[0033] A processor and a memory, wherein the memory stores computer-readable instructions that, when executed by the processor, are used to implement the method described in any of the preceding embodiments.

[0034] In one aspect of this disclosure, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, is used to implement the method described in any of the preceding descriptions.

[0035] In summary, the present invention has the following beneficial effects:

[0036] 1. This invention innovatively introduces a target matching module to specifically address the problem of the actual tracking target and interfering objects appearing in close similarity in appearance during tracking. With the help of the target matching module, effective matching of the tracking target and interfering objects is achieved across multiple frames of images. Simultaneously, by optimizing the matching strategy, the ability to suppress interfering objects is effectively improved, enhancing the stability of tracking.

[0037] 2. In this invention, to address the problem of insufficient accuracy in the measurement results of the bounding box scale, a feature-level fusion method and a corner position estimation method are proposed. These methods effectively integrate multi-level spatial information, make full use of feature details, enhance the target feature representation capability, and ensure that the bounding box size is more closely matched with the actual target, thereby further improving the accuracy of tracking.

[0038] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0039] The above and other features and advantages of this disclosure will become more apparent from the detailed description of exemplary embodiments thereof with reference to the accompanying drawings.

[0040] Figure 1 This is a flowchart of a single-target tracking method according to an exemplary embodiment of this disclosure;

[0041] Figure 2 This is a diagram illustrating the overall framework of the tracking network for a single-target tracking method according to an exemplary embodiment of this disclosure.

[0042] Figure 3 This is a diagram illustrating the target matching module of a single-target tracking method according to an exemplary embodiment of this disclosure;

[0043] Figure 4 This is a schematic diagram of the adjacent frame candidate target matching matrix of a single target tracking method according to an exemplary embodiment of the present disclosure;

[0044] Figure 5 This is a schematic diagram of the prediction network structure of a single-target tracking method according to an exemplary embodiment of the present disclosure;

[0045] Figure 6 This is a schematic block diagram of a single-target tracking device according to an exemplary embodiment of the present disclosure;

[0046] Figure 7 This is a block diagram of an electronic device according to an exemplary embodiment of the present disclosure;

[0047] Figure 8 This is a schematic diagram of a computer-readable storage medium according to an exemplary embodiment of the present disclosure. Detailed Implementation

[0048] To better understand the technical solution of the present invention, the following describes the exemplary embodiment in detail with reference to the accompanying drawings.

[0049] It should be understood that the described exemplary embodiments are merely some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention. Furthermore, those skilled in the art should recognize that the technical solutions of this disclosure can be implemented without certain specific details, or can be implemented by other methods, components, materials, devices, or steps. In some cases, to avoid obscuring various aspects of this disclosure, well-known structures, methods, devices, implementations, materials, or operations have not been described in detail.

[0050] The block diagrams shown in the accompanying figures represent functional entities only and do not necessarily correspond to physically independent entities. These functional entities can be implemented in software, or in one or more software and hardware modules, or some functional entities can be distributed across different network devices, processor devices, or microcontroller devices.

[0051] Example: A single-target tracking method, reference Figure 1 As shown, the single-target tracking method includes the following steps:

[0052] Step S110: Using the PrDiMP tracking algorithm as the basic tracking algorithm, when predicting the target score map of the current frame and the previous frame in the video, the position of the target with higher score is found through the preset constraint conditions to complete the selection of candidate targets, and two sets of feature vectors of all target candidate objects in the two frames are generated.

[0053] Step S120: Input the two sets of feature vectors of the candidate targets in the current frame and the previous frame of the video into the target matching module, and determine the pairing relationship between the candidate targets and interference objects between adjacent frames through preset constraints. After the pairing is completed, the target in the current frame can be inferred and used as the initial prediction result;

[0054] Step S130: Truncate the initial prediction result area to twice its size as the search area, and input it together with the template frame set into the Siamese network architecture with shared weights, extract the features of the two branches and perform hierarchical fusion;

[0055] Step S140: Estimate the corner positions of the fused features and generate the final tracking bounding box to complete the target tracking task.

[0056] In this example embodiment, the overall framework of the tracking network proposed in this invention is as follows: Figure 2 As shown below, a single-target tracking method in this example embodiment will be further described.

[0057] In step S110, the present invention constructs a candidate target set. This set includes the tracked target and interfering targets with highly similar appearance features. Assuming the number of candidate targets in each frame of the video is represented by N, the candidate targets in the current frame and the previous frame are labeled as set V and set V′, respectively. Based on the output of the basic tracking algorithm, the target score map s for each frame can be obtained. Using preset constraints, the coordinates c of the local maximum score are located in s. i Candidate target v can be extracted i The preset constraints are shown in formula (1).

[0058] φmax(s,c i )=1 and s(c i )≧τ (1)

[0059] When coordinate c i When the penalty is a local maximum in the score map s, φmax = 1; otherwise, φmax = 0. Meanwhile, τ is a threshold with a value of 0.2. Candidate target sets V and V′ can be constructed by searching for local maxima in the target score maps s and s′.

[0060] Next, a set of corresponding features is constructed for each extracted candidate target, including the target's position c in the image. i The score of the target classifier s i and appearance-based features f i =f(c i To achieve comprehensive reasoning on multiple vectors in this feature tuple, this invention designs a feature encoder that can convert this set of features into a single feature vector k corresponding to each candidate target. i The specific form is shown in formula (2).

[0061]

[0062] in, This represents a multilayer perceptron, used to convert s i and c i Mapping to f i The same dimensional space facilitates the execution of subsequent target matching tasks.

[0063] In step S120, to effectively resist interference from similar targets, the present invention introduces a target matching module, such as... Figure 3 As shown, this module can actively track interfering targets and match candidate targets and interfering targets between two adjacent frames of the video, thereby improving the robustness of tracking.

[0064] To calculate the allocation matrix H between two sets V and V′, this invention proposes the concept of "candidate bins," where unmatched candidate targets are assigned to candidate bins. To this end, an additional row and column are added to the allocation matrix H, representing the candidate bins respectively, thus improving the allocation mechanism. If v in the current frame... i (i = 1, 2, 3, ..., N) and v′ in the previous frame j If (j=1,2,3...,N′) matches, then H i,j =1, otherwise H i,j =0. When a new candidate target v is added to set V. i When, the corresponding H in the matrix i,N′+1 =1; similarly, when candidate target v′ in set V′ j When it no longer appears in set V, there is a corresponding H in the matrix. N+1,j =1.

[0065] Between two adjacent frames of the video, the candidate target v i and v′ j When matching, successful pairing requires the following two constraints to be met, as shown in formula (3).

[0066]

[0067] When there are unmatched candidate targets in sets V and V′, these targets must be matched with candidate boxes, as shown in Equation (4).

[0068]

[0069] in, This represents the number of times a candidate target is successfully matched between two adjacent frames. The assignment matrix of candidate targets between two adjacent frames of the video is as follows: Figure 4 As shown.

[0070] The target matching algorithm used in this invention will be described below. This algorithm includes a pairing process for candidate targets between adjacent frames, and a re-detection mechanism for lost tracking targets. The algorithm takes as input the target set O′ detected in the previous frame, the candidate target set V, and the selected tracking target o′ in the previous frame. Each candidate target contains two attributes: an ID number and a score list S, and supports the acquisition and setting of these attributes.

[0071] When performing target matching, the first step is to check the candidate target v. i Whether it matches a target detected in the previous frame. If the matching probability exceeds the set threshold ω = 0.7, then the candidate target v is selected. i The target is directly paired with the target corresponding to the previous frame, and the classifier score s of the candidate target is calculated simultaneously. i Add it to its score list S, and add the candidate target to the target set O of the current frame. If candidate target v i If a target fails to match any target in the previous frame, a new number is assigned to it, and it is added to the target set O. Conversely, if a target from the previous frame cannot find a matching candidate target in the current frame, it is removed from set O. Through these operations, all candidate targets in the current frame will be matched with either an existing target or a newly created target.

[0072] Next, the tracking target needs to be determined from the target set O detected in the current frame. First, check if there is an object selected as a tracking target in the previous frame. If so, and its number matches the candidate target number detected in the current frame, then the candidate target with the same number in the current frame is taken as the new tracking target. However, in real-world scenarios, the tracking target may be lost due to occlusion or leaving the field of view, which may lead to the mistaken selection of interfering targets as tracking targets. To solve this problem, it is necessary to further check if there are other candidate targets in the current frame. If so, and their target classifier scores are s... i If the score is higher than the highest historical score of the currently selected target, it will be replaced as the new tracking target.

[0073] Furthermore, if the tracked target selected in the previous frame is no longer visible in the current frame, it needs to be re-detected. In this case, check the target in the current frame. i Does the highest candidate target score exceed the set threshold η = 0.5? If so, the candidate target is temporarily selected as the new tracking target. This mechanism effectively improves the accuracy and robustness of target matching and tracking.

[0074] In step S130, to calculate the similarity between the search region (test branch) of the current frame and the template frame set (reference branch), this invention employs a feature-level fusion method to optimize target feature extraction. In the test branch, the tracking target output by the target matching module is first used as the initial prediction, and based on this, twice the size of the target region is cropped as the search region. Then, its size is adjusted to 256×256 before input. In the reference branch, a template frame set is constructed, containing the first frame of the video sequence and the frame with the highest confidence and above the threshold t=0.85 from the last thirty frames. For these two frames, similarity is calculated with the input of the test branch, and multiple similarity maps are concatenated using a concatenation operation. This preserves the initial tracking target information of the first frame while introducing the latest and highest-confidence target information from the tracking sequence.

[0075] In the template frame set, the first frame image is cropped to twice the size of the real target bounding box and resized to 256×256 before being input. For the other frame image, the target tracking bounding box is cropped to twice the size and resized to the same size before being input. Next, features from both branches are extracted using a Siamese network with shared weights, and the target features of the reference branch are extracted using a PreciseROIPooling layer. Finally, a feature-level fusion operation is used to integrate the features from the two branches, resulting in the target features optimized by the reference branch.

[0076] In the above process, the feature-level fusion process includes the following steps:

[0077] (1) Label the features of the search region in the current frame and the template frame extracted by the Siamese network with shared weights as C∈R respectively. C×H×W and F∈R C×H×W ;

[0078] (2) Decompose the template feature F stepwise into (H0-2)×(W0-2) 3×3 filters F m ∈R C×3×3 and (H0-4)×(W0-4) 5×5 filters F n ∈R C×5×5 ;

[0079] (3) Through filter F m F n F and F are used to calculate the similarity with the feature C of the search region, and the corresponding similarity maps are generated. and S∈R 1×H×W .

[0080] (4) For the generated multiple similarity maps S m S nSince the number of samples in the three categories differs, to balance the classification effect and ensure that the similarity map with fewer samples receives more attention, different weights are assigned to the three similarity maps. Then, the weighted similarity maps are concatenated and merged using a concatenate operation to obtain the target feature optimized by the reference branch.

[0081] In step S140, to obtain the final tracking bounding box, this invention employs a heatmap-based network output structure. This structure predicts the positions of the two corner points of the bounding box by generating a heatmap, thereby determining the complete target box location information. Specifically, as follows... Figure 5 As shown, the target feature vector obtained after feature layer fusion is first used as input information. After being processed by two parallel convolutional modules, two feature maps are generated. The convolutional module consists of the following core components: a convolutional layer that performs convolution operations to extract features and generate feature maps; a normalization layer that normalizes the feature maps to stabilize the training process and accelerate convergence; an activation function layer that introduces nonlinearity and maps the result of linear combination to a nonlinear space; and an upsampling layer that restores resolution to ensure that the spatial information of the feature maps is preserved.

[0082] Subsequently, these two feature maps are subjected to 1×1 convolution operations to achieve channel dimensionality reduction and feature integration, reducing the number of channels from 32 to 1 while maintaining the spatial resolution. Finally, the two generated heatmaps are input into a corner pooling layer to detect corner features of the tracked target boundary. The corner pooling layer performs max pooling operations in the vertical and horizontal directions respectively, concentrating the target features to the top-left and bottom-right corners of the bounding box, thereby determining the final tracking bounding box.

[0083] This example embodiment is merely an explanation of the invention and is not intended to limit the invention. It does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired result. Additional or alternative steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps.

[0084] Furthermore, in this example embodiment, a single-target tracking device is also provided. (Refer to...) Figure 6 As shown, this single-target tracking device 200 may include: a target feature extraction and modeling module 210, a target matching module 220, a feature-level fusion module 230, and a tracking result generation module 240. Wherein:

[0085] The target feature extraction and modeling module 210 is used to predict the target score map of the current frame and the previous frame in the video, and generate the feature vector of all candidate targets in the two frames.

[0086] The target matching module 220 is used to input the target candidate feature vectors of the current frame and the previous frame, and to determine the pairing relationship between candidate targets and interference objects between adjacent frames through preset constraints. After the pairing is completed, the target of the current frame can be inferred and an initial prediction result can be generated;

[0087] The feature-level fusion module 230 is used to enlarge the initial prediction result region by two times and then crop it into the search region, and input it together with the template frame set into a Siamese network with shared weights to extract the features of the two branches and perform hierarchical fusion.

[0088] The tracking result generation module 240 is used to estimate the corner positions of the fused features and generate the final tracking bounding box.

[0089] The specific details of the aforementioned single-target tracking device module have been described in detail in the corresponding single-target tracking methods, and will not be repeated here.

[0090] It should be noted that although several modules or units of a single-target tracking device 200 have been mentioned in the detailed description above, this division is not mandatory. In fact, according to embodiments of this disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0091] Furthermore, in an exemplary embodiment of this disclosure, an electronic device capable of implementing the above-described method is also provided.

[0092] Those skilled in the art will understand that the present invention can be implemented as a system, method, or program product, including the following forms: pure hardware implementation, pure software implementation (such as firmware, microcode, etc.), or a combination of hardware and software implementation, collectively referred to as a "circuit", "module", or "system".

[0093] The following reference Figure 7 To describe an electronic device 500 according to such an embodiment of the present invention. Figure 7 The electronic device 500 shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0094] like Figure 7 As shown, the electronic device 500 is manifested in the form of a general-purpose computing device. The components of the electronic device 500 may include, but are not limited to: at least one processing unit 510, at least one storage unit 520, a bus 530 connecting different system components (including storage unit 520 and processing unit 510), and a display unit 540.

[0095] The storage unit stores program code that can be executed by the processing unit 510, causing the processing unit 510 to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of the present invention. For example, the processing unit 510 can perform actions such as... Figure 1 Steps S110 to S140 are shown in the diagram.

[0096] Storage unit 520 may include a readable medium in the form of a volatile storage unit, such as random access memory (RAM) 5201 and / or cache memory 5202, and may further include a read-only memory (ROM) 5203.

[0097] Storage unit 520 may also include a program / utility 5204 having a set (at least one) program module 5205, such program module 5205 including but not limited to: operating system, one or more application programs, other program modules and program data, each or some combination of these examples may include an implementation of a network environment.

[0098] Bus 530 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.

[0099] Electronic device 500 can communicate with one or more external devices 570 (such as keyboards, pointing devices, Bluetooth devices, etc.) via input / output (I / O) interface 550, and can also communicate with devices that support user interaction or devices used to connect other computing devices (such as routers, modems, etc.). Furthermore, electronic device 500 can connect to a network (such as a local area network, wide area network, or the Internet) via network adapter 560, which communicates with other modules of the device via bus 530. It should be noted that, although not shown in the figure, electronic device 500 may also be equipped with other hardware or software modules, such as microcode, device drivers, redundant processing units, external disk arrays, RAID systems, tape drives, and data backup systems.

[0100] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, terminal device, or network device, etc.) to execute the methods according to the embodiments of this disclosure.

[0101] In exemplary embodiments of this disclosure, a computer-readable storage medium is also provided, on which a program product capable of implementing the methods described above is stored. In some possible embodiments, various aspects of the invention may also be implemented as a program product comprising program code that, when the program product is run on a terminal device, causes the terminal device to perform the steps of the various exemplary embodiments of the invention described in the "Exemplary Methods" section above.

[0102] refer to Figure 8 As shown, a program product 600 for implementing the above-described method according to an embodiment of the present invention is described. It may employ a portable compact disc read-only memory (CD-ROM) and include program code, and may run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited thereto. In this document, the readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, apparatus, or device.

[0103] The program product may employ any combination of one or more readable media, including readable storage media and readable signal media. The readable storage media may be an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or apparatus, or any combination thereof. Common readable storage media include electrical connections, portable hard disks, hard disks, RAM, ROM, flash memory, optical fibers, CD-ROMs, optical storage, and magnetic storage.

[0104] Readable signal media may include propagated signals carrying program code, such as electromagnetic signals, optical signals, etc., suitable for data signals propagated via baseband or carrier wave. It may also be wireless, wired, optical fiber, or radio frequency (RF) media for transmitting program code.

[0105] The program code can be written in various programming languages, including object-oriented languages ​​(such as Java and C++) and procedural languages ​​(such as C). The program can be executed fully, partially, or in collaboration with remote devices, or even run entirely on a remote server. Remote devices can connect to the user device via networks such as local area networks (LANs), wide area networks (WANs), or the Internet.

[0106] It should be noted that the above figures are merely illustrative of the method and do not limit the execution order. Related processing can be executed synchronously or asynchronously in multiple modules, and the specific implementation is not limited by the figures.

[0107] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and embodiments are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the claims.

[0108] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A single-target tracking method, characterized in that, The method includes: Using the PrDiMP tracking algorithm as the basic tracking algorithm, when predicting the target score map of the current frame and the previous frame in the video, the algorithm finds the position of the target with higher score through preset constraints to complete the selection of candidate targets and generates two sets of feature vectors of all target candidate objects in the two frames. The feature vectors of the current frame and the previous frame of the video target candidate object are input into the target matching module. The pairing relationship between the candidate target and the interference object between adjacent frames is determined by the preset constraints. After the pairing is completed, the target in the current frame can be inferred and used as the initial prediction result. The search region is truncated to twice the size of the initial prediction result region and input together with the template frame set into a Siamese network architecture with shared weights. The features of the two branches are extracted and fused hierarchically. Estimate the corner positions of the fused features and generate the final tracking bounding box to complete the target tracking task.

2. The method as described in claim 1, characterized in that, The method further includes: The target matching algorithm takes the set of targets detected in the previous frame, the set of candidate targets, and the selected tracking target in the previous frame as input; Verify whether the candidate target in the current frame matches any target detected in the previous frame, and check whether its allocation probability exceeds a preset threshold; if so, directly pair the candidate target with the corresponding target in the previous frame. Search for a target to track in the target set detected in the current frame. First, determine whether there is an object selected as a target to track in the previous frame. If there is an object and its number is the same as the number of a candidate target in the target set of the current frame, then select the candidate target with the same number as the target to track in the current frame.

3. The method as described in claim 2, characterized in that, The method further includes: If there is another candidate target in the current frame of the video with a score higher than the historical score of the currently selected target, then the other candidate target is selected as the target to be tracked in the current frame. When the selected tracking target in the previous frame of the video is no longer visible, the detection step needs to be re-executed, and it needs to be determined whether the target score with the highest target classifier score in the current frame exceeds the preset threshold; if so, the candidate target is temporarily selected as the tracking target for the current frame.

4. The method as described in claim 1, characterized in that, The feature-level fusion method further includes: The search regions in the current video frame and the template frame are input into a parameter-shared Siamese network for feature extraction. The extracted features are denoted as C and F, and F is decomposed into a first preset number of 3×3 filters F. m The second preset number of filters F (5×5) n ; Using the first preset number of filters F respectively m The second preset number of filters F n The similarity between the template feature F and the feature C of the search region in the current frame is calculated, and the corresponding similarity maps are obtained in sequence. Based on preset weights, the similarity maps are concatenated and merged to obtain target features optimized by the reference branch.

5. The method as described in claim 1, characterized in that, The method further includes: The corner positions are estimated on the fused features. Here, a prediction head based on the heatmap output is used for corner localization. The positions of the two corners of the bounding box are determined by the generated heatmap, and finally the tracking bounding box is generated to complete the target tracking task.

6. A single-target tracking device, characterized in that, The device includes: The target feature extraction and modeling module is used to predict the target score map of the current frame and the previous frame in the video, and generate the feature vector of all candidate targets in the two frames. The target matching module is used to input the target candidate feature vectors of the current frame and the previous frame, and determine the pairing relationship between candidate targets and interference objects between adjacent frames through preset constraints; after completing the pairing, the target of the current frame can be inferred and the initial prediction result can be generated. The feature-level fusion module is used to enlarge the initial prediction result region by two times and then crop it into the search region. This region is then input into a Siamese network with shared weights along with the template frame set to extract features from the two branches and perform hierarchical fusion. The tracking result generation module is used to estimate the corner positions of the fused features and generate the final tracking bounding box.

7. An electronic device, characterized in that, The device includes: A processor and a memory, wherein the memory stores computer-readable instructions that, when executed by the processor, are used to implement the method of any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, is used to implement the method of any one of claims 1 to 5.

Images