Multi-template based object tracking method, system, medium and electronic device
By introducing multi-template information and an adaptive update mechanism, the problems of target tracking loss and poor versatility in existing technologies are solved, and stable and accurate target tracking in complex scenarios is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SHANGHAI JIAOTONG UNIV
- Filing Date
- 2024-02-26
- Publication Date
- 2026-07-24
Smart Images
Figure CN118097714B_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of image processing technology, and more specifically, to a target tracking method, system, medium, and electronic device based on multiple templates. Background Technology
[0002] Visual object tracking can locate targets in images, greatly advancing the automated processing of video data in tasks such as camera surveillance, robot vision, autonomous driving, and human-computer interaction. Although many methods have been introduced to achieve real-time, high-precision tracking, significant challenges remain in handling real-world scenarios such as occlusion, deformation, and target disappearance.
[0003] In recent years, Siamese network architecture has become the mainstream choice for target trackers. In Siamese network architecture, a cross-correlation response map is calculated between the target template and the input image in each frame, and the target's location is extracted through similar responses. To reduce training complexity, the template and image share the same backbone encoder when mapping from RGB to the feature space. Recently, the integration of Transformers with attention mechanisms has further enhanced the capabilities of Siamese networks, achieving significant performance improvements.
[0004] However, despite advancements in network detail design, the utilization of inherent properties in the 3D world remains limited, leading to relatively severe target loss during tracking. Essentially, both human visual imaging and camera imaging of targets in the environment involve mapping from 3D to 2D, resulting in different target appearances. Human observation reveals that, on the one hand, when recognizing familiar people, humans can often quickly identify them regardless of changes in perspective or clothing; on the other hand, humans are inherently capable of distinguishing targets from objects that occlude them, and applying appropriate attention to remove occlusion interference, thereby predicting the target's actual appearance and completing tracking. Computer-based trackers should also mimic human processing mechanisms, starting with incorporating multi-view information and updating target templates accordingly, to achieve more stable, efficient, and accurate target tracking in complex real-world scenarios.
[0005] Chinese invention patent CN116612157A designs a video single-target tracking method, device, and electronic device. It mainly improves target tracking performance by directly modifying the underlying structure of a Siamese network, adding a Transformer-based feature fusion network, and directly using the target appearance of the previous frame as the target template for the next frame. Chinese patent CN116703980A designs a target tracking method and system based on a pyramid pooling Transformer backbone network. It mainly improves target tracking performance by directly modifying the Siamese network's underlying structure, changing the backbone network to a pyramid pooling Transformer-based backbone network. This allows for attentive capture of global contextual information when mapping from the RGB image space to the feature vector space.
[0006] The existing patents represented by the two patents mentioned above mainly involve direct and large-scale modifications to the underlying structure of the twin network. Therefore, on the one hand, the network needs to be retrained, which is time-consuming and labor-intensive, and on the other hand, it is impossible to select the most suitable target tracking network base for specific real-world scenarios, resulting in poor versatility.
[0007] Meanwhile, existing technologies do not incorporate multi-template information, but instead use the target appearance of the initial frame or adjacent frames as the target appearance template. This inherently discards the target appearance information under multi-view and multi-occlusion states during real-world scenarios such as long-term target movement, deformation, and occlusion, and also fails to adaptively balance short-term and long-term appearance memory. Furthermore, they primarily employ specially designed Siamese networks as target trackers, resulting in extremely high system coupling and extremely low modularity. Summary of the Invention
[0008] In view of the deficiencies in the prior art, the purpose of this disclosure is to provide a target tracking method, system, medium and electronic device based on multiple templates.
[0009] To achieve the above objectives, according to one aspect of this disclosure, a multi-template-based target tracking method is provided, comprising:
[0010] Obtain the appearance templates of the target from multiple perspectives and determine the target appearance template library;
[0011] Based on the actual appearance of the target in each frame image and the target appearance template library, a preset similarity function and a preset update function are used to determine the target appearance template library that has been updated at each frame time.
[0012] Based on the updated target appearance template library and the preset template fusion function at each frame time, the fusion template corresponding to each frame time is determined;
[0013] The fusion template corresponding to each frame time is input into a preset target tracking network to determine the tracking result of the target over the entire time period.
[0014] Optionally, the step of determining the updated target appearance template library at each frame time using a preset similarity function and a preset update function, based on the actual appearance of the target in each frame image and the target appearance template library, includes:
[0015] At each frame time, the preset similarity function is used to determine the similarity between each appearance template in the target appearance template library and the actual appearance of the target in the current frame image;
[0016] Using the preset update function, the target appearance template library is updated based on the similarity between each appearance template in the target appearance template library and the actual appearance of the target in the current frame image, thereby determining the target appearance template library that has undergone the update process at each frame time.
[0017] Optionally, the step of using the preset update function to update the target appearance template library based on the similarity between each appearance template in the target appearance template library and the actual appearance of the target in the current frame image, and determining the updated target appearance template library at each frame time, includes:
[0018] The actual appearance of the target in each frame image is used as a new target appearance template example. The appearance template in the target appearance template library corresponding to the largest similarity is updated, and the target appearance template library after the update process is determined at each frame time.
[0019] Optionally, determining the fusion template corresponding to each frame time based on the updated target appearance template library and a preset template fusion function includes:
[0020] At each frame, the preset template fusion function is used to fuse each appearance template in the target appearance template library that has been updated in the previous frame, thereby determining the fused template corresponding to each frame.
[0021] Optionally, the step of inputting the fusion template corresponding to each frame time into a preset target tracking network to determine the tracking result of the target over the entire time period includes:
[0022] Based on the fusion template corresponding to each frame time, the preset target tracking network is used to track the target in each frame of the environmental image and determine the detection box of the target in each frame of the environmental image.
[0023] Based on the target detection bounding box in each frame of the environmental image, the tracking result of the target over the entire time is determined.
[0024] According to a second aspect of this disclosure, a multi-template-based target tracking system is provided, comprising:
[0025] The acquisition module is used to acquire the appearance templates of targets in multiple view states and determine the target appearance template library.
[0026] The update module is used to determine the updated target appearance template library at each frame time by using a preset similarity function and a preset update function, based on the actual appearance of the target in each frame image and the target appearance template library.
[0027] The template fusion module is used to determine the fusion template corresponding to each frame time based on the updated target appearance template library and the preset template fusion function at each frame time.
[0028] The target tracking module is used to input the fusion template corresponding to each frame moment into a preset target tracking network to determine the tracking result of the target over the entire time period.
[0029] Optionally, the update module includes:
[0030] The first determining submodule is used to determine, at each frame time, the similarity between each appearance template in the target appearance template library and the actual appearance of the target in the current frame image by using the preset similarity function.
[0031] The second determining submodule is used to use the preset update function to perform the update process on the target appearance template library based on the similarity between each appearance template in the target appearance template library and the actual appearance of the target in the current frame image, and to determine the target appearance template library that has been updated at each frame time.
[0032] Optionally, the template fusion module includes:
[0033] The second determining submodule includes:
[0034] The third determining submodule is used to take the actual appearance of the target in each frame image as a new target appearance template example, update the appearance template in the target appearance template library corresponding to the largest similarity, and determine the target appearance template library after the update process at each frame time.
[0035] According to a third aspect of this disclosure, a non-transitory computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the steps of the method provided in the first aspect of this disclosure.
[0036] According to a fourth aspect of this disclosure, an electronic device is provided, comprising:
[0037] A memory on which computer programs are stored;
[0038] A processor for executing the computer program in the memory to implement the steps of the method provided in the first aspect of this disclosure.
[0039] Compared with the prior art, the embodiments disclosed herein have at least one of the following beneficial effects:
[0040] The above technical solution involves acquiring target appearance templates from multiple perspectives to establish a target appearance template library. This library incorporates multiple template information to maintain long-term memory of different target appearances. Based on the actual appearance of the target in each frame, a preset similarity function and a preset update function are used to update the target appearance template library, introducing short-term memory and maintaining the target's multiple template information. A preset template fusion function is used to determine the fusion template corresponding to each frame, adaptively balancing the target's long-term and short-term memory. The fusion template corresponding to each frame is then input into a preset target tracking network to track the target throughout the entire timeframe. In real and complex scenarios, this reduces the probability of target tracking loss, achieving stable, effective, and accurate target tracking.
[0041] The embodiments disclosed herein solve the problem of insensitivity of relevant parameters through multi-dimensional index design, and have stronger universal adaptability in highly variable real-world application scenarios. Attached Figure Description
[0042] Other features, objects, and advantages of this disclosure will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:
[0043] Figure 1 This is a flowchart illustrating a multi-template-based target tracking method according to an exemplary embodiment.
[0044] Figure 2 This is a block diagram illustrating a multi-template-based target tracking system according to an exemplary embodiment. Detailed Implementation
[0045] The present disclosure will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present disclosure, but do not limit the present disclosure in any way. It should be noted that those skilled in the art can make several modifications and improvements without departing from the concept of the present disclosure. These all fall within the protection scope of the present disclosure.
[0046] Figure 1 This is a flowchart illustrating a multi-template-based target tracking method according to an exemplary embodiment.
[0047] like Figure 1 As shown, this disclosure provides a target tracking method based on multiple templates, including S11 to S14.
[0048] S11, obtain the appearance templates of the target in multiple view states, and determine the target appearance template library.
[0049] The target appearance template library includes multiple appearance templates of the target and the image features corresponding to each appearance template. By storing images of the appearance templates of the target in multiple view states and the image features corresponding to each appearance template, an adaptive long-term memory of the target is established.
[0050] The appearance template of a target in a multi-view state can include the appearance template of a target in an occluded state.
[0051] As an example, the target appearance template library corresponding to frame t is T. t ={T 1，t T 2，t , ..., T N，t It captures and memorizes the target's appearance history under different viewing angles and various occlusion conditions.
[0052] Among them, element T in the target appearance template library i，t represents the appearance template of the i-th target in the target appearance template library at time t, and N represents the number of appearance templates in the target appearance template library.
[0053] Using an appearance template library maximizes the diversity between appearance templates for a target, ensuring that each element captures the unique aspects of the target's appearance.
[0054] The target appearance template library can be stored in memory.
[0055] S12, based on the actual appearance of the target in each frame image and the target appearance template library, a preset similarity function and a preset update function are used to determine the target appearance template library that has been updated at each frame time.
[0056] This disclosure employs a multi-template update mechanism, which includes: on the one hand, determining the appearance templates of targets that need to be updated in the target appearance template library based on the actual appearance of the target in the current frame image and a preset similarity function; on the other hand, updating the appearance templates of targets that need to be updated in the determined target appearance template library using a preset update function and the actual appearance of the target in the current frame image, thereby realizing the update processing of the target appearance template library.
[0057] In one possible embodiment, S12 may include S21 to S22:
[0058] S21, at each frame time, a preset similarity function is used to determine the similarity between each appearance template in the target appearance template library and the actual appearance of the target in the current frame image.
[0059] As an example, at the initial frame time, after detecting the actual appearance of the target in the initial frame image, a preset similarity function is used to determine the similarity between each appearance template in the target appearance template library and the actual appearance of the target in the initial frame image.
[0060] As another example, at non-initial frame times, after detecting the actual appearance of the target in the current frame image, a preset similarity function is used to determine the similarity between each appearance template in the target appearance template library that has been updated at the previous frame time and the actual appearance of the target in the current frame image.
[0061] In one possible embodiment, assuming the t-th frame time is the current frame time, the target appearance template library T corresponding to the (t-1)-th frame time in the historical template library can be used. t-1 And the actual appearance of the target in the t-th frame image O t Design a pre-defined similarity function.
[0062] The preset similarity function is:
[0063] s i，t =f sim (T i，t-1 O t )
[0064] Among them, s i，t f represents the similarity value between the i-th appearance template in the target appearance template library at time t-1 and the actual appearance of the target in the image of frame t. sim Let T represent the formal nominal form of the similarity function. i，t-1 O represents the appearance template of the i-th target in the target appearance template library corresponding to the (t-1)-th frame. t This represents the actual appearance of the target in the t-th frame image.
[0065] Among them, f sim The definition and substitution can be customized according to actual needs. In this disclosure, the mean absolute error f is specifically defined. sim The form of expression is used to extend the above formula:
[0066]
[0067] Where MAE represents the mean absolute error, x represents the dimension in the x-axis, y represents the dimension in the y-axis, z represents the dimension in the z-axis, and T... i，t-1 In (x, y, z), x represents T. i，t-1 The dimension in the x-axis, and y represents T. i，t-1 The dimension in the y-axis, and z represent T. i，t-1 The dimension in the z-axis is O. t In (x, y, z), x represents O. t The dimension in the x-axis, and y-axis represent O. t The dimension in the y-axis, and the z-axis represent O. t Dimensionality in the z-axis.
[0068] S22, using a preset update function, the target appearance template library is updated based on the similarity between each appearance template in the target appearance template library and the actual appearance of the target in the current frame image, thus determining the target appearance template library that has been updated at each frame time.
[0069] Specifically, the actual appearance of the target in each frame image is used as a new target appearance template example, and the appearance template in the target appearance template library corresponding to the highest similarity is updated to determine the target appearance template library after the update process at each frame time.
[0070] Following the example above, the target appearance template library T corresponding to the (t-1)th frame of the historical template library can be used. t-1 And a preset update function is designed to determine the similarity between each appearance template in the target appearance template library determined by the preset similarity function and the actual appearance of the target in the current frame image.
[0071] The default update function is:
[0072] T i，t =f upd (T i，t-1 s i，t )
[0073] Among them, s i，t f represents the similarity value between the i-th appearance template in the target appearance template library at time t-1 and the actual appearance of the target in the image of frame t. upd T represents the formal name of the update function. i，t This represents the appearance template of the i-th target in the target appearance template library at time t, that is, the appearance template of the target after the update process at time t.
[0074] Among them, f upd The specific definition and replacement can be made according to actual needs. In this disclosure, a weighted update replacement f is adopted. updAnd extend the above formula:
[0075] T i，t =f upd (T i，t-1 s i，t ) = 1 rep ·O t +(1-1 rep )·[(1-α)T t-1 +αT t-1 ]
[0076] Among them, 1 rep The binary operator indicating whether to use weighted update can be designed according to the actual situation: If 1 rep =1 indicates that the weighted update method was not used, and the actual appearance O of the target in the t-th frame image was directly updated. t Replace T i，t If 1 rep =0 indicates that a weighted update method is used. α represents the update rate.
[0077] From the above, the complete update expression for the multi-template update mechanism can be expressed as follows:
[0078]
[0079] The historical template library T is updated through a multi-template update mechanism. t-1 The appearance template of the target corresponding to the maximum similarity between each appearance template and the actual appearance of the target in the t-th frame image is updated to the actual appearance of the target in the t-th frame image, thereby determining the updated target appearance template library T at the current t-th frame time. t .
[0080] In one possible embodiment, since i = argmin j (s j，t If the condition is always true, then the traditional single-template paradigm can be regarded as a simple special case of the multi-template update mechanism disclosed herein.
[0081] The traditional single-template paradigm is as follows:
[0082] T t = (1-α)·T t-1 +α·O t
[0083] Where α represents the update rate, and 0 < α < 1.
[0084] In the process where the traditional single-template paradigm is considered a simple special case of the multi-template update mechanism of this disclosure, the single template T updated at time t is... tIt can be viewed as a template for target tracking using a target tracker at frame t+1.
[0085] In this disclosure, the target appearance template library T is updated at frame t. t A target appearance template library for target tracking using a target tracker at frame t+1.
[0086] The above technical solution can effectively address the shortcomings of traditional single-template update mechanisms, prevent the loss of target information features in the long-term memory of the target appearance template library, balance the target information in short-term memory and long-term memory during the update of the target appearance template library, ensure the effectiveness of template diversity in the target appearance template library, and thus improve the effectiveness of target tracking.
[0087] S13. Based on the updated target appearance template library and the preset template fusion function at each frame time, determine the fusion template corresponding to each frame time.
[0088] In one possible embodiment, at each frame time, a preset template fusion function is used to fuse each appearance template in the target appearance template library that has been updated in the previous frame time, thereby determining the fused template corresponding to each frame time.
[0089] Specifically, at the initial frame time, a preset template fusion function is used to fuse the appearance templates in the initially acquired target appearance template library to determine the fused template corresponding to the initial frame time.
[0090] At non-initial frame times, a preset template fusion function is used to fuse each appearance template in the target appearance template library that has been updated in the previous frame time to determine the fusion template corresponding to the current frame time.
[0091] For example, at frame t, a preset template fusion function is used to fuse each appearance template in the target appearance template library that has been updated at frame (t-1) to merge multiple appearance templates into a single fused template, which is used as the fused template corresponding to frame t.
[0092] The preset template fusion function is as follows:
[0093] T t =f fuse (T t-2 O t-1 )
[0094] Among them, T t f represents the fusion template corresponding to frame t. fuse T represents the formal name of the template fusion function. t-2O represents the updated target appearance template library corresponding to frame t-2. t-1 This represents the actual appearance of the target in the (t-1)th frame image.
[0095] Among them, f fuse The specific definition and replacement can be made according to actual needs. In this disclosure, the average replacement f is adopted. fuse And extend the above formula:
[0096]
[0097] Among them, 1 avg Indicates whether to use the direct averaging fusion method. If 1 avg =1 indicates that the direct averaging fusion method is used. avg =0 indicates that the fusion method of direct averaging was not used, s i,t-1 This represents the actual appearance O of the target in the (t-1)th frame image determined by a preset similarity function. t-1 The updated target appearance template library T corresponding to the (t-2)th frame. t-2 The similarity value between the appearance templates of each target, where N represents the number of similarity values, and T i,t-1 The target appearance template library T represents the updated target appearance template library at time t-2. t-1 The i-th element in.
[0098] The above technical solutions ensure a balance between target information features in long-term memory and target information features in short-term memory, guaranteeing the effectiveness of the target appearance template library in actual use, thereby improving the effectiveness of target tracking.
[0099] S14, input the fusion template corresponding to each frame into the preset target tracking network to determine the target tracking result over the entire time.
[0100] The preset target tracking network can use any common target tracker based on the Siamese network structure, and there is no need to retrain or adjust the parameters of the target tracker, providing relatively fast adaptation and deployment capabilities in real-world application scenarios.
[0101] The target tracker essentially uses a TransT deep neural network.
[0102] In one possible embodiment, S14 may include S31 to S32.
[0103] S31, based on the fusion template corresponding to each frame time, a preset target tracking network is used to track the target in each frame of the environment image and determine the detection box of the target in each frame of the environment image.
[0104] Following the example above, the TransT deep neural network can be used to detect the location of the target in the current frame's environment image based on the fusion template corresponding to the current frame, and a rectangular target detection box can be used for annotation to determine the target detection box in the current frame's environment image.
[0105] As an example, at the initial frame time, the fusion template corresponding to the initial frame time determined according to the initially acquired target appearance template library is input into the preset target tracking network to determine the target detection box in the initial frame environment image.
[0106] As another example, at non-initial frame times, a single fusion template determined by the target appearance template library updated in the previous frame time is input into a preset target tracking network to determine the target detection box in the current frame's environment image.
[0107] S32, based on the target detection bounding box in each frame of the environment image, determine the target tracking result for the entire time.
[0108] The method described in S31 predicts the position of the target in each frame of the environmental image through frame evolution and marks the target in each frame of the environmental image with a rectangular target detection box, thereby achieving the target tracking result in the entire time.
[0109] The above technical solution acquires multi-view target appearance templates to establish a target appearance template library. This library incorporates multiple templates to maintain long-term memory of different target appearances. Based on the actual appearance of the target in each frame, a preset similarity function and a preset update function are used to update the target appearance template library, introducing short-term memory to maintain the target's multi-template information. A preset template fusion function is used to determine the fusion template corresponding to each frame, adaptively balancing the target's long-term and short-term memory. The fusion template corresponding to each frame is then input into a preset target tracking network to track the target throughout the entire timeframe. In real-world complex scenarios, this reduces the probability of target tracking loss, achieving stable, effective, and accurate target tracking. Furthermore, through multi-dimensional index design, the insensitivity of relevant parameters is addressed, resulting in stronger universal adaptability in highly variable real-world application scenarios.
[0110] Figure 2 This is a block diagram illustrating a multi-template-based target tracking system according to an exemplary embodiment.
[0111] Based on the same concept, this disclosure also provides a multi-template-based target tracking system 100, such as... Figure 2 As shown, it includes: acquisition module 110, update module 120, template fusion module 130, and target tracking module 140.
[0112] The acquisition module 110 is used to acquire the appearance templates of the target in multiple view states and determine the target appearance template library;
[0113] The update module 120 is used to determine the updated target appearance template library at each frame time by using a preset similarity function and a preset update function, based on the actual appearance of the target in each frame image and the target appearance template library.
[0114] The template fusion module 130 is used to determine the fusion template corresponding to each frame time based on the updated target appearance template library and the preset template fusion function at each frame time.
[0115] The target tracking module 140 is used to input the fusion template corresponding to each frame into the preset target tracking network to determine the tracking result of the target over the entire time.
[0116] The above technical solution involves acquiring target appearance templates from multiple perspectives to establish a target appearance template library. This library incorporates multiple template information to maintain long-term memory of different target appearances. Based on the actual appearance of the target in each frame, a preset similarity function and a preset update function are used to update the target appearance template library, introducing short-term memory and maintaining the target's multiple template information. A preset template fusion function is used to determine the fusion template corresponding to each frame, adaptively balancing the target's long-term and short-term memory. The fusion template corresponding to each frame is then input into a preset target tracking network to track the target throughout the entire timeframe. In real and complex scenarios, this reduces the probability of target tracking loss, achieving stable, effective, and accurate target tracking.
[0117] Optionally, the update module includes:
[0118] The first determining submodule is used to determine, at each frame time, the similarity between each appearance template in the target appearance template library and the actual appearance of the target in the current frame image by using the preset similarity function.
[0119] The second determining submodule is used to use the preset update function to perform the update process on the target appearance template library based on the similarity between each appearance template in the target appearance template library and the actual appearance of the target in the current frame image, and to determine the target appearance template library that has been updated at each frame time.
[0120] Optionally, the second determining submodule includes:
[0121] The third determining submodule is used to take the actual appearance of the target in each frame image as a new target appearance template example, update the appearance template in the target appearance template library corresponding to the largest similarity, and determine the target appearance template library after the update process at each frame time.
[0122] Regarding the embodiments of the above system, the specific ways in which each module performs operations have been described in detail in the embodiments of the method, and will not be elaborated here.
[0123] Based on the same concept described above, in another embodiment of this disclosure, an electronic device is also provided, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor, wherein the processor executes a multi-template-based target tracking method when executing the program.
[0124] Optionally, the memory is used to store programs; the memory may include volatile memory, such as random-access memory (RAM), such as static random-access memory (SRAM), double data rate synchronous dynamic random-access memory (DDR SDRAM), etc.; the memory may also include non-volatile memory, such as flash memory. The memory is used to store computer programs (such as application programs, functional modules, etc. that implement the above methods), computer instructions, etc., and the aforementioned computer programs, computer instructions, etc., can be partitioned and stored in one or more memories. Furthermore, the aforementioned computer programs, computer instructions, data, etc., can be accessed by the processor.
[0125] The aforementioned computer programs, computer instructions, etc., can be stored in partitions within one or more memory locations. Furthermore, the aforementioned computer programs, computer instructions, data, etc., can be accessed by a processor.
[0126] A processor is used to execute a computer program stored in memory to implement the various steps of the methods involved in the above embodiments. For details, please refer to the relevant descriptions in the preceding method embodiments.
[0127] The processor and memory can be separate structures or integrated structures. When the processor and memory are separate structures, they can be coupled together via a bus.
[0128] In this embodiment of the disclosure, a non-transitory computer-readable storage medium is also provided, on which a computer program is stored, which, when executed by a processor, implements the steps of a multi-template-based target tracking method in any of the above embodiments.
[0129] Those skilled in the art will understand that embodiments of this disclosure can be provided as methods, systems, or computer program products. Therefore, this disclosure can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this disclosure can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0130] This disclosure is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0131] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0132] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0133] Although preferred embodiments of this disclosure have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this disclosure.
[0134] Obviously, those skilled in the art can make various modifications and variations to this disclosure without departing from its spirit and scope. Therefore, if such modifications and variations fall within the scope of the claims of this disclosure and their equivalents, this disclosure is also intended to include such modifications and variations.
Claims
1. A target tracking method based on multiple templates, characterized in that, include: Obtain the appearance templates of the target from multiple perspectives and determine the target appearance template library; Based on the actual appearance of the target in each frame image and the target appearance template library, a preset similarity function and a preset update function are used to determine the target appearance template library that has been updated at each frame time. Based on the updated target appearance template library and the preset template fusion function at each frame time, the fusion template corresponding to each frame time is determined; The fusion template corresponding to each frame time is input into a preset target tracking network to determine the tracking result of the target over the entire time. The step of determining the updated target appearance template library at each frame time using a preset similarity function and a preset update function, based on the actual appearance of the target in each frame image and the target appearance template library, includes: At each frame, the preset similarity function is used to determine the similarity between each appearance template in the target appearance template library and the actual appearance of the target in the current frame image. Using the preset update function, the target appearance template library is updated based on the similarity between each appearance template in the target appearance template library and the actual appearance of the target in the current frame image, thereby determining the target appearance template library that has undergone the update process at each frame time.
2. The method according to claim 1, characterized in that, The step of using the preset update function to update the target appearance template library based on the similarity between each appearance template in the target appearance template library and the actual appearance of the target in the current frame image, and determining the updated target appearance template library at each frame time, includes: The actual appearance of the target in each frame image is used as a new target appearance template example. The appearance template in the target appearance template library corresponding to the largest similarity is updated, and the target appearance template library after the update process is determined at each frame time.
3. The method according to claim 1, characterized in that, The step of determining the fusion template corresponding to each frame time based on the updated target appearance template library and the preset template fusion function includes: At each frame, the preset template fusion function is used to fuse each appearance template in the target appearance template library that has been updated in the previous frame, thereby determining the fused template corresponding to each frame.
4. The method according to claim 1, characterized in that, The step of inputting the fusion template corresponding to each frame time into a preset target tracking network to determine the tracking result of the target over the entire time period includes: Based on the fusion template corresponding to each frame time, the preset target tracking network is used to track the target in each frame of the environmental image and determine the detection box of the target in each frame of the environmental image. Based on the target detection bounding box in each frame of the environmental image, the tracking result of the target over the entire time is determined.
5. A target tracking system based on multiple templates, characterized in that, include: The acquisition module is used to acquire the appearance templates of targets in multiple view states and determine the target appearance template library. The update module is used to determine the updated target appearance template library at each frame time by using a preset similarity function and a preset update function, based on the actual appearance of the target in each frame image and the target appearance template library. The template fusion module is used to determine the fusion template corresponding to each frame time based on the updated target appearance template library and the preset template fusion function at each frame time. The target tracking module is used to input the fusion template corresponding to each frame time into a preset target tracking network to determine the tracking result of the target over the entire time. The update module includes: The first determining submodule is used to determine, at each frame time, the similarity between each appearance template in the target appearance template library and the actual appearance of the target in the current frame image by using the preset similarity function. The second determining submodule is used to use the preset update function to perform the update process on the target appearance template library based on the similarity between each appearance template in the target appearance template library and the actual appearance of the target in the current frame image, and to determine the target appearance template library that has been updated at each frame time.
6. The system according to claim 5, characterized in that, The second determining submodule includes: The third determining submodule is used to take the actual appearance of the target in each frame image as a new target appearance template example, update the appearance template in the target appearance template library corresponding to the largest similarity, and determine the target appearance template library after the update process at each frame time.
7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the steps of the method described in any one of claims 1-4.
8. An electronic device, characterized in that, include: A memory on which computer programs are stored; A processor for executing the computer program in the memory to implement the steps of the method according to any one of claims 1-4.